Loading...This site has...
3,838,717
...likes, shares & fans. Join us!
Media Mentions (Web and Books)
Other Technology Topics
- Computer Dictionary (33)
- Internet Dictionary (19)
- Network Dictionary (10)
- Project Management Definitions (18)
- Project Team Roles Definitions (20)
Hi. I have added the plugin. Problem, I do believe, is that when my site was setup (I inherited this site) they added /wp to the end of the url, which isn't within the robots.txt url. I get a 500 error without the /wp. Thanks.
This plugin won't work with sites that are in a sub-folder.
You can preview your robots.txt file here (opens a new window). If your robots.txt file doesn't match what is shown below, you may have a physical file that is being displayed instead.
When I click on that preview, mine says User-agent: *, Disallow: /. Does this mean the plug in is not working?
Yes it looks like something is interfering with the plugin.
I installed this plugin after searching on google for hours "how to add robots.txt" file to my wordpress site http://amazingnewsnetwork.com but could not find a reasonable solution. I came across this plugin through google and was surprised to see it can create this file in seconds.
Thanks for saving my time, here comes a 5-star review.
Hi, Thank you for your plugin,
I have installed it, happy with its content and easy application. However, I realized that it has not made any reference to the sitemap. I added
Sitemap: http://marshamehran.com/sitemap.xml to an empty robot.txt file through cpanel. still it is not working. I appreciate your assistance
Abbas
If you have an actual robots.txt file it will override this plugin. You should choose to use one or the other, not both.
Thank you for so quich reply
I did not have any robots.txt. I installed the plugin. I search it in Chrome browser, it was there. Then I checked the my site files through cpanel. There was an empty robots.txt file. If the plugin insert a virtual robots, why do I have that empty robots file. What do you suggest me to do? delete that empty one? or.. Thanks
I can't say why you have an empty robots.txt file. Perhaps your host created it for you. You can either delete the file and use this plugin or just use the file and make edits to the file.
I've installed your robots.txt plugin, however it's not affecting the robots.txt file - however wordpress is installed in a sub-folder ( ../web/ )
is there a fix for this?
There's no easy fix. The problem is that when a file is requested outside of the WP folder (robots.txt is always requested from the root), the WP engine doesn't "know" about the request and so it can't intercept it and return the virtual robots.txt file. You could get this to work with the right settings in the .htaccess file, but I don't have the exact entries handy.
Note that the issue with sub-folders refers to your site being in a sub-folder off of the domain e.g. domain.com/your-wp-site-under-this-folder/. A sub-folder on the server "behind-the-scenes" is perfectly fine.
I have a problem, I installed your plugin and since I can't rewrite my robots.txt. I remove your plugin and same problem. I add a physical robot.txt in my root and always same problem. I can't edit my robot.txt, that's a big problem.
If you've deactivated and deleted the plugin, then it can no longer be impacting your site in any way.
Sorry, it won't let me reply to your comment.
Yes I have, I still get the 404 error. My site isn't live, could that be the issue? I have a plug-in for my sitemaps but that seems to be working?
You should still be able to view your robots.txt file even if the site isn't live.
Unfortunately I'm not sure what the problem is. These things are hard to diagnose without errors.
If you'd be willing to let me log into your site, I'd could "look around". If you're game, email me the info: support@infolific.com. Don't post it here!
Hi there,
I've installed this on my Wordpress but whenever I click preview I get a '404 Not Found' error. What am I doing wrong?
Is your WordPress installation in a sub-folder like /blog/ or something similar?
I don't think so, it's in the public_html folder?
Have you tried accessing the robots.txt by typing in your site's URL followed by /robots.txt?
Sorry, I don't understand the "FAQ: Will it conflict with any existing robots.txt file?
If a physical robots.txt file exists on your site, WordPress won't process any request for one, so there will be no conflict."
I have a physical robots.txt file (blocking google my business) that is different from the one created by your plugin. Will your plugin OVERRIDE the physical robots.txt file?
A physical file will override the plugin.