Like a traveler coming to a country, the Robot needs a guide — which is, the robot.txt file. It's a specific guide, though, as it'll only tell the Robot which cities he shouldn't see.
Like, if you're at the crossroads and one of the roads leads to a private place you shouldn't visit, there will be a NO TRESPASSING sign on the way.
So, if there's something to hide, a robots.txt file is a must for your website. It helps you keep the Robot away from anything that's not good for your Search Engine rankings. Yep, just tell him not to go here or there — and he'll believe you.
You can make a robots.txt file yourself, though it's rather your webmaster's business. So ask your webmaster to write a robots.txt file for your site, or do it yourself.
Remember, if you rewrote the dynamic URLs we talked above, use robots.txt to forbid the old URLs like this:
http://www.myshop.com/showgood.php?category=56&good=54146
After you have the robots.txt file, run it through a validator to ensure it's written correctly.
Hundreds of robots.txt validators can be found on the web. You can apply Google's tool, or this one, for instance:
http://www.invision-graphics.com/robotstxt_validator.html
As soon as the robots.txt file's correct, you needn't worry, as it will only do you a lot of good, and no harm.
Like, if you're at the crossroads and one of the roads leads to a private place you shouldn't visit, there will be a NO TRESPASSING sign on the way.
No TRASSPSSING sign
A robots.txt file will prevent the Robot from going to some pages with sensitive material, web pages that you don't want to be found through Google search (for instance, the "shopping cart"), pages that are not important or can be negative for your rankings. And, you can direct the Robot to other, keyword-rich pages, instead.
So, if there's something to hide, a robots.txt file is a must for your website. It helps you keep the Robot away from anything that's not good for your Search Engine rankings. Yep, just tell him not to go here or there — and he'll believe you.
You can make a robots.txt file yourself, though it's rather your webmaster's business. So ask your webmaster to write a robots.txt file for your site, or do it yourself.
Remember, if you rewrote the dynamic URLs we talked above, use robots.txt to forbid the old URLs like this:
http://www.myshop.com/showgood.php?category=56&good=54146
After you have the robots.txt file, run it through a validator to ensure it's written correctly.
Hundreds of robots.txt validators can be found on the web. You can apply Google's tool, or this one, for instance:
http://www.invision-graphics.com/robotstxt_validator.html
As soon as the robots.txt file's correct, you needn't worry, as it will only do you a lot of good, and no harm.
No comments:
Post a Comment