We implemented a rating system on a site some time back, which included a link to a script. However, at 3/5 p.m. with many ratings on the site and more than 1-5 ratings, we have started to suspect that the search engine crawlers are doing it through. URLs look like this:
http://www.thesite.com/path/to/the/page/rate?uid=abcdefghijk&value=3 When we started, we add the following to our robots.txt:
User-agent: * Decline: / rate Are they wrong or googlebot and others are just ignoring our robots.txt?
You should use the post for those tasks which change things as search engines generally Apart from this, do not submit the form; In addition to this, it will prevent users who download your website from frequent votes to your website (for example, with wget).
Depending on your site, even after dealing with voting there may be a JavaScript solution.
About your robots.txt: This route path - that is - and if your rating system is at / blah / rate, then you need to reject : / blah / rate instead of reject: / rate
Comments
Post a Comment