Surfing As GoogleBot (no more "Free Tours"...)
Yes, GoogleBot has privileges and it is not enough to simply change the User-Agent string to Googlebot. Here are the five factors that are important:
1. Beat IP Delivery: Use Google Translate as a Proxy, translating from spanish->english even though the site is already in English. (http://www.google.com/language_tools?hl=en)
2. Beat User-Agent Cloaking: Use the FirefoxUser-Agent Switcher to spoof as GoogleBot (https://addons.mozilla.org/firefox/59/)
3. Beat Javascript Detection: Use the Firefox Web Developer Toolbar to turn off javascript. (https://addons.mozilla.org/firefox/60/)
4. Beat Cookie Detection: Use the Firefox Web Developer Toolbar to turn off cookies. (https://addons.mozilla.org/firefox/60/)
5. Beat Referer Detection: Use the Firefox RefControl Extension to prevent referer from being sent. (https://addons.mozilla.org/firefox/953/)
3 votes
