Google announced it is changing the user agent name of the current GoogleBot in December. We knew this was coming when Google launched the new Evergreen GoogleBot. Google didn't change the user agent ...

Google announced that starting in December Google will update Google’s user agent to represent the version of Chrome that GoogleBot is running. It is important to update any code you may have that may ...

There have been an ongoing discussions over the past few weeks across social media that Googlebot has dramatically reduced its crawling. For example, the founder of a web crawl analysis service ...

Google updated their Googlebot Developers Support Page to reflect that Google is now able to try downloading pages via the latest HTTP/2 protocol. This is effective November 2020. The Googlebot ...

There is no doubt that a ton of bot activity on one's sites are from rogue spiders. Spider or bots that pretend to be legit bots but are there to steal your content. We have covered several sessions ...

Google has added two new crawlers to the Googlebot family of web crawlers: Google said these two new crawlers are versions of GoogleOther optimized for fetching image and video bytes. GoogleOther was ...

In December 2011 Google announced that it now has a smartphone Googlebot-Mobile. Historically, Googlebot-Mobile has concerned itself only with mobile sites designed for feature phones, so this is a ...