Google Introduces New Web Crawler – Optimizing Google bot’s performance

1.25K viewsSearch Engine Optimization

Google Introduces New Web Crawler – Optimizing Google bot’s performance

Google has introduced a new web crawler called “GoogleOther\”. It is search index crawler which helps to alleviate strain on Googlebot. The new crawler will help Google to optimize and streamline its crawling operations.

Features of GoogleOther

  • GoogleOther helps to optimize crawling operations
  • The main purpose of the new GoogleOther crawler is to take over non-essential tasks like research and development (R&D) crawls, which are not directly related to search indexing and search indexing will be handled by Google bot
  • GoogleOther shares infrastructure with Googlebot ;it possesses the same limitations and features, including host load limitations, robots.txt (albeit with a different user-agent token), HTTP protocol version, and fetch size.
  • Introduction of this will not significantly impact the websites

It is unclear what would happen if you try to block this new bot. But keep an eye on it and if it is causing too much stress on your server, you can decide if you want to allow or deny access to your site for this new Googlebot.

Anuckshana Ganeshamoorthy Answered question April 23, 2023
1

Thanks for sharing this insightful information!
Google has introduced GoogleOther to optimize Googlebot’s performance. Some of the features include improved performance and efficiency, the ability to provide webpages more accurately, and better JavaScript support. This could lead to better indexing and improve the search results for users.

Anuckshana Ganeshamoorthy Answered question April 23, 2023
0