Google announced Caffeine update on August 10,2009. The update was so massive that Google provided months of what they called a “Developer Preview”. And on June 8,2010,it finally rolled out.
Caffeine was a new web indexing system.
This new system allowed Google to crawl and store data far more efficiently.
In fact, by Google’s own account they were able to not only increase their index but provide far fresher results (50 percent fresher by their estimates).
Basically in their old indexing system, pages and content types were put into a category based on the perceived freshness requirements (as it relates to this update at any rate).
Different crawlers were sent out, some looking for changes and others reindexing changed pages – all based on the classification of the content in question.
If a site was in the fresh category it was crawled by different bots that would add the content to the index quickly, but for most sites, their content would be reindexed every couple weeks.
Of course, this sets up a scenario where important and fresh content can be missing from the index due to a site classification.
With Caffeine, Google gained the capability to crawl, collect data, and add it to their index in seconds meaning far fresher information was available across a wider range of sites.
Why Google Launched Caffeine?
Caffeine wasn’t an algorithm update; in fact, it wasn’t an attempt by Google to impact rankings at all.
No, Caffeine was a complete rebuild of their indexing system.
Impact On Search
While the Caffeine Update had little direct impact on rankings, outside allowing for faster indexing of new content, it set the stage for some massive changes to come.
The pre-Caffeine index could not keep up with the ~1.3 billion websites on the internet today, nor could it deal with the variety of devices, data formats, and query input types we now take for granted.