Google Updates Search with Caffeine

Today is the official launch of Caffeine, a new method Google is using to index the web. Google is a little bit mum on if this is effecting the actual search ranking algorithms or if this update is designed more purely to increase the speed of updating the index.

Google’s description of Caffeine:

Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.

With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index. That means you can find fresher information than ever before—no matter when or where it was published.

I applaud their efforts to keep content fresh and relevant. Speed of index, however, is not always useful. As a consumer, I am not concerned with Google’s ability to quickly index, say, Twitter, especially when a good portion of Twitter is simply noise. What I rely on Google for — and should be the crux of any search engine — is matching my query to relevant information.

A few years ago, search engines used to publish the size of their index. There was a race among the search engines to say they had indexed larger portions of the Web than any other search engine. There was a sub-text that a larger index was inherently better, as if they were comparing the size of their manhood. But again, the measure of a search engine is about how relevant the search results are for the consumer.

This is how Google rose to power. Back in Google’s infancy (2001), they were providing more relevant results to consumers despite the fact that their index was significantly smaller than Yahoo’s and Excite’s.

Back in October of last year, Bing announced it was indexing Twitter in real time. Again, with the sub-text that speed makes Bing better. Now Google launches Caffeine geared at indexing the web faster. I applaud these efforts, as long as they do not forget that relevancy is king.