Google announced a new indexing system called Caffeine. Google said the new indexing system would be show 50% newer results in the web search. It is a great achievement as the content is shown to search as soon as it is created.
This is a great news for bloggers, as all bloggers want their content to be indexed in google as soon as possible and shown in the results page. Previously when you searched google server, the results were displayed from a list ofpreviouls indexed pages. As older and more refernced content is given more prefernce and newer content would take a lot of time to reach the same level of prestige.
But the new indexing scheme Caffeine is said to have a higher percentage of newer content. Google beore this employed live search using twitter and other real time search platforms to provide real-time web search. Google made a really good attempt to explain all this here.
To keep up with the evolution of the web and to meet rising user expectations, Caffeine was built. The image below illustrates how the old indexing system worked compared to Caffeine:
Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.
Some interesting facts google told us :-
Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles.
We hope we all love google but never actually know the real effort google puts behind the simple box search page we see.
Source - Googleblog