After a full weekend of World Cup action from South Africa, I began to wonder just how popular it was turning out to be in the United States. I often tell clients that search engines are the largest focus groups in the world. So, I pulled some keyword data to see where in the US the World Cup was the most popular.

Keyword density map showing where World Cup related searches are taking place.

I wonder how many of those searches in Ohio are from me checking scores while at work?

This video provide a great, yet simple explanation of how search engines work:

Today is the official launch of Caffeine, a new method Google is using to index the web. Google is a little bit mum on if this is effecting the actual search ranking algorithms or if this update is designed more purely to increase the speed of updating the index.

Google’s description of Caffeine:

Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.

With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index. That means you can find fresher information than ever before—no matter when or where it was published.

I applaud their efforts to keep content fresh and relevant. Speed of index, however, is not always useful. As a consumer, I am not concerned with Google’s ability to quickly index, say, Twitter, especially when a good portion of Twitter is simply noise. What I rely on Google for — and should be the crux of any search engine — is matching my query to relevant information.

A few years ago, search engines used to publish the size of their index. There was a race among the search engines to say they had indexed larger portions of the Web than any other search engine. There was a sub-text that a larger index was inherently better, as if they were comparing the size of their manhood. But again, the measure of a search engine is about how relevant the search results are for the consumer.

This is how Google rose to power. Back in Google’s infancy (2001), they were providing more relevant results to consumers despite the fact that their index was significantly smaller than Yahoo’s and Excite’s.

Back in October of last year, Bing announced it was indexing Twitter in real time. Again, with the sub-text that speed makes Bing better. Now Google launches Caffeine geared at indexing the web faster. I applaud these efforts, as long as they do not forget that relevancy is king.

Special thanks to Columbus AMA for inviting Jeff Ivany and myself to present on how to use Search and Social Media together. Below is our presentation.

Note: If you are interested in having someone from EnginePoint Marketing or Conrad Phillips Vutech present this topic at your organization or conference, please contact us. Our presentations are designed more for the personal touch with our presenters.

In February 2006, I stipulated that Google was on the verge of becoming the next Yahoo. I did not mean it as a complement. This analysis came about as I was (again) watching Yahoo’s product line fracture.

Yahoo had a strange history of creating competing products. It was either a case of one part of a large company not aware of what another part of a large company was doing, or there was no cohesive strategy from leadership. (Maybe both?) At one point Yahoo had it’s own image sharing service, only later to acquire Flickr. The services were never merged into one, single useful product. Yahoo Images (which is now a image searching engine)  limped along for awhile, then was eventually shut down. Similarly, Yahoo had MyWeb which was different than Yahoo Bookmarks, despite the fact they apparently did the same thing. Both of which were identical to Delicious, which Yahoo purchased and… you get the idea.

Along the way, Yahoo experienced lots of product drift and were usurped by Google. That we all know.  We also know that Google is a very, very smart company. So why are they repeating Yahoo’s mistakes?

Farhad Manjoo has a fantastic article on Slate.com (link: Déjà Google) pointing out that Google seems to be repeating the same folly of Yahoo. Google is spinning out products that directly compete with other products they have already created. A few highlights:

• Newly launched Google Buzz will tell your friends where you are… duplicating Google Latitude which was launched last year
• Orkut already offered social media online profiles, but that didn’t stop Google from launching Google Profiles
• Google offers to save your bookmarks so they are available on any computer, but it is not-compatible with Chrome’s bookmarking feature that allows you to save your bookmarks so they are available on any computer.

Which brings me back to my original post from February 2006, Google is the Next Yahoo!, that points out that Microsoft could beat Google if Google accidentally becomes too much like Yahoo.

Looks like that is happening. OK, Microsoft it’s your move.

Yahoo! and Microsoft… Microsoft and Yahoo!. Together at last. Despite what they may say about increasing innovation and the operating income of each company, this deal is about one thing: Google.

The basic terms of the deal include Microsoft running the search engine portion, in essence, making Yahoo! search an extension of Bing. Yahoo will run the ad platform. This makes perfect sense, as Yahoo! has not considered search to be a core business product for years, and didn’t really want to get its hands dirty fighting Google.

Microsoft, on the other hand, doesn’t mind getting its hands dirty. More than that, they hate being second at anything and has show a willingness to tap their deep pockets to fund a fight. Microsoft sank nearly $500 million dollars into the Xbox before seeing a profit. That’s how bad they wanted to beat the PlayStation.

This sort of agreement is not new to Yahoo!. Around 2000, Yahoo’s search platform was powered by Google. Then someone at Yahoo! woke up and realized Google was actually a competitor, so the rushed a search platform into development.

The only thing that surprised me was the length of the agreement. Ten years! That’s five complete product life cycles on the internet. It seems improbably that these two companies could get along that long to make that time line a smart move. A deal of that length says one thing to me.

Yahoo! and Microsoft are heading for a merger. This lengthy arraignment is the equivalent of dating before they get married.

I surf the web a lot, but I’m still stunned from time-to-time when someone tries to fool a search engine to get better rankings. Sites do this in different ways: spamming, link baiting, cloaking… there are hundreds of “black hat” search techniques.

In almost all cases the webmaster goes to lengths to hide these techniques. In other instances, they don’t. Or more likely, the person doesn’t know they have tipped-toed into questionable SEO waters.

Which leads us today’s installment of “things not to do when optimizing your site.”

A few weekend searches over the nuances of Soccer rules, brought my wife to this page: (found here: http://www.soccerhelp.com/Soccer_Field_Index.shtml)

If it is not good for the user, then it is not good for the engine.

A close look reveals the problem, specifically the section that says “Dear visitor, ignore these links, they are for the search engines.”

All content is equal.

The rule of thumb is that search engines want to experience a site the exact same way a real, human user would. If content not intended for human eyes, then a search engine doesn’t want to “see” it either. Even if you try to section it off in a fun, tongue-in-cheek way on your site.

With the launch of Bing this week, as well as the recent introductions of Wolfram Alpha and Zuula, I thought I’d share my tips for how I kick the tires on a new search engine to see how good it is.

1. Type in your own name.
You know who you are, so see how much the search engine knows about you. Unless you have a very common name, even a mediocre search engine should return your facebook page somewhere on the first page or two.

2. Search for Viagra.
Really. Do a search on Viagra. If a search engine wants to be good it has to know how to handle spam, and there is TONS of spam for Viagra. Everyone hates spammy search results, so this is something new engines must have solved before they open to the public. If you don’t see legit medical and product information for Viagra, then they haven’t figured out how to handle spam.

3. Do a search on Barack Obama.
Next to relevancy, engines must be able to return fresh results. A search on the President should have a decent number of results that are no more than 2 days old.

If a new search engine gets good marks on all three of those tests, then they have spent time and effort developing a worthwhile, sophisticated backbone.

Slate.com seems to think so in this interesting article. They touch upon some of the old snake oil perspectives of SEO that need to stay in the past. Certainly there were/are people/firms that are not highly ethical, but SEO better or worse than a financial planner. Some really want to help, some just want your money.

Ross Dunn has a great write up in which he intelligently questions Google’s acquisition of SEO firm Performics as part of the DoubleClick deal.

The fact of the matter is a leading search engine like Google who claims to highly value its "don’t be evil" mantra will rapidly lose any remaining credibility if it continues to operate a SEO/SEM company.