I’m a big fan of the Bob & Tom morning radio show. In fact, I’m a VIP member on their website. So I was highly interested when I found out they would be talking to Scott Jones the co-founder of ChaCha Search.

He mentioned a few interesting facts about the new search engine, such as the working name of ChaCha was SNI, or Scott’s New Idea. After a few months and a few thousand name ideas later, they came up with ChaCha based on the meaning of Chinese meaning of “Cha” which is “search.” Scott Jones also said that in the near future ChaCha will offer a custom home page as well. Similar to a My Yahoo, this MyChaCha page will allow a user to set up their own portal page with news, sports scores, and other items of interest.  Also coming: a mobile version of ChaCha.

ChaCha, still officially in beta, launched as a search engine with a guided search feature. You can search independently or you can search with the help of a guide. With an instant message-like window, you can talk to a human guide about what you are looking for and they can push search results to your screen.

ChaCha has gotten a fair amount of press coverage, as indicated by the Bob & Tom coverage this morning. Since ChaCha launched, a few colleagues and I have tested the service and found it lacking. Maybe Scott Jones should spend less time investing in PR and more time investing in training for their guides.

Essentially ChaCha reinforces the importance of good customer service. The nature of customer service is that its only as good as the person providing it. ChaCha search it is no different. Results can vary based on the knowledge and customer service ability of the guide helping you. Scott Jones claims 30,000 human guides behind the technology that “self-select” their area’s of expertise.

In order to test it out, myself and five other colleagues went to ChaCha at different times and conducted a guided search on the same topic to see how our results would vary. We all acted like we were in the market to buy a computer and was considering a laptop but didn’t know much about them. I wanted to see what types of questions we would be asked, such as if we considered a Mac or PC, a brand preference, or if we were directed to big brand sites like Dell, or toward informational sites.

Overall, the guides asked very few or no questions. Here is an example of a typical guided search conversation we experienced:

Status: Connecting…

Status: Looking for a guide …

Ring: …

Ring: …

Ring: …

Status: Connected to guide: Brian(73486)

Brian(73486): Welcome to ChaCha!

Brian(73486): Hi there!

Brian(73486): What can I help you with on this topic?

You: Hi Brian

You: I’m looking for a laptop

You: and wanted to get more info about what’s out there

Brian(73486): Please wait a moment while I search for your results.

Brian(73486): Are these results sufficient?

In all five instances, we found the experience lacking. Overall the guides did not ask any questions. The true X-factor behind all search technology is trying to determine user intent of the search. A user will find the search experience to be positive if the search results match their intent As an example, if I type in “flat panel TV” as a search term, the results are more helpful if they match the intent of my query. If I’m looking for brands that sell flat panel TVs, but brand information is not part of the search results, I’ll feel my search was not satisfied. This is whereChaCha could stand out, but only if the guides ask relevant questions to get to the nature and intent of the search.

The only question one of my colleagues was asked was about desired laptop battery life. I would think anyone would want the longest battery life possible for a laptop, so it seemed like a strange question. If these guides are self-selecting their areas of expertise, one should expect knowledgeable help.

But how about the actual search results? Here is a typical set of results that we were shown:

1. Notebook and Laptop Reviews

Notebook and Laptop Reviews

http://www.notebookreview.com/

2. Laptop Review – Compare prices from Office Depot, EcomElectronics and more at Brilliantshopper.com

Laptop Review – Compare prices from Office Depot, EcomElectronics and more at Brilliantshopper.com

http://www.brilliantshopper.com/results.php?q=laptop+review&src=NG&ntwk=OVT&k…

3. Laptop Reviews, Best Laptop, Laptop Computer Reviews
Laptop Reviews, Best Laptop, Laptop Computer Reviews

http://www.consumersearch.com/www/computers/laptops/index.html

Overall, these results contained the same basic information as results I received from Google when I typed in “Laptops.” The guides were not offering any more information that a user could have
easily found on their own by doing the most basic of searches elsewhere.

For all five guided searches we did, we where shown an average of four results. As one of my colleagues said: “When you search on Google, you get the feeling they are searching the universe for results. Seeing only four listings on ChaCha feels limiting.”

The experience a user has with ChaCha is only as good as the guides knowledge, much like walking into a BestBuy and inquiring about a product. The perceived positive/negative experience with BestBuy comes down to the specific salesperson you end up talking with. If they are knowledgeable  and have good customer service, then the experience is positive. In this case,ChaCha’s technology of guided search has
nothing to do with the experience – ultimately it comes down to the specific guide that helps you.

In short, ChaCha offers an interesting technology. But despite what may be happening behind the scenes, the true experience comes down to simple customer service and in our experience, even that was lacking.