ArticleSnatch Blog

You are browsing the archives of Scientist Andrew Tomkins.

Yahoo announced today the general public availability of their SearchMonkey program. This is a program that has been in beta testing with limited partners. It allows the partner to provide Yahoo with structured data that provides advanced information about a web page. This information is then used by Yahoo to influence the presentation of organic search listing results for that page.

This is a very powerful concept in that a modified search listing can surely influence click through rates. Imagine your search listing with an image and several related links built in. Let’s look at a quick example:

You can see additional examples in my interview with Yahoo Chief Scientist Andrew Tomkins. The interview was published this past Monday and focuses on SearchMonkey.

The basic process for creating SearchMonkey applications is straightforward. SearchMonkey supports multiple formats, including microformats, RDFa, eRDF, XML feeds, and APIs such as OpenSearch, so publishers have many options for exposing the data.

In addition, developers can build sophisticated applications into the search results. An example of this is the notion of an InfoBar. With an InfoBar, you can actually put an active control in your search listing result. When users click on the control, you mini application will run and can present additional data that displays inline right on the Yahoo search results page.

Here is what it looks like:

The InfoBar provides a very powerful mechanism for managing complex interactions with users right on the Yahoo search results screen. This should have significant value from a branding and click through perspective.

Here is a summary of the development process:

1. Application Type – Decide what type of app you want to build (Enhanced Result or Infobar) and enter basic info such as application name, description, and icon.
2. Trigger URLs – Decide the URL patterns that will trigger your app.
3. Data Services – Data Services are the structured data on which SearchMonkey apps are based. They can be created using data available in the Yahoo! Search index (via data feeds or page mark-up such as microformats or RDF) or by using APIs or page extraction.
4. Appearance - Use PHP to configure how structured data should appear in the application.

Commentary

Note step 2, the one in which your application gets activated. A critical part of the program will be determining when and where you would like your enhanced result to show up.

One key element of the program is that creating an enhanced result, or an InfoBar, does not mean that all users will be exposed to them. Users need to enable the enhanced listings on a publisher by publisher basis. In addition, users can change their minds later and remove your SearchMonkey application from their results.

I spoke to Amit Kumar, Director of Product Management at Yahoo, this past Tuesday, and he indicated that in the future that select SearchMonkey applications may get exposed to all comers. Applications that are adopted by lots of users, and not remove by many at all would be more likely to make this leap to general availability. This however, is not a certainty.

Amit also told me that Yahoo is going to setup a Gallery of such applications for users. This will be a place where the user can go to select an application and enable it. It will be interesting to see how much exposure the Gallery gets. This will play a critical role in the rate of adoption of these types of results. The publisher can, of course, promote their own application, and try to drive people to sign up for it.

Another thing that Amit emphasized during our conversation was that the effort level for developers to engage with SearchMonkey is quite low. The platform makes it really easy for them to engage. This could play a critical role in broadening adoption.

One thing I learned in my interview with Andrew, and also from his presentation at SES New York, is that building SearchMonkey applications will not help you improve your rankings. The program is not intended to be used for that purpose.

Personally, I’d like to see a stronger move towards exposing some of the applications to all users. This maybe a difficult thing to implement at some level, and it makes it far more susceptible to spam. But it would certainly accelerate the exposure of these types of applications to the general public.

The early action (in terms of users) will likely be driven by early adopters. Then we will need to see how widely it penetrates the market, and how aggressively Yahoo pushes it forward.

That said, this is exciting stuff. I have long been a believer that search engines should get more information from the publishers, in a structured format. Yahoo has taken a big step in that direction with this program.

SEW Expert Eric Enge published a terrific interview on his Stone Temple blog with Yahoo Chief Scientist Andrew Tomkins, who keynoted SES New York. What makes it a great read? Eric asks spot-on questions that cut to the heart of the matter.

Eric Enge: In New York you talked about the future of search, but the thing that really struck me in the conversation was the notion of “webmaster supplied content” communicated essentially directly to the search engine. Maybe you can tell me whether that notion resonates with you in just your general thoughts on the concepts that you laid out in the presentation?

Andrew Tomkins: I’ll start by saying that characterization of webmasters and publishers sharing more structured representation of their content is exactly what we are talking about. I guess it’s easy to think of it as sharing it with a search engine.

The exchange that really impressed me was late in the interview when Eric and Andrew discuss a site’s authority:

Andrew Tomkins: Understanding how authoritative a site is, then specifically for each part of the site; what they are about, how much you should trust them and how much people tend to believe them. How deep they go; all of this is very valuable from the ranking standpoint.

photo credit: Marc_Smith in Flickr

Eric Enge: You could have a site that has a million links, and that has many sections like I talked about, but the tennis section for some reason has very few inbound links from third party sites. Whereas, the camping section has half a million links, where you would actually allocate trust differently by site section.

Andrew Tomkins: That’s a great example of a good cue that you would want to pay attention to.

If you care about how search enignes work and where they’re headed in the future, this interview is a must-read.

Yahoo announced an agreement today with Computational Research Laboratories (CRL) to support cloud computing research. At Search Engine Strategies (SES) New York last week, Yahoo Chief Scientist Andrew Tomkins previewed the future of search in his keynote address. (For a video of his keynote click here soon.) No doubt cloud computing will one day make search engine innovations possible that we can only dream of today.

CRL, a wholly owned subsidiary of Tata Sons Ltd., a Mumbai, India-based industrial conglomerate, will lend one of the world’s top five supercomputers to Yahoo for joint research. The CRL supercomptuter has “substantially more processors than any supercomputer currently available,” according to Yahoo.

The first Data-Intensive Computing Symposium held at Yahoo’s campus this week will bring together leading industry and academic experts from all aspects of data-intensive distributed computing.

The symposium is part of a larger effort to explore opportunities for research and application of large-scale computing to benefit applications ranging from machine translation to genomic medicine.

So who in the world are CRL and Tata?

Tata Sons Limited–founded in 1868–could be the poster boy for The Big Switch, a brilliant book by SES keynote speaker, thought leader, and Mike Arrington BFF Nick Carr on the transformation of corporations and computing leading up to the Age of the Internet and beyond Google.

Click to read the rest of this post…