ArticleSnatch Blog

You are browsing the archives of Andrew Tomkins.

Yahoo announced today the general public availability of their SearchMonkey program. This is a program that has been in beta testing with limited partners. It allows the partner to provide Yahoo with structured data that provides advanced information about a web page. This information is then used by Yahoo to influence the presentation of organic search listing results for that page.

This is a very powerful concept in that a modified search listing can surely influence click through rates. Imagine your search listing with an image and several related links built in. Let’s look at a quick example:

You can see additional examples in my interview with Yahoo Chief Scientist Andrew Tomkins. The interview was published this past Monday and focuses on SearchMonkey.

The basic process for creating SearchMonkey applications is straightforward. SearchMonkey supports multiple formats, including microformats, RDFa, eRDF, XML feeds, and APIs such as OpenSearch, so publishers have many options for exposing the data.

In addition, developers can build sophisticated applications into the search results. An example of this is the notion of an InfoBar. With an InfoBar, you can actually put an active control in your search listing result. When users click on the control, you mini application will run and can present additional data that displays inline right on the Yahoo search results page.

Here is what it looks like:

The InfoBar provides a very powerful mechanism for managing complex interactions with users right on the Yahoo search results screen. This should have significant value from a branding and click through perspective.

Here is a summary of the development process:

1. Application Type – Decide what type of app you want to build (Enhanced Result or Infobar) and enter basic info such as application name, description, and icon.
2. Trigger URLs – Decide the URL patterns that will trigger your app.
3. Data Services – Data Services are the structured data on which SearchMonkey apps are based. They can be created using data available in the Yahoo! Search index (via data feeds or page mark-up such as microformats or RDF) or by using APIs or page extraction.
4. Appearance - Use PHP to configure how structured data should appear in the application.

Commentary

Note step 2, the one in which your application gets activated. A critical part of the program will be determining when and where you would like your enhanced result to show up.

One key element of the program is that creating an enhanced result, or an InfoBar, does not mean that all users will be exposed to them. Users need to enable the enhanced listings on a publisher by publisher basis. In addition, users can change their minds later and remove your SearchMonkey application from their results.

I spoke to Amit Kumar, Director of Product Management at Yahoo, this past Tuesday, and he indicated that in the future that select SearchMonkey applications may get exposed to all comers. Applications that are adopted by lots of users, and not remove by many at all would be more likely to make this leap to general availability. This however, is not a certainty.

Amit also told me that Yahoo is going to setup a Gallery of such applications for users. This will be a place where the user can go to select an application and enable it. It will be interesting to see how much exposure the Gallery gets. This will play a critical role in the rate of adoption of these types of results. The publisher can, of course, promote their own application, and try to drive people to sign up for it.

Another thing that Amit emphasized during our conversation was that the effort level for developers to engage with SearchMonkey is quite low. The platform makes it really easy for them to engage. This could play a critical role in broadening adoption.

One thing I learned in my interview with Andrew, and also from his presentation at SES New York, is that building SearchMonkey applications will not help you improve your rankings. The program is not intended to be used for that purpose.

Personally, I’d like to see a stronger move towards exposing some of the applications to all users. This maybe a difficult thing to implement at some level, and it makes it far more susceptible to spam. But it would certainly accelerate the exposure of these types of applications to the general public.

The early action (in terms of users) will likely be driven by early adopters. Then we will need to see how widely it penetrates the market, and how aggressively Yahoo pushes it forward.

That said, this is exciting stuff. I have long been a believer that search engines should get more information from the publishers, in a structured format. Yahoo has taken a big step in that direction with this program.

SEW Expert Eric Enge published a terrific interview on his Stone Temple blog with Yahoo Chief Scientist Andrew Tomkins, who keynoted SES New York. What makes it a great read? Eric asks spot-on questions that cut to the heart of the matter.

Eric Enge: In New York you talked about the future of search, but the thing that really struck me in the conversation was the notion of “webmaster supplied content” communicated essentially directly to the search engine. Maybe you can tell me whether that notion resonates with you in just your general thoughts on the concepts that you laid out in the presentation?

Andrew Tomkins: I’ll start by saying that characterization of webmasters and publishers sharing more structured representation of their content is exactly what we are talking about. I guess it’s easy to think of it as sharing it with a search engine.

The exchange that really impressed me was late in the interview when Eric and Andrew discuss a site’s authority:

Andrew Tomkins: Understanding how authoritative a site is, then specifically for each part of the site; what they are about, how much you should trust them and how much people tend to believe them. How deep they go; all of this is very valuable from the ranking standpoint.

photo credit: Marc_Smith in Flickr

Eric Enge: You could have a site that has a million links, and that has many sections like I talked about, but the tennis section for some reason has very few inbound links from third party sites. Whereas, the camping section has half a million links, where you would actually allocate trust differently by site section.

Andrew Tomkins: That’s a great example of a good cue that you would want to pay attention to.

If you care about how search enignes work and where they’re headed in the future, this interview is a must-read.

The official launch of the new version of Yahoo! Search crawler comes as no surprise to members of Digital Point forums, readers of Search Engine Watch and attendees of SES New York, who saw Yahoo Chief Scientist Andrew Tomkins‘ keynote speech on the future of search.

Yahoo! Slurp 3.0 won’t change the content Yahoo crawls on your site: the new Yahoo! Slurp 3.0 recognizes the same user-agent and all robots.txt directives for Yahoo! Slurp. The primary difference will be in your log files where you’ll see Slurp 3.0.

Yahoo shared that the new bot (phased rollout over several weeks) will start crawling from a different and smaller set of IP addresses. No change of location, though. Yahoo! Slurp 3.0 will originate from the crawl.yahoo.net domain. So any reverse DNS checks to ID Yahoo’s crawler will still work.

The big change: Yahoo’s recommendation to stop using IP-based recognition and use reverse DNS identification instead.

Here’s why:

Yahoo warns you might see a drop in crawl and coverage if you’re using IP-based recognition. Best bet: switch to reverse DNS-based identification of Yahoo! Slurp if you’re using any other method. The current set of IPs will disappear from your log files in the next few weeks.

Crawlers that similarly respect the Yahoo! Slurp directive but identify themselves more specifically, such as Yahoo! Slurp China and others, will not be impacted.

It’s Day 4 at SES New York 2008 and the folks from Pan Communications found more than 20 stories about the Search Engine Strategies conference that had been written before 1:45 p.m. when the press room was closed. So, I may be missing some of the additional coverage of the event that appeared later in the day.

Plus, Kevin Ryan, VP, Global Content Director, SES & Search Engine Watch, added a morning keynote to the schedule along with new sessions like “My Search is Better than Your Search.” So, there were plenty of sessions to cover.

Plus, there are so many search engine marketing news sites and SEO blogs that have been covering SES New York that I’m sure I’ve missed some of stories from the event – because its been called “SES New York,” “SES NY,” “SES NYC,” “Search Engine Strategies New York,” “Search Engine Strategies NY,” “Search Engine Strategies NYC,” the “Search Engine Strategies conference,” and the “Search Engine Strategies expo.”

So, while watching the NCAA Mens Final Four in my hotel room, I’ve looked through the news articles and blog posts that I could find from Thursday, March 20, 2008. Here’s the best list that I could assemble of the top ten stories on Day 4 of the event:

1. Yahoo Wants Sites to Play Ball on Semantic Web
Kenneth Corbin of InternetNews.com says, “In his keynote address here at the final day of the Search Engine Strategies conference, Andrew Tomkins, chief scientist of Yahoo’s (NASDAQ: YHOO) search division, described an industry at a tipping point. The search engines are only now beginning to adapt to the explosion of content and the increasing complexity of the tasks people perform on the Internet, he said.”

2. My Mahalo preview at SES New York
Jason Calacanis of Calacanis.com says, “Had a blast at SES New York this week… at the conference we previewed some new features called My Mahalo that build semantic relationships between our users and object in our database (fancy way of saying you and the thing you own, have seen, want to see, want to read, etc.).” He adds, “We also spent some time doing interviews with the SES team. I’ll post a couple here.”

3. WSJ Digital Network gets aggressive with search: SES NY Keynote
Ellen Keohane of DMNews says, “Gordon McLeod, president of the Wall Street Journal Digital Network, spoke at Wednesday morning’s keynote session at the Search Engine Strategies conference in New York, sharing insights about the network’s efforts to grow its site traffic and search presence.”

4. John Battelle on Google Universal Search at SES NY 2008

John Battelle, Founder/Chairman/CEO, Federated Media, is interviewed by Pauline Ores, SES Advisory Board and Senior Marketing Manager, Social Media Engagement, General Business, IBM Corporation.

5. Search Goodness in Bite-sized Chunks
Jason Lee Miller of WebProNews summarizes the highlights from the Search Engine Strategies conference, “Or, SES you can put in your pocket.” He says, “There’s a ton of information pouring out of the Search Engine Strategies Conference in New York City, but we’ve done our best to bring you the best of it so far, via text and via video. Below is a representation of all of that knowledge, boiled down until each little bit fits into its own little nutshell.”

6. Live from Search Engine Strategies New York 2008
Andrey Milyan of Search Marketing Standard summarizes the 16 sessions that his The SES conference in New York has begun and our correspondent, Kent Lewis, is on the scene, bringing you the latest from one of the biggest events of the year.

7. My Search is Better than Your Search
Charles Knight of AltSearchEngines says, “The entrepreneurs and those who dare to do things differently have shaped the web and search as we know it. Is innovation dead? We at Search Engine Strategies and AltSearchEngines don’t think so. Though most experts agree there will be no ‘Google Killer,’ there will be several emerging technologies that will shape the way we search, find and retain content.”

8. Blogging Boogeyman:WHAT Is Social Media Good for? PART 2
http://www.aimclearblog.com/2008/03/20/blogging-boogeymanwhat-is-social-media-good-for-part-2/
Charlene Jaszewski of the aimClear Blog says, “Are you afraid of bloggers? Sleep with the light on? You’ll find reasons to sleep above the covers after you read the next installment Social Media: What Is It and What Is It Good For? from Search Engine Strategies New York.”

9. Search Engine Strategies NYC 2008 Recap - my own thoughts
Marshall Sponder of The Analytics Guru says, “Many people are already familiar with Search Marketing and SEO and there needs to be a way to present material satisfying to all levels, but that might not always be possible.” He adds, “The Social Media/Social Search Tract was the most interesting for me this year and was more expanded than in the past.”

10. Totally Plugged In @ SES New York: 13 Undeniable Symptoms
Marty Weintraub of Search Engine Watch says, “Search marketing conference attendees seem to be the most plugged-in-public group of techno-comrades on earth. We rove in packs of iPhone and laptop-totting pied-pipers evangelizing link love, holistic patterns, authentic participation, conversion tracking, and good will.”

Where’s search heading? Ask Yahoo’s chief scientist, Andrew Tomkins. He’ll be giving the morning keynote today at SES New York. While most search engines for the past 10 years have relied heavily on analyzing anchor text, links, and content to determine relevance, researchers at the top search engines have recently begun to look at other signals that might indicate search result relevance. Tomkins reveals a few hints at the future in today’s SearchDay, “Where’s Search Heading? Ask Yahoo’s Chief Scientist.”

Jason Calacanis has owned the front page of Techmeme–the world’s most influential technology and Web 2.0 news aggregator–for the past 18 hours and counting.

Another Civil War in Silicon Valley? Well, it’s war anyway–even if not civil.

Aussie Duncan Riley of TechCrunch fame bodyslammed Jason after his Calacanis.com post on how to run a startup. Duncan said “Calacanis Fires People Who Have A Life.” So far, 164 comments on TechCrunch about firing anyone who’s not a workaholic …

Jason got up off the canvas, charged his opponent Valleywagged, and parried Duncan’s jabs by updating his post, How to save money running a startup by revising his VC deathmatch coda. http://www.calacanis.com/2008/03/07/how-to-save-money-running-a-startup-17-really-good-tips/

By then Duncan had tagged out: Allen Stern delivered a sidekick to Jason’s solar plexus that sent him all the way to Starbucks country: Working at Mahalo is Like Prison Except We Gots Better Coffee.

Jason Calacanis “Electrified Cage Deathmatch Bar Room Brawl” at SES New York in the SOLD OUT (Not Paris) Hilton in New York the day after St. Patrick’s Day.

Note: See it live! SES On Demand Video will be available to Search Engine Watch members only after the SES New York Deathmatch.

Celebrities on the front row: Kevin Ryan, John Battelle, Andrew Tomkins, Nick Carr, Gordon McLeod and many, many more.

Tickets on sale now!

Whether you work at one of the top 10 ad agencies on Madison Avenue, or you head up an Internet advertising agency in Silicon Alley, you folks could use “a little churching up.” Slide on down to SES New York, and catch Carol Kruse.

That’s right. Carol Kruse, Vice President of Global Interactive Marketing, The Coca Cola Company, is speaking at the Search Engine Strategies conference, which will be held the week of March 17-20, 2008, at the Hilton New York. She will address what it takes for big brand companies to shift their marketing mix to align with new customer behaviors, putting more ad budget to Search and digital advertising.

Other speakers at SES New York 2008 include:
• Jonathan Ashton, VP of SEO & Web Analytics, Agency.com;
• Paul Beck, Senior Partner, Worldwide, Executive Director, Interactive Marketing & Advertising, Ogilvy;
• Jeff Ferguson, Director of Online Marketing, Napster;
• William Flaiz, Vice President, Search Engine Optimization & Web Analytics, Avenue A | Razorfish;
• Kurt Garbe, Entrepreneur in Residence, Advertising, Adobe Systems;
• Bill Hunt, CEO, Global Strategies International;
• Steven Kaufman, SVP Media Director, Digitas
• Olivier Lemaignen, Group Manager, Global Search Marketing, Intuit;
• Bill Macaitis, VP of Online Marketing & SEO/SEM, Fox Interactive Media;
• Gordon McLeod, President, The Wall Street Journal Digital Network
• Ulli Muenker, Search Marketing Manager, BusinessWeek;
• Pauline Ores, Senior Marketing Manager, Community and Collaboration Strategy, Global Small & Midmarket Business, IBM Corporation;
• Susan Prater, Global Interactive Marketing Manager, Owens Corning;
• Nigel Ravenhill, Program Manager, McAfee;
• Irene Rigos, Senior Ecommerce Manager, Wyndham Hotel Group;
• Michael Sack, Director, SEM technology & Development, Idearc Media Corp.;
• Erica Schmidt, Global Director of Search, Isobar;
• Marshall D. Simmonds, Chief Search Strategist, New York Times / About.com;
• Don Steele, Director of Digital Marketing, Comedy Central;

And, if you look closely at the conference agenda for Search Engine Strategies New York 2008, you’ll see 10 Google speakers and eight from Yahoo! And that’s not counting Michael Rubenstein, Vice President & General Manager, DoubleClick, because it’s acquisition by Google is still pending.

Listen to what they’ve got to say.

Now, if you’re too busy to attend SES NYC 2008, don’t feel guilty. I’m confident that another one of the many fine advertising agencies in New York City will be more than happy to pitch in and help the corporate executives from the name brands above understand where the advertising industry is heading.

And, if market demand for magazine ads and television spots prevents you from getting over to Search Engine Strategies NY, don’t worry. I’m fairly certain that you’ll be able to catch up with what you missed in AdWeek or Advertising Age by reading about some other ad agency in New York has just landed a new client for its managed search engine marketing services.

“You get wise. You get to church.”

If you don’t need some “jive-ass preacher” talking to you about Search Advertising 101, then you’ll still find lots of other “must attend” sessions at Search Engine Strategies 2008 in the Big Apple. This includes:

Day 1 - Monday, March 17, 2008
• Creating Compelling Ads
• Analytics: Data Into Action
• Auditing Paid Listings and Click Fraud Issues
• Orion Panel: Getting Vertical Search Right

Nick Carr, SES New York 2008, The Big Switch

Day 2 - Tuesday, March 18, 2008
• Nick Carr, author of The Big Switch: Rewiring the World, from Edison to Google
• Ads in a Quality Score World
• Orion Panel: Universal Search
• Optimizing Search Marketing Campaigns
• Ad Copy & Landing Page Clinic

Day 3 - Wednesday, March 19, 2008
• Gordon McLeod: Search Has Changed Everything… And So Can You
• Ad Testing: Research & Findings
• Big Brand Search Strategies: Build Connections and Fuel Online Promotions
• Jason Calacanis, Founder & CEO of Mahalo.com, Inc.
• Managing PPC for Multiple Clients
• Ad Exchanges Are Changing Everything

Day 4 - Thursday, March 20, 2008
• Andrew Tomkins, Chief Scientist at Yahoo! Research
• Staffing Up for Search
• SEM Agencies: Working With Ad Agencies
• My Search is Better Than Your Search

Now, I should disclose that SES New York is a client. “A gig like that, you gotta prepare the proper exploitation.”

Besides, “your lazy butts are in this, too.” According to The New York Post, “The ad biz may lose its fizz.”

So, slide on down to the Search Engine Strategies conference, and get your four box lunches and a Coke.

Do you see the light?!

The biggest Search Engine Strategies conference of the year will be held the week of March 17 in New York City. Whether this will be your first SES New York, or the fifth one in a row that you’ve attended since 2004, you might appreciate some free advice on schedule optimization.

One look at the conference at a glance will tell you why. There are more than 70 workshops, keynotes, panels and sessions over the four-day Search Engine Strategies conference. And, on the fifth day, there are an additional six SEM training classes following SES New York.

Since there are five concurrent tracks during the four-day Search Engine Strategies conference and three concurrent workshops during the fifth day of SEM training, no one can attend everything. This is not daunting to the first-time attendee. It is also a challenge to someone like me, who attended SES New York in 2004, 2005, 2006 and 2007. I’m looking at last year’s conference handbook and this year’s conference at a glance and more than 70% of the sessions are new!

Now, Danny Sullivan programmed last year’s show and Kevin Ryan organized this year’s agenda. But, that’s only one of the factors driving the dramatic changes in the content at the event.

On the last day of last year’s Search Engine Strategies conference in New York City, Google announced its $3.1 billion acquisition of DoubleClick, which is still pending. A couple of weeks later, Yahoo! acquired the remaining 80% of Right Media for $680 million. Then, last May, Microsoft acquired aQuantive for $6 billion. Last July, Microsoft acquired AdECN for an undisclosed amount of money and in September, Yahoo! acquired BlueLithium for $300 million in cash. Then, on Feb. 1 of this year, Microsoft proposed to acquire Yahoo! for $44.6 billion.

That’s a lot of M&A news for the industry to digest – and our search engine marketing definition needs to be rewritten this year.

And our search engine optimization definition needs to be totally rewritten, too.

About a month after last year’s SES New York, Google announced its critical first steps toward a universal search model that offers users a more integrated and comprehensive way to search for and view information online. It was the biggest thing to hit the search engine marketing industry since Google’s Florida Update in November 2003.

In June 2007, Ask3D was launched. In September 2007, Microsoft launched its biggest update to Live Search since its debut in January 2005. And in October 2007, “an all new Yahoo! Search experience” was launched.

Meanwhile, comScore reports that YouTube, Google Image Search, Google Maps and Google News are approaching 1.6 billion searches a month, which is more than Live Search. In other words, Google (6.2 billion searches a month) is the #1 search engine, Yahoo! (2.4 billion) is #2, YouTube and all other Google “expanded” search sites (1.6 billion) would be #3, and Microsoft’s Live Search (1 billion) is #4.

So, is it any wonder that even SES alumni are heading back to New York?

So, to help industry veterans as well as search newbies, I’ve put together the optimized schedule below for the Search Engine Strategies conference that starts on Saint Patrick’s Day in the Big Apple.

Now, when you get to SES New York, you’ll make adjustments on the fly. As Bob Shirilla of Keepsakes Etc. told me at SES Chicago back in December, “I had a detailed game plan when I came to SES, but I’m calling a lot of audibles.”

Nevertheless, schedule optimization will help you get the return on marketing investment that you’re looking for. Here are the workshops, keynotes, panels and sessions that I’d recommend:

Day 1 - Monday, March 17, 2008

9:30-10:45am
Creating Compelling Ads
Organic Listings Forum

11:15am-12:30pm
Analytics: Data Into Action
Igniting Viral Campaigns

2:00-3:15pm
Web Analytics: Measuring Success
Auditing Paid Listings and Click Fraud Issues

3:45-5:00pm
Orion Panel: Getting Vertical Search Right

Day 2 - Tuesday, March 18, 2008

9:00-10:00am
Conference Welcome and Opening Keynote
Nick Carr, author of The Big Switch: Rewiring the World, from Edison to Google

Nick Carr, SES New York 2008, The Big Switch

11:00am-12:15pm
Introduction to Search Engine Marketing
Ads in a Quality Score World

1:30-2:45pm
Orion Panel: Universal Search

3:15-4:30pm (Sponsored Sessions)
Hitwise: Know about Your Competitors’ Paid and Organic Traffic
Google: What’s new with Google Analytics and Website Optimizer?

4:45-6:00pm
Search Engine Friendly Design
Ad Copy & Landing Page Clinic

Day 3 - Wednesday, March 19, 2008

9:00-9:45am
Morning Keynote
Gordon McLeod: Search Has Changed Everything… And So Can You

10:15-11:30am
Link Building Basics
Ad Testing: Research & Findings

1:00-2:15pm
Search Advertising 101
Top Search Trends

2:30-3:30pm
Afternoon Keynote
Jason Calacanis, Founder & CEO of Mahalo.com, Inc.

4:00-5:15pm
The New Face of In-House Search
Social Media Research: Informing Search Strategies

5:30-6:45pm
The Business Case for SEO Content Development: Turning Words Into Action!
Ad Exchanges Are Changing Everything

Day 4 - Thursday, March 20, 2008

9:00-9:45am
Morning Keynote
Andrew Tomkins, Chief Scientist at Yahoo! Research

10:00-11:00am
Usability & SEO: 2 Wins For The Price of 1
Podcast & Audio Search Optimization

11:15am-12:15pm
Beyond Linkbait: Getting Authoritative Online Mentions
Images & Search Engines

12:45-1:45pm
Meet the Crawlers
Video Search Optimization

But wait! There’s more!

On Friday, March 21, there are six half-day SEM training classes, which can be taken in addition to the SES New York conference or independently – at an additional cost.

Again, look over the descriptions of each workshop to see which ones are for you. But, here are the SEM training classes that I’d recommend:

8:00am-12:00pm
Link and Reputation Workshop
Optimizing for Universal Search

1:00-5:00pm
Search & Analytics Workshop: Using Analytics to Increase Search Effectiveness
The 7 Step RSS/Content Syndication/SEO Strategy

If you register for the Search Engine Strategies conference by tomorrow, Friday, Feb. 29, you can save $150. Conference attendees get free access to Market Motive training and Bruce Clay tools. And, if you attend SES New York, you could win a Scion xB! A free drawing will take place on Wednesday, March 19, in the Expo Hall.

I should disclose that Search Engine Strategies is now a client, but I’ve been writing about SES since 2002, when the March event was still held in Boston.

The search engine marketing industry has been totally transformed since then. For example, the keynote speaker at my first Search Engine Strategies conference was from Terra Lycos.

Remember them?

That’s why both industry veterans as well as search newbies will be heading to SES New York in a couple of weeks. The newbies will want to learn everything they can. And the veterans need to relearn most of what was being taught just a few years ago.

Greg Jarboe of SEO-PR and Amanda Watlington of Searching for Profit at SES London 2008