Semantic Web: Yahoo takes the first step

ReadWriteWeb has a very good summary of Yahoo’s announcement earlier this week that they’ll soon be indexing content marked up with "semantics", or "meaning".

If all this sounds like geek-speak, bear with me for a moment.   

The change that’s coming to the way we use the web will be quite significant, and impact web publishers (and consumers) greatly.  It’s not a question of if, but when this change arrives. 

My prediction is that most of the evolution will start to show up in the 3 year timeframe.

Today, the way search engines work is essentially:

  • Crawl through the web
  • Read the HTML markup on each page (the "View/Source" behind every webpage)
  • Look hard at this text with lots of "smart" algorithms.  Try to figure out which content is most important, and which is unimportant.  Make some educated guesses. 
  • Look at things like quality-of-sites-pointing-in, meta tags, keyword density, spam lists, etc. to try to assemble a directed graph of what-links-to-what.  Rank it (e.g., Google’s famous PageRank).  Make some educated guesses.
  • Make some educated guesses about what the most "authoritative" search results are for the user, based on the keywords entered
  • Finally, throw back a bunch of URL’s to the user reporting what you’ve found.  Pay the bills by making some educated guesses about what ads that match what the user meant and show those too.

Notice all the "make an educated guess" statements above?  It’s no accident.  Search engines, as good as they seem, are making educated guesses all the time about what we want, and getting it right only part of the time.  They have many tricks to use, but essentially they are reading text and trying to infer a lot more.

Also, note that information itself is not interconnected. 

In other words, you cannot issue a query to Google like this:

     "Show me all the Amazon.com reviews posted by my Facebook friends".   

or

   "Show me all the products my Facebook friends rated 4 stars or better."

or

   "Show me photos of all Maui condos that are available for spring break, have 2 bedrooms, and cost between $300 and $800 per night."

In a few years, these kinds of searches will be trivial, and a huge amount of economic value is going to be created and destroyed in the process.

Even though searching is vastly better than what we had just 5 years ago, 2008 is still the stone-ages in search, because in the end, computers are looking at text and making a bunch of guesses.  They are not looking at information in a well-structured, interconnected form. 

Sure, there is a lot of subjective content on the web, and prose is terrific, but in many circumstances that matter — whether you’re trying to research a medical ailment, book a hotel, value a property, or buy a car, you want to get at the data itself.

Search engines have been trying to infer meaning from HTML text. 

But what if web publishers took the time to encode their content jut a little bit further, with the meaning and structure of what they were really writing about?

The simplest and first thing that happens is that the search results get a lot more immediate to the question you just asked.

For instance, if you were shopping for a new digital camera, and typed in "Canon 40D reviews", you really probably want to know a good deal about the reviews themselves, rather than a bunch of URL’s leading you to reviews. 

The way this comes about is now quite apparent. 

Step 1:  Reviews sites publish their reviews in hReview format.  (Until earlier this week, there wasn’t a strong incentive to do so.  And the incentive will get stronger and stronger.) 

Step 2: The search engines look at that structure and present the results in a more orderly way, complete with filtering tools, etc. on the search engine pages themselves.   

If you type in "Maui condo for rent", wouldn’t it be nice if it knew that this was a reservable resource, and it asked you "when do you want to go", and "do you want beach or not"?  You could then immediately see a list of Maui condos for rent (on Yahoo or Google or MSN mind you), along with filtering tools?

As good as we think search engines are today, they are akin to parents helping toddlers on Easter-egg hunts.  "I think you might want to look over here …", "The answer might be here; why don’t you look here?", etc. Except they are not even as good, since search engines are, like the kids themselves, only guessing at what might lurk behind that tree stump.

Up until now, search engines don’t know how to look at a bunch of HTML and extract "meaning", e.g., the "kernel" of the review itself, such as the date it was written, what it was written about, how it scores on a standardized 5-point scale, etc., so they throw up their hands and send you off to various sites to help you assemble the answers yourself.

Over the "long" term (and that’s maybe 0-3 years), I think Yahoo’s groundbreaking announcement will eventually cause:

  • Google and MSN and Ask.com to respond in kind (as usuallook for Google to push the envelope further here — this is squarely in their strategic wheelhouse of "organizing the world’s information".  Listen up Microsoft — this is finally a real chance to differentiate here, and create a leading innovation in search.  But you need to start leading in microformat creation and toolset delivery.)
  • Web publishers to start publishing content in microformats, which is trivial for most publishers.  (My site, BigOven.com will start microformat markup in 2008, most likely.)
  • New microformats will emerge for nearly any important entity that can be visited, bought, sold, born, killed, or reserved.  Particularly lacking today is a microformat for reservable resources.
  • Which microformats will see quickest adoption?  My guess is that those entities that have a very high frequency and/or need-for-comparison and high fragmentation will likely gain the most value in microformat adoption, like restaurants, lodging, and more
  • Entire business models are at risk.  Pay-for Classified-ad-style companies will lose a lot of power and market cap, an inevitable trend I suggested we’d start seeing back in March ’07.
  • Owners of back-end supply that are structuring fragmented data will become strategically more valuable and visible, if only because their content will show up higher in the search process
  • Structured data will show up in both sponsored ads and the search results themselves.  I could see Google start displaying their top 3 or so microformat-structured results in the "sponsored results" section right at the top.  My suspicion is that structured search results have an opportunity to have a higher clickthrough than the kind of generic "click here for more" CPC ads we see today.
  • We’ll see the browsers themselves going further up-market and presenting structured data for the user.  Note today how the leading browsers highlight if there’s an RSS feed on the page.  Soon, you can expect them to also highlight content-marked-up-with-meaning as well.

It’s particularly interesting to think about who wins and who loses with these significant changes, because that is where wealth is created and destroyed.

Winners:

  • Companies that help enterprises publish their content in structured form.
  • Search engines, since more of the share-of-attention during search takes place on their site.
  • Companies which get out in front and help forge a company-friendly microformat that play to their strengths
  • … more

In the vacation rental world, my view is that back-end suppliers, like Escapia.com, get much more valuable (because they power the back-end, where the reservation data is), and classified-ad-style directories lose a lot of value, because they have effectively been reselling search engine presence. 

What does it mean in your industry?

Coming Soon: Print your own Cookbooks

BigOven.com is partnering with Sharedbook of NYC, to allow any cook to create their own professionally-printed cookbooks.

In a matter of weeks, anyone will be able to create personal cookbooks using BigOven.  Uses include:

  • Gift giving — archive Mom’s recipes and give them to your siblings
  • Personal reference — gather your favorite spring, summer, fall and winter recipes into bound journals
  • Fundraising — raising money for a school?  Get each family to post one favorite recipe to BigOven.com (free!), and mark it with a special tag.  Then, the publisher can search on that tag and create a customized recipe book in a matter of minutes!

Books will be available for purchase starting at less than $40.00, including free shipping and handling to any US location. 

Cookbookbanner

New Milestones for BigOven

Earlier this week, BigOven.com passed 80,000 registered members.

Growth is picking up — each week, thousands more sign up.   And I just noticed that the free Vista Sidebar Gadget that lets you search our 160,000 recipe archive, which was released this past summer, is now well-past 20,000 downloads.   More people mean more great recipes, photos and expertise shared with everyone. 

Are you using Vista?  Would you like a ready answer to the question "what should I make tonight?" right on your desktop?  Download the free BigOven Recipe Vista Gadget here:

Look up your favorite recipes from your cellphone!

How many times has this happened:

You’re at a grocery store or farmer’s market, and just don’t know what to make.  You see some fresh ingredients, and have some idea of what you’d like, but it’s not fully-formed.  It might be "chicken caesar salad", and you might want to know what you’d need to make one.  Or it might be "grilled salmon", but you might not have made it before.

Well, simply point your cellphone to:

http://www.bigoven.com/m

And you’re all set.  There, you can enter a list of keywords or ingredients or recipe name, and get simple results back, right away.  And, if you’re a free BigOven member, you can get quick access to your Favorites and Try Soons.

For more information, see the link: www.bigoven.com.

#22 on Seattle Web 2.0 list

I was pleased to see that BigOven.com moved up another 5 places to the #22 position in the Seattle Web 2.0 List for December of the 243 sites tracked by the list, just behind #21 Redfin.com, and beating out many fairly well-known (and/or well-funded) sites like Avvo.com, Cozi.com, Bag Borrow or Steal, Yapta, Earth Class Mail, TripHub, Jott, SmartSheet and more.  Even more promising is that if you just sort on Compete.com rankings, BigOven.com is now #13 on the list.

Thanksgiving & Christmas — Record Growth for the Site

Following up on my last post, now that we’re past the holiday season, it’s a good time to reflect on the trends in food search & recipe lookup.

Google shows the following trend for the term "recipe" over time.  Note the repeated and significant peaks at Thanksgiving and Christmas:

Viz_3

BigOven certainly saw these two significant peaks in new user signups, and also in software sales.   We now have over 72,000 registered users, and thousands join each month. 

The Thanksgiving Effect

Most retail stores get 50% of their net income or more in the last quarter of the year.

If you’re running a food & cooking website, I guess an equivalent is new user registrations leading up to Thanksgiving. 

Check out the pace of new BigOven.com signups overthe past 60 days — (vertical axis removed for competitive reasons): 

Image1_3 

BigOven Cooking Groups Now Live

Today, we’ve released BigOven Cooking Groups, an easy way for people with common interests to chat about food.  Its mission is to be the easiest way for people to congregate to talk about any food-related subject. 

Food is social. 

Several years ago, in the lead-up to kicking off the BigOven project, I was reflecting on how social and viral discussion about food really is.  It was that trigger — that food really is one of the most "viral" subjects (in the information sense) — that caused me to begin a site to let people share their cooking.

Recipes have a great information-flow — it’s almost as though the recipes themselves want to be shared.  People get passionate about food.  There’s a togetherness around the preparation and consumption of food.  People share food as expressions of love, joy, celebration, honor and ritual.  We’ll certainly see this during Thanksgiving week.

People also gather together to learn the latest diet craze, whether it’s Atkins, South Beach, Weight Watchers, Zone, or more.  Right now, in my neighborhood, I’m noticing dinner and cocktail chatter often revolving around people wanting to cook more organically; more locally.  But information is scarce on how to do this best. 

We may be moving from the era of the celebrity chef to the celebrity farmer; who knows.  But I have to say that information on how to do this best is quite scarce.  Who makes the best cheese in the northwest?  If I want to minimize my carbon impact this Thanksgiving, but still want a terrific, traditional yet elegant Thanksgiving, how do I do it? 

(I’ve created a Pacific Northwest Organic cooking group on BigOven.com as one of the first groups — hopefully, this time next year, we’ll have many experts you can call upon for answers to those questions.)

After a bit of development work to get this going, I’m pleased to report that tonight, the beta of BigOven Cooking Groups is live.  It allows any (free or paid) user to create or join a cooking club around any topic, contribute recipes to the club’s pool, and chat. 

(Notice I’m intermingling the terms club and group here?  We haven’t quite decided whether this should be called Cooking Clubs or Cooking Groups.  Feedback welcome.)

Perhaps you’re a fan of Italian food, or need some ideas for a special occasion.  Maybe you’ve got a finicky 3 year old and need some great suggestions from other parents of children of a similar age.  Maybe you’re in the Northwest and looking to cook more locally and organically.

BigOven Cooking Groups  makes all this possible, by providing a framework for the grouping of recipes and people.  (Videos will be added to this framework shortly.)  It’s better than newsgroups, because photos and even videos can be contributed.  Recipes can be tagged and linked together.  Hyperlinks and rich text can be added in your comments.  And more.

Since it’s a social network, we had to think through some important modeling aspects of this — e.g., if anyone can add recipes, can the group owner delete them?  Can Person X delete contributions by Person Y?  What if Person X is the one who started the group?  What happens if the one who started the group wants to leave the group?  Can you transfer ownership?  How?  Etc.)

Food is social… My goal is to make BigOven the easiest, friendliest social network about food, and groups were a vital part of the framework.  We at Lakefront Software are happy to get this major pillar of the platform out the door, even if it’s still in v1.0 beta stage. 

(More about BigOven Cooking Groups is discussed on the BigOven Message Board.)