Forget search: local is a recommendation problem

If I’m looking for local facts – an address, directions, a phone number – then it’s appropriate to frame the problem as a search or information retrieval one.

But otherwise, I need recommendations based on my preferences, situation and current need.  And here, the search paradigm fails spectacularly.

The Random Ranker: at Best Arbitrary…

The search paradigm starts by trying to match the words in my query with the words in the listings — and maybe with the words in reviews for the listings and maybe even the website of the listings.  It then retrieves the ones where there is some kind of match and ranks the results.  Often this default ranking is called something like ‘relevance’.  But there is no explanation of how this so-called ‘relevance’ is determined — and it certainly isn’t obvious by looking at the results.

Do you really have any idea how Google decides which 10 listings to include in the ‘featured box’ on a search results page?  From the point of view of the user, such an ordering is at best arbitrary and at worst suspicious.  This undermines user trust and confidence.

In web search, the path to relevancy usually involves a number of incremental refinements to my query.  After each query, I quickly scan the page and if the results are clearly not what I’m looking for, I re-formulate it and try again.

This approach isn’t applicable in local.  Quite often I get to a more or less reasonable set of results in short order.  In fact, the basic search problem – getting some reasonable listings – just isn’t that hard.

But my real problem, is finding a way to differentiate between the options.

Manual Filtering: Too Complicated, Doesn’t Work…

Now, it’s true that most services provide me with various tools to filter and re-order the results.  But the results are frequently less than satisfying:

  1. The tools are complex and time-consuming.  You’re making me – the user – do all the work.  So adoption of evaluation tools tends to be low.
  2. The information is often incomplete and inaccurate – so the filtering doesn’t work properly.
  3. There is too much information — too many reviews, too many conflicting opinions — and I have no basis on which to evaluate the alternatives – because I don’t know the reviewers or I don’t know the area.
  4. And often, I can’t actually do the evaluation on the criteria that matter to me — and sometimes I can’t even articulate those criteria.  I just want someone I’m going to like to cut my hair — is that too much to ask?

Recommendations

I need help in the form of meaningful and transparent recommendations.  Meaning — you need to explain to me how you arrived at the recommendations.  Are they based on other people’s opinions – how do I know I should trust these people?  If it’s based on expert assessment – what are the credentials of those experts?

And you need to learn my preferences and understand my situation in a painless way.  Don’t make me fill out a bunch of questionaires – they don’t work, I don’t know what I want and I’m not going to fill them out anyway.  Instead, learn what I need form what I do and what I’ve done and what I’ve told you I like.

Advertisements

11 Responses to “Forget search: local is a recommendation problem”


  1. 1 johnemcnulty May 18, 2009 at 8:43 pm

    Eric, you mentioned Twitter in a separate post as a source of leads. On the flip side you could potentially mine Facebook as a source of recommendations. Facebook allows users to “friend” things like restaurants, movies, shopping malls, bands, etc. I expect heavy frienders of inanimate objects are limited to certain demographics but you could be a hero for the ages or at least those demographics by combining (in a non-trivial way) the two hottest 2.0 services on the web today — use Facebook friending of stuff to provide recommendations to Tweeters. Doing it within Facebook itself makes sense as well but seems less useful without the inate mobility of Twitter. I’m sure it will happen someday. Cheers, John

    PS I checked — “Hatebook” to provide a counterweight to things you “friend” (and presumably like) is taken up by some teen angst site. oh well 🙂

    • 2 predictabuy May 18, 2009 at 9:10 pm

      Hi John,

      Yep, I agree that Facebook friending might be a good signal, but I’m also suspicious as to the ‘breadth of coverage’ of this behavior. In general, I think the biggest challenge in deriving good recommendations is to get sufficient high quality information on what people actually like. There are many services that provide ‘windows’ in to the user’s behavior — but they are tend to be flawed in one way or another.

      And you’re right, knowing what people ‘hate’ or even what they ‘just tolerate’ is equally interesting.

      Thanks,
      Eric

  2. 3 johnemcnulty May 19, 2009 at 11:49 am

    Eric,

    I agree that Facebook friending would be quite patchy, I have all sorts of opinions and preferences and have never once friended a non-human. And I also agree that one is unlikely to ‘find’ a source of high quality info much less one with any breadth. Correct me if I’ve read too much into your statement.

    Given this would the immediate challenge/opportunity be how to make use of high quality data where it can be found already? as opposed to finding or generating sources of high quality data?

    I’m thinking here of rich deep, sources of information like TripAdvisor that would seem a natural fit for some sort of preference-based searching and that has both ratings and text comments from users.

    What do you think is preventing usage of such data? Is it not high quality enough? Do algorithms for tapping that quality not exist? Or is the pay-off for solving the foregoing insufficient? None of the above?

    John

  3. 4 predictabuy May 19, 2009 at 5:31 pm

    Hi John,

    As always, good questions!

    I believe there are many sources of data that can be used – though the majority of them are not publicly available. I also think the necessary algorithms exist – though of course there is always significant effort in applying them and making them work well.

    I think it’s also possible to design user experiences that will be better at capturing helpful data — though I doubt you could deploy a new service solely on the basis of providing recommendations – more of a feature than a reason for being.

    So, the interesting question then is why isn’t it happening? I have several theories:
    – many ‘traditional’ players who probably have more than adequate data lack the expertise and perhaps even the inclination;
    – newer players have limited resources and are focused on other things they view as ‘foundational’ at the moment;
    – the big search engines have larger priorities — they are probably more focused on getting more local businesses online and advertising.

    There may be other cultural and historical barriers that are preventing various players form moving forward as well?

    What do you think?

    Cheers,
    Eric

  4. 5 johnemcnulty May 21, 2009 at 11:43 pm

    Eric,

    What’s fascinating about your theories (on why recommendation isn’t happening) is that they are almost identical to the theories at Agents, Inc (later Firefly) back in the late 90’s when we were pitching recommendations-in-a-box to book sites, movie sites, travel sites, directories (you remember those before portals and then search took over) — basically everyone was focused on their core businesses (as they should be) and once that once in order they would see the value of recommendations.

    But why issn’t recommendations technology closer to the core business?

    If I had to draw a Boston grid characterizing the ‘sweet spot’ for recommendations technology (rec tech) it would have “purpose of the technology” on axis and the categories “enhanced user experience” and “behavioral targeting” on that axis; and “explicit preference profiles” vs “implicit preference profiles” on the other axis.

    Then in that grid I would say the sweet spot is in the “implicit/behavior targeting” quadrant where Firefly spent most of its time pitching an explicit, enhanced user experience that might someday lead to better ad targeting. (To be fair internet advertising revenues were minimal back in the late 90s)

    As you said in your Microsoft post, behavioral targeting can improve click-thru-rates by more than 600%. That transforms a business model, that’s very close to the core business. Similarly, one could argue that profiling users based on search queries has had a much more dramatic effect on Google’s ad revenues than on its user experience.

    I believe that you also made the point in the MSFT post that what a user wants to *do right now* rather than who they *are* is more important for improving click-thru-rates. I think this is why Amazon played with both classic rec tech and ‘also bought’ and ultimately went with ‘also bought’ — it speaks to what a user is interested in right now rather than who they are based on some cumulative profile of their interest in books.

    I bring up your point here because rec tech tends to be associated with building up a profile of a user’s preference, what sort of user are they, and putting them in a category or a user segment. If that is the case then even if the rec tech is flawless you’ve just achieved a less valuable result — you’ve determined who they *are* but not what they want right now. A user might be a tenured professor of literature and all the preferences that come with that, sure, but maybe I just need a plumber at the moment.

    All this to say then that rec tech should be complementary to behavioral targeting. If rec tech made behavioral targeting more effective then rec tech would be more relevant to the core business of a lot more businesses.

    And when would rec tech be complementary to behavioral targeting? Well behavioral targeting takes care of determining what the user wants right now. Rec tech would come in if there were many, many ways to satisfy that need and user preferences played a big part in selecting from among those myriad options.

    Seen from this angle, the range of *lucrative* applications of rec tech starts to look pretty narrow. So we’re either looking for a market like the one described above (behavioral enhanced by rec tech) or, back to the original vision, a market in which the user experience would be totally transformed or even defined by recommendation technology (like movie recommendations that actually nailed it). My assumption is that simple rec tech can do the former but not the latter and that’s why it’s not yet the big business everyone has been predicting for years.

    A pessimist might say, to borrow a phrase, that recommendation technology is the technology of tomorrow and it always will be. What do you think?

  5. 6 predictabuy May 24, 2009 at 9:11 pm

    John,

    There are a number of companies selling ‘recommendation technology’ for e-commerce sites. Have a look at http://www.choicestream.com/ and http://www.richrelevance.com/ for example. I don’t know how well they are doing, but at least in the context of e-commerce this has a sort of ‘main-stream’ feel to it now. So, perhaps the timing on Firefly really wasn’t right – or perhaps there is just a new generation who will eventually reach the similar conclusions.

    Second, if by ‘rec-tech’ you mean collaborative filtering, I think you’d really enjoy reading some of the latest research coming out of the Netflix Prize. Have a look at the publications by the AT&T team for the most recent progress prize. There has certainly been a great deal of discussion around the relative merits of ‘item-based’ and ‘user-based’ recommendations.

    Finally, in the context of my statement that ‘local is a recommendation problem’, I’m really thinking of ‘recommendation’ in the broad sense of the word rather than in a narrow or specific way that might only be taken to include a specific sub-set of algorithms. And in the particular case of Predictabuy, I’m starting to think of what we do as more aptly described as ‘situational targeting’.

    That being said, I think collaborative filtering algorithms will still have some role to play in ‘local recommendations’, but they will be at best only a partial solution for certain classes of problems.

    In the case of the plumber for example, what I really want is a plumber who is available, who does emergency work, who will do a good job and charge me a fair amount of money. That’s still a kind of recommendation problem – though it’s not one where knowing my taste in music or literature will help much!

    Recommendations of local goods and services also have a very strong ‘social’ aspect to them. Over time these social interactions will be supported by or mediated through tools for filtering, evaluating and curating opinions.

    Cheers,
    Eric

  6. 7 johnemcnulty May 28, 2009 at 1:46 pm

    Eric,

    Based on your last post I think you would agree with the claim that local search is more a contextualization problem than a recommendation problem if we take ‘contextualization’ as a specific part of your generalized notion of ‘recommendation’ as ‘situational targeting’

    ‘situational targeting’ breaks down into two phases: ‘situational diagnosis’ and ‘situational recommendation’ Situational diagnosis has also been called ‘contextualization’ and refers to the problem of inferring what situation or context the user is in. Then, given an understanding of the user’s context, you are in a position to recommend solutions.

    As you point out recommendation in the narrow sense works well in some markets. Those markets, not surprisingly, are characterized by the fact that the user context is well understood or given in advance — getting a movie recommendation, getting a book recommendation, etc.

    Where the problem starts to get interesting and, incidentally, most relevant to local search is when the user context is not given but must be inferred from limited information. The problem here is first and foremost one of contextualization — how well can you reconstruct the user’s context? If that can be done with confidence then you are back on the well-trodden turf of evaluating one of the many recommendation technologies.

    The situation is somewhat reminiscent of the early days of AI in which some researchers naively thought they could slap a vision system and robotic arm on their block-stacking decision process. It turned out that ‘basic’ vision and manipulation were much hard to crack than the supposedly ‘harder’ and higher level cognitive processes of deciding how to stack blocks in a desired pattern.

    The vision system seems analogous to me to the contextualization problem. If you knew what the user situation was then, sure, you can recommend solutions using any number of algorithms. But it turns out that figuring out what the problem is is much harder than actually solving it.

    To my mind the key differentiator for applying recommendation algorithms to local search is in the quality of the contextualization solution or “situational diagnosis” and not the recommendation algorithm.

    The good news is that mobile local search provides more clues, more inputs than have ever been available before to help in situational diagnosis or contextualization. The question is whether those clues plus some savvy models are enough to support reliable contextualization.

    I think this is consistent with your expanded notion of recommendation but do you think that contextualization or ‘situational diagnosis’ is “critical path” for successful application of current recommendation solutions to local search?

    John

  7. 8 predictabuy May 29, 2009 at 6:33 pm

    Hi John,

    I’d phrase it this way: the best recommendation for a person depends both on context (as you note) as well as preferences. In addition, the relative weight of ‘context’ and ‘preferences’ varies depending on what I’m looking for (my situation). For example, if it’s midnight and cold outside and my furnace has quit, I’m probably pretty happy with anyone who will come to fix it! If I’m going out for a date and trying to select a restaurant, then preferences come more in to play.

    Lately, I’ve been thinking of the person’s ‘actual’ situation as a ‘hidden’ state – one that we are trying to infer from available information. Algorithmically, you can predict this hidden state directly and then make predictions from it OR make predictions directly from the context and preference information (i.e. don’t explicitly model the hidden state).

    As I get ‘further along’ in the deployment of early customers I’ll be sharing more on the things that work best from a local advertising perspective.

    Cheers,
    Eric

    • 9 johnemcnulty May 30, 2009 at 11:18 am

      Eric,

      ‘hidden state’ is how I think of the contextualization problem as well. In the case of local advertising I would lean toward an explicit model of the inferred state because I think the models are useful for communicating audiences to advertisers. For example if you were able to determine the state you described — midnight, very cold, boiler busted — then this is readily communicated to and compelling for an advertiser.

      It also works in reverse. For example an advertiser (agency) came to us in the UK asking whether we would be able to target university students preparing for the start of school so they could offer them credit cards (oh so pre-credit crisis! 🙂 Given that model you can look across the data points you have and ask which would be evidence for and against identifying a college student engaged in the target activity. You could for example aggregate all the postcodes within walking distance of a university, target clothing of interest to that age group and stationary shops, etc. Previous activity could also be used to identify users who are likely to be of the appropriate age group.

      Given the models you could use Bayesian techniques to update the probability that one or another is the best available hypothesis for explaining the current observations. That leaves a lot of unconstrained assumptions but I had some thoughts about a rational way to tie down those constraints while at PS. Happy to pursue in more detail if of interest for your current efforts.

      John


  1. 1 Netflix Prize has lessons for local search « Predicting What Consumers Want to Buy Trackback on May 7, 2009 at 12:40 pm
  2. 2 Thinking holistically about local search « Predicting What Consumers Want to Buy Trackback on May 9, 2009 at 11:49 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s





%d bloggers like this: