LP Login

Think Big. Move Fast.

On April 15th, 2007 an excellent article in the NY Times magazine asked “Is Justin Timberlake the product of cumulative advantage?“. It describes an experiment in which a group of unknown songs was exposed to different sets of people who were able to listen, rate and download them. One group of people had no exposure to what songs others were listening to, rating and downloading. The other eight groups were exposed to feedback on what others in their group (and only their group) were doing. Here is the conclusion:

… if people know what they like regardless of what … other people like, the most successful songs should draw about the same … market share in both the independent and social-influence conditions … And … the “best” ones … should become hits in all social-influence worlds.

… we found… exactly the opposite. In all the social-influence worlds, the most popular songs were much more popular … than in the independent condition. At the same time, however, the particular songs that became hits were different in different worlds… Introducing social influence into human decision making, in other words, didn’t just make the hits bigger; it also made them more unpredictable.

… intrinsic “quality,” which we measured in terms of a song’s popularity in the independent condition, did help to explain success in the social-influence condition. … But the impact of a listener’s own reactions is easily overwhelmed by his or her reactions to others.

… Because the long-run success of a song depends so sensitively on the decisions of a few early-arriving individuals, whose choices are subsequently amplified and eventually locked in by the cumulative-advantage process, and because the particular individuals who play this important role are chosen randomly and may make different decisions from one moment to the next, the resulting unpredictability is inherent to the nature of the market.

(This is heavily edited – read the whole thing. )

This is fascinating stuff.

Jordan Schwartz and Tim O’Reilly point to this as an example of the “Hive Mind”:

You can see these same processes at work in social bookmarking and ranking services like Digg and Del.icio.us. Individual users rate a site as being interesting, causing other users to visit it and, in turn, assess whether the site is interesting enough to rate it as interesting. Over time, certain sites gain momentum and rise to the top of the heap, even though most individuals only ever see a small fraction of the options.

There is a bit of an implication that the BEST sites rise to the top due to these social processes – a sort of “Wisdom of the Crowds” effect.

Josh Porter at Bodarko draws the opposite conclusion, that

This result has huge implications for all social web sites, especially those that show aggregate data. Digg, for example, shows aggregate data everywhere on the site. This experiment, in addition to several other issues that I wrote about in Digg’s Design Dilemma, suggest that the results there are socially influenced to such an extent that it would be hard indeed to know where the quality lies…

Others have posted in the past about why the Wisdom of Crowds fails on Digg.

Its worth going back to Surowiecki’s book to understand why. Summarizing Kottke:

In order for a crowd to be smart … it needs to satisfy four conditions:

1. Diversity. A group with many different points of view will make better decisions than one where everyone knows the same information.

2. Independence. “People’s opinions are not determined by those around them.”

3. Decentralization. “Power does not fully reside in one central location, and many of the important decisions are made by individuals based on their own local and specific knowledge rather than by an omniscient or farseeing planner.”

4. Aggregation. You need some way of determining the group’s answer from the individual (i.e. unbiased) responses of its members..

Digg fails on all four of these criteria; it is not a completely diverse group of users, Digg intrinsically influences its users’ opinions, there are certain Digger’s who have more power to push a story to the front page than others, and Digging does not occur in a vacuum.

Harvard Business review has another explanation for why crowds don’t always identify the best solutions by way of the Marquis de Condorcet, a 17th Century mathematician:

Groups will do better than individuals in choosing a correct answer, and big groups better than little ones, as long as two conditions are met: the majority response “wins,” and each person is more likely than not to be correct.

(Italics mine)

In a large enough sample set of options, the likelihood that a person can choose the “best” article or “best” song rapidly drops below 50%, so this second criteria fails.

Ultimately, there are a couple of key takeaways for people who run social media sites:

1. If you’re trying to iterate towards a “best answer” then keep feedback loops to a minimum, at least before users “vote” on their own. (e.g. Hotornot, espgame)
2. If you’re trying to create “hits” out of some of your content (and don’t care if it’s the “most worthy” content – you only care that they are hits), then display feedback and popularity constantly, as this will effect user behavior and exacerbate the size of the hits (e.g. Youtube, Digg, American Idol?
3. If you want to “guide” user behavior in a certain direction, provide feedback that validates or shows the popularity of that behavior. This is consistent with my prior post on game mechanics applied to social media: keeping score.

Can you think of more examples of these three best practices? If so, please add them in comments.

Continue Reading ...

I’ve posted in the past on applying game mechanics to social media. Robb Web’s blog pointed me to a fantastic lecture by Luis von Ahn about how to design games to take advantage of human computation. In effect, he gets people to solve problems that computers can’t by making them into a fun game that people want to play. His research interests also include Captchas, automated tests that humans can pass but that current computer programs cannot. The most common version of this is the distorted set of words and numbers that you sometimes have to type when getting a new account or entering a comment on a blog to prove that you’re human.

I believe this lecture was part of the Google Tech Talk series. It was given in July 2006 and was widely covered at that time, but I missed it.

Examples of the games that von Ahn has developed include the ESP game that gets people to tag images, the Peekaboom game that gets people to identify objects within an image, and the Phetch Game that gets people to find a particular kind of image on the web. I’ve played them all, and they are all pretty fun!

Put these pieces together and you have the ingredients for an image search engine…

(In the lecture he also mentions the Verbosity game that helps collect well known facts to help fill the corpus of knowledge for AI research, but this seems to not be working at the moment.)

This guy is a genius. The a video is about 40 minutes (with another 10 minutes of questions) but it really is time well spent. I don’t make a 40 minute recommendation lightly.

Continue Reading ...

Searchengineland points to an interesting article in PC World which pitted the top search engines against each other to determine which had the best (ie most relevant) search results.

Their conclusion was that Google had the best overall results across multiple query types (general search, image search, video search etc), but:

two other services topped it–barely–in our text-search tests.

(Text-search is general web search). These two services were Alltheweb and Altavista, both owned by Yahoo – the chart of results is here

These results don’t differ all that much from what we found in testing the various search options at AOL when I was there a couple of years ago. When stripped of UI and branding, most users couldn’t tell the difference in quality between search results. When UI and branding was returned, most people thought that Google was the best search engine.

This speaks to the power of branding in the third phase of competition of consumer internet companies, as I have blogged about in the past. First comes distribution, then product and finally branding. It explains why, even though Google’s general web search isn’t head and shoulders better than the other search engines any more, it continues to win market share.

Continue Reading ...

On Monday Myspace announced the results of a research study that they commissioned, showing the effectiveness of marketing campaigns within social networks. It was widely covered. The key results were that:

– More than 40 percent of all social

Continue Reading ...

Wagner James Au has a great post on GigaOm about Gaia. Gaia is a casual immersive MMORPG that describes itself as:

“The world’s fastest growing online world hangout for teens.”

In an interview with Gaia‘s CEO, Craig Sherman, …

Continue Reading ...

March’s Comscore numbers just came out. I took a look at the top 2000 web domains, but this time I ranked them by number of visits per month. The results were a little surprising. Here are the 41 sites that …

Continue Reading ...

I spoke today at the Web2.0 expo on the topic of how this generation of internet companies can make money under both media and e-commerce models (Show me the money). This is an expansion of previous posts I

Continue Reading ...

I’m particularly interested in social media sites with traction focused on topics appealing to endemic advertiers. Social media sites because of their extraordinary ability to grow without incurring marketing costs, and endemic advertisers because of their willingess to pay …

Continue Reading ...

I posted recently on the importance of context for social media sites; the need to be “easy to learn and hard to master”.

Two recent stories/posts have reminded me on the consequences of failing to adhere to this approach.


Continue Reading ...

Last week Kosmix announced that Jon Miller has joined their board of directors. I’m very excited.

Jon is one of the most visionary thinkers about the internet that I know. I was his chief of staff while he was CEO …

Continue Reading ...