I just read a fascinating NPR article about a three-year study called The Good Judgment Project, conducted by three psychologists and a few people within the intelligence community.
The study's purpose was to determine how accurately a group of regular citizens could predict the outcomes of important world events.
The group consisted of 3,000 people, none of whom had any government security clearance or access to classified information, and they were all asked to make predictions on issues, such as these:
- Would there be a significant attack on an Israeli territory before spring of this year?
- Would North Korea launch a new multi-stage missile prior to May 10, 2014?
This random group of 3,000 political neophytes predicted the outcomes of world events with incredibly high accuracy.
In fact, those people were so accurate that a smaller subset of this group was able to predict world affairs 30% more accurately than the CIA. No training, no special government clearance, no daily briefings. Yet the random group members were better judges of political, military, and economic outcomes than the world's most revered intelligence agency. How is this possible?
The answer lies in a theory founded by a British statistician named Francis Galton in 1906, called "The Wisdom of the Crowds." The theory states that when large groups of people make a prediction about something, their averaged collective answer is more likely to be correct than when one person (or a few people) with a vast amount of knowledge on that subject makes the same prediction.
So, what does this have to do with recommendations systems for online retailers? Well, to be honest, everything.
When shoppers visit a retailer's website, there is a good chance they will be presented with specific product recommendations as they surf around from page to page. "People Who Like That Product Also Like These Products" and "We Think You'll Love These Items" are common website merchandising blocks we have all seen on most stores' sites. And, for the most part, the same assertion that the three psychologists in The Good Judgment Project made—that large groups of people can predict something better than a few people can, no matter how much or how little information they have—is used every time a shopper visits a retailer's website.