The wisdom of crowds, and why your chat’s middle guess is so good

The ox at the county fair

The classic story comes from 1906. At a livestock fair in Plymouth, visitors paid to guess the weight of an ox once it had been slaughtered and dressed. The statistician Francis Galton collected the tickets afterwards and, in a short letter to Nature in 1907 titled “Vox Populi”, reported what he found. The middlemost guess, the one with as many guesses above it as below, was 1,207 lb. The ox weighed 1,198 lb. The crowd’s median was out by less than one per cent.

None of the individuals was especially expert, and many guesses were far off. But their errors pointed in different directions and largely cancelled out. That idea, popularised by James Surowiecki’s 2004 book The Wisdom of Crowds, has since been tested with jelly beans in jars, quiz questions and forecasts of all kinds.

Why it works

Every guess has two parts: some real knowledge, and some error. If the errors are random (one person too high because they pictured a bigger animal, another too low because they forgot the bones), then combining many guesses keeps the knowledge and lets the errors cancel. The more independent guesses you have, the more the noise shrinks.

Surowiecki described four conditions that help:

• Diversity of opinion: people bring different information and different ways of thinking.
• Independence: people do not just copy each other.
• Decentralisation: people draw on their own local knowledge.
• Aggregation: there is a sensible way to turn many answers into one.

A Twitch chat is, almost by accident, a good fit for the first three. Viewers come from different countries, ages and backgrounds, and in the FLUNG games every guess stays hidden until the lock, so nobody can copy. The game supplies the fourth.

Why the median, not the average

The obvious way to combine guesses is to take the average. On the internet, that is a mistake. Here is a set of nine guesses for a question whose answer is in the hundreds, with one viewer typing a joke number:

120, 250, 300, 400, 450, 600, 900, 2,000, 5,000,000

One silly number has dragged the average into the hundreds of thousands. The median barely notices, because it only cares about the order of the guesses, not how extreme the outliers are. To move a median, lots of people have to move.

The “log space” part matters for estimation questions, where answers span orders of magnitude. Guesses of 10 and 1,000 are equally wrong if the answer is 100 (each is ten times off), so the game treats them as balancing around 100, not around 505. Galton himself chose the middlemost guess for much the same reason: it is a vote in which no one can win by shouting louder.

The Chat Brain

In SIZE SHOCK and WORLD SHOCK, every valid chat guess goes into one combined answer, the Chat Brain. It is the median of all valid guesses in log space, worked out the moment the window locks. The Chat Brain then goes head to head with the streamer’s own hidden guess: whichever is closer wins the round for its side, and the scoreboard at the top reads CHAT against STREAMER.

That turns a guessing game into a team game without anyone having to organise a team. A viewer who would never type in a busy chat still contributes to the result, and a streamer who is confident about planets can still lose to three hundred people who are each a little bit right.

Party mode uses a related idea for teams. In Teams Quick mode a team’s answer is the geometric mean of its members’ guesses, so every member counts equally; in Huddle mode the team agrees one answer together.

When crowds go wrong

The wisdom of crowds is not magic. It fails in predictable ways:

• Herding. If people can see earlier answers, they anchor on them and the errors stop cancelling. That is why guesses are hidden until the lock.
• Shared blind spots. If everyone has the same wrong idea (say, an out-of-date population from school), the crowd is confidently wrong together. Some of the best rounds in WORLD SHOCK are exactly these.
• Too few people. With three guesses, one wild answer still has a lot of say. The median helps, but small crowds are noisier.

Watching for those moments is half the fun: the rounds where the streamer beats the crowd are usually the ones where the crowd shared a myth.

Try it yourself

The easiest way to see the effect is to play. On stream, the Chat Brain appears on every reveal. With friends, play a few rounds of FLUNG Party in teams and compare the team answers with individual ones. Or play DAILY SHOCK with your family each day and compare your scores with the group’s middle score.