Base Rates: A Short History of Starting With the Usual Case

A café opened a few streets from where I live, and I gave it a year. I had nothing to base this on except a feeling: the corner was a bit out of the way, the menu looked ambitious, and the owner seemed to be doing everything alone. I told a friend so, with more confidence than I'd earned. Later I realised I had never once asked the plainer question: how do new cafés usually fare? I had gone straight to the story and skipped the background.

That plainer question is the idea behind a base rate. A base rate is how often something happens across the whole group of similar cases. Asking for it before you look at the details gives you a steady place to start, and the details can then move you up or down from there. It is a small habit, but it has a long and interesting history.

What is a base rate, in plain words?

Suppose you want to know whether a new café will still be trading in a year. You can think about this café: its coffee, its location, how friendly the owner is. Or you can think about new cafés in general and ask what fraction are still open after twelve months.

The second question gives you the base rate. I don't have a trustworthy figure for it, and I'd be wary of anyone who quotes one without a source, because it will differ from place to place and from one period to another. For the example below I'll use a made-up number and label it as such. The point is the shape of the reasoning, not the figure.

The reason to start there is that the usual case carries a great deal of quiet information. Every café in the group faced rent, tired mornings and fickle footpaths. The group's track record already contains many of the forces that will act on your café too, including ones you can't see from the pavement.

Where did this way of thinking come from?

The mathematics behind base rates starts with a problem that sounds technical but is really quite human. Early probability theory mostly went forwards: if you know a coin is fair, how likely are seven heads in ten tosses? Jacob Bernoulli's Ars Conjectandi, published after his death in 1713, pushed towards the harder question that people actually face. You see the outcomes and want to know what is behind them. If you see seven heads in ten tosses, what can you say about the coin?

Thomas Bayes, an English minister and mathematician, worked on exactly this. His essay was published in 1763, after his death, because his friend Richard Price found it among his papers, tidied it and sent it to the Royal Society. Bayes imagined a ball thrown onto a table at an unknown spot. A second ball is then thrown repeatedly, and you are told only whether each throw lands to the left or right of the first. With each report, your idea of where the first ball lies sharpens. It is a tidy picture of learning: start with a rough guess, receive evidence, revise.

Historians are not sure why Bayes wrote the essay or why he never published it himself, and I'd rather say so than invent a motive. We know what the essay did, and that Price saw its value.

The credit for spreading the method goes largely to Pierre-Simon Laplace, who arrived at similar results independently in the 1770s and developed them over decades. In his Philosophical Essay on Probabilities (1814), he described probability as common sense reduced to calculation. His famous illustration was the sunrise: if the sun has risen every morning you know of, how confident should you be that it rises tomorrow? His answer, now called the rule of succession, says that after n unbroken successes, the chance of another is (n + 1) ÷ (n + 2). It grows toward certainty without ever claiming it.

Two cautions are worth adding. The phrase "base rate" is a much later one, and Bayes and Laplace did not use it. They spoke of the initial state of knowledge before the evidence, which modern writers call the prior. Also, Laplace himself noted that the sunrise figure was too naive, because we know far more about how the solar system works than a bare tally of mornings. The method is only as good as what you put into it.

Why do we forget the usual case?

For most of this history, the habit was a technique for mathematicians. It became a topic for everyone when psychologists noticed that people tend to skip it. In the 1970s, Daniel Kahneman and Amos Tversky ran studies in which people were given a description of a person and asked to guess their occupation or field. Participants leaned heavily on how well the description matched a type and gave far less weight to how common that type was.

You can feel the pull with a version of that puzzle. Imagine a man who is quiet, tidy and loves books. Is he more likely to be a librarian or a farmer? The description fits our idea of a librarian, so the answer seems obvious. But in most countries there are many more farmers than librarians. Even if a modest share of farmers are quiet and bookish, they may outnumber the bookish librarians. The vivid detail is doing a lot of persuading, and the quiet fact about how many of each there are is doing none.

This is the misconception worth correcting. Remembering base rates is not stereotyping, and it is not an instruction to ignore the individual. It is the opposite of declaring a verdict from a category. It gives you a starting estimate, which the individual details are then allowed to change. The details still count. They just count as adjustments, not as the whole answer.

How do I use a base rate on an ordinary day?

Return to the café. Here is a worked example with numbers I have invented purely for illustration.

Say that out of 100 new cafés in a town, 60 are still open after a year and 40 have closed. That is your starting estimate: about a 60% chance of survival.

Now look at the particulars. Suppose you notice that the owner has run a kitchen before and the café is near a busy bus stop. Let's say half of the cafés that survive have signs like these, and 30% of those that close do. (Again, invented numbers.) Count it out:

• Survivors with those signs: 50% of 60, which is 30 cafés.
• Closers with those signs: 30% of 40, which is 12 cafés.
• Among all cafés showing the signs, 42 in total, 30 survive. That is about 71%.

So the encouraging details moved you from 60% to roughly 71%. They made a real difference but did not turn a maybe into a certainty. If the signs had been discouraging, the same arithmetic would have pulled the estimate down, not off a cliff.

Working in counts, as above, is a gentle trick. Many people find "30 out of 42 cafés" much easier to hold in their heads than a percentage and a conditional probability. If a statement of odds ever leaves you tangled, try imagining a hundred cases and counting.

My own forecast for the corner café had no base rate in it at all. It was all detail, and a few of the details were just my mood. I'm still learning to catch that before I say it aloud to a friend.

Connecting the idea across fields

What I like about this idea is how often it turns up under other names. Weather forecasters, for instance, often judge a forecast against a simple baseline, such as the long-run average for that date or "tomorrow will be like today". A fancy prediction is only worth its trouble if it beats the plain one. The base rate acts as a bar to clear.

Project planners have a similar habit. Instead of building a schedule only from the particulars of this project, they look at how similar projects have turned out, a view that Kahneman and a colleague later called the "outside view". It is the same move: ask how the whole family of cases has gone before leaning on the story of this one.

Notice that in all these fields the base rate is a floor to stand on, not a ceiling. It protects against the commonest error, which is being carried off by a vivid detail.

What are the limits of the usual answer?

There are honest difficulties, and I don't think they have tidy solutions.

Choosing the group is a judgement. Is your café one of "new cafés in the country", "new cafés in a regional town", or "new cafés run by a first-time owner"? Each narrower group may be more relevant, but it also has fewer cases behind it, so the figure is shakier. Experienced forecasters seem to settle for a reasonable compromise rather than a perfect answer, and I haven't found a rule that removes the judgement.

Base rates can be out of date. Towns change, and the past is only a guide when the conditions are broadly similar.

Some cases really are unusual. If a situation has no meaningful group of comparable cases, the base rate has little to offer, and it is better to say so than to force one.

The numbers need a source. My figures above are invented to show the method. For anything that matters, find a real one, and be willing to say "I don't know the base rate" when you can't.

A question to carry around

I find it quietly remarkable that an idea from a minister's unpublished papers, shaped by a French mathematician's pen and later tested by psychologists, ends up as advice about cafés and kitchen-table guesses. The history suggests something else too. Bayes's table, Laplace's sunrise and the bookish stranger all ask us to hold two things at once: what usually happens, and what is special about this case.

Here is what I'm still wondering about. When you most trust your sense of a situation, how much of it is the particular story, and how much is the background you stopped noticing? Perhaps the next time you make a confident guess, you could ask the plainer question first and see how far it moves you. Thank you for reading, and I'd be glad to hear what you find.