Surface the Assumptions and Limitations Behind a Number
A number that fits your decision has cleared the first hurdle, but it is not home free. Every figure quietly rests on things it takes for granted, and each of those quiet assumptions marks a limit on how far the number can be trusted. This chapter shows you how to surface those assumptions and then turn them into a short, repeatable check for what a number leaves out.
Surface the Assumptions a Number Rests On
Every number arrives with a backstory it does not tell you. Someone decided who to count, over what stretch of time, and what the count would stand for, and all of that is folded silently into the single figure you end up seeing. Before you trust it, your job is to unfold it again.
Three assumptions hide behind almost every everyday figure:
That the few it counted stand in for everyone, so a handful of reviewers speak for all customers;
That a pattern from the past will carry on into the future, so last month’s rise keeps rising;
That the thing it measures means what its label suggests, so a star average equals real quality.
Naming the assumption is the whole first move. Once you can say out loud what a number is taking for granted, you can ask whether that thing is actually true, and that question leads straight to the number’s limits.
Know Who and What a Number Counts
Most everyday numbers are not built from everyone; they are built from a slice. Understanding that slice is what turns a vague unease about a figure into a specific, checkable doubt.
These four ideas explain most of the ways a number quietly misleads. A figure drawn from a tiny sample, or from people who selected themselves in, can be perfectly accurate about that slice and still say very little about the whole. Holding an assumption up against these ideas is how you find where it breaks.
Run the Limitations Checklist
Once you have named a number’s assumptions, you can run the same short checklist over any figure to expose its limits. Each item is a question, and a “yes, worth worrying about” marks a limitation.
Small or self-selected sample: was the figure built from just a few cases, or from people who chose to be counted? A sample size of a dozen, or a group of volunteers, cannot speak for a whole population;
Short time frame: over how long was it measured? The time frame is the stretch of time a number covers, and a single week or month can swing up or down by chance;
Omission: what got left out? An omission is anything relevant the number does not count, like the customers who never left a review at all;
Unfair comparison: is it lined up against something truly comparable? An unfair comparison sets a number beside one gathered differently, over a different period, or from a different kind of group;
Base rate ignored: how common is the thing anyway? The base rate is how often something normally happens, and ignoring it makes a number look more surprising, or more reassuring, than it really is.
Keep the base-rate check intuitive: before you react to a count, just ask “out of how many?”
Do I really have to run all five every time?
Not slavishly. With a little practice, the checklist collapses into a single habit: you glance at a number and its weak spots stand out on their own. Early on, running the five questions in full is the fastest way to build that instinct.
Watch It Work: The Restaurant’s Five Stars
Picture yourself deciding where to book dinner. One restaurant shows 4.7 stars, and you think, “people clearly love it.” Before booking, surface the assumption, then test its limits.
Start with what the rating takes for granted:
that the people who reviewed stand in for all its diners (representativeness);
that those diners chose to post rather than being asked at random (self-selection);
that a high star average means the food and service are genuinely good (the metric means what it says).
Now run the checklist against those assumptions:
Sample and self-selection: the 4.7 comes from 22 reviews, a small and self-selected batch, the handful of diners moved enough to post, often the delighted or the furious rather than the quiet middle;
Time frame: most of those reviews are from one busy holiday month, not the ordinary weeks you would actually visit;
Omission: nobody who booked, waited too long, and left without eating shows up in the count;
Unfair comparison: the place next door shows 4.3 from 900 reviews, and treating 4.7-from-22 as “better” ignores how differently those two numbers were built;
Base rate: how common is a high score anyway? If nearly every restaurant in the area already sits above 4.5 stars, a 4.7 barely stands out, and treating it as a mark of special quality ignores how ordinary that score really is.
None of this proves the restaurant is bad. It simply moves the honest reading from “everyone loves it” to “the few who posted, mostly during one busy month, rated it highly.” That is a real but modest claim, and it is exactly what the number can bear once its assumptions and limits are on the table.
Let’s Recap!
Every number quietly assumes something: that a few stand for everyone, that a past pattern continues, or that a metric means what its label says.
Naming the assumption comes first, and each assumption points to a limitation you can test.
A sample misleads when it is small, self-selected, or not representative of the whole group.
The limitations checklist covers sample, time frame, omissions, unfair comparisons, and the base rate.
Surfacing assumptions and limits turns an overstated claim into the honest, modest one the number can actually support.
Even a number with sound assumptions and clear limits can still be misread in one last way, as proof that one thing caused another or that a trend is bound to continue; the next chapter takes on both traps.
