Appraise Evidence From Start to Finish
You have now learned every move on its own: telling support from proof, judging fit, surfacing assumptions and limits, resisting the correlation and trend traps, and stating what the evidence honestly supports. This final chapter runs all of them in a single pass. First, watch one ordinary recommendation, backed by a small pile of mixed evidence, go through the whole method from first look to honest verdict. Then you will run the same method yourself on a fresh case and check your work against a model solution at the end of the chapter.
Meet the Recommendation and Its Evidence
The Rivera household wants to spend less each month, and someone floats a plan: “We should switch our mobile phone plan to BudgetMobile, it will save us money.” Before anyone signs up, they gather what they have. Three pieces of evidence sit on the table:
A price comparison: their current plan costs $40 a month, and BudgetMobile advertises a similar plan at $25 a month.
A personal usage log: the phone app shows the household used about 12 gigabytes of data last month.
A shared statistic: a switching website claims “94% of people who switched to BudgetMobile saved money.”
One confident recommendation, three numbers that seem to back it. The job now is not to accept or reject the plan on a hunch, but to run each piece through the method and see what the evidence, taken together, actually supports.
Judge the Fit of Each Piece
Start with fit: does each piece actually bear on the decision, which is whether this household will pay less overall on BudgetMobile?
The price comparison measures the advertised monthly price of each plan. That bears on the decision, but only partly, because the decision turns on what they would really pay, and a headline price can hide fees, taxes, and promotions. It fits, but treat it as partial.
The usage log measures how much data the household actually uses. That bears squarely on the decision, because a cheaper plan only saves money if it covers what they need. Strong fit.
The “94% saved” statistic measures how other switchers fared. It sounds directly on point, but it measures other people’s outcomes, not this household’s. It is, at best, weakly relevant.
Already the piece that sounds most persuasive, the 94% figure, is the one that fits worst, and the plainest piece, their own usage, fits best.
Surface the Assumptions and Name the Limits
Now surface what each piece quietly assumes, and run the limits checklist against it.
The price comparison assumes the $25 is what they would truly pay, every month. Check the limits: the advertized rate is a six-month promotion that rises afterward, and it leaves out a one-time setup fee. That is an omission and an unfair comparison, a promo price set against their current standard price, so the real gap is smaller than $15 a month.
The usage log assumes last month was typical. Its limit is a short time frame: one month, which may have been unusually quiet or busy. It is worth confirming with a second month, but it is real, personal data, which makes it the sturdiest piece here.
The “94% saved” statistic assumes those switchers resemble the Riveras and that “saved” was measured honestly. Its limits are serious: the figure comes from a self-selected sample, people who chose to switch and chose to answer a survey, and it ignores the base rate, since people who switch specifically to save will naturally report saving. It tells you almost nothing about whether this household would.
If a piece of evidence has limits, should I just throw it out?
No. Naming a limit does not delete the evidence; it tells you how much weight it can bear. The usage log has a limit (one month) yet still carries real weight, while the 94% figure has limits serious enough to make it nearly weightless here.
Check the Correlation and Trend Traps
Two quick trap-checks before the verdict.
There is a trend in the background: the household’s bill has crept up three months running, and the worry is that it will keep climbing unless they switch. But a trend is not a guarantee. Ask what drove the rises, and it turns out a phone-installment charge is ending next month, which will lower the bill on its own. The upward run was not a law of nature; it was about to reverse.
There is also a tempting cause-and-effect story: “the months our bill was highest were the months we streamed the most, so streaming is what makes switching worth it.” But moving together is not causing. A likelier third factor sits behind both: on months the family traveled, they streamed more and used more data of every kind. Streaming did not single-handedly drive the bill, so “switch to fix the streaming” misreads the link.
Reach an Honest Recommendation
Now put it together. The pivotal moment comes from the best-fitting piece, the usage log. At 12 gigabytes a month, the household sits well above the 5 gigabytes the $25 BudgetMobile plan includes, and beyond that allowance the plan charges steep overage fees. Run the real numbers and the “saving” likely disappears, because their actual usage would push them into charges that erase the $15 gap and possibly more.
So the honest verdict is not the confident one they started with. One accurate, highly relevant piece of evidence, their own usage, points against the recommendation, and hiding it to keep the tidy “we’ll save money” story would be the quiet form of overclaiming. Stated honestly:
“On its headline price the BudgetMobile plan looks cheaper, but that rests on a six-month promo and ignores fees, so I hold it at low confidence. Our own usage log, the most relevant evidence we have, points the other way: at 12 gigabytes a month we would exceed the plan’s 5-gigabyte allowance and pay overages that likely wipe out the saving. So the evidence does not currently support switching to this plan. It supports either staying put or finding a plan that actually covers about 12 gigabytes. My confidence is medium, capped by having only one month of usage, and I would confirm with a second month before deciding.”
That statement carries a claim, a confidence level, a reason, and the limit that bounds it, and it lets the inconvenient piece do its work instead of vanishing. That is the whole method, from start to finish.
Your Turn!
The method is only yours once you have run it cold, without a walkthrough beside you. Before you leave the course, take one fresh recommendation all the way through on your own.
Here is your case. A commuter says: “I should buy the $70 monthly transit pass instead of paying per ride, it will save me money.” They have three pieces of evidence:
A price comparison: a monthly pass costs $70, and a single ride costs $2.50.
A personal trip log: last month they took the bus 20 times.
A shared statistic: a transit-app banner says “most riders save with a monthly pass.”
Run the same method you just watched:
Judge the fit of each piece against the real decision (will this commuter pay less overall?).
Surface what each piece assumes, and name its limits.
Decide whether any well-fitting piece points against the recommendation.
State what the evidence honestly supports, with a confidence level, a reason, and its limits.
Keep everything to public information and the commuter’s own records, with no special tools beyond simple arithmetic. Give it more time than feels comfortable, and resist the urge to reach for the “most riders save” line as if it settled things.
Then check your answer against the model solution found at the end of this chapter.
Let’s Recap!
A full appraisal runs one recommendation through every move: fit first, then assumptions and limits, then the correlation and trend traps, then an honest claim.
Judging fit early often demotes the most persuasive-sounding number and promotes the plainest, most personal one.
Naming a limit sets how much weight a piece can bear; it does not delete the evidence.
Let a well-fitting piece that points the other way lower or flip the recommendation instead of hiding it.
State the verdict with a claim, a calibrated confidence level, a reason, and the limit that bounds it.
That completes the method, and the course. You can now take a real recommendation, judge what its evidence honestly supports, and say so with the right amount of confidence, which is exactly what this course set out to give you. Once you have compared your work against the model solution below and the course quiz confirms it is solid, the method is yours to use on the next number that tries to decide something for you.
Model Solution
Here is one strong way to appraise the transit-pass case. Your wording will differ and can still be right, so compare the shape of the reasoning rather than the exact words.
Judge the fit:
The price comparison ($70 pass versus $2.50 a ride) measures headline prices. It bears on the decision but is only useful once paired with how often the commuter actually rides. Partial fit.
The trip log (20 rides last month) measures this commuter’s real usage. It fits the decision squarely, because the pass only saves money above a certain number of rides. Strong fit.
The “most riders save” statistic measures other riders’ outcomes, not this commuter’s. It sounds relevant but bears only weakly on the decision.
Surface assumptions and limits:
The price comparison assumes the commuter rides often enough for a flat pass to beat per-ride fares. Its limit: on its own it says nothing about how many rides that takes.
The trip log assumes last month was typical. Its limit is a short time frame, one month, which may have been unusually light (a holiday, illness, or working from home). Worth a second month to confirm.
The “most riders save” figure assumes those riders resemble this commuter. Its limits: a likely self-selected sample and an ignored base rate, since heavy riders who buy passes will naturally report saving. It says little about a 20-ride month.
Find the counter-evidence and do the arithmetic:
The best-fitting piece points against the pass. At 20 rides, paying per ride costs 20 × $2.50 = $50, which is less than the $70 pass. On last month’s usage, the pass would cost more, not less.
The break-even point is 28 rides a month ($70 ÷ $2.50). The commuter would need to ride noticeably more than they did last month for the pass to pay off.
State what the evidence honestly supports:
“On last month’s actual usage, the monthly pass would cost me more than paying per ride ($50 versus $70), so the evidence does not currently support buying it. The pass only pays off above about 28 rides a month. My confidence is medium, limited to a single month of trips; if my riding is usually heavier than last month, I would log a second, more typical month before deciding, and reconsider the pass if I regularly cross the high-20s.”
Notice the move that matters: the “most riders save” banner, the most confident-sounding piece, carried almost no weight, while the commuter’s own 20-ride log, plainer but far better-fitting, flipped the recommendation. Letting that inconvenient number speak, instead of reaching for the reassuring one, is the whole point of the method.
