Weigh Evidence by Quality, Not by Count
In the last chapter, your synthesis matrix surfaced something telling: the source making the loudest, most unconditional claim was also the one selling the product. That is a clue about weight, and weight is what this chapter is about. When your sources disagree, you cannot settle it by seeing which answer turns up most often. You have to judge which evidence is actually worth more, and be able to say why. This chapter gives you a repeatable way to do exactly that.
See Why Counting Sources Misleads You
The tempting shortcut, when sources disagree, is to count heads: whichever claim the most sources make must be the right one. That move even has a name. Vote-counting is deciding a question by how many sources back each side, as if evidence worked like an election. It feels fair and objective, and it is neither.
Here is why it fails. Counting treats every source as one equal vote, but sources are not equal. Three blog posts that all copied their figures from the same press release are not three pieces of evidence; they are one press release, repeated three times. Meanwhile a single careful measurement that happens to disagree with them is not outvoted, it is simply better evidence being shouted down. When you count, you reward whichever claim got repeated most, which is often just the claim with the best marketing behind it, not the best support underneath it.
So if counting is out, what replaces it? Judging each piece of evidence on its merits, and letting the strong outweigh the weak, however many of each you happen to have.
Weigh Each Source by Quality, Relevance, and Independence
The thing you are judging is a source’s evidence weight: how much a piece of evidence should count toward your answer, based on how good it is and how well it fits your question. A high-weight source can settle a point on its own; a low-weight source barely moves it, even when you have several. Three cues decide weight, and you check all three.
The first cue is method quality: how the source actually got its claim. Ask what stands behind the statement. A measured test, a large survey, or a documented record is strong. A single person’s impression is weaker. A marketing claim, built to sell rather than to inform, is weaker still, especially when the wording hedges with phrases like “up to” or “as much as”.
The second cue is relevance: how closely the source matches your scoped question and its boundaries. A perfectly reliable figure about the wrong thing is worth very little. A national average does not answer a question scoped to your street, and last decade’s data does not answer a question about now. Weight rewards evidence that lands inside the boundaries you set in Part 1, not evidence that is merely true in general.
The third cue is independence: whether a source reached its claim on its own, or simply repeated someone else. Two sources that trace back to the same origin give you one piece of evidence, not two. Independent sources that agree are powerful precisely because they could have disagreed and did not.
Run a Weight-of-Evidence Check
To make this repeatable, put each source through the same short check before you lean on it. The check does not produce a score. It produces a judgment you could defend out loud.
For each source, ask three questions:
Method: how did this source get its claim, through a measurement, a record, a personal impression, or a sales pitch?
Relevance: does it fall inside my question’s boundaries of who, what, where, and when?
Independence: did it reach this claim on its own, or is it echoing another source I already have?
From the answers, place the source in a rough band: high, medium, or low weight. High means strong method, on-target relevance, and genuine independence together. Low means weak method, loose relevance, or a mere echo. Most sources land somewhere in the middle, and that is fine; the bands exist to rank your evidence, not to grade it out of ten.
Weigh the Air Fryer Sources
Return to the four sources from the last chapter and put each through the check. This is where counting and weighing finally part company.
Source B, the independent lab report: it measured energy and time directly, on exactly the question you are asking, with no product to sell. Strong method, high relevance, genuinely independent. This is your highest-weight source.
Source A, the manufacturer’s page: relevant to the claim, yes, but its method is opaque, its wording hedges (“up to 70%”), and it is the least independent source imaginable, since it exists to sell the product. Low weight for settling the disputed figures.
Source C, the magazine’s hands-on review: a real, independent trial, on-topic, but resting on impression rather than measurement where energy is concerned. Medium weight, and genuinely useful on the capacity limit it observed first-hand.
Source D, the aggregated user reviews: many voices, which is exactly the trap. They are independent of each other in one sense, but reviews often echo the marketing, their relevance swings with household size, and none of them measured anything. Treat them as a soft signal, not a heavyweight, no matter how large the crowd.
Now watch what weighing does that counting could not. By a head count, the “air fryer always saves a lot” side looks strong: the manufacturer says so, and plenty of enthusiastic reviews agree. But the single highest-weight source, the lab, is also the one showing that the advantage narrows for large meals. One strong source rightly outweighs a crowd of weak ones, so the honest reading follows the lab and treats the manufacturer’s headline number as the weakest evidence in the room, not the loudest-and-therefore-truest.
Let’s Recap!
Vote-counting decides a question by how many sources back each side, and it fails because sources are not equal votes.
Evidence weight is how much a source should count, judged together by method quality, relevance, and independence.
Relevance rewards evidence that falls inside your question’s boundaries, not evidence that is merely true in general.
Independent sources that agree are strong, while several sources echoing one origin are really just one source repeated.
Weighing is reasoned judgment you can defend out loud, never a score you compute and total.
You can now tell a heavyweight source from a lightweight one. But your strongest source often speaks in numbers while others speak in words, and the next chapter shows you how to reason across both, so a figure is neither ignored nor trusted simply because it looks precise.
