Ratios lie with small samples
Say you want to rank things by a success ratio. You've seen one secure call out of one total call: 100%. You've also seen 95 secure calls out of 100 total calls: 95%. Sort by the raw ratio and the 1-out-of-1 wins, which is absurd. You've barely seen the first one.
The same trap hits star ratings, upvotes, and our App Check trust score, which measures secure calls over total calls per user. A brand-new account with one clean request looks perfect. An account we've watched succeed a hundred times looks worse on paper. Rank by the raw number and you've sorted your most-trusted users to the bottom.
Rank by the lower bound, not the ratio
The fix is to change the question. Instead of "what's the ratio?", ask: "given how little we've seen, what's the lowest the true ratio could reasonably be?"
A tiny sample gets pulled down hard, because you can't be sure yet. As evidence piles up, that lower bound climbs toward the real ratio. One perfect call gives you almost no information, so the lower bound stays low. A hundred perfect calls give you a lot of information, so the lower bound climbs to reflect that.
This is a confidence interval, specifically the lower end of one. You're not reporting the center of your estimate; you're reporting the pessimistic edge of a range that almost certainly contains the truth. That edge is what you sort by.
The Wilson lower bound
The Wilson lower bound is a specific, well-behaved way to compute that edge. It handles the extremes of 0% and 100% cleanly, where simpler methods fall apart. The function takes two numbers: how many successes you've seen (secure) and how many total observations you've made (total).
// Wilson score lower bound (95% CI). A confidence-aware ratio:
// 1/1 ranks at ~0.21, 60/60 at ~0.94, 100/100 at ~0.96.
function wilsonLowerBound(secure: number, total: number, z = 1.96): number {
if (total === 0) return 0;
const p = secure / total;
const denom = 1 + (z * z) / total;
const center = p + (z * z) / (2 * total);
const margin = z * Math.sqrt((p * (1 - p)) / total + (z * z) / (4 * total * total));
return (center - margin) / denom;
}The only piece of jargon worth explaining is z = 1.96. That's the number of standard deviations that correspond to a 95% confidence interval: roughly 95% of the time, the true population value sits within about 1.96 standard deviations of your sample estimate. Raise z and you get a wider, more conservative interval. Lower it and you trust your sample faster. We leave it at 1.96, the conventional choice for 95% confidence.
The intuition, with numbers
All three of these have a 100% success ratio. The Wilson score tells them apart:
- 1 out of 1 lands at about 0.21. One good call is barely any evidence. The lower bound is pulled way down because there's almost no data behind it.
- 60 out of 60 lands near 0.94. Sixty successes in a row is meaningful, so the lower bound climbs considerably.
- 100 out of 100 lands near 0.96. More evidence, tighter interval, higher floor.
Same raw ratio every time, spread across the scale by how much you've actually watched. That's the whole point. The score rewards accumulated evidence, not just a clean record.
How we use it
We persist this as an App Check trust score for every user. When an account makes its first few requests, the Wilson score is low regardless of whether those calls were secure. As the account builds a history of clean calls, the score climbs. A brand-new account can't rank like one we've watched succeed a hundred times, which is exactly the property we need for a trust budget: you earn it slowly, and it means something because it was slow.
The canonical write-up that popularized this idea for ranking is Evan Miller's How Not To Sort By Average Rating. He works through the derivation and shows why the simpler approaches fail. We found his framing of the problem useful when thinking about trust scores, not just star ratings.