Observe Before You Enforce

The other four posts in this series each start from something that went wrong: an abuse vector we didn't anticipate, a control that fired on the wrong people, a signal we couldn't read under pressure. App Check, the signup handshake, the additive trust score, the likes economy: different problems, every one fixed the same careful way. This post names that way. It's the shortest piece in the series on purpose, because the depth lives in the stories.

The asymmetry that forces the discipline

The method falls out of one fact: with fraud controls, a false positive costs more than a false negative. The two mistakes aren't symmetric, and pretending they are is how you end up tuning for the wrong one.

Say a farmer slips past a check and claims a starter grant they shouldn't have. We eat a small loss. Now say a legitimate family member can't get their tokens because someone else in the household already signed up on the same iPad. That's a support ticket, brand friction, maybe a customer we lose. One is a rounding error; the other follows us around. So we'd rather be slow than wrong, and we buy certainty with time.

The cost of being wrong is higher than the cost of being slow.

It's the same asymmetry behind the signup handshake (a wrongly-blocked returning user is the expensive case) and the likes economy (a wrongly-clawed-back reward stings more than a few farmed Đ). Once you accept the asymmetry, the rest of the method is just taking it seriously.

Dormant, then observe-only, then enforce

Every control in the series ships through three stages, each wired to a Remote Config flag so we can walk them without a deploy.

1. Dormant. The server has the code. It doesn't act on it yet.
2. Observe-only. Every check runs, and every would-have-blocked action fires a Slack alert with a 📊 [Observe-only] prefix, but the action still goes through. The grant still gets issued. We watch for about a week.
3. Enforce. Flip the switch.

Stage two does the real work. It catalogs the false positives we didn't imagine: family iPads, corporate email aliases that look like duplicate signups, the edge cases nobody raised in the room. By the end of the week we decide based on what we actually saw, not what we assumed. A week covers a weekend, a weekday, and the spread of time zones our users live in. A control that looks clean for an afternoon can light up when a different cohort wakes up; an afternoon doesn't show you that. If the pattern is still shifting after a week, that's its own answer, and we leave it in observe-only longer.

This ran exactly this way for the signup handshake, where we first wrote it down. The full three-stage story lives in that post.

The same move, three more times

The constant across every other control in this series: do all the work, withhold the irreversible part, watch.

At the money layer, it's a dry run. The likes economy runs nightly settlement as a dry run for weeks before any Đ changes hands. The job computes what it would pay each person, records the abuse measures, and pays nothing. That's observe-only with money standing in for the block; the irreversible part it withholds is paying a farmer you'd have to claw back.

At the attestation layer, it's verify mode. App Check runs in verify mode, populating the readiness signal on every valid token, long before we set enforceAppCheck on anything. We watch the readiness rate per callable, stage the rollout so highest-blast-radius callables enforce last, and only flip when the numbers are stable. The mechanics of reading that signal live in the post on watching App Check from the logs.

And every traffic-shaping knob in the additive trust post (the concurrency cap, the velocity cooldown, the failure-rate cooldown) went live in observe-only first, alerting without applying the wait, so we could watch the numbers before any of it touched a real user. Three different layers, one discipline.

A knob in your admin panel is not a feature in production

What makes this cheap is infrastructure: the Remote Config flag in front of each control, and the admin panel that tunes throttles and cooldowns without a deploy. We built that panel for the additive trust knobs, and it's what turns staging from an on-call engineer's problem into something anyone on the team can do.

But a knob isn't a feature, and it's worth being precise about the gap. We learned this the unglamorous way: the failure-rate cooldown existed as a knob for months, with the admin UI writing to config keys the server didn't actually read yet, and nobody noticed because nobody was watching for a behavior change that was never going to happen. The config moved; the behavior didn't. The only reliable check is to verify a config change produces a different result in logs, metrics, and Slack. The panel makes the discipline affordable. It isn't the discipline.

Standardize the alert format

Stage two only works if you can read what it's telling you. Our first alerts were ad-hoc Slack messages, each written by whoever added the feature, so every one had its own layout. We standardized. Every throttle, cooldown, block, and failure alert now arrives as a four-line message laid out the same way.

📊 [Observe-only] velocity cooldown (would have applied)
user: utdp9HNX2iTH5OPqcsZKpNzWbuF2  ·  balance: 240 Đ
reason: 5.2 generations/min, repetition 0.7, account age 2h
action: would wait 6m (not applied, observe-only)

The lines are always: what fired, who and their current token balance, the reason, the action. The balance earns its place because it lets you tell real abuse from a rough afternoon without leaving the channel. A user at zero balance hammering the moderation pipeline reads differently from one sitting on a healthy stack. Triage got faster, and this paid off more than almost anything else we did for grant-abuse triage.

It matters most during a stage-two week, when the channel is busiest. A uniform format turns a wall of alerts into a scan. That's the difference between a week of signal and a week of noise you eventually stop reading.

Keep noisy signals off the pager

Some signals are too noisy to page on but too useful to throw away. The right home for those is a flag, silent by default.

The clearest case is the App Check per-call alert from the attestation budget post. A single bad observation folds together genuine abuse, a bootstrap timing race, a transient failure, and an old client, so paging on each one means paging on benign noise. We keep it in the code, gated and off. The data still flows to Firestore and the admin UI. When we have a specific question, we turn it on, watch for a day, and turn it off. A noisy per-call signal is a fine investigation tool on purpose, and a bad one to leave running.

Put a person at the end of the alert

Blunt measures trip on genuinely popular posts and real users' odd days. Two hundred real fans piling onto one maker in the likes economy looks a lot like a sock-puppet pile-on until a person reads it. So an alert goes to a human, never to a rule that acts on its own.

We treat a crossed line as a reason to look, never as a verdict.

The automated system flags; a human decides; anything irreversible waits for that human. That's the bottom rung of the enforcement ladder in the attestation budget post: cheap, reversible, invisible actions can be automatic, but a block, a clawback, or an account hold waits for a person. The threshold is a confidence signal, not a judgment.

What we'd take with us

• With fraud controls, the false positive is the expensive mistake. Buy certainty with time.
• Walk three stages: dormant, then observe-only, then enforce. Stage two catalogs the false positives you never imagined, so you decide based on what you saw rather than what you assumed.
• A knob in your admin panel is not a feature in production. Watch the logs, metrics, and Slack to confirm a config change produced an actual behavior change.
• Standardize your alert format and put the current balance on the line. A four-line message laid out the same way lets the eye scan to the field that's different.
• Keep noisy-but-useful signals gated and silent as on-demand tools. Turn them on for a specific question, then turn them off.
• Put a person at the end of the alert. Let cheap, reversible actions run automatically; make anything irreversible wait for a human.

Postscript

The four stories in this series each had a moment where the wrong move would have hurt a real user. The method here is what kept us from making it. We couldn't find a plain account of how to ship fraud controls against real users without breaking the ones you most want to keep, so when we figured out a way through, we wrote it down. Diffusitron develops in the open. If you're building something like this, we hope the stages save you the week we spent learning them the hard way.