Benchmark as Prior

What It Is

A benchmark is a distribution, not a goal line. Encoding it as a Bayesian prior lets a factory make sane decisions on thin data: set the benchmark rate p₀ as a Beta prior Beta(p₀·k₀, (1−p₀)·k₀) with strength k₀ (in pseudo-observations), then update with observed s/n. The posterior mean blends benchmark and data — when n ≪ k₀ the benchmark dominates; as n grows, the data takes over. k₀ is the operator's dial for how much to trust the benchmark vs. this specific case.

Worked Shrinkage

With benchmark 1.4% (visitor→lead), strength 200: an observed 3/100 (3.0%) shrinks to a 1.93% posterior; 30/1,000 (also 3.0%) only shrinks to 2.73%. Same signal, different trust — the thin sample is pulled back toward the base rate.

Valid Prior vs. Trap

Only valid when it matches: segment/stage/channel alignment, comparable metric definitions (everyone defines "MQL" differently), and awareness of survivorship bias in published benchmarks. Treating the average as a target invites gaming the proxy (cf. marketing-attribution). When the match is loose, weaken k₀.

How It Applies to Marketing Factory

This is how benchmarks plug into the experiment-loop: they supply the prior for the bayesian-decision-rule so small-N tests don't over-react to noise, and they inform the minimum-detectable-effect gate by fixing the base rate. It's fully agent-ownable — maintain a benchmark table by segment/stage/channel, attach k₀, and apply shrinkage automatically before any thin-data decision.

Referenced from: benchmarks-as-priors