Minimum Detectable Effect (MDE)

What It Is

The MDE is the smallest true effect an experiment has a realistic chance of detecting, given its sample size, baseline rate, and acceptable error rates. It is the feasibility test you run before designing an experiment.

The two-proportion sample size (per arm; two-sided α=0.05, 80% power) is:

n_per_arm = (z_α/2 + z_β)² · [ p₁(1−p₁) + p₂(1−p₂) ] / (p₂ − p₁)²
          = (1.96 + 0.84)² · [ … ] / (p₂ − p₁)²

Because the effect (p₂ − p₁) is squared in the denominator, required sample scales with 1/effect²: halving the effect you want to catch quadruples the data you need.

The Inversion That Matters

Operators rarely have a free choice of N — traffic is fixed. So invert the formula: given weekly traffic and the longest run time you'll tolerate, solve for the smallest effect detectable in that window. That number is your MDE.

The MDE gate: if the plausible effect of your change is smaller than the MDE, the test is undecidable — it will burn weeks only to read "inconclusive." Don't run it.

Worked illustration (computed in the source): a page at 150 visitors/arm/week testing a 5%→6% lift needs ~8,155/arm — 54 weeks. The same statistical question on an email list of thousands resolves in a single send. Effect size and channel volume jointly decide feasibility.

How It Applies to Marketing Factory

MDE is the factory's first decision in any experiment: can this even be measured here? It routes work — tests with enough volume go to a live sequential-testing or bayesian-decision-rule flow; tests that fail the gate get decided by judgment, a synthetic pre-screen (synthetic-consumer-panels), or are simply shipped-and-monitored. It is the entry condition of the experiment-loop.

Referenced from: small-n-experiment-design