Synthetic Consumer Panels

What It Is

Synthetic consumer panels use LLMs, prompted to impersonate demographically-conditioned personas, to simulate survey respondents — producing a pre-publication test layer ("wind tunnel") for drafts where marketing otherwise has no "tests pass." Validated by PyMC Labs + Colgate-Palmolive (arXiv:2510.08338) across 57 personal-care surveys / 9,300 human responses, reaching ~90% of human test-retest reliability (how consistent real humans are with themselves on retest — not 90% accuracy).

What's Validated vs. Not

  • Validated: realistic response distributions (KS > 0.85) + qualitative reasons, when personas are demographically conditioned (conditioning is load-bearing, not garnish).
  • Naive approach fails: asking an LLM directly for a number regresses to a "safe 3" with unrealistic distributions — semantic-similarity-rating is what fixes this.
  • Not validated: domain transfer beyond CPG purchase intent (see domain-transfer-risk); absolute-score prediction (comparative use only).

How It Applies to Marketing Factory

This is the factory's v1 experiment-loop gate — a pre-publication test for variants that have passed the voice/fact gates (the agent workflow is in agent-workflow-pattern). Use it strictly comparatively (rank variant A vs. B), pair it with a calibration-loop against real post-publication metrics, and treat it as advisory (a tiebreaker, not a veto) until calibrated. Cost is ~$0.12–0.35 per A/B test vs. ~$30–90K for the equivalent human panel.

Referenced from: ssr-synthetic-panels