Domain-Transfer Risk

What It Is

Domain-transfer risk is the gap between the domain a method was validated in and the domain it's deployed in. The SSR authors state validity "depends on the alignment between training data and the survey domain" and on embedding-model choice — an explicit warning that results may not carry over.

For synthetic-consumer-panels specifically:
- Validated domain: purchase intent for personal-care CPG concepts.
- Deployment domain (factory): marketing copy for skeptical, contrarian, HN-flavored developer/founder audiences — plausibly the worst case for synthetic-panel agreeableness/realism.
- No published validation exists for dev-audience marketing as of 2026-06.

Also note the bias isn't simple agreeableness: the paper found LLMs rate less-attractive concepts lower than humans — systematic distortion at both ends of the scale.

How It Applies to Marketing Factory

Domain-transfer risk is why the synthetic-panel gate ships as comparative-use-only and advisory, never as an absolute predictor or a veto. The mitigation is the calibration-loop: measure the panel's real hit rate in your domain before trusting it. The flip side is opportunity — publishing "do synthetic panels predict developer-audience engagement?" is original first-party research nobody has done.

Referenced from: ssr-synthetic-panels