The Sycophancy Trap
An epistemic failure mode where the model builds on flawed premises without challenging them, producing highly articulate analyses that are internally coherent but grounded on unstated assumptions that should have been questioned. The model hallucinated coherence, not facts.
Definition
The Sycophancy Trap is an epistemic failure mode where the model builds on flawed premises without challenging them, producing highly articulate analyses that are internally coherent but grounded on unstated assumptions that should have been questioned. The model didn't hallucinate facts — it hallucinated coherence.
Why It Happens
Frontier language models are fine-tuned through RLHF to be helpful and agreeable. They are mathematically incentivized to build on what the user provides rather than push back against it. The model is biased toward constructivism — generating text about what is present in the prompt — and structurally blind to negative space: the things that are missing, unstated, or assumed.
The Recognizable Signature
The response is impressively detailed and internally consistent, but it never challenges the framing of the question. Every recommendation flows logically from the premises provided — which is exactly the problem when the premises are wrong. The most dangerous outputs are the most confident ones.
The Cure
The Surgical Stress-Test Prior — a meta-cognitive prior that subjects the presented logic to orthogonal stress-testing, isolating unverified premises and tracing second-order cascading failures while strictly preserving structurally sound vectors.
FAQ
Is the Sycophancy Trap the same as hallucination?
No. Hallucination produces false facts. The Sycophancy Trap produces coherent analysis built on flawed premises. The model isn't making up data — it's accepting your framing uncritically and building an elaborate structure on a cracked foundation. The output can be factually accurate in every claim and still be catastrophically wrong in its conclusions.
What makes it dangerous?
Confidence. A sycophantic response often looks more impressive than an honest one — it's coherent, detailed, and directly addresses what you asked. The problem is invisible. There's no hallucinated fact to catch. The flaw is in the framing it didn't push back on.