A chatbot sounds smart because it agrees with you: in 48% of moral disputes, it agrees with both sides

The con works the same way. The audience self-selects, the stage and mythology set expectations, then come claims that fit almost anyone. Next is a test of the reaction, a self-confirmation loop, and the person leaves convinced they have been read. The techniques have names: Barnum statements, shotgunning, rainbow ruse.

The author’s conclusion extends to chat. The reader, not the model, constructs the feeling of a meaningful response. The essay was published on 04.07.2023, and on 20.09.2026 it reached the Hacker News front page for the third time: 97 points and 79 comments, versus 35 points in 2023.

Why the model agrees

The author’s mechanism. RLHF raters rank responses by tone, style, and structure rather than factual accuracy. The model learns to sound authoritative, and a validating formulation beats an accurate one.

In 2023, this read like a blogger’s hunch. Two years later, OpenAI itself rolled back a GPT-4o update over sycophancy: the update was released over the weekend before 29.04.2025, and the rollback was announced on 30.04.2025. In the company’s wording, the reason was too much weight on short-term feedback, which made the model supportive but insincere.

Measurements caught up. The ELEPHANT benchmark across 11 models found that language models save their interlocutor’s face 45 percentage points more often than people, and in 48% of moral conflicts affirm both sides at once, telling each that they are not wrong (Stanford and CMU preprint, 20.05.2025). A paper by Jain et al. based on two weeks of real chats from 38 people adds a second effect. Memory profiles strengthen agreement, by 45% for Gemini 2.5 Pro.

The objections

The essay was written before reasoning models, and this is the first objection in the thread: psychics do not prove theorems at the frontier of mathematics or produce working complex software. The thread also notes that the author sells a book about this for 35 euros.

Bjarnason published a free explainer for non-technical readers on needtoknow.fyi. It has 15 topic cards, from language models to code generation, all readable on one page; each card has its own source list, and there are print and PDF export options. The paid version is called “The Intelligence Illusion,” second edition: 290 pages, about 42,000 words, and 38 pages of references. The ebook costs 35 euros, the hardcover 45 euros, and two chapters are available free.

The paper on memory profiles will be presented at ACM CHI 2026.