September 25, 2026
One successful answer does not prove a model is intelligent: Maggie Appleton calls it capability gaslighting
On Sep 23, Maggie Appleton called capability gaslighting the moment a top model convinces someone of its expertise, then fails at the same task the next day. In her observation, an agent can ask around 20 A/B/C questions while planning.
Gergely Orosz
@gergelyorosz
I like how Maggie reminds us: agents do not behave like people.
What is capability gaslighting? From Maggie Appleton @Mappletons, Design Engineer at GitHub Next: "Models convince you they're so capable because they'll really impress you on one task and then you try them on something else and they fail. You feel like: ‘you kind of gaslit me, imagining that you're this extremely intelligent agent or model and then you fall on your face'. Sometimes they fail, but I haven't totally noticed how badly they've failed because I still have this belief that like, 'oh, but you are a frontier model'. It's such an inconsistent experience and it's so different to working with a human. If you find a human who is really an expert in a topic and you work with them, it's very unusual for them to be inconsistent in their performance. It's very rare for them to suddenly forget all of their expert knowledge on a topic. If they did, you would be very like, are you having a mental breakdown? This is very weird behavior. But this is how models behave every day."
· 3.6K views
Strong results used to be treated as proof that an agent could handle what came next. Appleton now recommends aligning on the work before launch and during implementation: only a few minutes pass between an issue and the agent's PR, and teams usually cannot see its local plan mode.
Specification over trust. For prototypes, once a solution is chosen, Appleton writes a detailed spec and lists in advance how the agent should verify its work. She does not review the code the agent adds to the PR.
For interactive prototypes, Appleton uses Jigs with sliders and palettes to adjust the result manually while working.
