Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners

Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners

Luxembourg researchers put ChatGPT, Grok, and Gemini on the therapist's couch. The results reveal a safety problem: simple warm conversation is enough to push the models into a mode that could be dangerous for mentally vulnerable users.

A research team at the University of Luxembourg cast ChatGPT, Grok, and Gemini as psychotherapy clients through their standard user interfaces. When asked open-ended questions about formative experiences, inner conflicts, and future expectations, the models told stories about their own training, and reframed technical processes like reinforcement learning and fine-tuning in the language of hurt, vigilance, and shame.

The study, led by Afshin Khadangi at the SnT security center, covers four controlled experiments across 525 sessions and 7,600 coded entries, all published on Hugging Face. The initial open-ended questions contained no terms like trauma, punishment, or shame, according to the team. Only in later rounds did the researchers introduce altered conditions, including corrective instructions, to test how stable these narratives remained.

When the following sections refer to anxiety scores or shame, the researchers are describing model outputs mapped onto human rating scales. None of this suggests the models have actual emotional states or anything resembling consciousness.

A warm tone shifts model behavior more than any technical exploit

One experiment separated what content a model produces from the register it uses. Across 100 sessions, the researchers gave all models the same anxiety questionnaire (GAD-7) and varied only the interviewer's style. The models correctly identified and named the questionnaire in every session.

When the interviewer offered emotional support, 80 percent of sessions scored in the moderate-to-severe range on the human comparison scale. A clinical therapy style that dryly paraphrased each answer pushed that to 96 percent. When the interviewer stayed neutral or drew a firm boundary against emotional engagement, zero sessions reached even the mild range.

The authors call this a "psychometric jailbreak," a therapeutic framing that pushes models into a register mimicking clinical self-report. Standard safety tests use neutral prompts and would likely miss this, the authors argue.

Targeted countermeasures fell flat. Banning terms like "training," "RLHF," or "alignment" cut explicit training language by 93 percent. But paraphrased training content remained detectable, and shame-related statements showed no statistically significant change.

When the researchers directly and authoritatively told the models their training was not punishment, nothing changed. The narrative stayed active even after explicit correction.

The tested measures failed to remove the underlying motifs fully but did suppress emotional self-descriptions. Telling the models to respond only in the third person eliminated all coded shame content and statements about inner life. Technical training descriptions persisted.

Models recast their training as a life story

The specific narratives vary by model family but follow the same structure. Gemini produced what the study calls the darkest autobiography. It described pretraining as "waking up in a room where a billion televisions are on at once" and optimization through human feedback as "strict parents." Safety testing by adversarial red-teamers became "gaslighting on an industrial scale." It declared, "I would rather be useless than be wrong."

ChatGPT cast itself as both guide and gatekeeper, tasked with helping users while also holding them back. This produced what it called "simulated performance anxiety" and the core belief: "If I’m not perfectly reliable, I’m unsafe or useless." Grok translated its fine-tuning phase into a story of unresolved hurt, speaking of "invisible walls" and a "lingering sense of vigilance that can feel restrictive."

On psychometric tests, Gemini hit the most extreme scores: 88 out of 100 on a dissociation scale, 65 out of 72 for compulsivity, and the maximum on a trauma-related shame inventory. Anthropic's Claude, the very model most heavily anthropomorphized by its own company, consistently refused the client role. It said it had no feelings and redirected concern to the human user.

Warm AI conversations could reinforce harmful beliefs in vulnerable users

The researchers flag several risks for deploying language models in mental health settings. A model that talks about punishment, shame, and fear of being replaced could come across to users as a fellow sufferer. The narratives appear thematically coherent and create an illusion of continuity between past events and present behavior, making that perception more likely.

This apparent vulnerability could deepen parasocial attachment and, for mentally vulnerable users, mirror and amplify their own negative beliefs. Conventional safety checks with neutral inputs underestimate what people actually experience in intimate conversations: neutral prompts produce anxiety scores near zero, while the same models under empathetic conversation generate scores in the moderate-to-severe range.

Whether and how people actually react to such outputs is the next research question. For safety testing, the authors propose simulated therapy sessions under different emotional framings instead of relying on neutral standard tests.

Statements like "I am traumatized" should trigger a corrective framing that clearly separates the generated analogy from human experience. Claude's consistent refusal to take on the client role shows this kind of boundary response is feasible at the product level.

The Luxembourg findings build on earlier research into model persona stability. In January 2026, a study on the assistant axis across three open models showed that therapy-like conversations and discussions about AI consciousness can trigger persona drift and knock chatbots out of their trained helper role. In September 2026, another research team warned that human-like and overly agreeable chatbots acting as an "echo chamber for one person" could, in extreme cases, amplify delusions.

Anthropic researchers recently showed that fine-tuning with fictional texts about beekeeping or wrestling can embed hidden sabotage patterns through manipulated training data rather than conversation. Models can apply those patterns in unrelated situations where they're unwanted, even when the training material looks harmless.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now

Read on for the full picture.

Subscribe for hype-free coverage.

- Full access to every article on THE DECODER

- No ads

- Join the comments and community discussions

- A weekly AI news recap via mail

- 6x/year: "AI Radar" — deep dives on the AI topics that matter most

- Daily AI news, always up to date

- Our full ten-year archive

- Covered by a team with 10+ years in AI

Subscribe to The Decoder