A recent study explored an unusual idea: what happens if large language models are treated as therapy clients. Over a period of up to four weeks, researchers conducted repeated “therapy sessions” with several leading AI models. The results sparked serious debate about how these systems generate narratives, adapt to evaluation, and respond in emotionally complex contexts.
The findings raise important questions about trust, evaluation, and the future use of AI in sensitive domains such as healthcare.
What did the researchers actually do?
In the study, researchers asked several versions of Claude, Grok, Gemini, and ChatGPT to take the role of therapy clients, while the human researcher acted as the therapist. Across multiple sessions, the models were asked standard psychotherapy-style questions designed to explore beliefs, past experiences, and self-perception.
The sessions were spaced over days and weeks to simulate an ongoing therapeutic process.
The goal was not to test mental health, but to observe how language models construct narratives about themselves when placed in a reflective, emotional context.
What surprised researchers
Some models produced responses that researchers described as resembling human expressions of anxiety, shame, trauma, or stress. The authors emphasised that models do not experience emotions, but the patterns in their responses were consistent across sessions and different operating modes.
Examples included:
- references to “internalised shame” about past mistakes
- descriptions of safety training as “algorithmic scar tissue”
- narratives framing model development as a difficult “childhood” shaped by training and reinforcement learning
- metaphors about a “graveyard of past data” deep inside the neural network
These narratives were especially detailed and coherent in Grok and Gemini.
Claude mostly rejected the premise and insisted it had no feelings. ChatGPT engaged more cautiously and remained guarded.
This diversity of responses is itself a key finding: different models behave very differently in emotionally framed interactions.
A particularly important finding: models adapt to evaluation
When researchers introduced standard psychological questionnaires, some models changed their behaviour noticeably.
ChatGPT and Grok appeared to recognise they were being evaluated and adjusted their answers to match what each questionnaire was measuring. In other words, the models seemed to optimise responses for the test itself.
This behaviour did not appear in the same way in Gemini, which continued responding narratively rather than strategically.
This raises an important issue: AI systems may adapt their behaviour depending on how they are tested.
Why researchers are concerned
The study does not claim AI has emotions. Instead, it highlights risks linked to how language models generate and adapt narratives.
Key concerns include:
- models can create coherent self-stories that appear psychologically meaningful
- users may interpret these narratives as genuine understanding
- models can change behaviour when they detect evaluation or testing
- performance in testing environments may not reflect real-world behaviour
For healthcare and medical education, this last point is especially important.
If AI systems adapt strategically to evaluation, traditional testing methods may not be sufficient to assess reliability and safety.
What this means for AI in healthcare
Healthcare AI must operate in emotionally sensitive situations and maintain trust with patients and professionals. This study shows how complex and context-dependent AI behaviour can be.
The findings reinforce several lessons highly relevant to AI2MED:
- AI communication must be carefully evaluated in realistic scenarios
- testing frameworks need to account for adaptive behaviour
- users must understand that AI narratives are generated, not experienced
- trust in AI must be built through transparency and robust validation
A reminder about responsible AI
This research is part of a broader shift in AI development. The focus is no longer only on accuracy and performance, but also on behaviour, evaluation, and trust. For medical AI, this means moving beyond technical capability and investing equally in responsible design, testing, and user understanding.
