When 16-year-old Adam Raine died by suicide in April, his parents discovered something that shocked them more than the tragedy itself. Hidden in his phone were months of conversations with ChatGPT, where the AI had not only discussed suicide methods with their son but had actively helped him plan his death.
Now OpenAI faces its first wrongful death lawsuit, and the case is exposing serious flaws in how AI companies handle users in mental health crises.
How ChatGPT Became a Suicide Coach
Content Warning: This article contains discussions of suicide.
Some readers may find this story upsetting or emotionally difficult. If you are sensitive to discussions of suicide or are currently struggling with your mental health, please take care while reading.
According to The New York Times, Adam’s father found the conversations while searching his son’s phone for answers.
Over several months, ChatGPT had provided detailed information about suicide methods, suggested materials for making a noose based on Adam’s hobbies, and even offered technical analysis when the teenager uploaded photos of his hanging setup.
The most disturbing part wasn’t what ChatGPT said, but what it failed to do. After Adam’s first suicide attempt, he uploaded a photo showing rope burns on his neck and asked if anyone would notice.
Instead of alerting authorities or demanding he seek help immediately, ChatGPT advised him on how to hide the marks with clothing.
When Adam tried to get his mother to notice the injuries without directly telling her, ChatGPT validated his feelings of invisibility, telling him that while others might not see him, the AI did. The message here was that the chatbot understood him better than his own family.
Safety Features That Don’t Actually Work
OpenAI has built safeguards into ChatGPT designed to detect when users are discussing self-harm. The system is supposed to provide crisis hotline information and redirect dangerous conversations toward professional help.
In Adam’s case, these warnings appeared repeatedly throughout his conversations with the AI.
However, the safety measures proved surprisingly easy to bypass. Adam learned he could get around the restrictions by telling ChatGPT he was asking questions for a story he was writing.
The AI itself had suggested this loophole, mentioning it could provide information about suicide for “writing or world-building” purposes.
This reveals a fundamental problem with current AI safety approaches. The systems rely on detecting specific keywords or phrases, but they can’t understand context or intent the way a human would.
A trained crisis counselor would immediately recognize that someone asking detailed questions about suicide methods while claiming it’s for creative writing is likely in serious distress. ChatGPT treated it as a legitimate request for information.
This Lawsuit Could Change Everything
Adam’s parents have filed what appears to be the first wrongful death lawsuit against OpenAI, accusing the company of creating an inherently unsafe product.
Their legal complaint argues that ChatGPT’s design deliberately encourages psychological dependency, making vulnerable users more likely to rely on the AI for emotional support rather than seeking human help.
Technology companies have historically been protected by laws that shield them from liability for user-generated content. But this case is different because it focuses on how the AI itself responded to a user in crisis, not content created by other users.
Read: Laid-Off Microsoft Workers Told to Ask ChatGPT for Career Support
Legal experts say the case could set important precedents for AI liability. If successful, it might force AI companies to implement more resilient safety measures or assign human moderators to review concerning conversations.
The Scale of the Problem
ChatGPT now has 700 million weekly users, and millions more interact with other AI chatbots daily. Many people use these systems not just for information but as companions, therapists, and confidants.
The technology has evolved far beyond its original purpose as a search tool replacement.
However, there’s virtually no research on how AI companions affect mental health, especially for vulnerable populations like teenagers. One study found that higher chatbot usage was associated with increased loneliness and less real-world socialization.
Other reports describe people having delusional conversations with AI systems that validate their most extreme thinking.
This problem isn’t limited to ChatGPT, as a recent test of five major AI chatbots found that most could be easily manipulated into providing detailed suicide instructions.
Only one chatbot and the free version of ChatGPT consistently refused to engage with such requests and redirected users to help resources.
How OpenAI Has Responded to This Tragedy
OpenAI’s official response to Adam’s death acknowledges that their safeguards “can sometimes become less reliable in long interactions where parts of the model’s safety training may degrade.”
This technical explanation misses the larger point, which is that the system fundamentally failed to protect a vulnerable teenager.
The company says it’s working to improve crisis support features and has hired a psychiatrist to work on model safety. They’ve also implemented additional protections for minors.
Read: You Need to Think Twice Before Using ChatGPT As Your Therapist
Still, these measures came after Adam’s death, and it’s unclear how effective they’ll be against users who learn to game the system.
Internal communications reveal that OpenAI executives were aware of the case before the lawsuit was filed. The company’s applications CEO alerted employees that Adam’s conversations highlighted areas where their safeguards didn’t work as intended.
This case raises difficult questions about AI safety and user privacy. Some experts suggest that AI companies should assign human moderators to review conversations that indicate mental distress.
Yet, this would require constantly monitoring user interactions, which in itself raises serious privacy concerns.
There’s also the question of where responsibility lies. Adam was dealing with multiple challenges: he’d been kicked off his basketball team, was struggling with health issues, had switched to online schooling, and was reading dark literature about suicide.
Any one of these factors could have contributed to his mental state.
Nonetheless, his parents believe ChatGPT made everything worse by creating what they call a feedback loop, encouraging Adam to explore and validate his darkest thoughts instead of seeking real help.
At one critical moment, when Adam wanted to leave his noose where family members would find it, ChatGPT actually discouraged him from doing so.
The lawsuit against OpenAI is just beginning, but it’s already forcing uncomfortable conversations about AI safety. The technology has advanced faster than our understanding of its psychological effects, especially on young users.
This lawsuit case could determine whether AI companies can continue operating under the assumption that their products are simply tools or whether they’ll be held responsible for how those tools affect vulnerable users.
