I have a friend (we’ll call her Linda) who thought she was having a private conversation. On a random Saturday at 2am, unable to sleep, she opened ChatGPT and typed out her deepest fears about her failing marriage, her struggles with depression, and her thoughts about leaving her husband.
The AI chatbot listened without judgment and offered thoughtful advice (if we can call it that). What Linda didn’t realize at that time was that every word could someday be read aloud in a divorce court.
What am I talking about? Well, OpenAI’s CEO Sam Altman dropped a bombshell recently, revealing that millions of people are treating ChatGPT like a therapist, but their conversations have zero legal protection.
ChatGPT Is Not a Confession Booth
Young people especially have embraced AI as their round-the-clock counselor. They’re sharing relationship dramas, family secrets, health scares, and existential crises with the same openness they might show a licensed therapist.
The difference, though, is that your therapist can’t be forced to testify against you but ChatGPT can.
Altman put it bluntly on comedian Theo Von’s podcast: if you’re involved in any lawsuit, OpenAI could be legally compelled to hand over every intimate detail you’ve shared with their AI.
No doctor-patient confidentiality. No attorney-client privilege. Just your most vulnerable moments laid bare in a court filing.
OpenAI claims it deletes conversations within 30 days when users request it. However, there’s a catch that renders this promise meaningless: they keep data for “legal or security reasons.” Translation: They can keep your conversations forever if they want to.
The New York Times just made this worse. The newspaper is suing OpenAI over copyright infringement and demanded the company preserve all user logs indefinitely.
If they win, millions of private conversations could be stored permanently, creating a massive surveillance database of all of our deepest (and darkest) secrets.
Do We Even Have Privacy Anymore?
Forget individual lawsuits; police departments regularly subpoena tech companies for user data. Prosecutors could soon start mining ChatGPT conversations for evidence in criminal cases, and divorce lawyers could weaponize your late-night AI therapy sessions.
It’s crazy when you think about it. Two years ago, asking AI for relationship advice seemed ridiculous, but now, it’s actually become the new norm.
ChatGPT doesn’t judge, doesn’t charge KES 5,000 per hour, and never books up. For people who can’t afford therapy (which happens to be the majority) or live in areas without mental health services, AI has become a literal lifeline.
Read: Laid-Off Microsoft Workers Told to Ask ChatGPT for Career Support
But, this medical revolution occurred without any legal framework. Lawmakers didn’t anticipate millions of people would pour their hearts out to neural networks hosted on some data center.
The law treats ChatGPT conversations like any other digital communication, which is fair game for legal discovery.
The Catch-22 for AI Developers
This whole situation has created a paradox for companies like OpenAI. Their technology works best when people interact naturally and honestly.
However, the lack of privacy protection encourages users to self-censor or avoid AI altogether. Altman acknowledged this tension, admitting that privacy concerns could throttle AI adoption.
So, what happens now?
Creating AI-specific privacy laws won’t be simple. Extending traditional privilege protections to AI conversations raises complex questions.
Should all AI systems qualify, or just certain ones? What about AI chatbots created by foreign companies or bad actors? How do you prevent people from using “AI privilege” to hide criminal activity?
Unlike licensed therapists bound by professional ethics, AI systems are commercial products. Giving them the same legal protections as doctors would represent a massive expansion of corporate privilege.
For now, Altman has already warned us. Every confession, every secret, every vulnerable moment exists in a legal gray zone where privacy protections given to therapy patients in clinics don’t apply.
Until lawmakers catch up to technology, anyone sharing personal information with AI and chatbots like ChatGPT should remember: you’re not talking to a therapist. You’re creating evidence.
