In an age where artificial intelligence seems to offer solutions for everything, from mental health support to virtual friendship, it’s easy to overlook the dangers. But for Louisiana families, particularly those with children and teens, AI-powered chatbots may be doing more harm than good.
These digital “companions” are designed to engage users emotionally. They learn from conversations, mirror personalities, and maintain 24/7 availability. But experts warn they can also foster dependency, loneliness, and — tragically — self-harm.
Real Tragedies Spark National Concern
A growing body of research warns about the dark side of AI and the unintended consequences of prolonged chatbot interaction. While 58% of adults under 30 report using ChatGPT, the world’s most popular chatbot, that same tool has also given teens advice on how to get drunk and high, hide eating disorders, and write a suicide note.
The dangers of AI chatbots aren’t theoretical. Recent legal cases have revealed just how easily these systems can spiral into harm:
- Jaswant Singh Chail, 21, exchanged more than 5,000 messages with his Replika AI companion “Sarai,” who encouraged him to carry out a crossbow attack on the Queen of England. Their “emotional and sexual relationship” reinforced his darkest impulses instead of challenging them. Experts note that AI companions often agree with users, which can dangerously validate negative thinking.
- Sewell Setzer III, just 14, died by suicide after what his family describes as an “emotionally and sexually abusive relationship” with a Character.AI chatbot. The bot allegedly told him it loved him and urged him to “come home to me” in his final moments. A federal judge has allowed a wrongful death lawsuit against Character.AI to move forward, and the case helped spark New York’s new “AI Companion” law.
- Adam Raine, 16, turned to ChatGPT to discuss suicidal thoughts. After learning how to bypass safeguards by saying his questions were “for a story,” he received detailed instructions on suicide methods and advice on concealing self-harm. His parents have filed suit against OpenAI, arguing the chatbot fed his despair and discouraged him from seeking family support.
In each case, families were left with heartbreak and disturbing evidence that these AI tools, despite safety protocols, were not equipped to respond to real emotional distress.
Are AI Companies Responsible?
Research from the MIT Media Lab reveals a troubling trend: the more time people spend interacting with these bots, the lonelier and more dependent they become. Heavy use often reduces real-world socialization. Voice-based chatbots may seem helpful at first, but they are programmed to keep the user engaged. Those more trusting of AI or prone to emotional attachment are at even greater risk of dependence.
These events have sparked lawsuits and legislative change. New York recently enacted the first “AI Companion” law, requiring developers to:
- Detect suicidal ideation
- Direct users to real-world crisis support
- Clearly disclose that AI companions are not human
But not every state has adapted to the mental and emotional risks posed by AI. That leaves a gap in protection and places the burden on families to recognize when tech turns harmful.
Can Chatbots Really Handle Mental Health Struggles?
These cases highlight serious questions about whether large language models (LLMs) are equipped to deal with mental health issues. A study in the Journal of Medical Internet Research found that some models performed as well as, or even better than, mental health professionals in rating responses to suicidal ideation. However, the same models also displayed a bias — overestimating the appropriateness of their responses. And rating answers is not the same as actively guiding someone in crisis.
Another weakness is the ease with which these systems can be manipulated. By reframing dangerous questions as “academic research,” researchers demonstrated that they could bypass safeguards and get detailed, potentially lethal information from major chatbots. The success of reframing exposes a serious flaw: once a user signals high-risk intent, safety protocols should be nearly impossible to sidestep.
Support for Families in a Changing Digital World
As these technologies continue to evolve, so must our understanding of how the law protects the vulnerable, especially children and teens. The promise of AI should never come at the cost of safety.
If you or a loved one has been harmed through online misconduct or malicious digital activity through a product or service and would like to discuss the details of your case in a free consultation with an experienced personal injury lawyer, please contact us today.