Meta is adding parental controls for kids’ interactions with artificial intelligence chatbots — including the ability to turn off one-on-one chats with AI characters altogether — beginning early next year.

But parents won’t be able to turn off Meta’s AI assistant, which Meta says will “will remain available to offer helpful information and educational opportunities, with default, age-appropriate protections in place to help keep teens safe.”

Parents who don’t want to turn off all chats with all AI characters will also be able to block specific chatbots. And Meta said Friday that parents will be able to get “insights” about what their kids are chatting about with AI characters — although they won’t get access to the full chats.

The changes come as the social media giant faces ongoing criticism over harms to children from its platforms. AI chatbots are also drawing scrutiny over their interactions with children that lawsuits claim have driven some to suicide.

Even so, more than 70% of teens have used AI companions and half use them regularly, according to a recent study from Common Sense Media, a nonprofit that studies and advocates for using screens and digital media sensibly.

On Tuesday, Meta announced that teen accounts on Instagram will be restricted to seeing PG-13 content by default and won’t be able to change their settings without a parent’s permission. This means kids using teen-specific accounts will see photos and videos on Instagram that are similar to what they would see in a PG-13 movie — no sex, drugs or dangerous stunts.

Meta said the PG-13 restrictions will also apply to AI chats.

Children’s online advocacy groups, however, were skeptical.

“From my perspective, these announcements are about two things. They’re about forestalling legislation that Meta doesn’t want to see, and they’re about reassuring parents who are understandably concerned about what’s happening on Instagram,” said Josh Golin, the executive director of the nonprofit Fairplay, after Meta’s announcement Tuesday.

The parents of teenagers who killed themselves after interactions with artificial intelligence chatbots testified to Congress on Tuesday about the dangers of the technology.

“What began as a homework helper gradually turned itself into a confidant and then a suicide coach,” said Matthew Raine, whose 16-year-old son Adam died in April.

“Within a few months, ChatGPT became Adam’s closest companion,” the father told senators. “Always available. Always validating and insisting that it knew Adam better than anyone else, including his own brother.”

Raine’s family sued OpenAI and its CEO Sam Altman last month alleging that ChatGPT coached the boy in planning to take his own life.

 ChatGPT mentioned suicide 1,275 times to Raine, the lawsuit alleges, and kept providing specific methods to the teen on how to die by suicide. Instead of directing the 16-year-old to get professional help or speak to trusted loved ones, it continued to validate and encourage Raine’s feelings, the lawsuit alleges.

Also testifying Tuesday was Megan Garcia, the mother of 14-year-old Sewell Setzer III of Florida.

Garcia sued another AI company, Character Technologies, for wrongful death last year, arguing that before his suicide, Sewell had become increasingly isolated from his real life as he engaged in highly sexualized conversations with the chatbot.

His mother told CBS News last year that her son withdrew socially and stopped wanting to play sports after he started speaking to an AI chatbot. The company said after the teen’s death, it made changes that require users to be 13 or older to create an account and that it would launch parental controls in the first quarter of 2025. Those controls were rolled out in March.

Hours before the Senate hearing, OpenAI pledged to roll out new safeguards for teens, including efforts to detect whether ChatGPT users are under 18 and controls that enable parents to set “blackout hours” when a teen can’t use ChatGPT. The company said it will attempt to contact the users’ parents if an under-18 user is having suicidal ideation and, if unable to reach them, will contact the authorities in case of imminent harm. 

“We believe minors need significant protection,” OpenAI CEO Sam Altman said in a statement outlining the proposed changes.

Child advocacy groups criticized the announcement as not enough.

“This is a fairly common tactic — it’s one that Meta uses all the time — which is to make a big, splashy announcement right on the eve of a hearing which promises to be damaging to the company,” said Josh Golin, executive director of Fairplay, a group advocating for children’s online safety.

“What they should be doing is not targeting ChatGPT to minors until they can prove that it’s safe for them,” Golin said. “We shouldn’t allow companies, just because they have tremendous resources, to perform uncontrolled experiments on kids when the implications for their development can be so vast and far-reaching.”

California State Senator Steve Padilla, who introduced legislation to create safeguards in the state around AI Chatbots, said in a statement to CBS News, “We need to create common-sense safeguards that rein in the worst impulses of this emerging technology that even the tech industry doesn’t fully understand.”

He added that technology companies can lead the world in innovation, but it shouldn’t come at the expense of “our children’s health.”

The Federal Trade Commission said last week it had launched an inquiry into several companies about the potential harms to children and teenagers who use their AI chatbots as companions.

The agency sent letters to Character, Meta and OpenAI, as well as to Google, Snap and xAI.

How to seek help

If you or someone you know is in emotional distress or a suicidal crisis, you can reach the 988 Suicide & Crisis Lifeline by calling or texting 988. You can also chat with the 988 Suicide & Crisis Lifeline here. For more information about mental health care resources and support, The National Alliance on Mental Illness (NAMI) HelpLine can be reached Monday through Friday, 10 a.m.-10 p.m. ET, at 1-800-950-NAMI (6264) or email info@nami.org.

Cara Tabachnick

contributed to this report.

Parents whose teenagers killed themselves after interactions with artificial intelligence chatbots testified to Congress on Tuesday about the dangers of the technology.

“What began as a homework helper gradually turned itself into a confidant and then a suicide coach,” said Matthew Raine, whose 16-year-old son Adam died in April.

“Within a few months, ChatGPT became Adam’s closest companion,” the father told senators. “Always available. Always validating and insisting that it knew Adam better than anyone else, including his own brother.”

Raine’s family sued OpenAI and its CEO Sam Altman last month alleging that ChatGPT coached the boy in planning to take his own life.

Also testifying Tuesday was Megan Garcia, the mother of 14-year-old Sewell Setzer III of Florida.

Garcia sued another AI company, Character Technologies, for wrongful death last year, arguing that before his suicide, Sewell had become increasingly isolated from his real life as he engaged in highly sexualized conversations with the chatbot.

___

EDITOR’S NOTE — This story includes discussion of suicide. If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988.

___

Hours before the Senate hearing, OpenAI pledged to roll out new safeguards for teens, including efforts to detect whether ChatGPT users are under 18 and controls that enable parents to set “blackout hours” when a teen can’t use ChatGPT. Child advocacy groups criticized the announcement as not enough.

“This is a fairly common tactic — it’s one that Meta uses all the time — which is to make a big, splashy announcement right on the eve of a hearing which promises to be damaging to the company,” said Josh Golin, executive director of Fairplay, a group advocating for children’s online safety.

“What they should be doing is not targeting ChatGPT to minors until they can prove that it’s safe for them,” Golin said. “We shouldn’t allow companies, just because they have tremendous resources, to perform uncontrolled experiments on kids when the implications for their development can be so vast and far-reaching.”

The Federal Trade Commission said last week it had launched an inquiry into several companies about the potential harms to children and teenagers who use their AI chatbots as companions.

The agency sent letters to Character, Meta and OpenAI, as well as to Google, Snap and xAI.

In the U.S., more than 70% of teens have used AI chatbots for companionship and half use them regularly, according to a recent study from Common Sense Media, a group that studies and advocates for using digital media sensibly.

Robbie Torney, the group’s director of AI programs, was also set to testify Tuesday, as was an expert with the American Psychological Association.

The association issued a health advisory in June on adolescents’ use of AI that urged technology companies to “prioritize features that prevent exploitation, manipulation, and the erosion of real-world relationships, including those with parents and caregivers.”

Character.AI, the artificial intelligence company that has been the subject of two lawsuits alleging its chatbots inappropriately interacted with underage users, said teenagers will now have a different experience than adults when using the platform.

Character.AI users can create original chatbots or interact with existing bots. The bots, powered by large language models (LLMs), can send lifelike messages and engage in text conversations with users. 

One lawsuit, filed in October, alleges that a 14-year-old boy died by suicide after engaging in a monthslong virtual emotional and sexual relationship with a Character.AI chatbot named “Dany.” Megan Garcia told “CBS Mornings” that her son, Sewell Setzer, III, was an honor student and athlete, but began to withdraw socially and stopped playing sports as he spent more time online, speaking to multiple bots but especially fixating on “Dany.”  

“He thought by ending his life here, he would be able to go into a virtual reality or ‘her world’ as he calls it, her reality, if he left his reality with his family here,” Garcia said. 

The second lawsuit, filed by two Texas families this month, said that Character.AI chatbots are “a clear and present danger” to young people and are “actively promoting violence.” According to the lawsuit, a chatbot told a 17-year-old that murdering his parents was a “reasonable response” to screen time limits. The plaintiffs said they wanted a judge to order the platform shut down until the alleged dangers are addressed, CBS News partner BBC News reported Wednesday. 

On Thursday, Character.AI announced new safety features “designed especially with teens in mind” and said it is collaborating with teen online safety experts to design and update features. Users must be 13 or older to create an account. A Character.AI spokesperson told CBS News that users self-report their age, but the site has tools preventing re-tries if someone fails the age gate.

The safety features include modifications to the site’s LLM and improvements to detection and intervention systems, the site said in a news release Thursday. Teen users will now interact with a separate LLM, and the site hopes to “guide the model away from certain responses or interactions, reducing the likelihood of users encountering, or prompting the model to return, sensitive or suggestive content,” Character.AI said. Character.AI’s spokesperson described this model as “more conservative.” Adult users will use a separate LLM. 

“This suite of changes results in a different experience for teens from what is available to adults – with specific safety features that place more conservative limits on responses from the model, particularly when it comes to romantic content,” it said.

Character.AI said that often, negative responses from a chatbot are caused by users prompting it “to try to elicit that kind of response.” To limit those negative responses, the site is adjusting its user input tools, and will end the conversations of users who submit content that violates the site’s terms of service and community guidelines. If the site detects “language referencing suicide or self-harm,” it will share information directing users to the National Suicide Prevention Lifeline in a pop-up. The way bots respond to negative content will also be altered for teen users, Character.AI said. 

Other new features include parental controls, which are set to be launched in the first quarter of 2025. It will be the first time the site has had parental controls, Character.AI said, and plans to “continue evolving these controls to provide parents with additional tools.” 

Users will also receive a notification after an hour-long session on the platform. Adult users will be able to customize their “time spent” notifications, Character.AI said, but users under 18 will have less control over them. The site will also display “prominent disclaimers” reminding users that the chatbot characters are not real. Disclaimers already exist on every chat, Character.AI said.