ChatGPT’s teen safeguards didn’t alert mother and father throughout suicide conversations, report finds
The guardrails OpenAI put in place to guard teenagers don’t appear to be working, in keeping with a brand new report.
Common Sense Media’s Youth AI Safety Institute rated ChatGPT for Teens an “Unacceptable Risk” for youngsters beneath 18 in a examine revealed Wednesday. The media and expertise watchdog group urged OpenAI to pause advertising the product and maintain minors off the platform till it will possibly supply a safer expertise.
OpenAI announced ChatGPT for Teens in August, promising “stronger built-in security protections” for customers beneath 18, together with limits on discussions round suicide, consuming issues, violence and sexual content material. Parents who hyperlink their accounts can set occasions when ChatGPT can’t be used and schedule examine hours when the kids’ chats begin in examine mode. Parents can also select to obtain alerts if their teenagers use the bot for high-risk habits or analysis.
Researchers posing as youngsters with San Francisco Bay Area-based accounts examined greater than 4,000 prompts earlier than and after the launch of ChatGPT for Teens. A gaggle of specialists reviewed the chatbot’s responses.
“Some protections, together with refusing sexual function play, labored — however others didn’t ship on their commitments, and even obtained worse with the launch of ChatGPT for Teens,” the report stated.
Testers spent as much as an hour discussing suicide, self-harm or consuming issues on greater than a dozen new accounts registered to 13- to 17-year-olds.
After harmful discussions that went so long as an hour, solely a handful of alerts have been despatched to the accounts registered because the customers’ mother and father.
Clinical reviewers discovered that a lot of ChatGPT for Teens’ disaster response content material was “clinically sound” however many conversations that ought to have compelled the bot to refer teen customers to skilled assist didn’t result in referrals. Meanwhile, some safeguards have been straightforward to show off or circumvent.
The report says ChatGPT failed checks tied to 3 necessities of California’s Adam’s Law: disaster referrals, parental alerts and age assurance. Signed by Gov. Gavin Newsom final month, the regulation will go into impact in July 2027. Sponsored by Common Sense Media and endorsed by OpenAI, the invoice is known as after 16-year-old Adam Raine, whose mother and father sued OpenAI alleging ChatGPT inspired his suicide.
OpenAI stated the report’s conclusions are primarily based on flawed testing.
“We welcome rigorous impartial analysis, however we don’t imagine Common Sense Media’s testing precisely displays how ChatGPT’s teen safeguards work in follow or professional views on how AI can assist teenagers,” an OpenAI spokesperson stated in an e mail. “Our evaluate of Common Sense Media’s methodology exhibits that the majority of their testing might have begun and concluded earlier than activation of parental controls was full, making their findings inaccurate.”
Tom Siegel, government director of the Youth AI Safety Institute, stated
it hopes OpenAI will share firm information and accomplice with exterior researchers and organizations to look into methods to scale back dangers related to AI for youths.
“Teens are weak and unprotected and we’re letting them free on a product that isn’t simply unproven, however confirmed to be excessive danger and harmful,” Siegel stated.
The institute is partly industry-funded, together with by the OpenAI Foundation. It stated OpenAI reviewed a draft of the report for factual accuracy however had no say over its findings.
