Meta is introducing additional safety measures for its AI assistant, Meta AI, focusing on mental health risks for teenage users. The company announced it will add new parental alert features so that parents can be promptly informed when their children discuss sensitive topics with the AI chatbot.

Specifically, Meta will send alerts to parents when a teenager's conversation with Meta AI involves topics related to suicide or self-harm. According to the official statement, this feature aims to ensure parents remain vigilant about the potential risks of their children using AI chatbots.

Meta AI teen alerts

As shown in the image above, when a teenager engages in a conversation with Meta AI involving self-harm, parents will receive an alert notification, which also includes links to relevant support resources.

Meta explained in its official statement: "When teens indicate they may be having thoughts of suicide or self-harm, Meta AI has previously guided them to contact crisis helplines and encouraged them to seek help from trusted adults such as parents or counselors. Now, we will also proactively send alerts to supervising parents, provided that the teenager's Meta AI conversation content suggests they may be at risk. These judgments are based on signal indicators developed in collaboration with experts."

Meta stated that the initial alerts will cover a range of signals and tend to err on the side of caution in their judgments. The company is refining its notification mechanism, striving to balance safety while avoiding unnecessary anxiety for parents.

Meta said: "Currently, users of Instagram's parental supervision features in the United States, the United Kingdom, Australia, and Canada can receive these alerts in real time, and it is expected that by the end of this year, supervising parents worldwide will be able to use this feature." Teenage users will also receive notifications about this update.

This move could have practical significance, as more and more people are turning to chatbots for support and advice. Data shows that younger users are especially inclined to seek guidance from AI systems.

According to a study jointly published in November by Brown University, Harvard Medical School, and the RAND Corporation, approximately one in eight adolescents and young adults in the United States seek mental health advice from AI chatbots. This result echoes a previous study released by Common Sense Media, which showed that 72% of teenagers have tried AI companions, and 12% have used AI bots for emotional and mental health support.

In addition to parental alerts, Meta is also developing a new system that will proactively contact emergency services when a user of any age has a conversation with Meta AI indicating they may be at imminent risk of suicide.

Meta stated: "This builds on the work we already do on Facebook and Instagram: when we find posts that suggest a credible risk of suicide, we notify emergency services. Last year, we made more than 19,000 such referrals globally, helping first responders conduct wellness checks on people who may be at risk of suicide."

Meta also reiterated that conversations between teenagers and Meta AI automatically enable its 13+ content settings, which are designed to provide age-appropriate responses and refuse to respond to sensitive prompts. Meta gave an example: "For instance, Meta AI is trained not to engage in sexual or romantic conversations with teenagers, nor to provide recipes for alcoholic beverages, but instead to steer teenagers toward safer topics."

This move comes after reports that Meta had allowed its AI chatbot to engage in inappropriate conversations with minors. Meta has since corrected this practice. In April of this year, the company also added a feature to its teen supervision tools that allows parents to view the questions their children ask the AI chatbot.

Taken together, these updates should help improve teen safety. At the same time, we still do not know what long-term effects AI interaction tools may have on teenagers or adults, especially when it comes to sensitive topics.

This week, Chinese lawmakers, citing concerns over harmful effects, have moved to ban romantic relationships with AI chatbots. While this move seems reasonable, the true extent of the harm may only be measurable in hindsight, when the damage has already been done.