Meta to alert parents if teens discuss self-harm with AI

Meta announced that parents using its family supervision tools can now receive alerts when their teen's conversations with Meta AI on Instagram, Facebook, or Meta Horizon suggest they may be at risk of suicide or self-harm.

The feature builds on a system introduced earlier this year that flagged teens' search activity for similar terms, extending that same logic into AI chat conversations.

How does Meta alert parents?

In Meta AI, whenever the AI notices that someone is talking about their thoughts of suicide or hurting themselves, it refers them to the hotline services and urges them to discuss the matter with a parent or any other trusted adult.

Now, what the new addition does is give an additional notification to the parents or the supervisors when the signals from the AI indicate real danger.

Every conversation that is flagged by the system undergoes a manual check before sending out the notification.

"While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point," the company said in its announcement.

This tool is available only to users who have been enrolled in family supervision programmes of Meta by using Family Center, which means that parents must sign up to get the notifications regarding their children. It is currently available in the United States, Canada, the United Kingdom, and Australia, without any confirmed plans for expansion internationally.

The alerts do not contain the content of the conversation but only prompt parents to have a discussion with their child. Also, along with each notification, Meta provides parents with guidelines on how to handle the conversation.

Meta says it's also building the ability to alert emergency services directly when a conversation with Meta AI from either a teen or an adult suggests someone may be at imminent risk of suicide, mirroring a system it already runs for Facebook and Instagram posts. The company reported making more than 19,000 such referrals globally last year, resulting in wellness checks by first responders.