Meta has announced a new safety framework for its AI chatbot that will alert parents if a teenage user’s conversation indicates signs of suicide or self-harm. The company said that when its artificial intelligence system detects such signs, the content will first be reviewed by Meta’s internal team. Depending on the assessed level of risk, parents will receive an immediate alert, and in severe cases, emergency services may also be contacted.

Meta emphasized that the protection of teenagers will take precedence over perfect accuracy. This means that alerts may be sent even when the system is not fully certain about the message’s meaning, to avoid potential risks. Although this could lead to some unnecessary notifications, Meta stated it will not compromise on safety.

Initially, the feature is being rolled out to families in the United States, United Kingdom, Australia, and Canada who use Instagram’s parental supervision tools. Meta plans to expand the feature to other countries later this year. The company has also tightened content controls to prevent its AI chatbot from responding to sensitive or inappropriate questions from teenagers.