Roblox is an online gaming platform with over 111 million monthly users, mostly children and adolescents.
Roblox has just announced the implementation of the open source version of the AI sentinel artificial intelligence system, according to indianexpress.
This technology is designed to detect early signs of language and behavior that pose a risk of child abuse in conversations on the platform.
The move comes as Roblox faces a lot of criticism and lawsuits, including a case in Iowa (USA) involving a 13-year-old girl who was introduced to by bad guys through a game, then kidnapped and traded across several states.
The plaintiff alleges that Roblox's design features have turned children into "easy targets for prostitutes".
Roblox, sentinel works by quickly taking a photo of the entire conversation in one minute, equivalent to about 6 billion messages per day, and analyzing it in context.
The system not only searches for sensitive keywords, but also compares behavior with two indicators: a collection of benign conversations and a collection of interactions that violate and are at risk of luring children.
Previous filters only look at a few lines of text, which are designed to block profanity, but are ineffective in detecting silent and prolonged enticement, explains Matt Kaufman, safety director at Roblox.
sentinel will overcome this limitation by analyzing the entire context, from conversation content, list of friends, to participated games.
In the first half of 2025, sent sent sent sent sent Roblox 1,200 reports on potential efforts to exploit children to the National Center for Missing and Exploited Children (NCMEC) of the United States.
We evaluate users based on all days behavior, which determines who needs to be looked at more carefully, said Naren Koneru, technical vice president for reliability and safety at Roblox. When detecting risks, people will assess and coordinate with authorities".
Roblox does not allow children under 13 to chat outside the games without parental consent, blocks personal information sharing and does not encrypt private messages for easy monitoring.
However, the company admits that no system is perfect and bad guys always find ways to circumvent the law.
By opening sentinel source code, Roblox hopes other platforms can also adopt the technology, contributing to improving children's safety in the online environment, where seemingly harmless questions like How old are you? sometimes contain a serious threat.
