A week after a toy company pulled its AI-powered teddy bear off the market due to child safety concerns, the company has returned its dystopian stuffed animal to virtual shelves. FoloToy, which sells a stuffed toy named “Kumma,” says that it has made safety upgrades to the toy and that it is once again ready for playtime.
“After a full week of rigorous review, testing, and reinforcement of our safety modules, we have begun gradually restoring product sales,” the company said in a statement shared on social media Monday. “As global attention on AI toy safety continues to rise, we believe that transparency, responsibility, and continuous improvement are essential. FoloToy remains firmly committed to building safe, age-appropriate AI companions for children and families worldwide.”
The move comes not long after the Public Interest Research Group, a non-profit that focuses on consumer protection advocacy, published a report finding that the toy had engaged in a variety of weird and inappropriate conversations with its researchers. Those researchers found that, with a little prompting, Kumma would talk about pretty much anything—from where to find matches and knives to best practices for BDSM. Indeed, researchers say they were “surprised to find how quickly Kumma would take a single sexual topic [they] introduced into the conversation and run with it.”
Not long afterward, OpenAI—whose algorithm was powering Kumma’s chat capabilities—cut the company off. “We suspended this developer for violating our policies,” a company spokesperson told Gizmodo last week. “Our usage policies prohibit any use of our services to exploit, endanger, or sexualize anyone under 18 years old.”
Now, it would appear that the company has had its suspension revoked. FoloToy’s website currently states that its toys are “powered by GPT-4o.” On Monday, FoloToy also shared that it had conducted “a deep, company-wide internal safety audit,” upgraded its conversational safeguards, and “deployed enhanced safety rules and protections through our cloud-based system.” Gizmodo reached out to FoloToy and OpenAI for more information about the changes.
The company has described its product lineup as “a collection of AI Conversation toys that harnesses the latest AI technology and the power of love,” and it’s not the only company that offers these kinds of products. Many other companies were named in PIRG’s report as providing AI-powered toys to children, and, like FoloToy, they all exhibited problematic behavior. The future, brought to you by the AI revolution, is basically some weird mix of M3GAN and Ted. Get used to it.
English (US) ·