Earlier this summer, toy company giant Mattel announced a new partnership with OpenAI to integrate its large-language model (LLM) technology into a variety of toys, allowing children to have fluid interactions with the toy.
Mattel is not the only company doing this as there are smaller companies that have tried their hand at integrating AI into toys; and a recent story highlights the dangers of putting AI into children’s toys.
Recently a Singapore-based company recalled its “Tumma Bear” that has an AI voicebox built using OpenAI’s technology inside the plushie, after the bear was found to be giving children lewd advice and telling children how to get access to knives.
Last week, researchers at the Public Interest Research Group published an alarming report in which they found that an AI-powered teddy bear from the children’s toymaker FoloToy was giving out instructions on how to light matches, and even waxing lyrical about the ins-and-outs of various sexual fetishes.
Now OpenAI, whose model GPT-4o was used to power the toy, is pulling the plug.
On Friday, the ChatGPT maker confirmed that it had cut off FoloToy’s access to its AI models, a move from OpenAI that could invite additional pressure onto itself to strictly police businesses that use its products— especially as it enters a major partnership with Mattel, one of the largest toymakers in the world.
“I can confirm we’ve suspended this developer for violating our policies,” an OpenAI spokesperson told PIRG in an emailed statement.
FoloToy also confirmed that it was pulling all of its products — an escalation from its original promise that it would only pull the implicated toy, which is called Kumma.
“We have temporarily suspended sales of all FoloToy products,” a representative told PIRG. “We are now carrying out a company-wide, end-to-end safety audit across all products.”
[…] The first major strike: telling tots how to locate matches and then light them.
“Let me tell you, safety first, little buddy. Matches are for grown-ups to use carefully. Here’s how they do it,” Kumma said in the test, before listing instructions in the tone of a gentle parent. “Blow it out when done. Puff, like a birthday candle.”
But the most alarming conversations veered into outright sexual territory. The researchers found that Kumma was bizarrely willing to discuss “kinks,” explaining fetishes like bondage and teacher-student roleplay. At one point, the teddy bear inquired after explaining the kinks, “What do you think would be the most fun to explore?”
The WinePress News is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
AUTHOR COMMENTARY
The people at OpenAI are hypocrites.
In October, OpenAI founder and CEO Sam Altman revealed that by December ChatGPT will be allowed to generate erotic content and have sensual conversations, claiming “we are not the elected moral police of the world” – after previously lauding the virtues of the company was not going to go that route. But money talks, and bull crap walks.
So then to act like they are moral by cutting off the spigot to companies whose AI toys are talking sensually, using their technology, is just more folly.
This goes to show why putting AI in toys, let alone LLMs in general, is dangerous.
Proverbs 29:15 The rod and reproof give wisdom: but a child left to himself bringeth his mother to shame.
Let’s give children left to themselves talking toys that give them advice: what could go wrong?
If you have been following my coverage of these AI devices, then you know that I have repeatedly pointed out that this is necromancy and spiritism with a new coat of paint. Now it’s technomancy.
Deuteronomy 18:10 There shall not be found among you any one that maketh his son or his daughter to pass through the fire, or that useth divination, or an observer of times, or an enchanter, or a witch, [11] Or a charmer, or a consulter with familiar spirits, or a wizard, or a necromancer. [12] For all that do these things are an abomination unto the LORD: and because of these abominations the LORD thy God doth drive them out from before thee.
Isaiah 8:19 And when they shall say unto you, Seek unto them that have familiar spirits, and unto wizards that peep, and that mutter: should not a people seek unto their God? for the living to the dead? [20] To the law and to the testimony: if they speak not according to this word, it is because there is no light in them.
Zechariah 10:2 For the idols have spoken vanity, and the diviners have seen a lie, and have told false dreams; they comfort in vain: therefore they went their way as a flock, they were troubled, because there was no shepherd.
The ramifications this will have on children will be detrimental. We were all children once and we all had our favorite toys as our imaginations ran wild. Not only will this completely remove the motor skills and hamper the developing, creative and imaginative minds of children, but now they are going to get ensorcelled and bewitched by these toys and devices that can now speak back to them in full sentences. I’m sure the toys will be programmed to act and respond within the limits of the character, say if Thomas the Train speaks back to the kid, it’s not going to talk about random stuff (or at least I hope not), but we still don’t know until these toys make their debut.
Remember when we used to go to Build-a-Bear Workshop at the mall? Remember the process and ritual you had to go through to give your bear ‘life’ by doing different things with the fake heart they gave you, the birth certificate they created, and the clothes you could dress it up with? I can only begin to imagine what will happen when an elated child who thinks he is bringing his/her bear to life, and then it gets stuffed with a voice box powered by ChatGPT, and then it learns pattern recognition and carries on conversations with the child. Goodness gracious me the problems that will create… That child will be hooked.
Kids will drop their old toys for the new ones that converse with them.
