Content warning: This post contains numerous images of AI-generated gore and AI-generated, mutilated children posted to Facebook
Last week, Facebook CEO Mark Zuckerberg told investors that the platform’s expanding “AI recommendation system,” which pushes posts into users’ feeds from all over Facebook, was leading to greater engagement on the platform. “Right now, about 30% of the posts on Facebook feed are delivered by our AI recommendation system. That's up 2x over the last couple of years,” Zuckerberg said.
Some of the posts Facebook’s recommendation engine is putting into users’ feeds are AI-generated images of starving, drowning, amputated, bruised, and otherwise suffering and mutilated children.
Two different 404 Media readers have told me that posts from accounts called “Little Ones” and “Cuddle Bugs” have been recommended into their feeds. “It’s my special day! Hoping for some extra love and good vibes today!” One of the images shows a child whose leg is amputated below the knee and holds a sign reading “Today is my birthlday pleaase like.” That image has 70,000 likes and 3,000 comments. Another image is of a girl face-down in the ocean wearing an oxygen mask that is connected to a floating birthday cake. Variations of this specific image have shown up on multiple pages; one version I saw has 5,000 likes and 211 comments, another version has 267,000 likes and 13,400 comments.
Other images on the pages feature a child with three mutilated legs, a missing hand, and a beard sitting in a shack surrounded by squalor; a child who is missing both hands and is emaciated with a cake that reads “Oday is my brhday;” a screaming infant in a diaper sitting on the side of a road with his arm blown off and huge wounds on his legs; and two children with burns, missing legs, and a cardboard sign that reads “Todax is my lie.” Another image features a young girl who is part butterfly or fairy, is wearing an oxygen mask, has had the skin melted off of her emaciated arms and legs, and is begging for money on the street.
Another 404 Media reader sent me one of these pages and said “it’s got quite dark now. There are now AI pages with generated mutilated kids farming for likes. Very disturbing IMO. I got this recommended—now I will have more of this stuff recommended :/“ The drowning girl image has also gone viral on Twitter.
The pages posting these images have been doing so for a few weeks. In recent days, people have begun posting in the comments that these are AI-generated images. But comments in older photos do not mention AI and mostly say “happy birthday” or have a lot of celebratory emojis.
Facebook has not, to my knowledge, taken action on any of the AI-generated spam that we have repeatedly sent to the company, nor has it deleted the pages of the engagement farmers who are pushing people off the platform or are otherwise trying to scam users. Facebook has published multiple blog posts saying that it will eventually label AI-generated content, but has not done so yet. Its most recent blog post has focused on the use of AI for disinformation, and claims they will begin to label AI-generated content in May, though Facebook has only said the labeling “will be based on our detection of industry-shared signals of AI images or people self-disclosing that they’re uploading AI-generated content.” In one of its blog posts, Facebook says that, “stakeholders in 34 countries” told Facebook that “generative AI is becoming a mainstream tool for creative expression.”
As Emanuel reported earlier this week, big tech companies are moving forward with their AI plans regardless of the human toll, and are hoping that they will be able to iterate their way out of the unpleasant truth that these tools are regularly being abused to create grotesque content. Meta is a signatory of a new plan collaboration to “responsibly” train AI models. But existing AI models trained on real images of real children have been used to generate AI child sexual abuse material, and that same technology is now also being used to create viral images of mutilated children. The AI-powered recommendation algorithm Mark Zuckerberg brags about to shareholders is pushing them to users.
Meta did not immediately respond to a request for comment.