Facebook’s recommendation algorithms are promoting bizarre, AI-generated images being run by spammers and scammers to an audience of people who mindlessly interact with them and perhaps don’t understand that they are not real, a new analysis by Stanford and Georgetown University researchers has found. The researchers’ analysis aligns with what I have seen and experienced over the course of months of researching and reporting on these pages, many of which have found a novel way to link to off-platform, AI-generated “news” sites that are littered with Google ads or which are selling low-quality products.
Last week the world was introduced to Shrimp Jesus, a series of AI-generated images in which Jesus is melded with a crustacean, and which have repeatedly gone viral on Facebook. The images are emblematic of a specific type of AI image being used by spammers and scammers, which I first wrote about in December but have repeatedly made the masses go “WTF” and “WHY?” when shared away from an audience of Facebook users who are seemingly unable to detect them as AI, or don’t care that they are AI. “WHAT IS HAPPENING ON FACEBOOK,” a viral tweet about Shrimp Jesus read.
What is happening, simply, is that hundreds of AI-generated spam pages are posting dozens of times a day and are being rewarded by Facebook’s recommendation algorithm. Because AI-generated spam works, increasingly outlandish things are going viral and are then being recommended to the people who interact with them. Some of the pages which originally seemed to have no purpose other than to amass a large number of followers have since pivoted to driving traffic to webpages that are uniformly littered with ads and themselves are sometimes AI-generated, or to sites that are selling cheap products or outright scams. Some of the pages have also started buying Facebook ads featuring Jesus or telling people to like the page “If you Respect US Army.”
"These images in total account for hundreds of millions of interactions and are shown through Facebook’s Feed to some Facebook users who do not follow the Pages," Renee DiRestra of Stanford's Internet Observatory and Josh A. Goldstein of Georgetown's Center for Security and Emerging Technology wrote about their research. "While Shrimp Jesus is (perhaps) obviously an artistic fantasy—created by a page that previously shared clickbait links to a content farm—comments on many of the AI-generated images of more mundane things, like housewares, homes, or artwork purportedly created by children, suggest many users are unaware of the synthetic origin."
