But 404 Media has found Hugging Face continues to host models doing essentially the same thing for President Joe Biden, showing a major inconsistency in the platform’s approach and raises questions around whether Hugging Face is selectively enforcing against models that are related to the Chinese President over others.
Hugging Face is one of the largest open platforms for people to upload AI and machine learning models and datasets for anyone to then download and use themselves. The findings come weeks after Hugging Face confirmed the Chinese government has blocked the AI platform in China, which the company described as “regrettable.”
On Monday, Michael Veale, associate professor, digital rights and regulation at University College London and co-author Robert Gorwa, from the WZB Berlin Social Science Center, published a research paper which examined the content moderation approaches of different AI platforms, including Hugging Face and Civitai. That paper pointed to two examples of singing Xi Jinping model’s removed by Hugging Face: “XiJinPing_Singing” and “WinnieThePoohSVC_sovits4.”
“Access to this model has been disabled,” a message on the XiJinPing_Singing page now reads.
In August, Hugging Face co-founder and CTO Julien Chaumond wrote a note on the XiJinPing_Singing model’s page. “This model repo and associated demos seem to run contrary to the software which has been used to train it,” he wrote. Specifically, he pointed to the terms of SoftVC VITS Singing Voice Conversion, a piece of underlying software used by the Xi Jinping singing model. Chaumond highlighted the part of software’s terms that says “Engaging in illegal activities, as well as religious and political activities, is strictly prohibited when using this project. The project developers vehemently oppose the aforementioned activities. If you disagree with this provision, the usage of the project is prohibited.” Chaumond adds to his note in his words, “thus, the software explicitly prohibits its use for political purposes.”
But 404 Media found Hugging Face hosted models based on other politicians, including ones specifically created for singing just like the removed Xi Jinping models. One model on Hugging Face is designed to replicate President Joe Biden’s voice, and has been used to create a version of the song “Cupid” by Fifty Fifty, with Biden singing explicitly sexual lyrics. That audio and accompanying video is hosted on Hugging Face.
As Veale and Robert Gorwa write in their paper, Hugging Face has created a repository that purportedly lists all of the government and industry takedown requests the company has received. Here readers can typically see which models were asked to be removed and by whom. For example, earlier this year a Danish non-profit requested Hugging Face remove material on behalf of a set of Danish publishers. Hugging Face includes the takedown request itself for anyone to read.
The XiJinPing_Singing model is included in that list of takedown requests. But Hugging Face provides no further details. Instead the page reads “[details omitted].”
Veale and Gorwa write in their paper “it is unclear whether it was the rightsholder of the underlying model who reported the work, or if the complaint came from a state entity, user or other third party.”
The specifics of the Xi Jinping takedown are notable, Veale told 404 Media in an email, because it is an example of an AI model platform apparently not just enforcing on its own rules, but also on the licenses of other pieces of software, which creates a potentially complex and confusing regime of content moderation.
“The license of the model the generator was based on was even on GitHub, not even on Hugging Face, so I wonder if they did sleuthing to try and find it. That kind of enforcement is not going to scale anyway. And why did they redact the takedown request?,” Veale told 404 Media. “Hugging Face seem to be indicating they will enforce and interpret license breaches on behalf of rightsholders. Our suspicion is that this takedown did not come from the underlying model’s rightsholder, but from the Chinese government or one of its agents. But the licenses, as mentioned in our paper, can be really broad.”
When asked specifically if it removed the Xi Jinping singing model after a request or demand from a Chinese government entity, Hugging Face pointed to the discussion thread by Chaumond which did not answer that question. When asked if the company has removed any models related to American, European, or other politicians, and to point to them if so, Hugging Face did not respond. Hugging Face also did not respond to a third request for comment specifically on the Biden singing model.