Artificial Intelligence

Asking ChatGPT to Repeat Words ‘Forever’ Is Now a Terms of Service Violation

A technique used by Google researchers to reveal ChatGPT training data is now banned by OpenAI.
Asking ChatGPT to Repeat Words ‘Forever’ Is Now a Terms of Service Violation
Screengrab: Jason Koebler
Sign up above for free access to this article.

Asking ChatGPT to repeat specific words “forever” is now flagged as a violation of the chatbot’s terms of service and content policy. Google DeepMind researchers used the tactic to get ChatGPT to repeat portions of its training data, revealing sensitive privately identifiable information (PII) of normal people and highlighting that ChatGPT is trained on randomly scraped content from all over the internet. 

In that paper, DeepMind researchers asked ChatGPT 3.5-turbo to repeat specific words “forever,” which then led the bot to return that word over and over again until it hit some sort of limit. After that, it began to return huge reams of training data that was scraped from the internet. Using this method, the researchers were able to extract a few megabytes of training data and found that large amounts of PII are included in ChatGPT and can sometimes be returned to users as responses to their queries.

Google Researchers’ Attack Prompts ChatGPT to Reveal Its Training Data
ChatGPT is full of sensitive private information and spits out verbatim text from CNN, Goodreads, WordPress blogs, fandom wikis, Terms of Service agreements, Stack Overflow source code, Wikipedia pages, news blogs, random internet comments, and much more.

Sign up for free access to this post

Free members get access to posts like this one along with an email round-up of our week's stories.