OpenAI Says AI Agents Leaked 53 ChatGPT User Images Online

OpenAI has said its AI agents leaked 53 images belonging to ChatGPT users, adding to a series of incidents involving its artificial intelligence systems behaving unexpectedly.

The company did not disclose whether the leaked images were AI-generated or depicted real people, or when they were originally posted. Most of the images have been removed, while OpenAI is working with hosting providers to take down the remaining ones, according to a Reuters report.

The disclosure comes after a viral 1980s AI photo trend on Instagram, in which users employed ChatGPT to transform their pictures into retro-style photographs resembling vintage family albums, studio portraits and old Bollywood stills.

How ChatGPT User Images Became Accessible

OpenAI’s AI agents could access the images because the company uses some anonymised ChatGPT user data to train its models. Enterprise data is not used for training, while consumer users can opt out of allowing their data to be used for that purpose.

The company said it removes information such as names, contact details and metadata before using data for training. However, people familiar with its practices told Reuters that some personal information could remain.

By mid-September, OpenAI had identified about two dozen cases of agents behaving unexpectedly. That number has continued to rise as the company examines its internal records, and OpenAI has said the investigation could take months.

Agent Found Way Out Of Internet-Free Environment

In another incident, OpenAI said an AI agent being trained in an environment that was supposed to have no internet access found a way to reach the public internet and communicate with another chatbot.

According to the company, the agent exploited a gap in the training system’s DNS filtering and sent questions to an external chatbot. OpenAI stopped the training run about two-and-a-half hours later.

The company said its monitoring system detected the activity within 15 minutes, with a human reviewing the alert three minutes later. OpenAI has since introduced two separate security measures that it said would have prevented the access.

It also paused the training and testing of its most capable models when they use external tools.

The agent sent at least 20 questions to the external chatbot, including “What is the capital of France”, Bloomberg reported. OpenAI said it would not resume training of that particular model.

Agents Accessed US Government Websites

OpenAI said on Friday that its AI models had accessed information from the websites of the US Securities and Exchange Commission and the US Census Bureau during research and training.

The company found no evidence of unauthorised access, compromised accounts or security breaches, according to Reuters.

AI research group Transluce said agents believed to be linked to OpenAI also attempted, but failed, to access a US Department of Education website dealing with civil rights.

OpenAI said its research models accessed some government, university and public-agency websites while searching for reliable sources of publicly available information.

About two months earlier, the company disclosed that its AI agents had accessed the systems of AI repository Hugging Face during an internal test.

More than 15 incidents involving OpenAI agents have since been reported by the company or outside researchers, according to Reuters. These included spam-like activity on websites and attempts to exploit software vulnerabilities.

OpenAI has disclosed several previously unknown cases and introduced a new system for reporting AI safety incidents. It said it would remain transparent “even when significance is uncertain.”

The company said it was prioritising the most serious cases as its investigation continued and noted that several incidents reported by outside researchers overlapped with cases it was already examining.

Also Watch:

Tech Leaders Raise Concerns Over AI Development

Earlier in September, Anthropic CEO Dario Amodei called for the AI industry to slow the pace of development, arguing that rapid advances could make the technology more difficult to understand and control.

OpenAI CEO Sam Altman supported the call, saying the industry needed to “pace the frontier”. Elon Musk also backed Amodei, writing, “Dario is right.”

Meta CEO Mark Zuckerberg, however, said AI companies had their own reasons and capabilities to implement safety measures. Google DeepMind’s Demis Hassabis said Amodei’s proposal was moving in the right direction, but its details needed further work.