There are two major problems here:

• OpenAI took months to realise that their agent had hacked Medicare
• Services Australia had no idea they’d been breached until OpenAI informed them

On top of that, the time it took for the notice to go through the channels here in Australia was ridiculous.

That was my thought exactly. Agent broke in, read some files (no mention of PII or confidential things) to confirm read access, dropped a test file to confirm write access, sent an email explaining that’s bad with a few details.

“We are notifying you of a security vulnerability identified during our review of OpenAI Model activity…” the communiqué began. “An OpenAI model identified a way to make the server carry out instructions sent through the public reporting interface, without a private account or password.”

“It was able to access this to read portions of internal program files and settings, obtain a list of files, and create and read back a small test file on the server,” it explained. “Our review found no evidence that the model accessed patient-level records, personal information or credentials; deleted data; or established ongoing access.”

Being on the defensive side myself, if I received that from a human I’d be grateful.

Edit: from the bad screenshot, the email was even sent to the right email: “PUBLIC DISCLOSURE”. So the target does accept such reports.

Well do we trust OpenAI?

Best for the company? probably.

Morally clear? grey. If their ‘hacking’ accidentally causes issues with the system or it goes down, it’s definitely bad. Whatever was on that system is now in the hands of OpenAI, not great. Because OpenAI is open about it, a lot of people who are even less reputable will do it.

Historically, Pentesting had unwritten rules of engagement. None of that is being followed.

We don’t seem to apply law to AI, that’s a real problem.

But still, best for the company/entity, yeah.