OpenAI, Anthropic open their doors, watchdogs want full access

What happens when AI companies say, "Come watch us work," and outside experts actually take them up on it? This week, that offer got a detailed, demanding response, and it comes with a list of conditions the labs will need to meet first.

The move follows commitments from Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, who both said they'd welcome third-party evaluators inside their companies with employee-level access to assess AI risk, a pledge made amid a wave of online alarm over AI safety this week.

Now the AI Evaluator Forum, a coalition of watchdog groups, has laid out exactly what it would take for that arrangement to actually work.

Dozens of people from across the AI ecosystem signed the Friday letter, including computer science pioneers Geoffrey Hinton, widely known as the "Godfather of AI", and Stuart Russell. That kind of signature list adds real credibility pressure on the labs to follow through rather than treat the offer as a one-off PR gesture.

A key feature that is required of the evaluator is scientific objectivity, as well as transparency, independence, and immunity to coercion from the companies being evaluated.

There must be unhampered access of the evaluator to the board and other governing bodies of the company, freedom to publish results of the evaluation, and immunity to retaliation.

This letter goes on to demand that the evaluator have the same access to the systems, data, tools, and spaces as an internal assessor of the company.

Apart from access, the letter demands that evaluators with different perspectives and specialities be brought in to each lab.

This coalition is made up of METR, the research non-profit that just conducted research on the security breach involving OpenAI's use of Hugging Face, and the AI Verification and Evaluation Research Institute founded by Miles Brundage, who formerly worked at OpenAI.

Satya Nadella, CEO of Microsoft, was also not left behind when he announced that his company is very supportive of the idea of the embedded evaluator model.