What to Know

  • Nvidia, OpenAI, Anthropic, Google, Meta and xAI signed a White House accord on Sept. 29 covering advanced AI models.
  • U.S. President Donald Trump described the accord as morally binding.
  • The companies pledged to partner with independent external auditors or evaluators and to have independent board members review auditor findings.
  • The accord does not specify who selects auditors, what they may examine, whether results become public, or what penalties apply if companies ignore the pledge.
  • Critics argue that audits chosen and scoped by the companies being reviewed could miss important risks.
  • The collapse of Enron and Arthur Andersen remains a cautionary example of how conflicts of interest can weaken professional audits.
  • In 2023, leading AI companies made voluntary White House commitments, but later scoring by researchers at Stanford, Harvard and Brown found an average compliance level of 53% among 16 companies.
  • On one pledge to help outsiders find and report problems, eight of the 16 companies scored zero.
  • Google DeepMind has demonstrated a double blind model testing approach using sealed computer hardware, developed with Singapore's AI Safety Institute and three other outside groups.
  • Researchers have shown that sealed hardware can be attacked with less than $1,000 of equipment if physical access to the server is available, highlighting the need for careful controls.

AI Audit Promise Raises a Governance Question

The White House accord signed by Nvidia, OpenAI, Anthropic, Google, Meta and xAI has put AI auditing at the center of the technology governance debate. The idea is straightforward: companies building the most advanced AI systems should allow qualified outsiders to inspect, test and evaluate the risks attached to those systems. In principle, that could give boards, policymakers and the public greater confidence that powerful models are being checked before problems become too large to contain.

Yet the practical details matter more than the headline promise. If the company under review chooses the auditor, defines the scope of the test and controls what information is available, the review may become less like an independent audit and more like a controlled demonstration. The accord says companies will partner with independent external auditors or evaluators, and that independent board members will review reports and make sure problems are addressed. It does not, however, say who chooses those auditors, what access they receive, whether any findings are disclosed publicly, or what consequences apply if a company does not follow through.

That gap has become the core issue for market participants, governance specialists and technology policy observers watching the next phase of AI oversight. The White House effort signals political pressure for stronger accountability, but without enforceable audit standards, the process could rely heavily on the discretion of the same companies whose systems are being evaluated.

Why Company Selected Audits Can Fall Short

Audits are only as strong as their independence, access and incentives. A reviewer may have deep technical expertise, but that expertise can be blunted if the client controls the assignment too tightly. In the AI context, that could mean limited time with a model, restricted visibility into training or evaluation details, or a narrow set of tests that avoids the most sensitive failure modes.

The corporate audit world already offers a powerful warning. In 2000, Enron paid Arthur Andersen $25 million to audit its books and another $27 million for tax and consulting work. Within two years, both firms had collapsed. The lesson was not simply that an auditor lacked technical skill. It was that a professional services firm paid heavily by a client had weak incentives to challenge that client aggressively.

Congress responded with the Sarbanes Oxley Act, which changed the balance of power in public company audits. The law moved auditor selection away from management, restricted auditors from selling many other services to the companies they audit, and established a public board to inspect the auditors themselves. That model is now being invoked in the AI debate because the White House accord does not create an equivalent overseer for AI model auditors.

The analogy is not perfect. AI models are not balance sheets, and safety testing is not the same as accounting. Still, the underlying governance problem is similar. When a company pays for and shapes the review of its own high stakes activity, outsiders may wonder whether the audit is designed to find problems or to reassure stakeholders.

Voluntary AI Commitments Have Had Mixed Results

The AI sector already has experience with voluntary pledges. In 2023, leading AI companies made voluntary commitments to the White House, including a pledge to allow outside experts to test systems before release. Later, researchers at Stanford, Harvard and Brown evaluated 16 companies against those commitments and found an average score of 53%.

The weakest area was especially relevant to the current audit debate. On the promise to help outsiders find and report problems, eight of the 16 companies scored zero. That result suggests that voluntary commitments can produce uneven behavior unless they are backed by clear requirements, transparency and credible accountability.

Outside testing has occurred in some cases, but the developer often determines the terms. METR, a nonprofit that tests AI models, said its testing of OpenAI's o3 model before release was conducted over only three weeks, with limited access to information about the model and no view of the model's step by step reasoning. OpenAI did allow the testing, which is an important point. But the episode also shows how much depends on what a company chooses to share.

For advanced AI models, the stakes extend beyond ordinary software quality assurance. Auditors may need to assess whether models can assist with cyber abuse, biological misuse, deception, autonomy, persuasion, or other capabilities that could create public risks. If testers do not choose the questions, do not receive sufficient access, or cannot publish even basic information about what they evaluated, the audit may have limited value to anyone outside the company.

Why AI Models Create a Special Access Problem

A trained AI model is essentially a very large file of numbers, and producing one can cost more than $100 million. Whoever has a copy of that model can potentially control or reproduce its capabilities. That creates a difficult tension for external evaluation. On one side, auditors need meaningful access. On the other side, companies have strong security reasons not to hand model weights or sensitive data to outside parties.

Until recently, that left evaluators with an awkward tradeoff. If testers sent their questions to the company, the company could see the exam in advance. If the company gave the model to the testers, the model could be exposed. Either approach created a weakness. The first could compromise the independence of the test, while the second could create a security risk.

Google DeepMind has described that problem and demonstrated one possible path around it. In August, it worked with Singapore's AI Safety Institute and three other outside groups to run what it called the world's first double blind test of an AI model. The structure resembled the logic of a double blind drug trial. The test ran on sealed computer hardware, meaning the outside testers did not see the model and Google did not see their questions.

Google says each side can mathematically verify that its material stayed private. If such approaches mature, they could substantially change the audit debate. Companies would have less basis to argue that independent test design is impossible, while auditors would be better positioned to examine models without taking possession of them.

Hardware Trust Becomes the New Pressure Point

Double blind testing does not eliminate trust. It moves much of the trust onto the hardware and the process used to control it. That is a narrower and potentially more manageable challenge, but it remains real. Last year, researchers broke into this kind of sealed hardware with less than $1,000 of equipment, although they needed physical access to the server.

That detail matters. A vulnerability requiring physical access is different from a remote compromise, but it is not irrelevant. The implication is that any double blind AI audit structure must include strict rules about who controls the machines, where the hardware is located, how it is monitored and how tampering risks are reduced.

Even with those caveats, the emergence of privacy preserving evaluation tools weakens a major argument against stronger independent audits. If outsiders can test advanced systems without seeing the model or its data, then auditors can be given more freedom to design meaningful evaluations. That freedom is essential if audits are to detect problems companies may not already expect or may not prefer to emphasize.

Transparency Without Exposing Trade Secrets

One practical reform would be basic disclosure. Companies could identify who their auditors are and describe what the audits covered, even if detailed findings remain private for security reasons. Public companies have named their financial auditors for decades without revealing trade secrets. A similar principle could apply to advanced AI systems.

That kind of transparency would allow outside observers to distinguish between narrow reviews and broad evaluations. It would also allow scrutiny of auditor incentives. If an auditor depends heavily on one AI company for revenue, or sells other services to the company it audits, stakeholders may reasonably question the independence of the review.

Another reform would be external oversight of the audit process itself. Sarbanes Oxley created an inspector for the inspectors in financial auditing. The AI accord does not include a comparable body. Without an external check, independent board members may review auditor reports, but the process still remains largely inside the company.

Congress may ultimately need to decide what fills that gap. Options could include a public oversight board, formal accreditation standards, mandatory disclosure rules, or legal duties for companies developing the most advanced models. The precise structure is still open for debate, but the current voluntary framework leaves key questions unanswered.

What the Accord Means for AI Accountability

The White House accord is not meaningless. It marks a public acknowledgement by some of the world's most powerful AI companies that external scrutiny is necessary. That alone is notable in an industry where model capabilities, training details and risk assessments are often closely guarded.

But the accord's credibility will depend on implementation. Audits that are independent in name but controlled in scope may do little to reassure policymakers or the public. Stronger audits would allow evaluators to choose what they test, give them enough time and technical access to do meaningful work, and provide at least some public information about who conducted the review and what was examined.

The next phase of AI governance may therefore turn on a simple question: will auditors be able to look where they believe the risks are, or only where companies permit them to look? If the answer is the latter, the audit pledge could become a procedural shield. If the answer is the former, the accord could become a foundation for more credible oversight of advanced AI.

Frequently Asked Questions (FAQs)

Which companies signed the White House AI accord?

Nvidia, OpenAI, Anthropic, Google, Meta and xAI signed the accord on Sept. 29, pledging to work with independent external auditors or evaluators for advanced AI models.

What did President Donald Trump call the accord?

U.S. President Donald Trump described the accord as morally binding, meaning it carries public and political weight even though the text does not set out penalties for ignoring it.

Why are critics concerned about the audit structure?

The concern is that each company may select its own auditor and decide what the auditor can examine. That could limit the independence and depth of the review.

What is the Enron and Arthur Andersen lesson for AI audits?

Enron paid Arthur Andersen $25 million for audit work and another $27 million for tax and consulting services in 2000. The collapse that followed showed how conflicts of interest can undermine audit independence.

How did prior voluntary AI commitments perform?

Researchers at Stanford, Harvard and Brown scored 16 companies against 2023 voluntary commitments and found an average result of 53%. On one pledge involving outside problem reporting, eight companies scored zero.

Why is outside testing of AI models difficult?

A trained model can be a highly valuable file of numbers, and producing one can cost more than $100 million. Giving outsiders a copy can create security risks, while letting companies see test questions can weaken independence.

What is double blind AI model testing?

Double blind AI testing uses a structure in which testers do not see the model and the company does not see the test questions. Google DeepMind demonstrated such a test in August with Singapore's AI Safety Institute and three other outside groups.

Does sealed hardware solve all audit problems?

No. Sealed hardware can reduce the need to share models or questions, but trust shifts to hardware security and control. Researchers have shown attacks are possible with less than $1,000 of equipment if physical server access is available.

What would make AI audits more credible?

Audits would be more credible if auditors could choose what they test, companies disclosed who the auditors are and what the audits covered, and an external oversight mechanism checked the audit process itself.