Brook Stefanou

Writing

What Australia should ask of frontier AI labs that train here

Australia should attach two low-cost conditions to frontier AI training here: labs should share the safety information they already give other governments, and let Australia's safety agencies test new models early. Labs that meet them should be rewarded.

7 min read

This develops my submission to PM&C’s consultation Getting it right: Building AI infrastructure that works for Australia, which closed on 9 October 2026.

Australia should ask labs for what they already give others

The first condition is information parity. A developer training here should give the Australian Government, in confidence, the same reports it already gives California, New York and the EU: critical safety incident reports, and summaries of the risks from using its own models internally.

The second is early access. The Australian AI Safety Institute (AISI) and the Australian Signals Directorate (ASD) should be able to test new frontier models before release, on the same terms the lab gives other Five Eyes countries’ safety and security agencies. Where a lab can’t do that, it should explain why in writing.

Labs that comply should get faster decisions, and an authorisation that covers their future training runs too.

Australia has little leverage, so the conditions must be low-cost

The government wants frontier training onshore. Its consultation paper says hosting it should improve Australia’s ability to “understand, influence, manage and respond to emerging risks”.1 Labs have shown interest. Anthropic has raised the idea of “up to 5GW of AI training capacity in NSW”.2

But Australia has little to bargain with. We don’t build frontier models and our market is small. The labs also answer first to Washington, which opposes foreign rules it sees as slowing American AI firms.3

Labs do walk away over regulation. In April, OpenAI paused its Stargate UK project until “the right conditions such as regulation and the cost of energy” were in place.4 The consultation paper makes the same point when it warns that onerous conditions may lead firms to “direct investment elsewhere”.1 Whatever Australia asks for has to be cheap enough not to tip that decision.

Both conditions borrow from existing law

The paper itself wants to keep the burden on industry down “by leveraging existing frameworks and policy levers whenever possible”.1 Australia doesn’t need its own size threshold for which labs are covered. It can apply the conditions to developers that California’s law already classes as large frontier developers, or as systemic-risk model providers under the EU AI Act.56 Australia then doesn’t have to set and check its own threshold for computing power or energy use. Those lists cover OpenAI, Anthropic and Google DeepMind today. If they change, Australia’s list changes with them. California lets developers meet its incident-reporting duty by reporting under an equivalent federal scheme.7

The conditions should apply to the developer that holds the training authorisation, not the company that owns, runs or rents space in the data centre. The paper accepts that “AI training obligations may be more appropriately applied to organisations undertaking AI training activities”.8 Both conditions concern the developer’s models, so they work whoever owns the building.

Information parity

The government has committed to part of this. The paper says labs “granted authorisation to undertake large scale AI training in Australia will be required to adhere to specific, minimum-security and safety expectations, such as by disclosing defined reportable AI incidents to relevant Australian authorities”.1 It doesn’t yet say what counts as a reportable incident. The cheapest option is to borrow the definitions labs already report against.

California requires a critical safety incident report within 15 days, or 24 hours if there’s an imminent risk of death or serious injury. The duty covers “any critical safety incident pertaining to one or more of its frontier models”, wherever it happens.5 New York and Illinois both require reports within 72 hours from 1 January 2027.910 The EU has a similar duty.6

Large developers also send California a confidential summary of the catastrophic risks from using their own models internally. They send it every three months or on another agreed schedule. New York requires the same.9 Summaries for released models are already public, in transparency reports,11 so Australia only needs to ask for the internal-use summaries.

The bar for all this is high. California, New York and Illinois define catastrophic risk as a material risk that a model contributes to the death or serious injury of more than 50 people, or more than $1 billion in damage, in a single incident.510 Incidents also include theft of a model’s weights (the trained model itself) or loss of control that causes injury, and a model deceiving its developer in a way that raises catastrophic risk. These reports should be rare.

Early access on Five Eyes terms

AISI and ASD should be able to test new frontier models before public release, on the same terms the lab gives the equivalent bodies in other Five Eyes countries. The paper calls Australia “a trusted Five Eyes member”.1 AISI has “commenced frontier model testing”,1 and it works with ASD as a technical partner.1213

The US and UK have voluntary testing arrangements with the major labs.141516 AISI also works with labs under memorandums of understanding,12 but Australia’s access is patchier. Australia only got access to Anthropic’s Mythos model in the second wave of countries.17

This asks for nothing the lab doesn’t already give the US and UK. That makes it easier for Washington to accept, and harder for host countries outside Five Eyes to cite as a precedent.

Comply or explain

Some gaps in access come from US law rather than the lab’s choice.18 A legal bar like that should never count as a breach. If a lab withholds a model for any other reason, it should give the government a private written explanation. The government would weigh that explanation when the lab’s authorisation comes up for renewal.

A case like this came up in September. According to Politico, the White House Office of the National Cyber Director asked OpenAI and Anthropic to hold new frontier models back from the UK AI Security Institute until a US security review was finished. The request was informal and had no legal force. Anthropic complied and kept Mythos 5.1 within its US-only Project Glasswing group.1920 Under comply-or-explain, a lab that followed a request like that would owe Australia a written reason.

Shared information needs safeguards

Shared information should be confidential and seen only by staff who need it, as in California and the EU.216 California exempts these reports from its Public Records Act,22 and Australia should exempt them from freedom-of-information requests in the same way. Unreleased models and their outputs need ASD-grade security. The information should be used only for safety and security, never for commercial or unrelated regulatory purposes. The government shouldn’t publish findings about a specific model without consulting the developer.

Labs that comply should get faster, longer-lasting authorisation

Labs that meet both conditions should have their training authorisations decided first. They should also get a standing authorisation, so each new training run isn’t reassessed from scratch. In its March 2026 Expectations of data centres and AI infrastructure developers, the Commonwealth says it “will prioritise proposals most closely aligned” with those expectations.23 This would write that priority into the authorisation itself.

The body that issues training authorisations, whichever one the government designates, should grant this benefit and enforce the conditions. AISI and ASD would receive the access and information. AISI’s role is to test models and support regulators, so enforcement belongs with the authorising body.12

A lab that stops complying should lose the benefit and face steps of increasing severity: a formal notice, being named as non-compliant in a report, a civil penalty, and suspension of its authorisation for new training runs. The data centre could keep running its other work.

Objections

“Labs will train elsewhere.” The conditions copy what labs already do, or soon will do, for California, New York, Illinois and the EU. For those reports, Australia would be one more recipient, and a compliant lab would get its authorisation sooner.

“Other countries will want the same access.” The benchmark is what other Five Eyes members get, so a host country outside the group has no matching standard to point to.

“Even the UK got cut out.” Anthropic withheld Mythos 5.1 from the UK AI Security Institute after an informal White House request.1920 Matching other Five Eyes terms sets a minimum. It can’t guarantee access when Washington leans on a lab. But the UK, relying on a voluntary agreement, couldn’t even ask why. Under comply-or-explain, Australia could.

“Confidential reports give the public nothing.” Each year, the government could publish an anonymised count of how often labs gave or withheld access, and why. California will publish a similar anonymised report on critical safety incidents from 2027.24

Set these terms in the national AI standards

The Prime Minister has said the government will introduce AI laws, and that the Commonwealth will legislate national AI standards.25 Both conditions and the reward for meeting them belong in those standards, and should be settled before the first large training run is approved.

Footnotes

  1. Department of the Prime Minister and Cabinet, Getting it right: Building AI infrastructure that works for Australia, consultation paper, September 2026, pp. 1, 21, 24. https://www.pmc.gov.au/sites/default/files/resource/download/ai-consultation-paper.pdf ↩ ↩2 ↩3 ↩4 ↩5 ↩6

  2. ABC News, 26 August 2026. https://www.abc.net.au/news/2026-08-26/claude-maker-anthropics-massive-data-centre-plans-nsw/107075532 ↩

  3. Nextgov, June 2026. https://www.nextgov.com/artificial-intelligence/2026/06/us-opposes-global-ai-standards-sees-value-coordinating-real-world-harms-state-official-says/413926/ ↩

  4. Yahoo Finance, 9 April 2026. https://finance.yahoo.com/sectors/technology/articles/openai-pauses-uk-data-centre-135030881.html ↩

  5. California SB 53 (Transparency in Frontier Artificial Intelligence Act), Bus. & Prof. Code §§22757.11(c)–(d), 22757.12(d), 22757.13(c). https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53 ↩ ↩2 ↩3

  6. EU AI Act, Arts 2(1), 55(1)(c), 55(3). https://artificialintelligenceact.eu/article/55/ ↩ ↩2 ↩3

  7. SB 53, §22757.13(h)–(i). ↩

  8. Getting it right, p. 9. ↩

  9. New York RAISE Act, as amended. Summaries: Davis Polk, https://www.davispolk.com/insights/client-update/new-york-joins-california-regulating-development-frontier-ai-models ; Morrison Foerster, https://www.mofo.com/pdf/resources/insights/260403-new-york-amends-the-raise-act-to-align-more-closely ↩ ↩2

  10. Illinois Artificial Intelligence Safety Measures Act, Public Act 104-0538. https://ilga.gov/documents/legislation/PublicActs/104/104-0538.htm ↩ ↩2

  11. SB 53, §22757.12(c)(2). ↩

  12. Department of Industry, Science and Resources, “Australia’s AI Safety Institute”, accessed October 2026. https://www.industry.gov.au/science-technology-and-innovation/technology/artificial-intelligence/ai-safety-institute ↩ ↩2 ↩3

  13. InnovationAus, 10 July 2026. https://www.innovationaus.com/ai-safety-institute-teams-with-asd-to-test-unreleased-models/ ↩

  14. NIST CAISI testing agreements, May 2026, via HPCwire. https://www.hpcwire.com/off-the-wire/nists-caisi-announces-new-frontier-ai-testing-agreements-with-google-deepmind-microsoft-xai/ ↩

  15. UK–Google DeepMind memorandum of understanding, December 2025. https://www.gov.uk/government/publications/memorandum-of-understanding-between-the-uk-and-google-deepmind-on-ai-opportunities-and-security/memorandum-of-understanding-between-the-uk-and-google-deepmind-on-ai-opportunities-and-security ↩

  16. UK–Anthropic memorandum of understanding. https://www.gov.uk/government/publications/memorandum-of-understanding-between-the-uk-and-anthropic-on-ai-opportunities/memorandum-of-understanding-between-uk-and-anthropic-on-ai-opportunities ↩

  17. TechCrunch, 2 June 2026. https://techcrunch.com/2026/06/02/anthropic-scales-claude-mythos-to-critical-infrastructure-in-15-countries/ ↩

  18. ASPI Strategist, 16 June 2026. https://www.aspistrategist.org.au/as-trump-restricts-frontier-ai-australia-needs-options/ ↩

  19. Politico, 24 September 2026. https://www.politico.com/news/2026/09/24/white-house-asks-openai-and-anthropic-to-hold-new-models-from-uk-testers-until-u-s-review-01091769 ↩ ↩2

  20. Lucy Fisher and Madhumita Murgia, Financial Times, 9 September 2026. https://www.ft.com/content/560e1c8b-f163-4fd6-b604-e905550ac870 ↩ ↩2

  21. SB 53, §22757.13(b)(2). ↩

  22. SB 53, §22757.13(f). ↩

  23. Department of Industry, Science and Resources, Expectations of data centres and AI infrastructure developers, March 2026. https://www.industry.gov.au/publications/expectations-data-centres-and-ai-infrastructure-developers ↩

  24. SB 53, §22757.13(g). ↩

  25. Reuters, “Australian PM says to enact laws to govern AI” (15 Jul 2026), via The Star — https://www.thestar.com.my/aseanplus/aseanplus-news/2026/07/15/australian-pm-says-to-enact-laws-to-govern-ai ; also cited in the consultation paper’s “Purpose and scope”. ↩