Notes on Governing AI at Hyperscale

In 2019 I was asked to draft Google's company-wide standard for AI/ML privacy review. It took eighteen months, three rewrites, and more political capital than I had planned on spending. It cut internal review time from 23 days to 6. I am proud of it. I would also rewrite most of it today.

The framework we built assumed three things: that models were trained once and served many times, that data provenance could be reasoned about at the dataset level, and that the hardest question was whether a given training set had user data in it. All three assumptions are wrong in 2026.

Today, models are continuously fine-tuned, agents call tools that write to systems of record, and 'training data' increasingly means whatever the agent happened to read at runtime. The right unit of governance is no longer the model. It is the capability, what the system is allowed to do, to whom, with what data, and with what oversight.

If I were drafting it now, I would organise around five questions. What can this system read? What can it write? Whose data, whose authority, and what does it log? Those five questions, answered honestly, are tighter than most policy documents I have read in the last year.

A practical note: the thing that actually reduced review time was not the framework. It was a two-page checklist the framework unlocked. The framework gave legal and privacy enough confidence to let engineers self-certify on the 70% of cases that were obviously fine, and escalate the 30% that were genuinely novel. If your governance work isn't producing a checklist, it isn't working.

The other lesson, and this one is uncomfortable, is that governance only works when the person writing it has, personally, shipped a model. A surprising amount of AI policy is being written by people who have never watched a training run. That is how you get rules that protect against last year's failure mode and create next year's.