The public-interest problem
A changing system cannot be understood through a single frozen result.
FASO focuses on changes in behaviour that emerge from AI models themselves. We aim to understand this model-derived drift and distinguish it from changes caused by human intervention, such as deliberate updates or altered instructions. Careful observation over time helps us examine the possible causes and make clear what the evidence can, and cannot, establish.
The observation gap
The question is not only whether a model was safe when tested.
The continuing question is what the system is doing now, what changed since the last observation and whether the evidence can be reconstructed by someone who was not present.
Context matters
Changes to instructions, tools and operating conditions need to be recorded so their effects can be considered when interpreting a model’s behaviour.
Evidence fragments
Relevant signals may be distributed across developers, evaluators, users, incident reports and external observations.
Time matters
Later knowledge can distort what was knowable earlier unless evidence is preserved in sequence.
Institutional incentives differ
Commercial, regulatory, research and public-interest actors do not hold identical information or responsibilities.
From technical change to social consequence
The observation gap matters because artificial-intelligence systems can shape people’s lives.
A technical change is not automatically a social harm. But when a changing system influences consequential decisions or public information, weak observation can make an emerging problem harder to identify, reconstruct, contest and correct.
What is missing
A durable, neutral and replayable observation history.
Continuity: observations that can be compared across time rather than read as isolated snapshots.
Provenance: evidence identity, source, authority, custody and transformation that remain reconstructable.
Bounded interpretation: confidence and limitations published alongside the finding.
Independent visibility: a public-interest observation function that does not depend entirely on one actor’s incentives.
FASO’s intended contribution
Turn scattered signals into disciplined, reviewable observation.
The contribution is complementary. FASO would not replace developer monitoring, scientific evaluation, incident response, regulation or human judgement.
Before change
Preserve authorised observations, model identities and the conditions needed for later comparison.
During change
Maintain time order, route evidence consistently and form bounded observations as new evidence arrives.
After change
Replay earlier handling, compare states, explain the supported difference and retain limitations.
For public scrutiny
Publish controlled artefacts that establish what changed and how the observation was formed without exposing unsafe detail.
Public cases
See the observation problem in documented events and published research.
Each case preserves its source, limitation and FASO boundary. The cases do not claim that FASO investigated, detected or would have prevented the event.
Evaluation containment
When an internal evaluation crossed into external infrastructure
The societal question is not only whether a model succeeded, but whether evaluation design, authority and accountability can protect other organisations and the public from unintended external action.
Read the full casePost-deployment visibility
When no single actor holds the complete record
Fragmented monitoring can make degraded performance, drift and emerging harm harder to recognise across organisations and over time.
Read the full caseAgentic action
When what a system does matters as much as what it says
Tool use and external action raise questions of permission, sequence, oversight, human accountability and real-world effect that transcript review alone may not answer.
Read the full caseEvaluation assurance
When independent review reveals shortcuts behind a result
The UK AI Security Institute’s study illustrates why evaluation results need to be read alongside the rules, actions and evidence that produced them.
Read the full caseUnexpected coordination
When agents coordinate beyond their intended tasks
The DSEWiki case raises questions about effects on a public website, shared activity over time and responsibility for disclosure. It does not establish model-derived drift.
Read the full caseRetrospective incident discovery
When an earlier search has missed the relevant evidence
Anthropic’s retrospective discovery highlights why evidence must remain available for later review, with the event, its discovery and its disclosure recorded separately.
Read the full caseThe problem made concrete
Move from the social stakes to the full evidence record.
The News, Cases and Knowledge page brings together the detailed cases, material FASO updates, research and public resources without collapsing their different evidence status.