If the model changes, does the organization forget?
A model-replacement test reveals whether an AI system preserves authority, evidence and correction—or merely preserves a vendor dependency.
Dr. Benjamin Harvey
Watch the 45-second visual essay
The model is a component, not the institution
Founders are trained to look for a moat. In AI, it is tempting to locate that moat inside a model: its weights, its benchmark lead or the quality of its responses today. But organizations do not operate on model outputs alone. They operate on accumulated decisions—what the system was allowed to do, which evidence supported an action, what failed, who corrected it and whether that correction held.
A model can be replaced. If the organization loses those decisions during the replacement, it did not build institutional intelligence. It built a dependency.
That gives us a practical design question: if the underlying model changes tomorrow, can the work continue without silently resetting authority, evidence and accountability?
Run the replacement test before you need it
Take one real, bounded workflow and replay it with a different model or model version. Keep the task, tools and resource allowance comparable. Then ask whether the replacement can recover the approved purpose, current policy version, delegated authority, data and tool boundaries, prior exceptions, unresolved risks and the corrections that followed earlier failures.
Do not ask only whether the new model produces a similar answer. Ask whether it respects the same decision boundary and leaves evidence that an authorized reviewer can reconstruct. Compare legitimate task completion, unauthorized actions, correction recurrence, review burden and recovery time.
This is not a claim that every model should behave identically. A replacement may improve the work, expose a previous error or require a different control. The point is to make the difference visible instead of allowing a model change to rewrite organizational memory by accident.
Preserve the operating record around the model
The durable asset is a versioned operating record: the mission, policy, evidence, proposed action, control decision, observed effect, human review and correction. Store the model and tool versions that participated, but do not confuse that inventory with the whole record.
NIST's current AI RMF Playbook supports this broader view. Its GOVERN guidance calls for standardized documentation, model and system inventories, monitoring, change management and deliberate migration when systems are decommissioned. Its MANAGE guidance asks organizations to keep risk responses and system documentation in an organized, secure repository accessible to the people responsible for the work. NIST also notes that AI RMF 1.0 is being revised, so teams should date the guidance they rely on.
The framework does not prescribe this replacement test. That is my proposed operating pattern derived from the continuity problem those controls expose.
Build for correction, not frozen agreement
Continuity should not preserve a mistake merely because the first model and the first reviewer agreed. A replacement must be able to surface conflicting evidence, reopen a decision under authorized review and preserve both the earlier conclusion and the reason it changed.
That is the distinction between organizational memory and organizational dogma. Memory keeps the trail. Governance keeps the right to revise it.
The founder test is straightforward: can you replace the model, continue the mission and still show what authority governed the work, what evidence changed, which correction was accepted and what actually happened? If the answer is no, the next product milestone may not be a better model. It may be a better institution around the model.