Define what the twin represents
A model represents a declared slice of behavior. Start with supported interfaces, relevant business objects and explicit invariants.
Specify the business objects and transitions needed for the decision. A payment model might represent initiation, processing and settlement under authored rules, while excluding an unsupported exception path. Those exclusions are part of the model definition. Without them, a reader may infer coverage that was never evaluated.
- Record modeled and unmodeled paths.
- Distinguish source-derived values from authored assumptions.
- Keep version and source pins with each model.
Protect independent evidence
A deterministic replay does not prove general correctness. Define independent assertions and keep held-out cases separate from training and refinement.
Make the split reflect dependence between cases. Closely related scenarios, repeated source worlds or shared underlying events can make nominally separate cases less independent than their count suggests. Record the sampling frame and grouping assumptions, and protect expected outcomes from the process that proposes the candidate.
Separate model refinement from independent verification
Define the cases, conditions and exclusions the model represents.
- Split cases using declared criteria.
- Preserve protected evidence and expected results.
- Do not retune against the same hidden cohort until it passes.
Read failures as information
Inspect mismatches, unsupported cases and model sensitivity. A source-linked failure is more useful than an unqualified confidence score.
Classify a mismatch before changing the model. It may expose a mapping error, an incomplete business rule, an unsupported case or an incorrect expected result. Keep the original evidence and explain any correction. Repeatedly tuning against a revealed verification set changes the meaning of that evidence.
Does a synthetic settlement twin preserve the expected accounting outcome?
Freeze the model and declared payment population. Use independent expected outcomes for settlement, exceptions and ordering, keeping protected cases out of parameter refinement.
Investigate a failed accounting assertion before accepting a faster candidate. Retain the failure even if a later model revision resolves it.
- Retain exceptions and reconciliation detail.
- Review proposed refinements with a domain specialist.
- State the conditions under which the result is valid.
Use assertions that challenge the model
An assertion should express a business or operational property independently of the implementation under test. Examples include valid state transitions, preserved identity, reconciled totals and explicit handling of expected exceptions.
A deterministic run can reproduce the same mistake every time. Pair repeatability with assertions that can fail, and inspect whether the assertions cover the decision being considered. Large case counts alone do not establish representative coverage.
Three different kinds of evidence
| Evidence | What it can tell you | What it does not establish |
|---|---|---|
| Repeatable replay | The same declared setup reproduces its behavior. | General business correctness. |
| Independent verification | The frozen model meets specified assertions on declared cases. | Coverage of untested customer conditions. |
| Domain review | The result is meaningful for a stated decision. | Automatic production or professional approval. |
- Declare the expected outcome before the candidate run.
- Include exceptions and ordering-sensitive cases.
- Keep the assertion owner separate from automated proposal generation.
Read coverage and uncertainty together
Describe what was tested, what failed and which conditions remain unobserved. Separate training results from protected verification results, and retain the exact model and source identities associated with both.
Sensitivity analysis can reveal dependence on an authored parameter, but it does not turn that assumption into an observed fact. Explain which changes alter the conclusion and which new source evidence would reduce the uncertainty.
Keep the evidence and its conditions together. A later success does not erase an earlier failure or broaden the original scope.
- Retain every qualification outcome.
- Record dependence and sampling limitations.
- Review numerical results and domain usefulness separately.
Prepare a validation review
A validation review should reach a conclusion about a declared model version and intended use. It may accept a bounded claim, reject a candidate or request additional evidence.
Bring the model definition, independent assertions, complete outcome set and exclusions to the review. Customer-specific model acceptance and production authority remain separate from internal synthetic reference tests.
Review preparation
0 / 6 prepared- Pin the exact candidate and evidence.
- Document failed and unsupported cases.
- Name the reviewer and permitted use of the conclusion.