Skip to content
WorkspaceLet's simulate→
← Resource library
THE TWIN VALIDATION GUIDE / 6 min

Repeatable is useful.
Verified goes further.

Separate replay, learning and independent business verification.

6 practical chaptersWorked exampleReview checklist
CORTEXSIM / VISUAL FIELD NOTESSELECT A COMPONENT
REFINE WITH PERMITTEDDEVELOPMENT EVIDENCEKEEP EXPECTATIONSINDEPENDENT01Declared population02Development cases03Protected cases04Frozen twin05Verification
Declared population

Define the cases, conditions and exclusions the model represents.

GUIDE / CORTEXSIM

Separate model refinement from independent verification. Select a component to explore its role in the evaluation.

CHAPTER 01

Define what the twin represents

A model represents a declared slice of behavior. Start with supported interfaces, relevant business objects and explicit invariants.

Specify the business objects and transitions needed for the decision. A payment model might represent initiation, processing and settlement under authored rules, while excluding an unsupported exception path. Those exclusions are part of the model definition. Without them, a reader may infer coverage that was never evaluated.

DESIGN TAKEAWAYS
  • Record modeled and unmodeled paths.
  • Distinguish source-derived values from authored assumptions.
  • Keep version and source pins with each model.
CHAPTER 02

Protect independent evidence

A deterministic replay does not prove general correctness. Define independent assertions and keep held-out cases separate from training and refinement.

Make the split reflect dependence between cases. Closely related scenarios, repeated source worlds or shared underlying events can make nominally separate cases less independent than their count suggests. Record the sampling frame and grouping assumptions, and protect expected outcomes from the process that proposes the candidate.

EXPLORE THE METHOD

Separate model refinement from independent verification

Declared population

Define the cases, conditions and exclusions the model represents.

DESIGN TAKEAWAYS
  • Split cases using declared criteria.
  • Preserve protected evidence and expected results.
  • Do not retune against the same hidden cohort until it passes.
CHAPTER 03

Read failures as information

Inspect mismatches, unsupported cases and model sensitivity. A source-linked failure is more useful than an unqualified confidence score.

Classify a mismatch before changing the model. It may expose a mapping error, an incomplete business rule, an unsupported case or an incorrect expected result. Keep the original evidence and explain any correction. Repeatedly tuning against a revealed verification set changes the meaning of that evidence.

ILLUSTRATIVE WORKED EXAMPLE

Does a synthetic settlement twin preserve the expected accounting outcome?

REHEARSE

Freeze the model and declared payment population. Use independent expected outcomes for settlement, exceptions and ordering, keeping protected cases out of parameter refinement.

REVIEW

Investigate a failed accounting assertion before accepting a faster candidate. Retain the failure even if a later model revision resolves it.

Synthetic evaluation illustration. No customer result or benchmark is claimed.
DESIGN TAKEAWAYS
  • Retain exceptions and reconciliation detail.
  • Review proposed refinements with a domain specialist.
  • State the conditions under which the result is valid.
CHAPTER 04

Use assertions that challenge the model

An assertion should express a business or operational property independently of the implementation under test. Examples include valid state transitions, preserved identity, reconciled totals and explicit handling of expected exceptions.

A deterministic run can reproduce the same mistake every time. Pair repeatability with assertions that can fail, and inspect whether the assertions cover the decision being considered. Large case counts alone do not establish representative coverage.

Three different kinds of evidence

EvidenceWhat it can tell youWhat it does not establish
Repeatable replayThe same declared setup reproduces its behavior.General business correctness.
Independent verificationThe frozen model meets specified assertions on declared cases.Coverage of untested customer conditions.
Domain reviewThe result is meaningful for a stated decision.Automatic production or professional approval.
DESIGN TAKEAWAYS
  • Declare the expected outcome before the candidate run.
  • Include exceptions and ordering-sensitive cases.
  • Keep the assertion owner separate from automated proposal generation.
CHAPTER 05

Read coverage and uncertainty together

Describe what was tested, what failed and which conditions remain unobserved. Separate training results from protected verification results, and retain the exact model and source identities associated with both.

Sensitivity analysis can reveal dependence on an authored parameter, but it does not turn that assumption into an observed fact. Explain which changes alter the conclusion and which new source evidence would reduce the uncertainty.

WHAT TRAVELS WITH THE RESULT
01Input references
02Model + scenario
03Complete outcomes
04Review + limits

Keep the evidence and its conditions together. A later success does not erase an earlier failure or broaden the original scope.

DESIGN TAKEAWAYS
  • Retain every qualification outcome.
  • Record dependence and sampling limitations.
  • Review numerical results and domain usefulness separately.
CHAPTER 06

Prepare a validation review

A validation review should reach a conclusion about a declared model version and intended use. It may accept a bounded claim, reject a candidate or request additional evidence.

Bring the model definition, independent assertions, complete outcome set and exclusions to the review. Customer-specific model acceptance and production authority remain separate from internal synthetic reference tests.

Review preparation

0 / 6 prepared
Selections stay in this page session. They do not approve an evaluation, grant permissions or establish production readiness.
DESIGN TAKEAWAYS
  • Pin the exact candidate and evidence.
  • Document failed and unsupported cases.
  • Name the reviewer and permitted use of the conclusion.
TURN THE GUIDE INTO A PLAN

Bring the question.
Build the evidence.

Define one meaningful decision, the conditions it depends on and the evidence your reviewers need.

Create your evaluation brief ↗