Skip to content
WorkspaceLet's simulate→
← Resource library
THE EVALUATION PLAYBOOK / 6 min

A better question.
A stronger decision.

A practical path from an important business question to an evidence-based acceptance decision.

6 practical chaptersWorked exampleReview checklist
CORTEXSIM / VISUAL FIELD NOTESSELECT A COMPONENT
NARROW THE QUESTIONSTRENGTHEN THE EVIDENCE01Question02Baseline03Candidate04Evidence05Decision
Question

Name the decision and the person who owns its consequences.

PLAYBOOK / CORTEXSIM

From business question to acceptance evidence. Select a component to explore its role in the evaluation.

CHAPTER 01

Start with one decision

Choose a workflow with a named owner and a concrete decision: a migration, a recovery policy or a capacity change. Bound the interfaces and operating conditions before expanding the scope.

A useful scope can be expressed as a comparison: under the same declared demand, does this candidate preserve the required business result while changing an operational outcome? Choose a boundary the team can inspect. A broad ambition such as “improve resilience” needs a specific workflow, interruption and acceptance condition before it becomes an experiment.

DESIGN TAKEAWAYS
  • Record the baseline and the proposed change.
  • Define business correctness separately from performance.
  • Name the people responsible for acceptance.
CHAPTER 02

Agree the evidence contract

Identify what is observed, what is authored and what remains unknown. Use independent assertions that can fail even when the simulation is repeatable.

Write the evidence contract before observing results. Include the source population, workload, expected outcomes, execution mode and resource limits. Label synthetic cases and authored rules. Agree which evidence can support the immediate decision and which questions still require measurements from the intended deployment.

EXPLORE THE METHOD

From business question to acceptance evidence

Question

Name the decision and the person who owns its consequences.

DESIGN TAKEAWAYS
  • Pin source snapshots and model revisions.
  • Keep training and verification evidence separate.
  • Record excluded interfaces and untested conditions.
CHAPTER 03

Make the review actionable

Compare the candidate against the baseline under the same declared workload. A useful outcome can be a rejected proposal or a better-defined qualification gap.

Review the complete outcome set, including failed, unsupported and withheld cases. A candidate that improves a headline metric but violates a business invariant has not passed. Make the review explain what changed, why the result matters and the conditions under which the conclusion could fail.

ILLUSTRATIVE WORKED EXAMPLE

Would a revised processing path improve a synthetic payment workflow?

REHEARSE

Hold the declared payment population and correctness assertions fixed. Compare the current path with one supported candidate change, retaining completion, exceptions and timing context.

REVIEW

A useful result explains both any improvement and the cases that regressed. Native payment-system compatibility and production performance need separate evidence.

Synthetic evaluation illustration. No customer result or benchmark is claimed.
DESIGN TAKEAWAYS
  • Retain scenario inputs and result references.
  • Review exceptions with domain and platform owners.
  • Set the next acceptance gate before broader adoption.
CHAPTER 04

Choose a comparison that can fail

Good acceptance criteria can reject an attractive proposal. Separate necessary business conditions from desired operational improvements, and agree how exceptions are handled. The candidate should face the same declared workload as the baseline.

A successful rehearsal may demonstrate a narrow improvement, expose a missing model dependency or show that the proposed change is not useful. Keep all three outcomes available; otherwise the evaluation becomes a search for a preferred answer.

Read the result as a decision

ObservationInterpretationNext step
Correctness and target passEvidence supports the declared scenario.Review usefulness and scope before expanding.
Faster, but an invariant failsThe candidate breaks a required business condition.Reject or revise it with a new recorded comparison.
Inputs or coverage are incompleteThe conclusion is limited or inconclusive.Close the evidence gap before making a broader claim.
DESIGN TAKEAWAYS
  • Freeze correctness assertions before the run.
  • Retain the declared population and exclusions.
  • Review tradeoffs rather than collapsing them into one score.
CHAPTER 05

Build a reviewable evidence package

A reviewer should be able to trace the decision from inputs to results without reconstructing the experiment from memory. Keep the source snapshot, baseline and candidate model identities, scenario settings, execution records and verification outcomes together.

Attach the assumptions that materially affect the conclusion. If a result depends on synthetic demand, a declared repair time or an unsupported interface, carry that limitation into the review. A local experiment does not establish customer production acceptance.

WHAT TRAVELS WITH THE RESULT
01Input references
02Model + scenario
03Complete outcomes
04Review + limits

Keep the evidence and its conditions together. A later success does not erase an earlier failure or broaden the original scope.

DESIGN TAKEAWAYS
  • Preserve failed runs and unresolved differences.
  • Name the domain reviewer and platform owner.
  • Record the next evidence needed to broaden the conclusion.
CHAPTER 06

Prepare the evaluation brief

Turn the review into a bounded next step. The brief should identify one accountable owner, one decision, a declared environment and the evidence required to judge the result.

Use the checklist to prepare the conversation. Its selections describe your planning progress only. They do not change platform controls, approve an operation or establish that an evaluation has passed.

Review preparation

0 / 6 prepared
Selections stay in this page session. They do not approve an evaluation, grant permissions or establish production readiness.
DESIGN TAKEAWAYS
  • Agree what the evaluation can resolve.
  • Document what remains outside its scope.
  • Choose the next acceptance gate explicitly.
TURN THE GUIDE INTO A PLAN

Bring the question.
Build the evidence.

Define one meaningful decision, the conditions it depends on and the evidence your reviewers need.

Create your evaluation brief ↗