Skip to content
WorkspaceLet's simulate→
← Resource library
THE AI CHANGE REVIEW PLAYBOOK / 6 min

A proposal is a start.
Evidence is the gate.

Move from a plausible proposal to an independently testable decision.

6 practical chaptersWorked exampleReview checklist
CORTEXSIM / VISUAL FIELD NOTESSELECT A COMPONENT
WITHHOLDTHE PROPOSAL CANNOTREDEFINE ITS OWN TEST01Objective02Proposal03Bounded trial04Independent gate05Independent review
Objective

Constrain the question, relevant evidence and request budget.

PLAYBOOK / CORTEXSIM

Proposal, independent verification and human review. Select a component to explore its role in the evaluation.

CHAPTER 01

Constrain the question

Give the planning process a defined objective, relevant evidence and protected inputs. Permission to propose does not grant permission to execute.

Express the objective as a bounded question with supported parameters. Record what the planner may change, which inputs are protected and how many trials or requests are permitted. Giving a model more context does not give it more authority over an environment.

DESIGN TAKEAWAYS
  • Set trial and request bounds.
  • Keep acceptance assertions outside model control.
  • Record the provider and proposal provenance.
CHAPTER 02

Challenge the alternative

Review proposed values and assumptions before running bounded trials. Preserve the original and edited versions so a reviewer can understand the change.

Inspect the proposal for unsupported fields, uncited assumptions and changes outside the declared objective. Keep the original response and any reviewed edits distinct. The evidence should explain which candidate was actually evaluated, not just which explanation sounded convincing.

EXPLORE THE METHOD

Proposal, independent verification and human review

Objective

Constrain the question, relevant evidence and request budget.

DESIGN TAKEAWAYS
  • Inspect unsupported or uncited claims.
  • Keep baseline and targets fixed.
  • Evaluate training and independent held-out evidence.
CHAPTER 03

Keep abstention visible

A malformed response, provider outage or failed verification is not a recommendation. Withhold alternatives that do not satisfy the declared gate.

Abstention is a useful result when the provider is unavailable, the response is malformed or the candidate fails verification. Do not substitute a new successful response for a consumed failed outcome and then report only the success. Keep the full decision history visible.

ILLUSTRATIVE WORKED EXAMPLE

Should an AI-proposed scheduling change be considered further?

REHEARSE

Limit the planner to supported parameters and a declared trial budget. Freeze the baseline and assertions, then retain the candidate values, trial results and independent verification outcome.

REVIEW

A passing numerical result goes to domain review. A failed or unsupported proposal remains withheld with its evidence, rather than being presented as an accepted recommendation.

Synthetic evaluation illustration. No customer result or benchmark is claimed.
DESIGN TAKEAWAYS
  • Retain failed and withheld outcomes.
  • Require independent review where the operation demands it.
  • Separate simulation evidence from real-world authorization.
CHAPTER 04

Keep the test outside the proposal

A candidate should not be allowed to redefine the assertion that judges it. Freeze the baseline, independent expected outcomes and permitted parameter bounds before qualification. Separate development trials from protected evaluation.

A plausible explanation can help a reviewer understand intent, but it is not a substitute for executable checks. Evaluate the actual candidate values and retained results. If evidence is incomplete, report the gap rather than upgrading the proposal into a recommendation.

Keep outcome states meaningful

StateMeaningReview response
Malformed or unsupportedThe proposal cannot be evaluated within the contract.Retain the response and withhold execution.
Verification failedThe candidate did not meet the declared gate.Inspect the failure without rewriting its history.
Checks passedThe candidate met specified numerical conditions.Review usefulness and any separate action authority.
DESIGN TAKEAWAYS
  • Protect acceptance assertions from candidate control.
  • Pin model and evidence versions.
  • Keep development and qualification outcomes distinct.
CHAPTER 05

Review usefulness independently

Numerical success and practical usefulness answer different questions. A domain reviewer should judge whether the change addresses the intended problem and whether its tradeoffs are acceptable under the stated assumptions.

External action requires its own authority. The surrounding platform uses independently approved identities and supported action profiles; the AI cannot approve its own plan or turn a simulation result into customer production permission.

WHAT TRAVELS WITH THE RESULT
01Input references
02Model + scenario
03Complete outcomes
04Review + limits

Keep the evidence and its conditions together. A later success does not erase an earlier failure or broaden the original scope.

DESIGN TAKEAWAYS
  • Inspect the actual outcome and its assumptions.
  • Retain the reviewer decision separately from numerical checks.
  • Bind any later action to the required independent approval.
CHAPTER 06

Prepare the proposal review

Bring the objective, original proposal, candidate values, fixed baseline and complete outcome set to the reviewer. Include malformed, failed and withheld cases where they affect the conclusion.

The checklist records review preparation, not approval. A bounded internal experiment can inform the next decision while leaving customer integration, domain acceptance and production action authority unresolved.

Review preparation

0 / 6 prepared
Selections stay in this page session. They do not approve an evaluation, grant permissions or establish production readiness.
DESIGN TAKEAWAYS
  • Name the reviewer and permitted decision.
  • Retain provenance and failures.
  • Record remaining qualification and action boundaries.
TURN THE GUIDE INTO A PLAN

Bring the question.
Build the evidence.

Define one meaningful decision, the conditions it depends on and the evidence your reviewers need.

Create your evaluation brief ↗