Design the pilot around the next commitment

The pilot has a decision to serve

Our Diagnostic, Working Sprints and Live Pilot connect investigation to an operating change. The diagnostic identifies the bottleneck and relevant uncertainty. The sprints build what is needed to test a response. The pilot examines whether it works under the agreed conditions. The depth and sequence depend on the engagement.

Within that work, the pilot agreement connects the next commitment to the evidence required for it. We start by writing what someone will be asked to approve after the test. More users, more spend and a client-facing launch are different commitments. Each needs evidence relevant to its consequences.

We also draw a boundary around what the result can establish. A successful test in one team may justify a broader test rather than an organisation-wide rollout. We record which conditions must remain true and which still need investigation. This keeps an encouraging local result from becoming a claim the evidence cannot support.

Make the uncertainty explicit

A proposal often contains several assumptions disguised as one initiative. A new workflow may assume that people will use it, that it reduces effort, that quality is preserved and that the receiving team can absorb the change. A pilot should investigate those assumptions rather than merely demonstrate that the workflow can run.

Write a hypothesis that connects a change to an observable effect. For example, using one shared evidence request should reduce duplicate requests without increasing missing information. This is a testable proposition, not a promise of success.

Identify the uncertainty most likely to change the investment decision. There is limited value in refining the interface if the underlying process requires authority nobody has agreed to delegate. Test the critical assumption early.

Record the current operating position

Before changing the process, define the measures and collect a usable baseline. Specify where the data comes from, which cases are included and how the calculation is made. A turnaround measure should say when the clock starts, when it stops and how waiting time is treated.

Record variation as well as averages. A process that is usually quick but occasionally fails badly may need a different intervention from one that is consistently slow. Look for differences in complexity, volume, personnel and client inputs.

Where feasible, compare similar cases or use an appropriate comparison group. Where that is not practical, document the limits of a before-and-after comparison. A change in workload, staffing or demand can affect the result independently of the initiative.

Write a pilot agreement

Set thresholds before the team sees the results. A threshold is a decision rule chosen for the context, not a universal benchmark. Explain why it is commercially or operationally meaningful. Avoid selecting a convenient number merely because it is easy to measure.

Use proportionate safeguards. In consequential work, preserve required controls and provide a recovery path. A pilot is not permission to ignore professional standards or expose clients to unapproved experiments.

Measure the whole effect

An improvement in one step may move effort elsewhere. Faster drafting may create more review work. A simpler intake form may increase follow-up questions. Track the receiving team as well as the team making the change.

Combine operating measures with direct observation and feedback. Ask participants what they actually did, where they worked around the new process and what remained unclear. Distinguish enthusiasm for the idea from evidence that it works.

Include the cost of setup, training, management and ongoing support. A time saving is not automatically cash saved. It may release capacity, reduce overtime or improve responsiveness, but the financial effect depends on how the organisation uses that capacity.

Decide whether to continue, adapt or stop

Review the evidence against the agreed rules. If the threshold is met, identify what must remain true at a larger scale. More users, different case types and new dependencies can change performance. Plan the next commitment rather than assuming the pilot has proved universal applicability.

If the result is mixed, isolate the assumption that failed. Adapt the intervention or run a more informative test. If the evidence does not justify continuation, close the initiative and record the learning. A stopped pilot can be a useful result when it prevents a larger unsupported commitment.

The Magenta Book is relevant because it treats evaluation as something to design into an intervention. It does not prescribe the same evaluation method for every business pilot. Choose a method proportionate to the decision and the consequences of getting it wrong.