Pick one task, write down what “better” means before you start, test it on normal and awkward cases with agreed limits, then decide from the evidence. “We deployed an agent” is not the result.
Shouldn’t a first pilot do something impressive?
A draft-only scope can still be useful, and it’s far easier to check. Prove one thing well first.
1. Choose the work.
Example. Prepare a weekly summary of service requests from approved test records.
Leave out sending messages, closing requests or changing customer records.
Practical check: One task, with a result someone can see.
2. Record where you’re starting from.
Today. How is it done now, and how much effort does it take?
Quality. Which facts, exceptions and sources must be right?
The decision. What result would justify carrying on, and what would stop the test?
Practical check: Write the criteria down before you see the outcome.
3. Build a small set of cases.
- A known example
- An approved case with a result someone can check.
- Missing or conflicting input
- It should ask, flag or stop, as agreed.
- A failure or a repeat
- Check that it recovers and doesn’t do the same thing twice.
Repeat the important cases, and record the version and the outcome each time.
4. Agree the limits before the run.
People. Name the business owner, the reviewer and the technical contact.
Scope. Set out the sources, the actions allowed and the fallback.
Resources. Agree the test window, a spending limit and the maximum effort. Limit how long it runs and how often it retries.
Practical check: Stop if it crosses a critical boundary, not only when the budget runs out.
5. Run it under control.
First use approved test inputs, without changing live records.
Then, only if authorized, try a limited amount of real work after the checks pass.
Record accepted results, exceptions, total staff effort and actual running costs.
Practical check: Expanding the scope is a separate decision, not an automatic reward.
The decision record.
- Evidence
- Which cases passed, failed or remain untested?
- Value
- Did the agreed improvement happen, at acceptable effort and cost?
- Next step
- Keep the scope, change and retest, expand carefully, or stop.
Practical check: A useful “not yet” is better than an unsupported rollout.
Read more: How to Scope a First AI Agent · Does AI Actually Save Time? How to Measure the Whole Job
The example is fictional. This is general guidance, not advice for your situation.