Decide which decision the forecast has to support. Estimating demand and deciding what to order are two separate jobs, with different data and different tests.
Can we start before our data is fixed?
It depends on the decision. First define the output, the time horizon and what the pilot has to support. “Improve forecasting” isn’t a complete brief.
1. Two separate jobs.
The demand question. What quantity is likely to be needed, for which item and location, over which period?
The supply question. Given that estimate, what should be ordered or made, considering stock, open orders, lead times and constraints?
Practical check: A forecast is an input, not a purchase instruction.
2. Find the gaps that change the answer.
Definitions. Agree units, dates, item mappings and what is being forecast.
History. Identify missing periods, returns, promotions and stockouts. Recorded sales may not show demand you couldn’t meet.
Scope. Start with a suitable subset. A clean pilot isn’t evidence that every item is covered.
Practical check: Don’t label missing observations as zero demand.
3. Test without hindsight.
Compare it with the current method and a simple benchmark. Test on later periods that weren’t used to build the method, using only what was known at the forecast date. Judge it over the horizon the decision needs.
Practical check: Fitting history well is not proof of forecasting well.
4. Keep the roles visible.
The forecasting method. Name the model and its assumptions. A conversational explanation alone is not a tested forecast.
AI assistance. Prepare data checks, summarize exceptions or help develop analysis that is then validated.
Planning review. Look at sustained over- or under-prediction alongside service, inventory and operating constraints.
A pilot question small enough to test.
For this item group, horizon and data window, does the proposed method improve the agreed forecast measure against the baseline? Then test separately whether the planning decision that follows improves the business outcome. Record what stayed outside the test.
Read more: How to Run a Small AI Agent Pilot · Inventory Doesn’t Match the System: Can AI Help Find Out Why?
Forecast accuracy and supply performance are related, not identical. This is general guidance, not advice for your situation.