How to monitor forecast accuracy: lag and granularity

Measuring the forecast quality is never neutral. 

Two organizations can obtain very different results simply by changing : 

• the chosen lag 

• the level of granularity 

These choices determine how the teams are evaluated, the perceived robustness of the process, and the operational decisions that follow. 

Lag: a key choice for demand planners

Lag is the time gap between when a forecast is produced and the demand period used to assess its accuracy.

In other words, assessing accuracy at a given lag amounts to answering a simple question: 
at what point is the forecast considered to drive binding business decisions ? 

This choice is closely linked to the decisions that the forecast supports. 
Indeed, production, procurement, allocation, or S&OP management do not operate on the same time horizons. The longer the lead time, the further out the relevant lag becomes. Conversely, a very short-term context often li;its the relevance of distant lag analysis. 

The higher the lag, the greater the uncertainty, which generally translates in lower accuracy scores.

The ideal lag does not exist 

In organizations with a mature S&OP process, companies typically do not rely on a single reference lag to measure forecast accuracy, but on a multi-horizon view, for example: 

• Long lag (M-6 to M-3) 
  → useful for industrial plan, capacity, CAPEX, raw material contracts 

• Intermediate lag (M-2 / M-1) 
  → useful for procurement, production, upstream transport 

• Short lag (M-0) 
  → useful for operational trade-offs, prioritization, backorders 

An organization that relies only on the short lag may tend to overestimate its performance. Conversely, an organization that evaluates only the long lag may underestimate its operational constraints. 

Focus on cumulative lag
The cumulative lag involves assessing accuracy not for a single period, but over the sum of the forecasts across the decision horizon. 

In practical terms, this involves comparing: 

• the sum of forecasts over N periods,

• to the sum of the actual demand observed over the same periods,

where N corresponds to the horizon beyond which decisions become difficult or costly to correct.

The cumulative lag allows to measure the actual financial exposure associated with decisions taken over a given horizon. Good period-to-period accuracy may conceal a systematic bias in total volumes, which can lead to significant deviations in inventory levels, tied-up cash, and service level.

By making the cumulative gap visible between forecast volumes and the demand actually observed, this approach allows for more precise adjustments to supplier commitments, the sizing of safety stocks, and the level of risk accepted, while limiting the costs related to excess inventory, supply shortages, and product obsolescence.

Summary

Thus, combining several lags with a cumulative view often enables a more robust understanding of forecast quality, directly linked to the decisions they support.

Examples of the use of different lags

Consider an industrial company with a formalized forecasting process used to plan production, commit to suppliers, and manage inventory. Depending on the horizon at which accuracy is analyzed, the discrepancies observed do not tell the same story and do not call for the same decisions. 

Long lag 
Over a long horizon, the company observes that accuracy deteriorates significantly beyond six months, regardless of the product family. This suggests issues with the underlying forecast assumptions, notably long-term trends and the seasonality applied to end-of-life products. The long lag is used here to adjust foundational choices for top management: modelling rules, supplier commitment horizons, and the level of capacity flexibility. 

Intermediate lag 
At an intermediate horizon, accuracy is satisfactory in the short term but systematically deteriorates between two and four months, just after the monthly reviews. The analysis reveals inconsistent manual adjustments and a lack of coordination between sales and supply. The intermediate lag then becomes a lever to structure the process, clarify arbitration rules, and strengthen collective discipline. 

Short lag 
In the very short term, sudden deviations appear during specific periods. They indicate operational effects that are poorly integrated into the forecast, such as weather, local promotions, or substitution behaviours. Accuracy at short lag is used as an early warning signal for operational teams to trigger rapid replenishment adjustments, rather than to assess the quality of the model. 

Cumulative lag 
Measured in isolation, the overall accuracy appears satisfactory. However, the analysis in cumulative lag over the next four months reveals a recurring underestimation of the total volumes to be covered. This indicator highlights a governance model relying on inventory buffers and late corrective actions, and enables to recalibrate supplier commitments as well as the level of risk accepted over the decision horizon. 

Granularity: different levels of detail for different decision choices 

Granularity corresponds to the level of detail at which forecast accuracy is measured: 
product, customer, store, size, channel, region, or any combination of relevant dimensions. 

In other words, choosing the right level of granularity means answering: at what level are decisions actually made? 

An accuracy metric calculated at an overly aggregated level can give an impression of satisfactory performance, while masking critical errors where operational decisions are made. Conversely, a level of granularity that is too fine can make the analysis noisy, difficult to interpret, and not very actionable if it is not linked to a clear decision-making lever. 

As with lag, the most mature organizations generally combine several complementary readings. 

For example: 

• an aggregated granularity to assess the overall consistency of the plan, 

• an intermediate granularity to manage trade-offs between families, customers, or channels, 

• a fine granularity to identify operational pressure points and trigger targeted actions. 

The finer the granularity, the more demanding the metric becomes and the more forecast accuracy tends to decrease, as errors no longer offset each other. 

Examples of using multiple granularities

Let us consider a company with a formalized forecasting process, used to plan production, engage suppliers, manage inventories, and guide commercial decisions. 

The level of granularity at which accuracy is analyzed determines the variances observed do not tell the same story and do not lead to the same decisions. 

Aggregated granularity 

At an aggregated level (by product category or at the overall company level), accuracy appears generally satisfactory and relatively stable over time. 

This analysis helps quantify the overall consistency of the plan at senior management level: market trends, general seasonality, capacity sizing, or overall level of supply. 

It is a good first step to ensure that the company’s strategy is properly configured. However, this assessment may conceal significant imbalances that only become visible at a more granular level. 

Intermediate granularity (TSU × Client) 

At an intermediate level, for example TSU × Client, the company observes recurring variances on certain product-client pairs. 

The analysis points to specific behaviors: some clients overconsume during commercial campaigns, others have stronger variability related to their own market, and certain product launches perform differently for different clients. 

This granularity becomes a lever for managing business trade-offs: prioritization of allocations between clients, adjustment of commercial commitments, improved coordination between sales and supply regarding the volumes that are actually secured. 

It allows to directly link forecast performance to concrete commercial decisions. 

Fine granularity 

At a very fine level (for example TSU × Client × Point of sale or warehouse), accuracy becomes more volatile but reveals operational tensions that are invisible elsewhere. 

Discrepancies appear in very local situations: local weather effects, occasional supplier stockouts, in-store promotional effects, substitutions between similar references. 

A simple framework for structuring measurement choices 

The choice of lag and granularity should not be interpreted separately. 
Lag indicates at what point in time the error becomes critical for the decision. 
Granularity indicates at which level this error actually produces its effects within the organization. 

In short, operational and strategic decisions always rely on the combination of these two dimensions.
The same performance can lead to very different conclusions according to the horizon at which it is measured and the level at which it is observed.

Ideally, analytical tools should make it possible to vary these parameters without rebuilding all the analyses, in order to align measurement with the actual decision rather than technical constraints. This is the logic behind the design of forecast accuracy at Pawa. This is the approach used at Pawa across all forecast accuracy measures.

Thus, for each scope, it may be beneficial to take the time to ask the following questions before selecting these parameters :

1. Which decision is based on the forecast?

2. From which horizon is this decision made, and at what point does it become difficult to revise?

3. At what level of aggregation is this decision actually made?

4. Which type of error has the greatest impact (volume, value, service)? 

5. What uncertainty / flexibility trade-off is acceptable? 

Related articles