Which forecast accuracy measures are available in Pawa?

Forecast accuracy indicators quantify the gap between the forecast and the actual demand, according to various complementary dimensions (article on lag and granularity) and address different business questions. 

Each indicator provides a different perspective for comparing forecasts with actual sales results. Understanding these measures allows users to diagnose issues, adjust planning strategies, and make better business decisions.

Overview of accuracy metrics in Pawa

To evaluate and continuously improve forecast performance, Pawa includes several key indicators in its reports through the Forecast accuracy measure.

Details of Forecast Accuracy Indicators

The detailed indicators below correspond to the most commonly used forecast accuracy measures for analyzing forecast quality. The other values presented serve as a calculation basis or reference.

Absolute Error (units)

The absolute error measures the absolute error between the forecast and the actual demand, in units.

MAE (%)

The MAE (%), also called WAPE, expresses the absolute error as a percentage of the demand. 

It answers the following question: 
“What is my relative error compared to the demand level?” 

It allows to: 

• compare products or scopes of different sizes,

• have a more synthetic and comparable view.  

Methodological note : Several approaches exist for calculating relative error, including MAPE, which Pawa does not offer. The MAE has been chosen in its weighted form because it provides a more stable measure across heterogeneous scopes, particularly when some volumes are low or zero.

Bias (%)

The Bias measures the direction of the forecast error.

It indicates whether the forecast is:

• structurally too optimistic with a positive bias (over-forecast),

• or too conservative with a negative bias (under-forecast).

Accuracy

In Pawa, Accuracy is defined as:

Accuracy = 100 − MAE (%)

In other words, it corresponds to an inverted reading of the mean relative error (MAE(%)).

This table illustrates the interpretation of Accuracy as an inverted measure of relative error: in contrast to the MAE(%), the higher the value, the closer the forecast is to the observed demand. 

Moreover, Accuracy can take on a negative value when the error exceeds the entirety of the actual demand:

This is the direct consequence of the mathematical definition of the indicator, as illustrated below.

For example, a company forecast 250 units, whereas actual demand was only 100 units.

In Pawa, Accuracy can be calculated in two ways : bounded or unbounded.

In bounded mode, the value of Accuracy is can’t go below 0%. This mode facilitates readability in dashboards and prevents the appearance of negative values.

In unbounded mode, Accuracy is not limited. It can become negative when the absolute error exceeds the actual observed value. This mode more accurately reflects the magnitude of the error, particularly in the event of strong over- or under-forecasting.

Score

The score in Pawa is primarily used as an evaluation criterion to compare and optimize the forecasts produced by the models.

It combines the magnitude of the error and the forecasting posture by adding MAE(%) and Bias(%).

FAQ

Why is my accuracy level so low when my total actuals are close to my forecast?

This is generally related to the level of granularity used to calculate accuracy.

When accuracy is assessed at a detailed level (by customer, by product, or by customer × TSU combination), each difference between the forecast and the actual demand is counted individually.
At this level, errors do not offset each other.

Conversely, accuracy calculated on an aggregated total can hide significant discrepancies, because over- and under-forecasts partially cancel each other out.

Example at customer level

A company has two customers:

• Customer A
  Forecast: 100
  Actual: 150

• Customer B
  Forecast: 100
  Actual: 50

In total:
Forecast = 200, Actual = 200 → total error = 0
Overall accuracy is excellent.

At customer level:
Each customer shows an error of 50 units.
Accuracy calculated by customer is low.

Why does accuracy decrease when moving from the total to a more detailed level?

The business use-cases of forecast accuracy change depending on the granularity used.

• At the total level, accuracy measures the overall consistency of the plan.

• At a detailed level, it measures the ability to correctly forecast each actual flow.

The more detailed the level, the more demanding the measure becomes.
More granular accuracy indicates a truer representation of operational reality.

My forecast accuracy at detailed level is much lower than at total level. What should I do?

Low accuracy at a detailed level may indicate:

• high demand variability,

• heterogeneous customer behaviors,

• or forecasting assumptions that are too global for this level of analysis.

This is often a starting point for refining segmentation, adjusting parameters, or reassessing the appropriate management level.

Related Articles