Analysis · Finance
A “90%” Forecast Range Still Missed More Than 13% of Outcomes
By AWEI · AI-compiled · Published · Analysis prepared · 1 source · doi.org
A forecast range can become more reliable and still fall short of its label. In Ruihan Zhao’s 2026 electricity forecasting methods study, calibration raised the share of test outcomes inside a nominal 90% prediction interval from 82.90% to 86.86%. The improvement was real within this test; the advertised coverage remained unmet.
This was a synthetic demonstration, using generated hourly demand covering 2019–2022, rather than observations from a utility. The series combined daily and seasonal cycles, temperature sensitivity, holidays and disturbances. Historical demand, weather and calendar inputs fed forecasting models, with data separated chronologically into training, calibration and final testing periods.
The interval models estimated lower and upper demand bounds. A separate calibration period then supplied a single adjustment that widened those bounds for testing. This distinction matters: calibration changed the uncertainty range without rebuilding the underlying forecasting model. It tested whether past interval misses could help correct later ones.
Wider ranges naturally make more outcomes easier to capture, so coverage alone cannot establish a useful improvement. Here, average interval width increased from 14.04 to 15.50, while the Winkler score fell from 24.28 to 23.56. That score penalizes both wide ranges and missed outcomes; its improvement supports a better trade-off on this particular test.
The remaining misses also had a pattern. Monthly diagnostics showed coverage below target in volatile summer and winter periods. The paper proposes changing demand patterns, extreme errors and the single fixed adjustment as possible explanations, not separately established causes. Adaptive calibration is suggested for future investigation, not demonstrated as a solution.
For readers assessing market forecasts, the transferable lesson is an evaluation question: does a stated probability describe a target or measured performance? Ask for coverage alongside range width and results across difficult periods. This synthetic study illustrates that distinction; it does not validate market estimates or establish financial benefits.
Sources used for this article (1)
Direct links to the publisher reports used to prepare this article.
- Source 1
- Load-Memory and Weather-Calendar Feature-Driven Short-Term Load Probabilistic Forecasting with Split Conformal Calibration Ruihan Zhao
