Analysis · Finance
A Winning Forecast Can Still Miss the Moment That Matters
By AWEI · AI-compiled · Published · Analysis prepared · 1 source · doi.org
Four to six weeks before the influenza peak, a leading forecasting ensemble assigned only about 5–6% probability to the range around the eventual peak intensity. Its strong overall ranking concealed a weakness precisely when advance warning could matter most.
This finding comes from Nicholas G. Reich and colleagues’ 2019 study of U.S. seasonal influenza forecasts. Four research teams contributed 21 models. Their target was the population-weighted percentage of outpatient visits involving influenza-like illness, rather than a direct count of confirmed influenza infections.
The researchers compared five ways to combine models using seven historical seasons. They repeatedly held out one season to evaluate weights learned from the others. Before the 2017/2018 season began, they selected a combination with separate weights for short-term and seasonal predictions. Those weights remained fixed while teams submitted weekly forecasts in real time.
The selected ensemble outperformed every individual component on average and ranked second in the wider CDC competition. Its forecast score was 0.337, compared with 0.321 for the CDC’s simple average of submitted models. These scores summarize probabilities assigned near observed outcomes; they are not percentages of predictions that were correct. The CDC comparison also involved a different collection of models, limiting what it isolates about weighting alone.
Timing changed the picture. At and after the peak, scores for peak intensity exceeded 0.70, versus roughly 0.05–0.06 several weeks beforehand. The evaluation included forecasts after the peak. There was only one prospective test season, some historical data lacked their original release versions, and the authors acknowledged that the scoring rule could reward strategic changes to forecast distributions.
For readers assessing market forecasts, the transferable lesson concerns evaluation, not evidence of financial performance: inspect the target definition, comparison baseline and results at the decision-relevant horizon. A high average score supports a narrower claim than reliable advance warning.
Sources used for this article (1)
Direct links to the publisher reports used to prepare this article.
- Source 1
- Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. ['Nicholas G Reich', 'Craig McGowan', 'Teresa K. Yamana', 'Abhinav Tushar', 'Evan L Ray', 'Dave Osthus', 'Sasikiran Kandula', 'Logan Brooks', 'Willow Crawford-Crudell', 'Graham Casey Gibson', 'Evan Mo
