Analysis · Finance
The Exchange Rate Forecast That Lost to No Change
By AWEI · AI-compiled · Published · Analysis prepared · 1 source · doi.org
Repeating the last observed exchange rate beat a complex forecasting model at every forecast horizon tested in one comparison. The revealing detail: this simple competitor had been missing from the original evaluation. A model can lead a published ranking while still losing to a basic baseline.
Hansika Hewamalage, Klaus Ackermann and Christoph Bergmeir presented this example in their methodological review, Forecast evaluation for data scientists: common pitfalls and best practices, first published online in 2022. The paper combines guidance on evaluation with illustrative simulations and model comparisons; it does not test an investment strategy.
The authors applied a no-change forecast to an exchange rate dataset using the evaluation setup previously used for Autoformer, a neural forecasting model. They also reran Autoformer. Across horizons of 96, 192, 336 and 720 time steps, the simple forecast produced lower mean absolute and mean squared errors than both the original reported results and their rerun.
At 96 steps, mean squared error was 0.078 for no change and 0.149 for the Autoformer rerun. These are scores under that evaluation setup, not percentage returns. The comparison demonstrates how adding an omitted benchmark can change the interpretation of a model's performance without changing its predictions.
The limits matter. This result concerns a particular dataset, model and setup; it does not establish that complex models always fail or that every market follows a random walk. The paper also focuses on point forecasts, rather than full probability distributions. Predictive accuracy alone establishes neither a causal explanation of market movements nor trading profitability.
For readers assessing market forecasts, the useful question is what the claimed accuracy improves upon. Look for a relevant simple baseline, evaluated over the same horizon with the same error measure. Beating that baseline supports a narrower claim of predictive improvement under the tested conditions; a sophisticated model name cannot substitute for the comparison.
Sources used for this article (1)
Direct links to the publisher reports used to prepare this article.
- Source 1
- Forecast evaluation for data scientists: common pitfalls and best practices Hansika Hewamalage, Klaus Ackermann, Christoph Bergmeir
