Analysis · Finance
Why 94.56% Classification Accuracy Does Not Validate an Inflation Index
By AWEI · AI-compiled · Published · Analysis prepared · 1 source · doi.org
An algorithm can classify electronics correctly almost 95% of the time without establishing that its inflation estimate is accurate. A 2025 methodological study by Martin Berki, Vanesa Andicsova and Milos Oravec illustrates the distinction: sorting products and measuring comparable price changes are separate challenges.
The researchers collected nearly 10,000 product records from a Slovak price comparison portal, including descriptions, specifications and price histories. They scheduled daily collection and trained the BERT language model to assign products to COICOP consumer spending categories. They used 80% of the dataset for training and 20% for validation.
Validation accuracy reached 94.56%, but performance was uneven. Macro precision, which gives each category equal importance, was 79.41%. More than half the records were labeled unclassifiable under the statistical office criteria used. Small categories were harder to distinguish: the model frequently confused SLR cameras with digital compact cameras. The headline accuracy therefore concealed weaknesses relevant to individual product groups.
The team then constructed electronics price indices using means or medians, with and without category weights, setting December 2021 to 100. By early 2024, the unweighted median index exceeded 280, while its weighted counterpart reached approximately 125. The official Harmonized Index of Consumer Prices for the category stayed near 100. Aggregation choices substantially changed the resulting picture.
Those differences do not establish that official inflation understated household costs. The study covered one portal and one sector, and its reported method does not demonstrate that changes in product mix or quality were fully separated from price changes. Suggested explanations involving sales, product launches and supply disruptions were hypotheses, not tested causal findings.
The practical lesson is to examine both stages of an automated benchmark: how reliably products are classified, and how their prices become an index. This method offers a supplementary view of online electronics prices; broader inflation claims require further validation.
Sources used for this article (1)
Direct links to the publisher reports used to prepare this article.
- Source 1
- NLP-enhanced inflation measurement using BERT and web scraping M. Berki, Vanesa Andicsová, Miloš Oravec
