Research Affiliates’ Harvey: Backtesting is a huge problem in the industry

Tom Eckett

a man in a suit

Campbell Harvey, Professor of Finance at Duke University and senior adviser to Research Affiliates, has warned investors need to be very careful when investing in smart beta products even when there is academic research supporting the factor.

Speaking to ETF Stream, Harvey (pictured) said there is a huge problem in both academia and the asset management industry around backtesting and data mining.

The reason for this issue, he stressed, is academics are incentivised to find factors that work otherwise their results will not be published.

“Academics know the journal they submit to will only publish the results if they are interesting,” he continued. “Therefore, there is an incentive for academics to data mine as they need results published.”

Harvey, who joined Research Affiliates in 2017, explained this problem was rife across the whole of academia and not just in finance. “In academic literature, you have this skewed distribution of discoveries because in most cases there is nothing there.”

This then becomes an issue for investors as providers package these factors based on the suspect academic research into a product. Harvey added he has documented around 400 factors that have been published by academia.

"If investors see a factor that does not have a plausible economic story attached to it then you should be very suspicious," the Duke University professor said.

Backtesting, he added, is a “huge problem” within the industry as firms can pick out any of these factors based on academic research and turn them into a product.

Research Affiliates: Alice’s Adventures in Factorland

“We have a backtesting problem in both the academia and the practice,” he continued. “Even though it might look good in your backtest, it is unlikely to survive in the long term.

“Academic research is used for marketing and given the evidence we have about false findings, the industry needs to be more sceptical.”

The Research Affiliates senior adviser said this was a major “blunder” investors made when looking at the different strategies available.

“It is very dangerous,” Harvey continued. “Investors need to take this into account when choosing a product.”

Inverted yield curve

Harvey, who was the first person to link yield curve inversions to recession in his 1986 dissertation, said this is the most dangerous yield curve inversion in history because it is acting as a self-fulfilling prophecy.

Because investors and executives are now aware of the inverted yield as a recession indicator, he explained this is causing them to reduce spending potentially leading to a slowdown.

"It is brutally dangerous to ignore an indicator with a perfect track record. However, because consumers are aware of this indicator it could lead to a softer landing."

Featured in this article


No ETFs to show.