The pitfalls of using simplified measures to score ESG products

Detlef Glow warns on the drive to break down complex measures into a simple ratio

The finance industry is known for its high capabilities when it comes to models that predict future developments and/or measure the quality of an issuer or product.

So, it is not surprising that rating agencies, asset, and wealth managers have started to develop models and quantitative metrics to evaluate, forecast, and measure the impact of various ESG-related factors such as climate change within their portfolios with a single ratio or score. As a result, these ratios or scores are simplifying a highly complex network of different measures with the intent to make them understandable for a wider audience.

Generally speaking, the drive to break down a complex, and to some extent, unknown relation between different measures into a simple ratio or score is caused by human nature, since our brains like to simplify complex proceedings. This means it is quite understandable why asset managers and fund selectors are looking for simple measures to help them to find funds which suit their needs and meet the expectations of their customers.

Nevertheless, breaking down complex relations into a simplified measure has numerous pitfalls. First, one has to mention that everybody who wants to use a simplified measure needs to understand in detail the factors are taken into consideration and how they are incorporated in the respective measure, as this is necessary to use a simplified measure for the purpose for which it was made. In other words, one needs to do more than just look at the name of a measure to use it appropriately.

Secondly, every model needs data to run the respective calculations. Therefore, the developers of these models need to ensure that the calculations are based on high quality real data and not on appraised values, as otherwise the assumptions of the model are based on the assumptions of an analyst or model of a third-party data vendor. Unfortunately, there are more assumptions in ESG data than one would expect, since most data vendors want to close gaps in their data offerings by using approximated values or industry averages. This practice helps to make data sets more complete but adds another level of complexity and uncertainty to models which use this kind of data.

With regard to this, any regulatory initiative to standardise reporting duties at the fund level, as well as at the level of the underlying securities, should be much appreciated by all industry participants and investors. This is because standardised data will help retrieve more reliable results from the models to help investors make educated decisions. The standardisation of data and reporting duties may in addition help to prevent greenwashing since standardised data and reporting will increase the comparability of the measures taken and reported for the different financial products.  

That said, regulators should not try to regulate the models themselves, as any asset manager may have their own opinion on how they define sustainability and need individual models to evaluate and select the securities that suit their approach best. This means, in turn, investors will have to do proper due diligence to find those funds which use an investment approach that suit their needs best.

In other words, some ESG approaches will be more successful than others and the resulting fund flows will show which approach and/or model investors prefer.