For a video overview of this update with IBISWorld's Chief Economist, Dr. Rick Buczyncski, watch IBISWorld Risk Recalibration 2021.

We regularly review our methodology for all our products to ensure you have the most accurate and up-to-date information.

Our most recent review of our Risk Rating model methodology involved a steering committee including a core group of Industry Risk Rating clients and an anonymous American bank regulator. This effort was part of an ongoing regimen we employ for quality control that includes data entry procedures, macro-driver selection, driver forecast accuracy, IT architecture, etc. This is all part of our GRC (Governance, Risk and Compliance) adherence.

Using various statistical procedures and our offline risk rating macro-scenario model, we ran numerous economic scenarios with some limited Monte Carlo simulations.

We uncovered 3 areas where recalibrations would enhance our risk model/methodology performance. These involved the 3 core components of our Total Industry Risk Ratings: sensitivity, structure and revenue growth scores.

Here is a summary of the 3 recalibrations incorporated into our risk rating model methodology. This unequivocally improved model performance in testing and was accepted by the steering committee. We implemented these changes on our website on July 1, 2021, and in our Industry Early Warning System on July 31, 2021.

The issue is whether to use raw-level data or percentage changes for drivers that determine sensitivity scores. The solution is categorizing key drivers based on the “stationarity” of their respective time series via the Augmented Dickey-Fuller test.

If the series is found to be “non-stationary,” then percent change is used. If stationary, then level data is employed. Testing uncovered 91 requiring external driver modifications, where 70 switched to percent change and 21 were shifted to raw-level data. This resulted in 525 industries in which sensitivity scores were revised.

The issue is how to incorporate shocks like the pandemic into structure scores that may lose relevance in times of unprecedented stress. The solution is to use our COVID-19 Shock Assessment Tool in concert with the macro-scenario model, allowing the creation of a shock-adjusted structure score that can be reactivated/redefined for future unanticipated shocks.

This was facilitated by incorporating qualitative and quantitative factors such as social distancing and essential business policies, trade exposures and other key macro drivers while taking account of how the pandemic’s forces can resonate along an industry’s supply chain. While the COVID-19 pandemic recedes, weights of its impact will diminish. If a new shock arises, we’ll engineer a new tool with assistance from a steering committee.

The issue is that during times of extreme turning points in industry revenue growth, overly positive growth scores can result. The effect has been to overstate the impact of partial recoveries in revenue growth, creating an upward bias on total risk scores. The solution is to systematically dampen the weight of growth scores in the formula in the overall risk score. The primary objective of this dampening will be to adjust growth scores for industries where recent growth rates mask a significantly lower pre-shock output by an industry.

We analyzed several other procedures in depth but did not uncover compelling judgmental or statistical reasons that warranted additional recalibrations. Regardless, all recalibration candidates will be reviewed annually or when dramatic shocks occur.

As the effects of COVID-19 wane, recalibrations #2 and #3 will self-adjust, and the process will be monitored on a monthly basis.

For technical documentation on the Risk Rating model recalibration, please reach out to your Client Relationship Manager.

For additional questions regarding our Risk Rating Model methodology update, please contact your Client Relationship Manager. If don’t have an IBISWorld account, please contact us to learn more about our membership options.