Could an Early Warning System Have Predicted the Financial Crisis?

The 2008 global financial crisis has led to renewed calls for “early warning models” to reduce the risks of future crises. But this column says that few of the characteristics suggested as potential causes of the crisis actually help predict the intensity and severity of the crisis across countries. That bodes poorly for the performance of future early warning models.

The 2008 global financial crisis is notable for a number of reasons, including most obviously its severity and speed. The international span of the crisis has also been remarkable; essentially all the industrialised countries have been affected, as well as a large number of developing economies. The crisis has led to renewed interest in the creation of early warning models capable of predicting and hopefully mitigating severity of future crises of this type. IMF Managing Director Dominique Strauss-Kahn (2008) recently noted, “We at the Fund have already begun intensifying our early warning capabilities and will be strengthening our collaboration with others involved in this area.”

The renewed interest in early warning models raises the question of how these models would have actually performed in predicting the current crisis. Historically, economists have not had a particularly good track record at predicting the timing of crises, which is one of the objectives of an early warning system. However, economists have done somewhat better at modelling the incidence of crises across countries (e.g. Berg, et al 2004). That is, it has been easier to predict crisis intensity across countries than across time.

New research on early warning models

In a recent paper, we empirically model the cross-country incidence of the financial crisis (Rose and Spiegel 2009). Because the time-series component of an early warning system has proven harder to predict, we view the ability to predict relative performance in the cross section as a necessary, but not sufficient, condition for early warning models to be successful. We estimate a “MIMIC” (Multiple-Indicator Multiple Cause) model (Goldberger 1972), which we apply to a cross-sectional data set of 107 countries. The MIMIC specification explicitly acknowledges that the severity of a financial crisis is a continuous, rather than a discrete phenomenon, and one that can only be observed with error.

Our results yield a plausible set of estimates for the severity of the crisis across countries. That is, we find that Iceland and Latvia were hit more severely in 2008 than China. However, we have less success in linking crisis severity to its causes. We examine over sixty factors that have been advanced in the literature as potential causes of the 2008 credit crisis, but few emerge as robust predictors of its severity. Indeed, we find only one variable – the size of the equity market run-up prior to the crisis – that is a robust predictor of crisis severity. While the performance of this variable is intuitive, other equally plausible variables fail to perform well, such as the magnitude of real estate price appreciation or the quality of the regulatory environment. As early warning models must predict both the cross-country incidence of crises as well as their timing, our analysis bodes poorly for their success.

Measuring the incidence and severity of the crisis

Identifying the incidence of a financial crisis, let alone determining its severity, is no simple matter. Potentially serious measurement error is inherently present. We consider four observable indicators of the crisis, and model the incidence and severity of the crisis as being a latent variable that can be linked to these variables. Our first measure of the crisis is 2008 real GDP growth. We also consider three financial variables:

1. the percentage change in a broad measure of the national stock market over 2008;
2. the 2008 percentage change in the SDR exchange rate; and
3. the change in a country’s creditworthiness rating from Institutional Investor.

Our four different variables measuring the severity of the crisis are strongly positively correlated with each other and deliver broadly similar rankings. A number of countries have been particularly hard hit by the crisis, and these show up at the top of our list. These include Iceland, whose fall was particularly striking, as well as a number of other countries that have also been hit hard, including the Baltic countries (Estonia, Latvia and Lithuania), the Ukraine, Ireland, Korea, New Zealand, the UK, and Hungary. All these countries appear towards the top of our list of crisis countries; the plausibility of the extreme cases lends credibility to this exercise.

Potential determinants of the crisis

We include a myriad of potential causal variables, addressing most of the posited causes advanced in the voluminous literature on the origins of the global financial crisis. Throughout our analysis, we condition on a country’s size and income.

We then identify a number of categories of characteristics that may have affected performance during the global crisis. These include countries’ financial policies and conditions, appreciation in local equity and real estate markets, global imbalances, domestic macroeconomic policies, domestic institutions, and geographic characteristics.

For each of these categories of characteristics, we introduce a number of alternative observable indicators, dated from 2006 or earlier, one at a time. For example, as a measure of the quality of a nation’s financial regulatory regime, we include the share of bank deposits held in privately owned banks, measures of credit market controls, and a summary score on the quality of regulation in credit markets. We also include a number of measures of overall capital stringency, the ability of regulators to take prompt corrective action, a capital regulatory index, and indices of official supervisory power, restructuring power, and the power to declare insolvency. In all, we consider 65 potential causal variables.

Linking potential causes of the crisis to its incidence

Using our MIMIC specification, we estimate our latent variable from the four underlying crisis indicators and simultaneously link it to size and income as potential causes of the crisis. We find that size has no significant impact on the incidence of crises across countries, while income has a significantly negative impact; richer countries experience more dramatic crises. We then add each of our potential causes to the default MIMIC model one by one, retaining size and income as causes throughout.

Our results are disappointing and weak, as few of our potential causes have a statistically significant impact on crisis incidence. For example, the percentage change in real estate prices between 2003 and 2006 does not have an effect that is statistically different from zero at conventional levels. The same is true of almost all of the causes we consider.

It should be stressed that this observed weakness is not an artefact of the MIMIC methodology. For example, Figure 1 plots one of our measures of the adequacy of the financial regulatory framework – the capital regulatory index of Barth, Caprio, and Levine (2003) – against each of the four crisis indicator variables. Regulatory conditions are commonly cited as determinants of the relative performance of countries during the economic crisis. However, even in a simple scatter plot, it is apparent that there is no systematic relationship between this commonly-cited variable and our crisis indicators. The other figures are analogues that consider a number of other potential causes of the crisis that have been much discussed, including domestic credit growth, real estate price appreciation, and bank capital adequacy.

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

There are a few exceptions to our generally weak results. Countries that experienced a large run-up in the stock market were more likely to be hit by the 2008 crisis. Countries with larger current account deficits and fewer reserves relative to short-term debt were also more vulnerable. There is weaker evidence that high credit growth and a more levered banking sector are also associated with the severity of the crisis. We also know that some of the Eastern European and Baltic countries have been hard-hit, and this is apparent when we include geographic dummies.

Nevertheless, few of our potential causes have strong effects that are robust across slightly different specifications of our MIMIC model. Overall, our results suggest that measurable pre-existing conditions across countries had little common impact on the relative severity of these countries’ crisis experiences. These results indicate that creating an empirically viable early warning system will be challenging – such a system must conquer all the problems we faced, while also being able to predict the timing of future crises out-of-sample.

Conclusion

Success in predicting crises in the cross-section is a necessary (but far from sufficient) condition for any reliable early warning system, which must also confront predicting the timing of crises out-of-sample. We examine a large number of potential explanatory variables for the current crisis that have been discussed in the literature. However, almost none of our posited variables are statistically significant determinants of crisis severity. While we can model the incidence of the crisis reasonably well, we are unable to link the severity of the crisis across countries to its causes.

There can be three reasons for our predictive failure. First, the causes of the 2008 crisis might differ across countries. Alternatively, the 2008 crisis might be the result of a truly global shock, so long as its incidence varied across countries in a way that is unrelated to the regulatory, financial, and macroeconomic “fundamentals” we consider. Finally, the shock might be a national one (plausibly originating in the US) that spread contagiously across countries.

All these interpretations bode poorly for the success of early-warning models going forward. If the causes of the crises differ across countries, there is little hope of finding a common statistical model to predict them. The same holds if common or contagious shocks are critical, but a country’s ability to withstand a global or spreading shock is unrelated to fundamentals. We conclude that it will be challenging to build a plausible statistical model that can predict financial crises similar to that of 2008.

References

•Barth, James, Gerard Caprio, and Ross Levine, (2005), Rethinking Bank Regulation: Till Angels Govern, Cambridge University Press.
•Berg, Andrew, Eduardo Borensztein, and Catherine Patillo, (2004), “Assessing Early Warning Systems: How Have They Worked in Practice?” IMF Working Paper no. WP/04/52, March.
•Goldberger, Arthur S. (1972) “Structural Equation Methods in the Social Sciences” Econometrica, 40, 979-1001.
•Rose, Andrew K. and Mark M. Spiegel, (2009), “Cross-Country Causes and Consequences of the 2008 Crisis: Early Warning,” CEPR Discussion Paper 7354
•Strauss-Kahn, Dominique, (2008), “Letter from IMF Managing Director Dominique Strauss-Kahn to the G20 Heads of Governments and Institutions,” 9 November.

• Become A Funded Futures Trader