I don’t understand the business of political risk assessment.
Risk assessment, I get. I know something about politics, too. What I don’t get is the success of a business model that seems to boil down to: “Trust us, we know our stuff.”
Political risk consultancies offer various services, only some of which involve forecasting. Here, I’m talking specifically about the forecasting parts. As far as I can tell, none of the major purveyors of political risk assessment systematically and transparently assesses the accuracy of the forecasts they produce.
(If you know I’m wrong about this, please use the Comments section to set me straight—preferably with links to relevant documents.)
This kind of assessment should not be hard to do, especially in cases where forecasts are quantified, as a probability or in some other scalar form. The Economist Intelligence Unit, for example, assesses risks over the next two years in each of 10 areas on a 100-point scale, where 100 is maximum risk. Do bad things really happen more often in countries ranking higher on that scale? Is an 80 twice as risky as a 40, or is risk distributed differently across that range? These are questions that can be answered in a transparent way with statistics comparing those rankings to data on relevant events. As far as I know, though, this analysis either isn’t happening, or the results from it aren’t being shared with the consumers of those forecasts.
I don’t mean to pick on the EIU here. In fact, they are unusual in the field for making their risk assessments available for free and describing the process they use to produce those forecasts (the judgments of analysts working in regional teams) in some detail.
What puzzles me is that this opacity on past performance is standard practice and the customers don’t seem to mind. Basing business decisions on forecasts without knowing anything about how accurate the forecasts are is like continuing to take investment advice from a financial adviser without ever checking to see how your portfolio is doing. The stakes may be quite high, and there is an empirical answer to your question, but you just don’t bother to find out what it is.
Perhaps the clients paying for these services should take a page from the book on arms control in the 1980s and adopt a new slogan of their own: “Trust, but verify.”