William Nordhaus is one of the top experts on the economics of climate change and for a long time has worked on his series of DICE models (Dynamic Integrated Climate-Economy models) that examine how standard predictions of the effects of climate change on climate variables (such as global mean temperature) will impact the economy. His work is important because DICE is one of the three Integrated Assessment Models (IAMs) of climate-economic interaction that the government’s Interagency Working Group on Social Cost of Carbon uses in its analysis of the impacts of climate change. In other words, DICE is one of the three models used to calculate the Social Cost of Carbon (SCC), which is meant to represent the damage caused by one additional ton of CO₂ in the atmosphere. The SCC was devised so that the government could weigh different strategies to slow down CO₂ emissions and (in theory) reject any that cost society more than the damage caused by the CO₂ emissions they prevent. In this post I highlight a problem economist Robert Murphy had found in Nordhaus’s 1999 version of DICE and add an observation of my own. To be clear, Nordhaus has since updated his model, and the issue likely does not persist, though I have not yet read the 2013 DICE specifications.

A long time ago, I was reading Murphy’s 2009 article Rolling the DICE: William Nordhaus’s Dubious Case for a Carbon Tax, and I was struck by the section discussing the possible economic damages from low-risk catastrophic climate scenarios (such as the shutting down of the thermohaline circulation of heat and salt among the world’s oceans). These are, again, doomsday scenarios that scientists do not believe will happen, but to which they still assign a low probability of occurring. These numbers are needed to create the model’s damage function, which describes the loss of Gross World Product (GWP – also referred to as world GDP herein) due to increased temperatures. How did Nordhaus get his probabilities for these catastrophic scenarios? He relied on a survey of experts he published in 1994, worded as follows:

“Some people are concerned about a low-probability, high-consequence output of climate change. Assume by ‘high-consequence’ we mean a 25 percent loss of global income indefinitely, which is approximately the loss in output during the Great Depression. (a) What is the probability of such a high consequence outcome for scenario A, i.e., if the warming is 3 degrees C in 2090 as described above? (b) What is the probability of such a high consequence outcome for scenario B, i.e., if the warming is 6 degrees C in 2175 as described above? (c) What is the probability of such a high consequence outcome for scenario C, i.e., if the warming is 6 degrees in 2090 as described above?”

Yet the results from 1994 will not do in 1999, since Nordhaus claims that new research has placed increased emphasis on the possibility of these catastrophic events. What does he do, then? And this is where Murphy picks a fight with him. To cite Nordhaus,

“To reflect these growing concerns, we assume [that] the probability of a catastrophe with 2.5
 C warming is double the estimated probability for a 3
 C warming from the survey, that the probability associated with a 6 C warming is double the survey estimate, and that the percentage of global income lost in a catastrophe is 20 percent higher than the figure quoted in the survey.”

Murphy summarizes what has happened as follows:

“To restate the issue: Nordhaus in 1994 asked experts to estimate (among other things) the probability of global GDP loss of 25 percent in the event of 3.0
 C warming […]. The surveyed experts gave him their answers, from which he computed the mean. By 1999, further research had made these scenarios seem more plausible or catastrophic. So Nordhaus and Boyer took the original average of probabilities reported by the experts, doubled it, and then assigned this new figure as the probability for a 30 percent loss of GDP rather than the 25 percent the experts had been told to consider, for a less significant warming of 2.5
 C rather than the 3.0
 C mentioned in the original survey. […] More recent research suggests that at least some of these catastrophic scenarios were false alarms”

Why does it matter that Nordhaus apparently arbitrarily took the results of the survey of experts, doubled their answers, and then used these as probabilities for a different change in temperature and a different economic impact? Nordhaus claims that a 2.5C temperature increase would result in a loss of 1.50 percent of world GDP. And yet, 1.02 percentage points of these 1.50 percent are due to the low-probability catastrophic scenarios discussed above. This means that 68% of all the economic damages are based on numbers that Nordhaus arbitrarily manipulated.

So far so good – this has been Murphy’s critique (which I consider strong enough of a challenge to the model’s results). Yet I decided to dig a little deeper and look at how the 1994 numbers were obtained in the first place. They come from Nordhaus’s 1994 paper “Expert Opinion on Climate Change” in the American Scientist. In this paper, Nordhaus asked 22 experts their opinions on various questions relating to the effects of climate change on the economy. Three refused to participate, and of the nineteen remaining, nine were a mixture of economists, four were other social scientists, five were natural scientists and engineers, and the last one was Nordhaus himself. Again, it must be emphasized that this is not a review of literature of studies attempting to calibrate damage functions to observed warming, or at least to come up with plausible costs for disasters whose workings are scientifically described. No – the study merely asked nineteen experts in the field what they thought of the question [1].

We need to stop and consider how difficult the scenarios posed are: For example, in scenario A, respondents are asked how likely it is to have a 25% drop in world GDP in the year 2090 if there is 3 C warming. In a simple survey, respondents are asked to tackle a problem worthy of the whole literature – they are expected to take into account immigration flows, capital accumulation, technological adaptation, and a variety of low-probability catastrophic events we know of (and even ones we have not yet even conceived of) through the year 2090. They’re expected to be able to model the complex dynamics of the global market, advancements in science, and the catastrophic events due to precisely 3 C of warming. Nordhaus mentions in several places that there is much disagreement even among these experts on the economic impacts of catastrophic events, as seen in the spread of guesses for his questions. He notes that

“[o]ne respondent suggested whimsically that it was hardly surprising, given that the economists know little about the intricate web of natural ecosystems, whereas scientists know equally little about the incredible adaptability of human economies.”

And yet this does not seem whimsical at all, but a fundamental flaw in the methodology of the study. Natural scientists know very little about modeling economies and the ability of economies to respond to shocks. Similarly, economists are not intimately aware of the details of the consequences of climate change. Let’s look at what it would take to give an answer for Scenario A. One would first need to decide on the probability of various catastrophic outcomes due to 3 C in 2090 (a difficult enough endeavor). Then, one would need to estimate the economic impact of each catastrophic outcome to see whether it makes the 25% cutoff. The ones that do are all added together and this resulting probability is given as the answer.

In this deconstruction of the problem, we can see why the “whimsical” comment of the respondent above is not so whimsical at all – the question requires one to answer two very difficult subquestions, where each is focused on an entirely different field – one on climatology, the other one on economics. Only when these two are combined can the answer given be fully informed. How can an economist accurately answer the first part of the question – on the probabilities of different known (and even unknown!) catastrophic events. How can a natural scientist adequately guess the economic damage due to a climate event? Not only this, but the respondents were asked to give their subjective probabilities to these catastrophic events without having conducted any studies on this specific question, but merely as a matter of opinion. Yes – expert, informed opinion, but no matter how much of an expert someone is, when giving a probability to a 25% reduction in GWP due to 3 C warming in the future of 2090, which has unknown adaptive technology, her best guess will still be a fairly uninformed guess. Some respondents do mention technology as the big unknown:

“What is missing most is an understanding of the role of technology, of how society will change as technology advances. If we had been concerned with global warming in the 1890s, it would have concerned transportation by horses rather than automobiles. It is impossible to contemplate what society will be like a century from now as technology changes.”

Another writes

“Technology will develop to adjust to and accommodate many of the climatic changes and even provide approaches to countering warming effects. However, projecting technological changes a century or two into the future is hazardous at best. All we can really say is that there will be technological changes and that as in the past they will probably offset adverse effects to some degree.”

I cite this not to dismiss concerns over warming, but to show just the enormity of the task the experts were asked to perform – “impossible” by the words of some of them (in the case of forecasting technological changes). A major rainstorm in the distant past would have done much more damage to the livelihood of the public than it would today, for example. Why? We have sturdier buildings. This shows that the damage by natural events depends on the technological progress of a society very strongly. Nordhous quotes one of the respondents, who shares my main concern:

“I must tell you that I marvel that economists are willing to make quantitative estimates of economic consequences of climate change where the only measures available are estimates of global surface average increases in temperature. As [one] who has spent his career worrying about the vagaries of the dynamics of the atmosphere, I marvel that they can translate a single global number, an extremely poor surrogate for a description of the climatic conditions, into quantitative estimates of impacts of global economic conditions.”

Combining this with Murphy’s point that Nordhaus simply decides to double the values acquired from the survey and use them as answers to a different question, Nordhaus’s model (and its policy implications) becomes dubious. To reiterate what has happened: Nordhaus created a model where two-thirds of the damages from climate change are estimated from a survey asking a mix of scientists and economists their subjective opinion on how likely it is for a 3 C increase in 2090 to cause a 25% drop in GWP given unforeseeable technology, unforeseeable geographical population spread, unforeseeable gradual adaptation, and unforeseeable economic conditions, averages the opinions of these people, some of whom point out the precariousness of asking experts to weigh in on issues they know little about, and then takes this average, doubles it, and uses it to represent the chance of a catastrophic event resulting in a different scenario – a 30% drop in GWP due to a 2.5 C increase. In this case, “unsubstantiated guesswork” might be putting it lightly.

Once again, Nordhaus has a 2013 DICE model out, which likely does not have the above fatal mistakes. I did not intend this post to make light of the danger of climate change (with which I am not well-acquainted, not being a climate scientist myself), but merely to show the importance of looking at assumptions in our models, especially when they have such large impacts on the results of the model – and when the world looks to you for guidance on the issue.

Notes:

[1] Here’s an analysis I would consider to be an honest attempt at such an endeavor: Say we’re trying to assess the damages due to an enormous rise in sea levels. We’d begin with a map of the inundated areas and assess the current property values. This amount would then get adjusted to account for possible appreciation in the housing market. Afterwards, we might try to set up a model of housing investment, price changes, and population growth and migration to look at how much the demand for housing would increase across the entire world, and, hence, how much a reduction of usable housing area would cost the economy. We’d also need to take into account adaptive behavior that some areas might take to prevent the damages from a sea level rise (walls?). And so on. And this is just for sea level rise! In short, the survey respondents were asked to create a general equilibrium model of everything in the world and extend its predictions out to the year 2090 – a task worthy of a team of hundreds of economists and years of planning and debate (and even then, the model would still fall short).