‘Model Risk’ is Nothing New

Since the vivid failure of financial risk models in 2007-2008, regulators, auditors, IT professionals, and others have begun pressuring for estimates of “model risk.” Looking back, it is clear that financial engineers underestimated the risk of a crash in housing prices and, more importantly, did not model the fragility of the interconnections among leveraged banks and the shadow finance system. Put simply, “model risk” is the risk that a risk model is wrong.

The whole idea of “model risk” comes from financial engineering’s quasi-religious belief that their models can be made to match reality by sufficiently vigorous tweaking of assumptions and parameters. By contrast, real engineers know that models are always wrong—that is why they are called “models.”

In 1964 I was an engineer at NASA’s Jet Propulsion Laboratory (JPL). One of my responsibilities was estimating the reliability of the Mariner Mars spacecraft. This estimate was part of NASA’s management system and the U.S. Congress’ review of NASA. The methodology was to estimate the reliability—one minus the chance of failure—of each component of the spacecraft and to then combine these numbers into an overall reliability. As a very simplified example, if the radio had a reliability of 0.99 and the power supply had a reliability of 0.98, then the two together had a 0.99 x 0.98 = 0.9702 reliability. Looking at thousands of components, my estimate was that the whole spacecraft had a 72% chance of completing its mission to Mars.

Mariner 3 and 4 were both launched in the fall of 1964. Mariner 3 failed because the cover (the shroud) protecting the craft during its ascent to orbit collapsed and would not come free once the vehicle was in space. Looking over the telemetry data, engineers realized that although the shroud had been tested for resistance to vibration and pressure, it had not been tested for resistance to both vibration and pressure occurring together at the same time! JPL’s top engineers flew to Cape Canaveral and quickly designed and installed reinforcements for the shroud cover. Three weeks after the loss of Mariner 3, Mariner 4 was launched. Almost one year later, it sent back the first close-up photographs of Mars. Soon after, I left JPL to enroll in the doctoral program at the Harvard Business School. (The almost identical Mariners 6 and 7 successfully completed their missions in 1969.)

In the fall of 1967, I received a surprise telephone call from Carl, the JPL engineer who had inherited my task of doing reliability calculations for the Mariner project. “You are taking advanced courses on math and management,” Carl said, “maybe you can help me out on something.”

“What is it?” I asked.

“You know about the fire in Apollo 1 . . . ?”

“Yes, I know. Everyone knows. It was terrible.” The Apollo project aimed to put an American on the moon by the end of the 1960s. The conical command module was designed to function with a pure oxygen atmosphere, but pressured at only one-fifth of normal seal-level pressure. That way, the lowered pressure would put less stress on the structure (in the vacuum of space) and the astronauts would still get the normal amount of oxygen per breath since oxygen is one-fifth of sea-level air.

On January 27, 1967, NASA conducted a test of command module systems at Cape Canaveral, with the module pressured to one atmosphere with pure oxygen and sealed. For unknown reasons, a spark set all the plastic and fabric aboard afire. Three astronauts— Virgil Grissom, Edward White, and Roger Chaffee—died in the quick but very hot fire. Although oxygen at one-fifth atmosphere is no more dangerous than air, no one had thought through its dangers during a full-pressure on-the-ground test.

“Well, one upshot of the disaster is that NASA is concerned that the reliability estimation procedures are faulty. So, they have sent a memo to everyone responsible for reliability calculations. It asks us to `adjust our estimates for unanticipated contingencies.’ How in the devil do you adjust for something that is unanticipated?”

Carl’s question drove me nuts for weeks. Finally, I wrote him a memo:

The figure we have been calculating is technically a “component reliability.” It is not the probability of mission success because lots of things can go wrong that are not in the reliability model. For example, we measure the chance of failure over 10,000 hours of a particular type of transistor, but that does not include the chance that a technician will drop a blob of solder on it during assembly. Then there are system inter-relationships we do not model. For example, could particles from mid-course correction engine degrade the camera or the solar panels? Could a piece of reflective gold foil could break loose and let heat build-up in the controller? What is the chance of being hit by a fast-moving rock out in Mars’ orbit?

We don’t model the chances of all these sorts of things because (1) we have no idea of the chances of their happening and (2) we can only think of some of them. If we had experience with hundreds of flights to the planets, we would begin to have a better idea about what can go wrong. Of course, even then, we might not really know about what happens in a solar storm.

Here at HBS I am learning the art of Bayesian probability assessment from Prof. Howard Raiffa, the world expert on the subject. One way to approach the question raised in the memo is to make an informed judgment. In Bayesian terms, would you rather bet on a mechanical gamble with 72% odds of winning, or would you rather bet on Mariner getting to Mars? If you prefer the mechanical gamble, push the odds down until you are indifferent. If your indifference point is, say, 50%, then you are implicitly saying that there is a 30% chance that something outside the reliability model will go wrong. [1 - 50/72 = .3].

This kind of judgment is what an entrepreneur implicitly makes. But your problem is that you need a bureaucratic response and officialdom is not going to accept a personal judgment. There really is no logical way to get an “official” estimate of the chance of unanticipated events, but I am sure that there are companies who would be happy to take a contract to invent some official-looking procedure.

The truth is that going into space is exploration and we don’t know exactly how our technology will function out there. The reasonable course is to send out a vehicle and take measurements and see what happens. And, not to send 100 school-children to Mars until we are damn sure we know what we are doing. Finally, if the politicians cannot stomach any failures, even of robot spacecraft, then the reasonable thing is to spend the money on something else.

Viewed from 2009, my advice still seems to make sense. When you invent new things—like a security backed by liar-loans or a Mariner-Mars spacecraft, you simply do not know exactly what is going to happen. Attempts to estimate “model risk” in such situations are no better than orders to “adjust for unanticipated contingencies.”

Mariner 4

The Mariner 4 spacecraft flew by Mars on July 15, 1965. It was designed to take twenty-two pictures. Each picture’s resolution was 200 x 200 pixels and each pixel was one of sixteen shades of grey—a low-quality thumbnail image on a modern Web-site. The pictures were to be stored on a digital tape recorder and played back slowly because the distance limited the data transmission rate to only 8.33 bits per second, sixty-thousand times slower than today’s typical broadband Internet connection. At that rate, it would take just over five hours to transmit each picture.

At JPL we put a large piece of graph paper on the wall, marked into a 200 by 200 grid. As the data on each pixel arrived every thirty seconds, an engineer announced the pixel’s coordinates and shade—then a technician colored in the appropriate grid box with a crayon. Starting from the left top of the grid, the first row across was all black. The second row was all black as well. We thought we had pointed the camera at the center of the planet, so this did not look good. Gloom settled over the assembled staff as it began to look as if the camera had failed.

After an hour and forty minutes, there were 39 rows of black pixels. Then, on row 40, a grey pixel appeared in the last column. On row 41, more grey pixels in the last columns. Gradually, over hours, the arc of the planet took shape. The black pixels in the upper left of the photo had been the black of space.

Mariner 4′s photographs of Mars showed a surface that was heavily cratered like the Moon. Studies of the brightness of stars just showing at the planet’s edge provided a measure of the atmosphere and it turned out to be ten times thinner than originally believed. My master’s thesis, “Optimal Control of Planetary Lifting Entry,” became irrelevant—we would not be flying any vehicles in the thin atmosphere of Mars.

That first hazy photo from Mariner 4 showed the region called Elysium Planitia to the west (left). A modern image of the region is shown below (European Mars Express orbiter.)