Vaclav Smil: The (In)Accuracy of Long-Range Forecasting

I should really be grateful that I don’t need to continually build a case for Quantitative Complexity Management because many, more learned, individuals than me keep on doing it! However, the problems remain…WHAT WILL IT TAKE FOR PEOPLE OF INFLUENCE TO STOP: MAKING ASSUMPTIONS IN AN ATTEMPT TO PREDICT THE FUTURE; TALKING ABOUT THE IMPACT OF INCREASING COMPLEXITY; IGNORING THE INTERDISCIPLINARY CASE FOR CHANGE; WASTING TIME WITH AGREEMENTS AND REGULATIONS TOO COMPLEX TO EFFECTIVELY MANAGE…AND EVERY OTHER SODDIN’ THING THAT RELATES TO BUILDING A MORE STABLE AND SUSTAINABLE ECONOMIC BASIS???

Vaclav Smil offers some thoughts on the accuracy of long-range energy forecasting. Smil is truly a goldmine of thought. The excerpts below are taken from The Perils of Long-Range Energy Forecasting: Reflections on Looking Far Ahead.

On Complexity

Greater complexity that was required to make the forecasts more realistic also necessitated the introduction of longer chains of concatenated assumptions—and this necessity was defeating the very quest for greater realism…

On the Failure of Experts

What strikes me most when looking back is the extent of individual and collective failure. Perhaps the most remarkable aspect of the lack of imagination underlying this failure—or, conversely, of exaggerated expectations, and of often surprisingly swiftly disproved quantitative predictions—is that so many erroneous forecasts have come from eminent innovators or from individuals (or, more recently, from institutions) considered to be the leading experts in their field….
Perhaps the most precious example of failed national long-range energy forecasting—remembered fondly, I am sure, by all those who have been around energy matters for some time—is the goal of U.S. energy independence charted by the Nixon administration for the 1980s. Felix thought that the self-sufficiency can be realized by the year 1985, despite the fact that his forecast called for the consumption of some 3000 Mtoe in 1985. A reality check: in 1999 the USA imported more than a fifth of its total primary energy use, which was about 2400 Mtoe, and just over half of its demand for liquid fuels!

On others being wrong about long term forecasts

Quoting more than just a couple of examples from the 19th and the first half of the 20th century is irresistible, as those predictions turned out to be so spectacularly wrong. In 1879, just 3 years before T. A. Edison began selling electricity for lighting, the Select Committee on Lighting by Electricity of the British House of Commons heard an expert testimony that there is not “the slightest chance” that electricity could be “competing, in general way, with gas.” Exactly a decade later Edison himself was making a big blunder: “My personal desire would be to prohibit entirely the use of alternating currents. . . . I can therefore see no justification for the introduction of a system which has no element of permanency and every element of danger to life and property”
In his biography, Henry Ford reminisced that his employer objected to his experiments with the gas engine, believing that electric car was the coming thing, and that the Edison Company offered him the general superintendency but “only on the condition that I would give up my gas engine and devote myself to something really useful”. In 1901, 3 years before the Wright brothers took off, Rear Admiral George W. Melville concluded that even if man should “succeed in building a machine small enough to fly and large enough to carry himself, then in attempting to build a still larger machine he will find himself limited by the strength of his materials in the same manner and for the same reasons that nature has”

Predicting results by accident

In 1983, predictions of global primary commercial energy consumption submitted by the participants in the International Energy Workshop for the year 2000 ranged from 5330 million tonnes of oil equivalent (Mtoe) by Amory Lovins to 15240 Mtoe by the International Atomic Energy Agency; my forecast was 9500 Mtoe [17]. Using the actual consumption statistics for 1999 [18] and projecting just 1 year ahead (on the basis of average growth during the past 5 years, and by converting both hydro and nuclear electricity to Mtoe by using the average efficiency of fossil fuelled generation) brings the global consumption to about 9200 Mtoe by the year 2000, or 73% above Lovins’s 1983 projection and 40% below the IAEA forecast.
My forecast will be off by a mere 2–3%—but I do not call your attention to it to congratulate myself for having done better than both the hard and the soft energy protagonists, but rather to tell you how wrong I was. Although I had nearly nailed the overall demand, I was much less successful on forecasting the makeup of the world’s primary energy consumption. I underestimated both the use of natural gas and crude oil (by, respectively, 25 and 12%), and I overestimated the contributions of coal and renewable energies. If my breakdown would have been used for calculating the future emissions of CO2, or emissions of SO2 and NOx, the gases responsible for nearly all anthropogenic acid deposition, the errors would have been considerable.

Lessons That Will Be Ignored

As with any accumulated experience, lessons arising from failures of long-range energy forecasting will be largely ignored. Having no illusions about the usefulness of my advise I will, nevertheless, offer the following conjoined axioms: no truly long-range forecast can be correct in all of its key aspects; most of these forecasts will be wrong in both quantitative and qualitative terms; some forecasts may get a few quantities approximately right, but they will miss the real qualities arising from subtly to profoundly altered wholes.
Simulations of highly dynamic natural systems have shown that models of growing complexity are moving slowly toward reasonable replications of reality: global climate modeling is a fine example of this slow, but indisputable, trend. In contrast, forecasts of interactions of social, economic, technical, and environmental developments are not going to improve by making models more complex. This is because so many critical variables determining eventual outcomes cannot be either anticipated or, when they get considered, their probabilities cannot be confidently placed within bounds narrow enough to generate a restricted fan of possible outcomes that might be used in confident decision making. Once the inherent uncertainties make the outcome fan too wide, there is little point in building more complex models: we might have obtained pretty much the same results with a small electronic calculator and the proverbial back of an envelope.

What to Do Instead

…a new century will make little difference to our ability of making point forecasts: we will spend more time and money on playing the future game—but our predictions will continue to be wrong.
But acknowledging these realities is not the same as advocating a complete abstention from looking far ahead. There is a fundamental difference between decisions that are good only if a particular prediction turns out to be correct—and the ones that are good for a range of alternative futures: scenarios, rather than point forecasts, are thus much more valuable, both from heuristic and from practical points of view. As the future is inherently unpredictable, it is the decision analysis or contingency planning under a range of alternative scenarios that should be pursued most diligently. Techniques comprising this approach range from narrative and normative scenarios to Monte Carlo simulations and to stochastic programming.

Continue Reading

Related items:

Nature always extracts justice

Nassim Taleb on the impact of debt in uncertain times and lessons from nature

“Too Big To Manage?”: Or how NOT to interpret or tackle complexity…ever!

The Enigma of Capital and the Crises of Capitalism

Nature’s 10 Simple Rules for Business Survival

Risk: Things that Cannot be Modelled Should not be Modelled

Economics Nobel Prize winner advocates co-operative working for a better future

What’s next for global banks – Is it a tent and some dodgy headgear?

Fund Strategy Magazine: Complexity lessons from nature for a better economic future…

“We need to start thinking about the economy as a holistic, natural system”.