Projections are usually wrong when they are based solely on what has already happened — but this method is considered the most reasonable approach. When the projections are then considered predictions we start making bad decision — in other words, the status quo tends to rule the way we think about what we can do. We need to step away from raw data and factor in common sense. This may seem contradictory from someone who regularly argues that we must consider what has happened in the past as a lesson for what might happen in the future; it is not.
Today I read or skimmed a 360 page report by BP titled US Energy In Perspective: Data & Analysis of US Energy Supply, Production & Consumption (pdf). This report is truly incredible as a resource, and despite my occasionally less-than-favorable views of oil companies, this report generally seems to present the full picture of where we have been, and where we are with US energy with very little bias. The report also presents some projections, typically through 2030.
What struck me was that 2030 will play out an environmental (and political, economic and other) disaster if we actually do follow the path of the projections.
It also struck me that the very analytical, solid, fact-based analysis presented could easily be read as a foregone conclusion of what will happen. Or, if you are of a careful mindset, that which might be skeptical of “bold new plans” and other things that politicians have been heard to say from time to time, you might ask for the most reasonable set of facts and predictions available — the CEO of a major oil company might take this approach. The BP report cites numerous government studies, as well as many university and other scholarly works. There’s absolutely nothing wrong with the methodology, that I can see.
Except that its implied predictions are dead wrong.
To elaborate, I said earlier “projections are usually wrong,” and in truth that’s a simplification. Of course any non-trivial statistical analysis is limited by the precision and size of its data, but this is not what I mean by “wrong” — projections, in particular predictions of the future tend to avoid making assumptions about what might happen (but have not happened yet) if they are to be considered authoritative. Careful analysis uses history, raw data, and factors in as many variables that were past predictors of the current outcome, and in so doing develops a model of expected future outcomes. Once the model fits the way things have already happened, we let it run a little and see where we’re going. That’s what a projection is.
But actually projections tell us only where we might be going. Models tend to work well as predictors for physics, geology, biology, astronomy and other sciences. But when you’re trying to figure out the behavior of people, things get a little messy. Economists and other social scientists qualify the limitations of their models with the Latin ceteris paribus or loosely, “all other things being the same”.
But other things are not the same, and neither are the things that were the same. There is a simple (and systematic) reason why the things that have always been the same in the past tend not to be that way in the future. It’s because we’re humans — we’re smarter than we look. In the new knowledge gained from any given analysis, as well as from other sources or even just from our inaccurate fears, we tend to start changing the way we respond and mess up the data. This effect is most pronounced when the knowledge tells us something important that we didn’t understand before (e.g. “melting polar ice caps will flood Florida and NYC”). In other words, we don’t just react, we respond.
But even though we react, scientists don’t predict how we’ll respond very well, mainly because there’s so much error in projections to start with, injecting any “soft” data just makes the outcome even less reliable. So, we tend to plan based not on what the actual outcomes might actually be, but on what our projections are. Projections based on the past seem so much more prudent, don’t they?
For example, the BP paper shows many charts projecting through 2030. Many relate to the clear reduction of oil supply, but I picked the following chart showing the relative changes of different energy supplies used for electricity generation in the US as one typical example:
Does it alarm you to see that the projection shows coal growing about 40%, which in absolute terms is about 800 billion kWh compared to 2006? It seems little consolation that while renewables show significant growth, they are still only 20% of our coal electric production after 20 years from now. How can this square with the kinds of plans that Gore and others have promoted which would have us getting all of our electricity from renewables (and 0% from coal) by 2020, almost half the time of their projections?
The answer is: they don’t align. BP’s projections are “the best solid data”, yet they effectively proscribe actions that must be initiated now. And this is what really, really scares me. I wrote the other day about Exxon-Mobil’s projections, which are even less forward looking, asking how they could be basing their long-term business decisions based on what is “obviously” wrong. As I thought about this, I realized that everyone is using the same fundamentally flawed thinking.
A key outcome of this thinking is 1) it is disastrous, and 2) when major players like Exxon, BP and the US Government all base decisions on the results, we tend to cause the projected outcome to come to pass. Our projections create self-fulfilling prophesies! BP’s report chides us for having repeatedly failed to take sustained action to address energy independence. But as I read it, the report makes no compelling argument for why we shouldn’t make mostly incremental changes to business as usual.
Raw data is of no value. Analysis of raw data can provide the foundation for good decisions, and to be fair, this is what the BP report seems intended to do. But it is only the skilled and intelligent interpretation of analytical results that lead to appropriate decisions. We don’t seem to have much of that in much of business or government.
The report frankly assesses our repeated failures to enact effective policies, indicating “we failed to learn the lesson that hope is not a strategy”. The report starts with an assessment of why it is so important for us to follow through with our policy plans for energy independence. We have to get it right this time, they say. I agree with BP’s position on this goal. But the remaining 300+ pages of their report present only the best available data of what might be possible, and discounting that this is a report created by an oil company, the scope of their imagination seems not just limited, but terrifyingly counter-productive.
Yet, I cannot imagine the scenario BP predicts coming to pass; too many (bad, bad, bad) things would happen in that 20-year period to make any past data have any significant validity. (Indeed, an early discussion in the report explains why we have been completely incapable of predicting oil prices; I believe the same argument applies to their larger set of results).
Of course it may be that my fundamental assumptions are different than BP’s. For example, I assume that the 2007 IPCC findings regarding the impacts of greenhouse gasses are close enough to being correct that we must take action. BP’s projection could not be consistent with taking action of this sort: increasing coal output by 40% in the US would continue our acceleration of greenhouse gas emissions (the report acknowledges that CO2 sequestration would need to be proved on a massive scale to be even considered as a viable solution). But in truth, what either of us believes is not really relevant; the main issue is that the report doesn’t consider much, if any conclusions about the future because the methodology of projection based on modeling can only rely upon what has already happened.
So perhaps it’s this simple: BP makes projections based on the most significant risks they can measure from past history. Any future risk must be discounted as unreliable.
Yet in this method is the fundamental failing I noted at the outset. Humans certainly have the ability to react or respond to that which we expect to happen; indeed, our pre-historic survival was based not on analysis, but on a sort of intuition — an educated guess about what seems likely, including information that may not yet be fully quantifiable. A caveman might think, “In the past when I have heard a heavy thudding sound, and it quickly gets louder, it has been an animal that wants to eat me. I have survived by anticipating the arrival of whatever it is by running away. Perhaps this is not the same animal, but I am still going to run.”
I guess this is what some people call “common sense”. It’s not much to prop up a 360 page report, but it seems conspicuously absent from so many human activities.
I propose that instead of ignoring the approaching beast we at least strongly consider the possibility in any projections we make, common-sense possibilities be at least considered as possible alternatives. How we respond to these kinds of projections would be dramatically different if we factored in a little common sense.