The past year or so has not been a good one for forecasters, your truly included. I’ve submitted two successive business budgets that have been wildly inaccurate: in our most recent year our forecast had the same potency of an Obama bowling frame…gutterball after gutterball.
We’re not alone in this inaccuracy, however. Two compelling conclusions from the experience of the past year are: Past events do not necessarily forecast future events; over time, performance will tend toward the mean (as long as you measure a long enough period of time the right way).
A recent abstract in Interfaces that I came across provides a detailed insight into how seemingly scientific sets of facts can provide an unreliable basis for forecasting.
The subject: Whether or not Polar Bears should be declared an endangered species.
For most of us, it’s easy to believe Polar Bears might be endangered. We don’t stumble across them often in our daily life, when we do see them, it’s usually on TV accompanied by serious, stentorarian voice-overs, and because they appear so serious and cool, it’s easy to imagine they must be going extinct. Doesn’t that happen to all cool things?
Nine different reports were provided to the U.S. Fish & Wildlife Service, all concluding that environmental and demographics trends portend a demise for the currently plentiful Polar Bear population.
However, the authors conclude, “We assessed these reports in light of evidence-
based (scientific) forecasting principles. None of the reports referred to works on scientific forecasting methodology.” They go further:
The reports each provided forecasts of polar bear populations for 45, 75, and 100 yearsfrom the year 2000. The reports make recommendations with respect to the polar bear listing decision. These do not follow logically from their research in that they simply made forecasts of the polar bear
population. To go from the forecasts to policy recommendations, the following assumptions are required:
(1) “global warming” will occur and will reduce the extent of summer sea ice; (2) polar bears will not adapt and will thus obtain less food by hunting from the sea ice platform than they do now; (3) listing polar bears as a threatened or endangered species will result in policies that
will solve the problem without serious detrimental effects; and (4) other policies would be inferior to those based on the Endangered Species Act.
Given my uneven forecasting record for the last couple of years, I was particularly interested in several of the principals the authors determined were contravened in the nine different reports.
- Make sure forecasts are independent of politics;
- Match the forecasting methods to the situation;
- Be conservative in situation of high uncertainty or instability;
- Ensure that information is reliable and that measurement error is low;
- Forecast for alternative interventions.
Bad forecasting can create bad outcomes. In business, you have to lay bets and make moves on the best available information. You try to hedge and react, where appropriate, but ultimately, you have to move forward.