“I discovered a flaw in the model that I perceived is the critical functioning structure that defines how the world works.” This, infamously, was uttered by former chairman of the Federal Reserve Alan Greenspan, who admitted that his view of how economies work was deeply flawed, and yet refused to issue an apology for years of federal intervention (or lack thereof) based on his “flawed” model. Hundreds of millions of people worldwide are suffering economic hardship as a result of someone making policy decisions on the basis of a flawed assumption.

How is this possible? Economics is supposed to be a solid discipline, founded on complex mathematical models (and we all know math is really, really difficult). They even give Nobel prizes to economists, for crying out loud! And yet, economics has always had to fight off the same reputation of being a “soft” science that has plagued sociology, psychology, and to some extent even some of the biological sciences, like ecology and evolutionary biology. Indeed, like practitioners in those other fields of inquiry, some economists admit of being guilty of “physics envy,” that is, of using the physical sciences as the model for what their field ought to be like. Turns out even the assumption that a good science should be modeled on physics is “flawed,” to use Greenspan’s apt phrase.

A recent article by Chelsea Wald in Science (12 December 2008) puts things in perspective by asking how it is possible that so many smart people in the financial sector made irrational decisions over a period of years, despite clear data showing there was a problem, and eventually leading to a worldwide economic crisis that is at the least poking at, if not shaking, the foundations of capitalism itself.

Part of the answer is to be found in the persistent idea in economics that “markets” work because people are rational agents who act in their own self-interest and have perfect, instantaneous access to relevant information about the businesses they are considering investing in. Economists are not stupid, and they know very well that perfect rationality, complete information and instant access are all light years away from the reality of how markets operate. And in fact recent models have relaxed these assumptions to some extent. But it is so much more tractable to model things that way! After all, physicists do it too: remember those problems in Physics 101 that started “consider a spherical cow…”?

But now there is a new kid on the block: “behavioral finance” takes seriously the idea that people act somewhat rationally some of the time but can spiral into downright panic at other times. The new approach draws from an interdisciplinary milieu that includes “soft” sciences like psychology and sociology, since those are the fields that tell us the most about the idiosyncrasies of human behavior. Perhaps not surprisingly, there is another science that has been inspiring economists for some time now: evolutionary biology. The old “efficient markets hypothesis” underlying classical models is being replaced by the “adaptive markets hypothesis,” where Adam Smith’s invisible hand becomes more directly analogous to natural selection.

As evolutionary biologists have found out, natural selection is not an optimizing process, but a satisficying one, meaning that it produces whatever outcome happened to be achievable at a particular historical moment and that works “well enough” for the problem at hand. Moreover, it does so while “wasting” a lot of resources and often marching straight into dead ends (just think that over 99% of the species that ever existed went extinct). The emerging picture is much more realistic than the rationalist paradigm, but it sure is a lot more messy too.

There is another lesson to be learned from evolutionary biology that will not make economists, or the public at large, particularly happy: when complex systems evolve over time the paths they take is contingent on historical accidents (as opposed to being deterministic, like the laws of macro-physics, outside quantum mechanics). Sociologists, psychologists, ecologists and evolutionary biologists will readily tell their economic colleagues that it is certainly possible toexplain past events (the extinction of the dinosaurs, the dot-com bubble) by the use of sufficiently complex causal-historical models. What seems to be out of reach, however, is precisely what economists want most: predicting the future, the hallmark of “good” science.

I’m talking here about specific, actionable predictions, not general ones. Meteorology, for instance, is predictive in the sense that it will tell you that there is a high probability of low temperatures in New York City in the period December-February, no matter what the year. But it is barely capable of telling you whether tomorrow you’ll need a heavy coat or an umbrella. Similarly, ecologists can say that the likelihood of extinction goes up when certain environmental parameters change, for instance the size of the available territory, or its links to similar territories nearby (which affects metapopulation dynamics). But they will be hard pressed in predicting which individual species will go extinct when and where.

Similarly, the new economic models “work” in the sense that they can say, for instance, that stocks that are difficult to value will do well in “optimistic” times, while easy to value stocks will do better in “pessimistic” times. Heck, they can even predict that the variance in how well stocks do is higher for the first type than for the second one. But none of that is going to be of any use when you talk to your broker later today and have to make decisions about what to buy or sell here and now.

The moral of the story is that all of the above is not a failure of economics, sociology, psychology, ecology or evolutionary biology. It is the predictable outcome of the fact that these sciences deal with complex, historical systems, unlike much (though not all) of physics. The real assumption we need to get rid of is the highly persistent and pernicious one that physics is the golden standard by which all other sciences ought to be measured. Now if we only could convince federal funding agencies of that...