This issue's contents Current issue Index Search
Every few years, I happen on a book which expresses an idea lucidly that is at once simple, powerful and easy to understand. These books never seem to be by Jacques Derrida, but by authors who comprehend that no powerful idea needs to be expressed in mystifications. When my reaction to a book is, "That is so simple, why did I never think of it myself?", the book's thesis is about to become part of my own rule-set for examining life. One such book which has come to be very significant in my understanding of the world around me is Charles Perrow's Normal Accidents, first published in 1984.
Perrow analyzes a series of high technology accidents. He was writing before Chernobyl or Challenger, but his insights apply well to these as well. He starts from the premise that technology systems have become so highly complex that they can no longer be understood in their entirety by one human. Imagine even the least expensive notebook computer or Ipad before you: problems with it must be analyzed by a committee consisting of the specialists in the operating system and the various bundled applications, the CPU, machine language, its various ports and interfaces, its Internet connectivity, the chips of which it is constructed, etc.
Perrow's second premise is that many of these systems or subsystems are "tightly coupled", connected to one another or to human outcomes so that their output, even when highly erroneous, causes some result without the opportunity for human intervention. Most fatal technology accidents are caused by the tightly coupled interaction of two features either of which singly would have been manageable, but together are unexpected and overwhelming. The most classic of these couplings is an out of control condition--heat too high, or airspeed too low, or something is on fire--while the gauge intended to spot it is malfunctioning and fails to advise us.
For example, the 1982 crash of an Air Florida plane into the Potomac River shortly after takeoff was a "normal accident". The out of control condition was ice on the wings, and the gauge that malfunctioned reported airspeed. With heavy wings, the plane cannot achieve lift-off, but the gauge gave a falsely high reading, reassuring the pilot he was fine. The plane stalled and the pilot and most of the others on board died in the result.
Since Perrow first published, "normal accident" theory has been effectively applied to a surprising variety of other incidents, including medical interactions and the 2008 market meltdown. In a hospital, systems are intended to be loosely coupled. A computer or transcription error can order the wrong medication, but human intervention is usually enough to prevent it from being administered to the patient: a nurse spots the change, or the patient herself rejects the pills because they are a different shape and color than what she is used to. Medical systems become tightly coupled when the wrong medicine or dosage can be given without human intervention. As far back as the '80's, I remember reading about faulty software in new generations of radiation machines, which either gave patients a multiple of the dose set by the human operator or which disregarded the current setting and gave a dosage at whatever setting had been used for the last patient. These systems were tightly coupled, because the ultimate dose was determined not by the human, but by the machine.
In the financial markets, some low level occurrences are clearly normal accidents, such as market hiccups caused (as early as the '80's, as recently as a few months ago) by flawed trading software operating too rapidly for human supervision and able to drive markets or individual stocks up or down in a matter of seconds. What is more interesting, however, is the application of normal accident theory to broader disasters doing more profound damage. In this view, mortgage-backed securities and the mass market in resale of mortgages, were two highly complex and tightly coupled subsystems, so that a failure in one brought down the other (and also a third subsystem in which the companies isued derivatives betting against their own mortgage-backed securities). The failure of these then brought down, or almost capsized, many other tightly coupled subsystems, including investment and commercial banks which had made big bets in the securities, and insurance companies which had insured them, and which went out of business or had to be bailed out. These failures then had ensuing ripples of damage across everyone else's lives around the globe: poverty, homelessness, an epidemic of evictions, loss of health coverage, closure of businesses large and small, and even less predictable forms of suffering (horses and domestic pets abandoned to starve by owners who could no longer care for them).
Once, in the study of any accident, you have confirmed Perrow's two pillars of complexity and tight coupling, you can then look at the psychological elements which play a role. Before the accident can happen, you must first have the human trust in technology allowing the design of so many crucial systems in which decisions are made by the machine, without the possibility of a human intervening. Of course, much of this is driven by the need to do things very rapidly, in "real time" as the computer folk say: the moment after the launch of a shuttle, or the trading of stocks in fractions of a second, where the minutes are lacking for a human to think about what is happening and decide what to do next. Technological determinism dictates that whatever can be done with technology, we will do, so we tend to skip the debate as to whether the particular solution is necessary or useful (the financial markets could have survived without offering traders the opportunity to buy and sell the same stock in under a minute). In other cases, we were able somehow to achieve things which had to be done at high speed before the invention of the technology: humans flew planes successfully before computerized autopilots existed. Previously, you could crash into a mountain because of poor visibility, or because your radar wasn't working properly. Today you can do so because you inadvertently set the autopilot to the wrong three letter acronym for a city, selecting one that is on the other side of a mountain rather than your intended destination (an actual case from Latin America).
Trust of technology so complex no one human can understand it is the backdrop to normal accidents. Human psychology is a part of the history of almost every accident, because there is usually a moment at which someone understands the truth and is unable to communicate it to someone with the authority to avoid the accident. In the case of the Challenger, there were scientists who understood that O-rings fracture at low temperatures who spent an entire night fruitlessly trying to persuade those in authority to cancel the launch. In the case of the Potomac crash, the very deferential co-pilot, who was actually flying the plane, kept suggesting to the cocky, oblivious pilot, that there was too much ice on the wings, and that the plane felt sluggish and slow as it ascended from the runway. The pilot's response: everything is fine. And they both died.
Robert Trivers has written a book on self deception, The Folly of Fools, which analyzes both incidents. (Fascinating irrelevant factoid: Trivers became first mentor to, then friend of, imprisoned Black Panther leader Huey Newton, who obtained a graduate degree with Trivers' help while in prison; they co-authored a paper on the Potomac crash.) He discusses the "confirmation bias" which leads us to cherry pick facts which support a preconception, and ignore those which contradict it. He roots this behavior in the fact that human and animal nervous systems must exclude certain information in order to function effectively, and also organize data into recognizable models. Trivers describes how certain optical illusions support the insight that our brains organize the data received by our eyes to create recognizable patterns and objects, some of which are false. Another form of accident Perrow described was a very common one in shipping, where two freighters on parallel courses, which would have missed one another, turn and collide. One or both perceives the other as going away when they are headed towards each other, and is merely intending to cross its path at a distance.
Our planet itself is a highly complex system, tightly coupled to certain human subsystems, and is in the course of experiencing a normal accident. The International Geosphere-Biosphere Program, founded in 1987 to "coordinate international research on global-scale and regional-scale interactions between Earth's biological, chemical and physical processes and their interactions with human systems", states on its web site:
Earth behaves as a complex system. Complex systems can respond abruptly to changes within the system - these abrupts changes can be highly non-linear. There is strong evidence that the Earth system is prone to such abrupt changes.Human-induced global change has pushed the Earth system into a no-analogue state — where climatic and other environmental conditions are outside of the range of the last half million years (at least), increasing the likelihood of unpredictable changes with potentially harmful consequences.
The Gaia theory of Earth as an organism, first proposed in the 1970's, is an extension, either anthropomorphizing or metaphorical, of the insight that the planet is a complex system. We can regard the earth as the latter without finding that it is a form of life, but Gaia as a metaphor may, if used carefully like any other metaphor, be useful in reaching understanding or consensus.
Earth's complex atmosphere is tightly coupled with other systems. Before humans came along, the atmosphere was already tightly coupled to the ecosystem of plants and trees which scrubbed the atmosphere of some of its carbon dioxide and added oxygen. When humans first evolved, their ability to affect world-wide systems was paltry, but has steadily become dominant through technology. As a layperson with a very imperfect science education, it seems to me that an example of an early world wide effect, man's first ventures in changing world wide ecosystems, was the extinction of the competing Neanderthals through mass murder and of the woolly mammoth through over-hunting with spears.
Although there are still numerous powerful humans denying that human-caused climate change is occurring, nobody can reasonably deny that human technology has acquired the ability to affect the planet: we have known since the 1950's that a significantly widespread, uncontrolled nuclear war would turn substantial portions of the world into slag, irradiate the rest and possibly also cause a "nuclear winter" effect.
At the point where human technology becomes sufficiently powerful to change the entire planet, human processes like nuclear armament, manufacture of consumer goods, nuclear power and use of fossil fuels all become tightly coupled to the planetary processes in ways that are too complex for one human to understand. You wouldn't expect one human to be an expert in the combustion engine and in the world-wide flow of air currents. You need a consortium, the same way in solving much smaller Ipad problems you need experts in systems software and in microchips.
Here's another example of great importance: the unexpected connection between the Internet and forests, which seems to be a very tight coupling. The World Wide Web at its advent was predicted to be a savior of Earth's ecosystem, because instead of embodying all knowledge in paper books, we would have an eternally available, instantly retrievable electronic repository, and no need for paper. Twenty years later, it now seems evident the opposite is true. Instead of buying one book containing, let's say, Haldane's immortal essay "On Being the Right Size", and keeping it for a lifetime, we now reprint the essay from the Internet each time we need it.
Time here for a poignant, only slightly relevant story: when I came out of the subway under the World Trade Center a few minutes after the second plane hit on September 11, 2001, I looked up at the burning tower and saw beautiful incomprehensible spirals glinting in the sun, slowly twisting down towards me. Only a day or so later did I learn these were reams of printer paper blown out of the building by the explosion.
The Internet and fossil fuel use have also turned out to be tightly coupled, as companies like Google maintain immense server farms that use more electricity than small cities, with a corresponding need for the coal or nuclear power used to generate the electricity. Finally, the spread of the Internet (which had only 75,000 users at a point in history when I was already an adult) has corresponded with a proliferation of electronic devices that can access it. My wife and I, who are really not gadget people, own two desktop computers, one notebook, one tablet, and two cell-phones, on all of which we can surf the web and read email. Each of these devices contain toxic chemicals which you would not want in a landfill, still less incinerated, yet society (by which I mean either government or the private market) has provided no widespread, easily accessible means of disposing of electronic devices.
Please note that in each of these cases, the Internet as a system is surprisingly tightly coupled to the atmosphere. More use of paper means more destruction of forests. All that paper has to be disposed of, and some of it will be burned, adding ash and carbon dioxide to the air. Increases in coal use to generate electricity to power the Internet have an obvious effect. And computers and other electronic devices disposed of in incinerators also have a direct effect, while I suppose chemicals might also get into the air from landfills.
Regarding Earth as a complex system tightly coupled to human subsystems in Perrow-vian terms led me to a further insight, that the Potomac crash is a pretty good metaphor for global warming. Human-induced climate change is the ice on the wings, and climate scientists are the co-pilots warning, perhaps much too politely, that the system will crash. In the oblivious pilot seat are presidents and politicians who either do not believe in climate change, or feel helpless to do anything about it, so find it easier to disbelieve or ignore warnings of danger.
One interesting difference is that in the Potomac crash, the air speed gauge gave a falsely high, therefore reassuring reading. Today, in the world of climate change, we have the opposite phenomenon; we have in recent years developed gauges that, in IGBP's terms, are starting to show out of range numbers with startling accuracy. In a scenario which fits well within Trivers' analysis of human bloody-mindedness, we have powerful people--Congressfolk, Senators, Rush Limbaugh, the Cato Institute, the billionaire Koch brothers--denying those gauges are accurate, or that the readings really support the inference that humans are causing the problem, or that there's anything we can do about it anyway.
The saddest piece of it, and the one that inspires the most personal despair, is the fact that our republic includes legislators who in the 21st century are still capable of statements like the following, which I also quoted in a recent essay on the paradigm shift brought about by hurricane Sandy (who ever imagined Wall Street could be under water in any but a metaphorical sense?):
Genesis 8:22 [says] that ‘as long as the earth remains there will be seed time and harvest, cold and heat, winter and summer, day and night,’ my point is, God’s still up there. The arrogance of people to think that we, human beings, would be able to change what He is doing in the climate is to me outrageous.
That's Jim Inhofe speaking, senior Republican Senator from Oklahoma, who sits on the Senate Environment and Public Works Committee.
Epigraph:
" There is not a risky system on this earth that we need, essentially, for life."-- Perrow in 1989 on the Open Minds show on PBS.