The Perfect Swarm: The Science of Complexity in Everyday Life

THE PERFECT SWARM

ALSO BY LEN FISHER Rock, Paper, Scissors Weighing the Soul How to Dunk a Doughnut

THE PERFECT SWARM The Science of Complexity in Everyday Life

Len Fisher, Ph.D.

New York

Copyright © 2009 by Len Fisher Published by Basic Books, A Member of the Perseus Books Group All rights reserved. Printed in the United States of America. No part of this book may be reproduced in any manner whatsoever without written permission except in the case of brief quotations embodied in critical articles and reviews. For information, address Basic Books, 387 Park Avenue South, New York, NY 10016-8810. Books published by Basic Books are available at special discounts for bulk purchases in the United States by corporations, institutions, and other organizations. For more information, please contact the Special Markets Department at the Perseus Books Group, 2300 Chestnut Street, Suite 200, Philadelphia, PA 19103, or call (800) 810-4145 ext. 5000, or e-mail special [emailprotected]. Designed by Timm Bryson Library of Congress Cataloging-in-Publication Data Fisher, Len. The perfect swarm : the science of complexity in everyday life / Len Fisher. p. cm. Includes bibliographical references and index. ISBN 978-0-465-01884-0 (alk. paper) 1. Swarm intelligence—Social aspects. 2. Group decision making. 3. Group problem solving. 4. Social groups—Psychological aspects. I. Title. HM746.F57 2009 302.3—dc22 2009031018 10 9 8 7 6 5 4 3 2 1

To Wendella, who has now survived four books, and has helped me to do likewise.

CONTENTS

Patterns of the Perfect Swarm: Visions of Complexity in Nature ix Acknowledgments xix

Introduction 1 ONE

The Emergence of Swarm Intelligence 9

T WO

The Locusts and the Bees 23

THREE FOUR FIVE

SIX SEVEN

EIGHT NINE

Ant Logic 37 A Force in a Crowd 49 Group Intelligence: The Majority or the Average? 67 Consensus: A Foolish Consistency? 83 Crazes, Contagion, and Communication: The Science of Networks 105 Decision Rules OK? 133 Searching for Patterns: Imagination and Reality 155

vii

viii

Contents

T EN

Simple Rules for Complex Situations 167 Notes 173 Index 247

PATTERNS OF THE PERFECT SWARM: VISIONS OF COMPLEXITY IN NATURE How Complex Patterns Emerge from Simple Rules in Physical and Living Systems

Pattern of Rayleigh-Bénard cells formed by convection in a layer of oil in a frying pan heated from below (see page 3). COURTESY OF BEN SCHULTZ

Patterns in coral at Madang, New Guinea, formed by walls of calcium carbonate secreted by individual polyps competing for space. PHOTO BY JAN MESSERSMITH

Pattern of reaction products formed in a petri dish by an “oscillating” chemical reaction known as the Belousov-Zhabotinsky reaction. The black dots are adventitious air bubbles. COURTESY OF ANTONY HALL, WWW.ANTONYHALL.NET

Patterns formed by tens of thousands of the soil-dwelling “slime mold” amoeba Dictyostelium discoideum, growing on the surface of a gelfilled petri dish. Each individual is responding to chemical signals from its neighbors that warn of a lack of bacteria that are its main food. Ultimately the cells will aggregate to form a “slug,” technically called a grex, which can crawl through the soil much faster than the individual amoebae to find new bacterial pastures (see page 18). COURTESY OF PROF. CORNELIS WEIJER, UNIVERSITY OF DUNDEE, UK

xii

Stripes in the Algodones sand dunes of Southern California, formed by a combination of wind driving the sand up and the force of gravity pulling sand grains down (see page 2). © KONRAD WOTHEA/FLPA

xiii

Stripes in the developing larva of a fruit fly (Drosophila melanogaster ). The stripes are formed by the selective differentiation of cells in response to the presence of distinct neighbors. Each stripe will ultimately develop into a different part of the adult body—wings, thorax, eyes, mouth, etc. COURTESY OF JIM LANGELAND, STEVE PADDOCK, SEAN CARROLL. HHMI, UNIVERSITY OF WISCONSIN

Stripes on a tiger, once thought by some biologists to emerge from a balance of chemical gradients in a manner reminiscent of the Belousov-Zhabotinsky reaction. This tiger was photographed in Pench National Park, Madhya Pradesh, India in 2004. © ROGER HOOPER

COURTESY OF DR. HOWARD F. SCHWARTZ, COLORADO STATE UNIVERSITY, WWW.BUGWOOD.ORG

Spiral pattern in a sunflower head, an arrangement that allows for “optimal packing” of the individual parts. This design is described mathematically by the Fibonacci sequence, in which each number is the sum of the two previous numbers (i.e. 0, 1, 1, 2, 3, 5, 8, 13, and so on).

xiv

COURTESY OF NATIONAL AIR AND SPACE ADMINISTRATION (HTTP://HUBBLESITE.ORG/GALLERY/ALBUM/ENTIRE/PR2005001A/NPP/ALL/)

Spiral galaxy, whose formation is dominated by Newton’s Law of Gravity and his three laws of motion. Barred Spiral Galaxy NGC 1300; image taken by Hubble telescope.

Self-organization in a school of fish, produced by each animal following Reynods’ three laws. Shoal of Robust Fusilier, Caesio cuning, German Channel, Micronesia, Palau (see page 12).

xvi

xvii

Self-organization in a herd of wildebeest crossing the Serengeti plain. © ALICE CAMP, WWW.ASTRONOMY-IMAGES.COM

Self-organization produced by “social forces” of people spontaneously forming “lanes” as they walk along a crowded street (see page 50). COURTESY OF DIRK HELBING

ACKNOWLEDGMENTS

It is a pleasure to acknowledge the many people who have acted as advisors, mentors, and muses in my efforts to produce a simple guide to complexity. Amanda Moon, my editor at Basic Books, has done her usual extremely thorough and helpful job, as has Ann Delgehausen, who has performed marvels with the copyediting. Particular thanks must also go to my wife, Wendella, who has painstakingly examined every chapter from the point of view of the nonscientific reader, suggesting a multitude of interesting examples and pointing out where new ideas needed a clearer explanation. The book has also benefited greatly from the advice of world experts in the various fields that it covers. I have named them individually in the notes to the appropriate chapters and here record my collective thanks, along with my thanks to other friends and colleagues (scientific and otherwise) who have gone to a great deal of trouble to read the manuscript and offer suggestions that have contributed considerably to its gradual improvement over the course of writing. I can’t blame any of them for errors that may still have crept in. Those, unfortunately for my ego, are my responsibility alone. Here are the names of the people who helped, in alphabetical order, accompanied by the offer of a drink when we next meet: Hugh Bray, Matt Deacon, John Earp, David Fisher, Gerd Gigerenzer, Dirk

xix

Acknowledgments

Helbing, Jens Krause, Michael Mauboussin, Hugh Mellor, James Murray, Sue Nancholas, Mark Nigrini, Jeff Odell, Harry Rothman, Alistair Sharp, David Sumpter, Greg Sword, and Duncan Watts. If I have omitted anyone, I can only apologize, and extend the offer to two drinks.

Introduction

Shortly after Star Wars hit box office records, a group of ninety-seven locusts sat down to watch the film. They didn’t have much choice in the matter; they were strapped in place with their heads firmly clamped while experimenters monitored spikes of electrical activity in their brains as they reacted to the fleets of spaceships zooming at them from either side. The scientists were trying to work out how locusts in a dense swarm manage to avoid colliding with each other. Studies on this aspect of swarm behavior have provided valuable information about our behavior in the human swarm, from working our way through crowds to the design of collision avoidance systems for cars. There are many other lessons that we can also learn from the behavior of animals in groups, such as swarms of locusts, flocks of birds, and schools of fish. This book is about how we can use such lessons to make better group decisions and better decisions for ourselves as individuals within a group. The individual animals in a swarm, flock, or school follow rules that help them to get the most from the group. Some of these rules help them to stay together as a unit. Others allow them to act as if 1

THE PERFECT SWARM

they were components of a superorganism, which has no individual leader, and where the whole becomes greater than the sum of its parts as the group develops swarm intelligence and uses it to make collective decisions. The modern science of complexity has shown that collective behavior in animal groups (especially those of insects such as locusts, bees, and ants) emerges from a set of very simple rules for interaction between neighbors. It has also revealed that many of the complex patterns in human society arise from similarly simple rules of social interaction between individuals. My ultimate aim in this book is to explore how the process works and, more importantly, to help find simple rules that might guide us through the fog of complexity that so often seems to enshroud our lives. The process by which simple rules produce complex patterns is called “self-organization.” In nature it happens when atoms and molecules get together spontaneously to form crystals and when crystals combine to form the intricate patterns of seashells. It happens when wind blows across the sands of the desert to produce the elaborate shapes of dunes. It happens in our own physical development when individual cells get together to form structures such as a heart and a liver, and patterns such as a face. It also happens when we get together to form the complex social patterns of families, cities, and societies. There is no need for a central director to oversee the process. All that is needed is an appropriate set of simple local rules. Individual sand grains form themselves into dunes under the combined forces of gravity, wind, and friction with nearby grains. Atoms and molecules experience forces of attraction and repulsion with nearby atoms and molecules, and these forces are sufficient in themselves to produce long-range order that can extend for billions of atomic diameters in all directions.

Introduction

Our society is made up of billions of individuals also, and the forces of attraction and repulsion between us can act to create social structures among us as well. These structures, however, are not nearly as regular as those of atoms in a crystal. To use the complexity scientist’s picturesque but slightly misleading phrase, they are on the edge of chaos. The meaning of the phrase can be unclear because edge implies that our social structures are forever in danger of descending into anarchy. What it really means is that their degree of organization lies somewhere between complete order and complete chaos. Complete chaos is rather hard to achieve, although my wife claims that the disordered piles of paper on my desk come pretty close. I argue that there is order within the chaos, even if I am the only one who can see it. There is order within most forms of chaos. This is dynamic order, which you can see by simply pouring some cold milk into a cup of hot black coffee. Patterns appear on the surface that are an indication of what is going on underneath, where the mixing of the hot and cold liquids produces a set of swirling vortices that rapidly selforganize into a remarkably regular arrangement. These are called Rayleigh-Bénard cells. You can find them in a fraction of an inchthick layer of liquid in a shallow dish and in the miles-thick layer of the Earth’s atmosphere. Systems on the edge of chaos, including animal groups and human societies, also have dynamic order, but it lasts a lot longer than the vortices in a cup of coffee. The order arises from rules of interaction between individuals that produce large-scale dynamic patterns of interaction. The resulting set of emergent patterns characterizes the society as a whole rather than its individual members. These patterns span a variety of time scales. Some, such as those of cities, can be very long lasting. Others, such as those of a moving

THE PERFECT SWARM

crowd, may be as evanescent as clouds in a windy sky. Still others, such as those of human relationships, can be anywhere in between. Two sorts of dynamic pattern are possible in a system on the edge of chaos. In one, the system cycles endlessly between different positions, as sometimes happens in domestic arguments that go round and round without any resolution. In the other pattern, a much more productive one, the system adapts to meet changing circ*mstances, as does the shape of a school of fish when confronted by a predator. When individuals in a group are able to respond collectively to changes in circ*mstances, the group becomes a complex adaptive system. The rules that produce such systems and govern their behavior are of considerable interest, and not just to students of human society but also to students of the whole of nature. Successful ecosystems are complex adaptive systems, as are successful cities and societies. According to the scientist James Lovelock’s Gaia concept, the Earth as a whole is a complex adaptive system. One of its long-term adaptations that should be of concern to all of us may well be to get rid of our species to protect itself. Whether that happens or not could come down to whether we are able to understand the rules that govern its complexity, and whether we have the wisdom to adapt ourselves and conform to those rules. For a complex adaptive system to evolve and grow, the interactions between its individual members must be of a special kind. Instead of being simply additive, in the manner of a number of individuals pulling on a rope in a tug-of-war competition, the interactions must be nonlinear, meaning that an action by one individual produces a disproportionate response in other individuals or in the group as a whole. Just one person clapping at the end of a concert, for example, can stimulate several others to start clapping, and they in their turn can each stimulate a number of others until soon the whole audience is applauding.

Introduction

Collective clapping can sometimes fall into synchrony, which is a property of the audience as a whole, not of any individual member. Such emergent properties arise in complex adaptive systems when there are several types of nonlinear action going on at once. One of the most important emergent properties a group can have is swarm intelligence, which allows a group to tackle and solve problems in a way that its individual members cannot. In this book I examine the simple rules that give rise to swarm intelligence in nature. I ask whether we can use swarm intelligence and its underlying rules (or other equally simple ones) to help us to steer our way through the complexities of life. Our journey of discovery begins with the animal kingdom and progresses in nine stages. The first three cover the evolution of swarm intelligence in the natural world and what we can learn from its underlying rules. The following four focus on developing group intelligence in human society and using it to solve complex problems. The final two look deep into complexity itself to uncover new and simple rules that we can use to make the best decisions we can when we are trapped in its web. Chapter 1 is an overview of swarm intelligence: What is it? How does it arise from nonlinear interactions? What sorts of animals use it? What advantages does it convey to the individuals within a group and to the group as a whole? The following two chapters cover the rules that locusts and bees use when flying in swarms, and the land-based logic of ants. These three types of insect use the basic rules of complex adaptive systems to implement swarm intelligence in very different ways. We can learn something from each of them. The collision avoidance strategies of locusts have implications for driving in traffic and walking in crowds. Bees use “invisible leaders” to direct the movements of the swarm. We can do the same, and also take advantage of such leaders when traveling in

THE PERFECT SWARM

unfamiliar surroundings. Ants use a specialized form of group logic that allows them to find shortcuts and optimum routes. We can follow their example while walking or driving. You may be surprised by the ways their problem-solving approach is being applied in many other situations. After the chapters on insect logic, I look at individual behavior in human crowds and describe how recent research into complex crowd dynamics has revealed optimum strategies for making our way through them and handling ourselves in dangerous crowd situations. In the next two chapters I focus on group decision making. In the first I ask whether we should follow an average course, one that takes equal account of everyone’s opinion, or whether we should go with one endorsed by the majority. In the second I show how we can use group intelligence to achieve the best consensus and how we can best avoid the perils of groupthink while doing so. One way to implement group intelligence is through networking. In chapter 7 I explore different methods of networking, including those that lead to the famous six degrees of separation. I show how new understanding has led to more effective strategies for networking and communication, and contributed to our ability to prevent the spread of disease through the human network. The penultimate chapter is devoted to the ways in which we can use simple rules to make the best individual decisions when we are confronted by complex problems. Some of the best approaches are very simple indeed and provide surprising insights into the amount and type of information that we need to make the best decisions. Finally, in chapter 9 I explore one further way in which we might make decisions: by looking for patterns within the complexity. Sometimes these can guide us in the right direction, but, as the science shows, the whole of society is frequently greater than the sum of the parts and we need to be aware of times when overall complexity can

Introduction

overwhelm the simplicity that lies buried within. Simplicity is OK, but complexity rules. OK?

A Note on the Notes I uncovered many fascinating anecdotes, references, and points of interest during my research for this book that did not quite fit into the main text without disrupting the flow of the story. I have put these into a set of notes that are designed to be dipped into, enjoyed, and read quite independently of the main text. Several readers of my previous books, which I arranged the same way, have written to me to say that the notes section is where they start! If you would like to know more about any of the topics mentioned in the main text, there is very likely more detailed information in the notes. Some of the references point you to the underlying scientific literature (usually accessible through websites such as Google Scholar). I have done my best to choose articles that are both seminal and easily readable. Enjoy!

ONE

The Emergence of Swarm Intelligence

The behavior of animals in a swarm used to be seen as almost magical. Some early scientists even thought that swarms of insects, schools of fish, and flocks of birds could produce their wonderfully coordinated movements only through some sort of extrasensory perception, or perhaps through the development of a group consciousness that required each animal to sacrifice its individuality and become a puppet of that consciousness. Animal behaviorists, informed by the science of complexity, have now proved that swarm behavior does not need such outré explanations. It emerges naturally from simple rules of interaction between neighboring members of a group, as happens with a wave generated by a crowd of spectators at a football game. The wave might look to a visiting Martian like a complicated exercise in logistics, but its dynamic pattern emerges from a simple rule: stand up and stick your hands in the air (and then put them down again) as soon as you see your neighbor doing it.

THE PERFECT SWARM

Such a wave involves rapid transmission of information from individual to individual, and this is a key feature of swarm behavior. It happens in the human swarm in the form of gossip—neighbor chats with neighbor and additional information is channeled back along the same route until everyone knows what is going on and can act on the information. My wife and I once turned up at a country fair, to which we had been invited by friends, and were greeted at the gate by a perfect stranger who took one look at us and said, “Your friends are over in the beer-tasting tent.” She hadn’t actually seen our friends, but she had heard through the grapevine where they were and that they were expecting people who fit our description. Swarm behavior becomes swarm intelligence when a group can use it to solve a problem collectively, in a way that the individuals within the group cannot. Bees use it to discover new nest sites. Ants use it to find the shortest route to a food source. It also plays a key role, if often an unsuspected one, in many aspects of our own society, from the workings of the Internet to the functioning of our cities. Swarm intelligence is now being used by some people in surprising and innovative ways. Companies are being set up that are run by swarm intelligence. Computer programmers are using it in a radical approach to problem solving. There is even an annual event, Swarmfest, where scientists swarm together to discuss new applications of swarm intelligence. Groups that use swarm intelligence need no leader, and they have no central planning. What, then, allows them to maintain their coherence and to make seemingly rational decisions? How do individual interactions translate into such complex patterns of behavior? To make the best of our own individual interactions, we need to understand the answers to these questions. The answers have come from three sources: the real world of animals, the imaginative world of science, and the virtual world of the computer. Here is a brief background for each one.

The Emergence of Swarm Intelligence

Learning from the Animal Kingdom Animals use swarm intelligence to hunt for food and find shelter as a group, and to avoid predators. The scientific study of their behavior, ethology, has uncovered the simple rules they use to engender swarm intelligence. It has also caused the scientists concerned to face some unusual perils on occasion. German ethologist Martin Lindauer was caught in a particularly bizarre situation in the mid-1950s while he was trying to understand how honeybee swarms find their way to new nest sites. His practice had been to run along underneath a swarm through the outer suburbs of war-ravaged Munich, always wearing a white lab coat while doing so, perhaps to publicize his scientific credentials. Unfortunately, his coat resembled the uniform that patients in a nearby hospital for the dangerously insane were forced to wear. One day, guards from the hospital mistook him for an escaped patient and gave chase. Luckily, he was able to run faster than the guards, which showed not only how fit he was but also how fast swarms of bees can fly. We owe much to Lindauer, and to other ethologists who have exposed themselves to risk in the cause of science. When two Brazilian scientists decided to follow schools of piranhas by snorkeling directly above them, they dismissed the dangers of being attacked by the fish and were probably right to do so. They were on less sure ground in casually dismissing the danger of attack from caimans that were hunting nearby. With typical scientific understatement, they simply complained in their report that the caimans hampered their nighttime observations by muddying the waters with the lashing of their tails. The Brazilians were not the first to use snorkeling as a means of following schools of fish. That honor appears to belong to the Greek philosopher Aristotle, who is believed by some historians to have donned a face mask and thrust his bearded visage under the waters

THE PERFECT SWARM

of the Aegean Sea to observe that “sea basse [Dicentrarchus labrax ] and the grey mullet school together despite the hostility between the kinds.” But Aristotle risked no more than a wet beard. A scientist with whom I was studying coral reef ecology on Australia’s Great Barrier Reef risked rather more when he teased a supposedly harmless gummy shark that was lying on the bottom by poking it with his flipper. He explained to us that the shark had weak jaws and tiny blunt teeth. The shark proved him wrong on both counts by biting though his flipper and hanging on to it grimly. The water was 6 feet deep; the scientist was 5'10". The only way that he could escape drowning was to bend down, undo his flipper, and leave it with the shark. All of these scientists made original discoveries about the animals they were studying. University of Miami biologist Brian Partridge, however, was the first scientist to make real progress in understanding how groups of animals can sometimes move, act, and make decisions as though they were a single superanimal. His chosen species was the saithe. Saithe are also known as pollock (or pollack), and they are increasingly finding their way onto Western menus following the decline of the cod and haddock fisheries. There are two species: Alaskan pollock (Theragra chalcogrammais), said to be “the largest remaining source of palatable fish in the world,” and Atlantic Pollock (Pollachius virens). Both are around three feet long and weigh up to forty-five pounds. Partridge was studying Atlantic pollock. Like many fish, it exhibits schooling behavior. Partridge realized that to understand how the school stayed together and moved as a unit he would have to be able to identify and follow every individual fish. The first bit was easy—he simply branded each fish on the back (using a freezing brand rather than a red-hot one to make the mark). Following the fish was going to be rather more difficult. To do it he arranged to get schools of twenty to thirty saithe swimming around

The Emergence of Swarm Intelligence

a thirty-three-foot diameter doughnut-shaped tank at the University of Aberdeen in Scotland. While the fish were swimming, the experimenter lay on a rotating gantry above the tank and followed the movement of the school, recording a continuous race-track commentary on how the individual saithe were performing. Since the school was swimming at around one foot per second, this meant that the experimenter was being swung around, head down, at one revolution per minute. This doesn’t sound like much, but when I reproduced it on a merry-go-round at a child’s playground, I found it to be a fairly dizzying experience. Dizziness, however, was the least of the experimenter’s worries. After the experiments had been completed and the fish released (or eaten—his paper does not say which), Partridge and his helpers sat down to painstakingly measure the relative fish positions in more than twelve thousand frames of film. At the end, they discovered the key rules that permitted the school to move as a unit. There were just two, which each fish obeyed: follow the fish in front (if there is one*) and keep pace with the fish beside you. These two simple rules, expressed in various forms, are now known to underlie all sorts of complex group movements, from the wonderfully unified flashes caused by changes of shape or direction in schools of fish to the movements of flocks of birds, swarms of insects, and crowds of humans. But how does the complexity actually arise? What processes are involved? To find answers to these questions we must turn to another source, the world of physical science.

Learning from Science My first encounter with the application of science to the problems of complexity came while I was playing bridge in the early 1970s.

* If there isn’t one, then obviously only the second rule applies!

THE PERFECT SWARM

My partner was Robert (now Lord) May, and I couldn’t understand the squiggles and marks he was making in a notebook on the corner of the table when he was not playing a hand. I had no idea at the time that he was making history. Robert was puzzled by the behavior of a very simple equation called the “logistic difference equation,” which mathematicians use to describe how animal populations grow. It was a perfectly respectable equation, and it gave perfectly respectable answers. It predicted, for example, that populations would initially grow exponentially but that when food, space, or other resources became limited the population would plateau at a level that the environment could sustain. Robert had discovered a paradox, though. At certain rates of population growth, the equation went crazy. Instead of predicting smooth changes, it predicted cyclic or chaotic transitions between boom and bust, indicating that populations could appear to thrive but then suddenly crash. The equation produced these results because it contained elements of positive and negative feedback, elements that are now known to be central to the emergence of all sorts of complexity, including dramatic population fluctuations in nature, equally dramatic fluctuations in the stock market, and stable patterns such as those involved in swarm intelligence. Positive feedback is a cyclic process that is responsible for the squealing of a PA system at a concert when the amplification is turned up too high: the sound from the speakers is picked up by the microphone, which feeds it back to the speakers through an amplifier that makes it even louder, which sends it back to the speakers in a vicious circle that eventually overloads the system so that it produces a howl of protest. The scientific intelligence expert R. V. Jones observed a wonderful example at a lonely airfield during World War II. A microphone and loudspeaker had been set up on opposite sides of the landing strip, and someone happened to laugh near the microphone. The amplifying system was just on the edge of positive feedback, and the laugh was very slowly amplified after the person had

The Emergence of Swarm Intelligence

walked away, leading Jones to speculate that a human was no longer needed, and here was a machine that could laugh by itself. The credit crisis that began in 2008 provides a less humorous example of the effects of positive feedback, which in this case amplified a mistrust of financial institutions until the worldwide financial system was in danger of collapse. Many individual financial institutions have already failed under the stresses caused by positive feedback, which makes its presence known in the form of a run on these institutions. One example is the collapse of Washington Mutual on September 25, 2008. Over ten days, more and more investors had realized that others were withdrawing their money, and they rushed to withdraw theirs as well. The total reached $16.7 billion. Strong preferences, or fashions, can also arise from seemingly insignificant beginnings through the operation of positive feedback on a small random fluctuation. Say, for example, that most of your friends own either a Ford or a Toyota, and you are trying to choose between the two when it comes time to buy a new car. You ask around, and just by accident the first three people you encounter own a Ford and are perfectly happy with it. So you buy a Ford. After you buy a Ford there is one more person in the group who owns a Ford, which slightly increases the chance that the next person who asks around will talk to Ford owners rather than Toyota owners. If she buys a Ford, too, there will be two more people in the group who own Fords. The “Ford effect” amplifies, and eventually most of the group owns Fords. (It would have been the “Toyota effect” if you had talked initially to several people who owned Toyotas.) The pattern of everyone owning a Ford (or a Toyota) is one that has emerged from the application of a simple local rule (choose the car that the first three people that you meet own and are happy with) together with the action of positive feedback on a chance fluctuation (the first three happened to own the same brand). Positive feedback is not the only way to produce a runaway effect. Such effects can also arise from a chain reaction, such as the one

THE PERFECT SWARM

described by James Thurber in “The Day the Dam Broke,” part of his autobiography. Triggered initially by the sight of just one person running, the entire citizenry of the East side of Columbus, Ohio, fled from a nonexistent tidal wave, despite reassurances that there was no cause for alarm. Thurber and his family were among those running. “We were passed,” says Thurber, “in the first half-mile, by practically everyone in the city.” One panicking citizen even heard the sound of “rushing water” coming up behind him; it turned out to be the sound of roller skates. The panic arose because the sight of the first person running got several other people running, and the sight of each of these got a few more people running, and so on. The process continued until everyone was running. It is the same process that is at work in an atom bomb, when the disintegration of an atomic nucleus releases energetic neutrons with enough energy to break up several nearby nuclei, and each of these produces enough neutrons as it disintegrates to break up several more. The ongoing cascade produces rapid exponential growth in the number of neutrons and the amount of energy being released until there is a giant explosion. The chain reaction is controlled in nuclear power stations by the insertion of cadmium rods into the disintegrating material. The rods absorb a sufficient number of neutrons to block the chain reaction and produce a controlled release of energy instead. One of the great discoveries of complexity science is that a similar stabilizing outcome can be produced in many social situations by introducing negative feedback to counteract the destabilizing effects of chain reactions and positive feedback.* The result is a complex dynamic pattern with its own inherent stability, but also with the potential for evolution and growth. * To avoid confusion, we should note that the physical scientist’s meaning for the terms positive and negative feedback are not the same as those of a psychologist. To a psychologist, negative feedback is destructive and destabilizing, while positive feedback is good and desirable. To a physical scientist, the implications are usually the reverse.

The Emergence of Swarm Intelligence

Negative feedback exerts its balancing effect by acting to preserve the status quo. A simple example is a governor on a motor, which acts to progressively reduce the rate at which fuel is supplied as the engine speeds up, so that the engine can never run out of control. Negative feedback is frequently used to “correct” errors. When an error starts to creep in, the change from the status quo initiates a feedback process that acts to correct the error. When you are driving your car, for example, and you start to drift slightly to the right, your brain automatically applies negative feedback to get you to turn the steering wheel slightly to the left so as to bring you back on course. Positive feedback, which progressively amplifies small effects, would have you turning the wheel farther to the right and sending you farther off course. In economics, Adam Smith’s concept of the invisible hand, which says that the marketplace is self-regulating and will always return to equilibrium after a disturbance, is based on the idea that the institution of the marketplace has built-in negative feedback. As we shall see, modern complexity theory recognizes that this is far from being the case in practice, and that our complex economic system is governed by an intricate balance of positive and negative feedback, with the occasional chain reaction thrown in. The balance ultimately depends on the rules of interaction between individuals (these rules are known technically as “behavioral algorithms”). Two of the key problems in understanding the emergence of collective properties like swarm intelligence are identifying the patterns of interaction that individual animals (including ourselves) follow and detecting how information flows between the animals. Much of this book is concerned with the former, and working out how we can use these patterns to our advantage. For a group to have collective adaptability (the ability of the group as a whole to respond to changing circ*mstances) nonlinear rules alone are not usually sufficient. Complexity theorists John Miller and Scott Page list a total of eight criteria for collective adaptability, based loosely but respectfully on the Buddhist eight-fold path:

THE PERFECT SWARM

Right View The individuals in a group (complexity scientists call them “agents”) must be able to receive and make sense of information from other individuals in the group and from the world in general. Right Intention Agents must have some sort of goal that they want to achieve. Fish may want to avoid being eaten, for example, while people may want to act collectively to achieve political change. Right Speech Agents must be able to transmit information as well as to receive it. This need not involve actual speech. Cells in the communal slime mold Dictyostelium discoideum, for example, communicate by sending chemical messages, and the neurons in our brains communicate via electrical impulses. Right Action Agents must be able to influence the actions of nearby agents in some way. Right Livelihood Agents must receive some sort of payoff for their actions within a group, such as a salary for a task performed or the threat of a punishment like dismissal if the task is not performed. Right Effort Agents need strategies that they can use as they anticipate and respond to the actions of others. Right Mindfulness There are many kinds and levels of rationality. Our task as agents in a complex society is to choose and use the right level of each. Right Concentration To understand how complexity emerges, we sometimes have to go back to the old scientific approach of concentrating on one or two important processes, temporarily ignoring the rest.

The Emergence of Swarm Intelligence

All of these criteria are covered in the pages that follow, sometimes in very different contexts. Right mindfulness, for example, covers the level of detail that we need to have to make good individual decisions and also the ways of thinking that we need to adopt to reach consensus as a group.

Virtual Worlds To understand how these criteria influence our choice of behavior in complex situations, we often need to resort to computer modeling. Predicting their consequences can be virtually impossible without the aid of such simulation, both for practical and ethical reasons. One practical reason is that the human mind simply cannot encompass all of the variations and variability inherent in complex adaptive systems. This is why science has progressed almost exclusively in the past by making severe simplifications that allow us to abstract the essentials of a problem. Even when it comes to the relative movements of the sun, the Earth, and the moon, we can calculate the orbits of any pair around each other only by ignoring the effects of the third body. An exact calculation of the three together (known as the “threebody problem”) is beyond our analytical powers, and we have to rely on computer simulations just to get a close approximation. Interactions in society are more complex, and it is only with the advent of powerful computers that we have been able to model how complexity can emerge from simplicity. Such models are now used to understand aspects of crowd behavior, networking, and other aspects of our complex society. (Studies of crowd behavior in particular frequently preclude experimentation because of ethical concerns, especially if an experiment would involve putting individuals in dangerous situations.) The models are rather similar to those of games like Tomb Raider, where virtual individuals are given specific rules of behavior. In the

THE PERFECT SWARM

world of complexity science, though, there is no outside player to control what then happens. Instead, the virtual individuals are released in their virtual world, armed only with rules for interaction, while the programmer watches to see what happens. The rules might be guesses about how people interact with each other in a crowd, for example, and the outcome would be the behavior of the crowd when the individuals follow those rules. By

The Logistic Difference Equation The equation below looks incredibly simple at first sight, but it has probably driven more mathematicians crazy than any other equation in history. It was first applied to population growth. If a population of p individuals can grow without limit, and it does so at a constant rate r, then we can simply write: ppresent = r x pprevious

If the population were growing at 3 percent per year, for example, and the population were measured on the same date each year, then the value of r would be 1.03. This is called exponential growth, and it is quite clear that our planet cannot support it indefinitely. No matter what adaptations we make, there must be some upper limit. Let’s call K the largest population that the Earth could sustain, and follow the clever idea of the Belgian mathematician Pierre François Verhulst, who in 1838 proposed a simple equation to describe the way in which population growth must slow down as it approaches its upper limit, and even become negative if it overshoots that limit. Verhulst’s equation, the logistic difference equation, is:

ppresent = r x pprevious ((K– pprevious)/K )

This simple-looking equation (note that it is nonlinear, because pprevious gets multiplied by itself) has produced some truly extraordinary insights. If you really

The Emergence of Swarm Intelligence

tweaking the rules, the programmers can come up with reasonable suggestions for the most productive way for individuals to behave in crowds, and also for the best designs for the environments in which crowds might gather, such as city streets, stadiums, and nightclubs. One other use of computer programming is to mimic the way in which social animals (particularly insects) use swarm intelligence to solve problems. A swarm of virtual individuals is let loose in the

The Logistic Difference Equation (continued)

hate algebra, just skip the next two paragraphs, but do have a look at what follows. At first sight, the equation looks really neat. When populations are far from their limit, pprevious is much smaller than K, and the equation simplifies to the exponential growth equation. When populations get close to their limit, the growth slows right down as (K–pprevious) gets closer and closer to zero. This equation neatly describes the growth of bacteria in a petri dish and algae on a pond (so long as the food or light doesn’t run out). If you draw a graph of population total as a function of time, it comes out as a classical sigmoid shape, with exponential growth at the beginning and an asymptotic plateau after a longer time—so long as the rate of growth is not too high. Everything stays normal until we reach a tripling rate of growth (r = 3), and then strange things start to happen. The smooth population growth curve breaks into oscillations between two values that correspond to “boom” and “bust.” By the time that the growth rate reaches 3.4495, the curve is oscillating between four values. When the growth rate reaches 3.596, there are sixteen states, with the population oscillating rapidly between them. A little above that, complete chaos ensues. The mathematics of boom and bust accurately describes many events that happen in the real world. Unfortunately, this doesn’t always make them easier to predict (as demonstrated by the history of the 2008 credit crisis), partly because the model is always a simplified system compared to reality, but also because the behavior of the system can depend very sensitively on the precise conditions.

THE PERFECT SWARM

artificial computer environment, but this time the environment is designed to reflect the problem to be solved. The individuals might, for example, be given the task of finding the quickest routes through a network that mimics the arrangement of city streets or the routes in a telecommunications network. Amazingly, the solutions that the swarm comes up with are often better than those produced by the most advanced mathematics. All of these uses of computer programming, scientific rules, and lessons from the animal kingdom are covered in the following pages. We begin with the lessons that locusts, bees, and ants have to offer. Each of them uses subtly different forms of swarm intelligence, and each of their approaches has something different to tell us about group problem solving in our own world.

TWO

The Locusts and the Bees

Locusts and Us Locusts are distinguished from other types of grasshoppers because their behavior changes radically when conditions become crowded. Normally shy and solitary, the close proximity of other locusts turns them into party animals. In the case of the African desert locust, this is because the proximity stimulates them to produce the neuromodulator chemical serotonin, which not only makes them gregarious but also stimulates other nearby locusts to generate serotonin as well. The ensuing chain reaction soon has all the locusts in the vicinity seeking each other’s company. The locusts also become darker, stronger, and much more mobile. They start moving en masse, first on the ground and then in the air, gathering other locusts as they go and forming dense swarms that can eventually cover an area of up to 500 square miles and contain a hundred billion locusts, each eating its own weight in food each day over their lifetime of two months or so.

THE PERFECT SWARM

Descriptions of such locust plagues appear in the Qur’an, the Bible, the Torah, and other ancient texts, and modern plagues affect the livelihoods of 10 percent of the world’s population. It is little wonder that scientists want to understand what it is that drives the locusts to mass together and travel in such huge numbers. The behaviors they have uncovered give us vital clues about the self-organization of other animal groups, from social insects to human crowds.* When individual locusts first start moving, they are still in their juvenile, wingless form. At first their movements are more or less random, but as the population density increases, their directions of movement become more and more aligned. When the population density becomes very high (around seven locusts per square foot), a dramatic and rapid transition occurs: the still somewhat disordered movement of individuals within the group changes to highly aligned marching. Rather similar transitions happen in human crowds. At low densities, the movement of individuals can be likened to the random movement of molecules in a gas, as engineer Roy Henderson discovered when he monitored the movements of college students on a campus and children on a playground. In both cases, he found that the movements fit an equation (called the Maxwell-Boltzmann distribution) that describes the distribution of speeds among gas molecules. When he applied the theory to the observed movements of students and children, he found that their distribution of speeds followed the same pattern. The only difference between the students and the children was that the children had much more energy and consequently moved at much higher average speeds. Video studies of pedestrians show that their movements have a similar random component, although an overall direction is super-

* Luckily for us, of the twelve thousand known grasshopper species, fewer than twenty are locusts, although there are species of locust endemic to every continent except Antarctica.

The Locusts and the Bees

posed on their movements by the desire to reach a goal. When the pedestrian density reaches a critical value, however, spontaneously self-organized rivers of pedestrians start to form and flow past each other, with everyone in a particular river walking at the same speed, just like marching locusts. How does such self-organization occur? Are the basic mechanisms the same for locusts and for people? Can the collective behavior of locusts and other insects tell us anything about the behavior of human crowds? Over the course of the next four chapters I give answers to these questions, beginning here with the fundamental one: what are the forces that produce swarm behavior? In the case of marching locusts, one of those forces is the simple desire not to be eaten by the locust behind! Marching locusts are in search of food, and the locust in front provides a tasty temptation. The way to avoid being eaten is to keep marching and to keep your distance, just as the way to avoid being pushed from behind in a human crowd is to keep moving steadily forward. But keep-your-distance is not in itself sufficient to explain the selforganized synchrony of a group of marching locusts. If that were all there was to it, the group would simply disperse. There must be a balancing force to hold the group together. That force is provided by the serotonin-induced drive for company, which increases disproportionately with the number of similarly inclined locusts nearby. It is, in other words, nonlinear, and positive feedback is thrown in as more and more locusts are recruited, increasing the gregarious drive of those already in the group—just the conditions that are needed for the emergence of complex collective behavior. To understand how that collective behavior emerges, we need to turn to computer simulation. The first such simulation (produced in 1986 by animator Craig Reynolds) used small triangular objects called “boids.” The original animation is still worth a look. It laid the foundation for all of our subsequent advances in understanding complex collective behavior.

THE PERFECT SWARM

Boids Reynolds’ boids are small isosceles triangles. They wheel, dive, and disappear into the distance in a manner highly reminiscent of flocks of real birds. The audience that experienced their first public presentation, at a conference on “artificial life,” was particularly impressed by the way in which the flock split into subflocks to go around a pole (a circle on the screen) and then unified itself again on the other side. They were even more impressed when one boid crashed into the pole, fluttered for a while as though stunned, and then flew on to rejoin the flock. Such lifelike behavior would seem to require very complicated, very sophisticated programming. But in fact the program is quite short, and the individual boids follow just three simple rules: •

Avoid bumping into other individuals.

•

Move in the average direction that those closest to you are heading.

•

Move toward the average position of those closest to you.

These can be more succinctly described as: •

Avoidance (separation)

•

Alignment

•

Attraction (cohesion)

Next time you find yourself in a crowd at an airport, a train station, or a football game, take some time to watch those around you as they walk. You will usually find that most people are obeying the same three rules. Reynolds’ goal was to demonstrate that lifelike collective behaviors can emerge from simple interactions between individuals. Al-

The Locusts and the Bees

though he did not know it at the time, the three rules he used corresponded to the empirical rules discovered by Brian Partridge in his studies of fish schools. Partridge didn’t mention avoidance (probably taking it as obvious), but the other two rules he identified are equivalent to Reynolds’ rules of alignment and attraction. The optimum way for all fish to maintain pace with those alongside them and simultaneously following those fish in front is to move in the average direction of the nearest individual fish and toward their average position (concomitant with not actually bumping into them). Reynolds’ original model was taken up enthusiastically by the computer animation industry, where it is still used today. Its value to that industry is undoubted, but its deeper worth lies in the help it continues to give us as we unravel the secrets of collective behavior, such as that of locusts marching in synchrony.

Locust Logic Computer simulations have shown that synchrony emerges because each locust acts as a self-propelled particle whose velocity (i.e., speed and direction) is determined by those of its neighbors according to a specific built-in rule. This sounds like one rule rather than three, but a closer look reveals that this single rule can be decomposed into three rules that are similar to those proposed by Reynolds: follow the locusts in front, keep pace with the locusts alongside, and keep your distance from the ones behind. When all locusts in the group obey the same rules, the result is synchrony. (I show in chapter 4 that a similar synchrony can emerge in dense human crowds.) When locusts develop wings and start to fly, things change. Now the whole sky is at their disposal, and they have more to fear from birds and other predators than from their fellow locusts. The direction the swarm takes is determined by the wind, but the urge to stick

THE PERFECT SWARM

together is still strong, since flying with the group reduces the risk of predation on any individual. But when flying, locusts need more space, because a midair crash could damage their delicate wings, leaving them on the ground in an area where every vestige of food has already been consumed by the swarm. The new balance of forces still reflects in Reynolds’ three rules, but the relative importance of the rules is different. The last two rules become relatively weak (although still strong enough to keep the swarm together), while the avoidance rule becomes stronger. Implementing the avoidance rule starts with increased sensitivity to the presence of moving objects, especially to those coming from the side. The early Star Wars experiments showed that locusts flinched mentally when they noticed such objects approaching. Later experiments, in which the locusts were allowed to fly freely (although tethered to a length of cotton) showed that their response to an object coming from the side was to close their wings and go into a brief diving glide. This strategy gives them the best chance of avoiding a collision, and of protecting their wings if there is a collision. We adopt a rather similar strategy when walking in dense crowds. Instead of folding our wings, we keep our arms close to our sides. Instead of going into a diving glide, we shorten our steps or even stop moving. The overall effect, as with locusts, is to strengthen the avoidance rule as much as possible. This and other minor modifications of Reynolds’ three rules are sufficient to explain many aspects of swarm behavior, but no simple modification of the rules is sufficient to explain the emergence of true swarm intelligence. Reynolds’ rules explain how a group can collectively respond to external circ*mstances, but swarm intelligence needs something more—the ability to learn. This requires some additional form of communication within the group—the sort of communication that is shown, for example, by bees.

The Locusts and the Bees

Bee Logic Individual bees in swarms follow the basic rules of avoidance, alignment, and attraction, but the swarm as a whole has something that locust swarms don’t—an ability to fly directly to a target that has been identified by scouts. The way the swarm does this provides the first clue to the processes by which swarm intelligence emerges. “Well,” you might think, “it’s pretty obvious how they find the target. They use the well-known waggle dance. It’s the method that bee scouts use to tell the others where something is, such as a food source or a site for a new home. The scouts dance like teenagers in a disco, waggling their abdomens while moving in a tight figure eight. The overall direction of the dance points in the direction of the target, and the speed of the waggling tells how far away it is. Unfortunately this explanation doesn’t provide a full answer. The dance is performed in a hive that is almost as dark as some discos, so only those bees nearby (about 5 percent of the total) see the dance. The majority doesn’t see it, so most bees start flying in complete ignorance. Those that have seen the dance aren’t even out in front, showing the others the way. They are in the middle of the swarm, flying with the rest. So how does the swarm find the target? There seem to be two main possibilities: (1) the bees who know where the target is might emit a pheromone, and (2) those bees may behave in a way that flags them as the leaders. To check out the first possibility, scientists covered the Nasonov glands (the ones that emit pheromones) of each bee in a swarm with a spot of paint. They found that the swarm still flew straight to the target, which disproved the pheromone hypothesis. The swarm-following scientist Martin Lindauer discovered a clue that pointed to the possibility of leaders when he looked closely at swarms flying overhead. (Presumably he was running on flat ground.

THE PERFECT SWARM

If it had been me, I would have tripped over the nearest tree root and fallen flat on my face.) He noticed that a few bees were flying much faster than the others in the swarm, and that they seemed to be flying in the direction of the target. Fifty years later, other scientists confirmed his observation by photographing a swarm from below, leaving the camera aperture open for a short length of time so that individual bees appeared as dark tracks against the sky. Most of the tracks were short and curved, but a few tracks were longer (indicating that the bees were flying faster), and also straighter, with the lines pointing toward the target. The bees that produce the straight tracks have been evocatively named “streakers.” It seems at first that these would be the bees that know where the target is and that their behavior is intended to guide the other bees. It still remains to be proved whether the streakers are those that have received information from the scouts, but computer simulations of bee swarms have produced a stunning surprise—it doesn’t matter. Simulations have revealed that the knowledgeable bees do not need to identify or advertise themselves to the rest of the swarm to lead it successfully. Just a few informed individuals can lead a much larger group of uninformed individuals simply by moving faster and in the appropriate direction. Guidance is achieved by way of a cascade effect, in which uninformed individuals align their directions with those of their neighbors. Even if only a few bees know their way, Reynolds’ three rules—avoidance, alignment, and attraction— ensure that the whole swarm moves in the direction that those knowledgeable bees take. Leadership by these few individuals arises, according to the computer modelers, “simply as a function of information differences between informed and uninformed individuals.” In other words, it needs only a few anonymous individuals who have a definite goal in mind, and definite knowledge of how to reach it, for the rest of the

The Locusts and the Bees

group to follow them to that goal, unaware that they are following. The only requirements are that the other individuals have a conscious or unconscious desire to stay with the group and that they do not have conflicting goals. The purposeful movement of bee swarms, in other words, is an example of an emergent complex behavior that arises from simple local interactions guided by appropriate rules. Japanese scientists have already taken advantage of this discovery to design robots that will swarm around a human leader and follow the leader happily across a factory floor as they are led to perform a task. The robots have no goals of their own, just a built-in desire to stay with the group, using only Reynolds’ three swarm rules to stay together and follow the leader. Could rules that apply to robots also apply to us? Surely if we were in a group, we wouldn’t blindly follow unidentified “leaders” to a goal that only the leaders knew about? Oh yes we would.

Invisible Leaders Volunteer groups of university students were asked to participate in an experiment in which they were instructed to walk randomly in a circular room that had labels with the letters A to J distributed uniformly around the wall. The students were instructed to walk at normal speed, and not to stop until they were told to. They were allowed to walk anywhere in the space, required to stay within arm’s length of at least one other person, and forbidden to talk or gesture. This way, they met the swarm criteria of staying with the group but not having any particular goal in mind. A few of the students were given an additional secret instruction: go to a specific label, but without leaving the group. By the time the students were told to stop walking, most of them had ended up near

THE PERFECT SWARM

the same label. They were led there, but they did not know that they had been led. We tend to think of leaders as being highly visible and needing specific qualities in order to lead effectively. Leadership guru John C. Maxwell, author of the best-selling books The 21 Irrefutable Laws of Leadership and The 21 Indispensable Qualities of a Leader, lists qualities such as charisma, relationship, and vision as being essential. The above experiments show, however, that there is another possibility: we can lead a group simply by having a goal, so long as the others in the group do not have different goals. Leading from within is, of course, a well-known strategy, encapsulated in the phrase “the power behind the throne” or the term éminence grise. This sort of leading, though, is not what the experiments or the simulations were about. Persons such as Dick Cheney, Edith Wilson, and Cardinal Wolsey have exercised an influence that was not always visible from the outside; but George W. Bush, Woodrow Wilson, and Henry VIII—their respective puppets—certainly knew who was pulling the strings. What the computer models predict, and what experiments show, is that members of a group can be totally unrecognized as leaders by those whom they are leading. Furthermore, computer simulations have shown that “the larger the group, the smaller the proportion of informed individuals needed to guide the group with a given accuracy.” In the case of the students, the group needed only ten informed people out of two hundred (just 5 percent of the group) to have a 90 percent chance of success in leading the rest of the group to the target. Sometimes the target doesn’t even need to be a real one. In 1969, the famous social psychologist Stanley Milgram arranged for groups of people to stand in a New Haven, Connecticut, street and stare up at a sixth-floor window, an experiment that has become a classic. With just one person staring up, 40 percent of passersby stopped to stare with them. With two people, the proportion rose to 60 percent,

The Locusts and the Bees

and with five it was up to 90 percent. His results conform beautifully with later discoveries about invisible leaders. Maybe Connecticutians are unduly gullible, but when I repeated the experiment on a busy street in Sydney, I found that Australians are equally gullible, or at least biddable. I made my leaders even less visible, having them melt away when the crowd became large enough, leaving a crowd that stared by itself. On a more serious note, the presence of a few knowledgeable individuals in a swarm can make a world of difference to its performance. Without such informed individuals, the group can only react to external circ*mstances, as fish do when their school reacts as a unit to the approach of a predator, or as locusts do when they fly as a group in the direction of the prevailing wind. Swarm intelligence in the absence of individual knowledge and goals keeps a group together and allows it to react to situations, but it is difficult, if not impossible, for the group to be proactive. Bee logic changes all that. The Grammy Award–winning Orpheus Chamber Orchestra provides a real-life example. Audiences at Carnegie Hall who have not previously seen the orchestra can be surprised to see the black-clad members take their place on stage and begin to play—without a conductor. The orchestra, also the winner of a WorldBlu—a “Worldwide Award for the Most Democratic Workplaces”—appears to produce its beautifully coordinated sounds by democracy alone. How does this work? The orchestra does it by using invisible leaders. The music does not degenerate into an anarchic mess because a core of six out of the thirty-one players sets the musical agenda for each piece. The leaders are not invisible only to us in the audience. They are also effectively invisible to the other players during the performance. Those players are aware of who the leaders are, but they are not consciously watching them in the same way that they would watch and obey a conductor. They have, however, set aside their own agendas so that

THE PERFECT SWARM

they are free to be swept along by those few in the group who do have a specific agenda. The idea of an invisible leader working within a group is as ancient as civilization itself. According to a Chinese proverb often attributed to Laozi, the founder of Taoism, “a leader is best when people barely know he exists . . . when his work is done, his aim fulfilled, they will say, ‘We did it ourselves.’” What is new here is the proof, both theoretical and practical, that a leader (or group of leaders) can guide a group toward an objective from within and never be recognized. This suggests a rule that we can use as individuals within a group: Lead from the inside (if possible with a coterie of like-minded friends or colleagues), but take care not to let other members of the group know what you are doing. Just head in the direction that you want to go, and leave it to the rules of the swarm to do the rest. The process works in groups of individuals who have an innate or learned tendency to follow the example of those nearby. Just a few individuals taking the lead instead of copying is sufficient to induce a chain reaction of copying, and soon the whole group will be following their example. Any deviations will quickly be brought into line by negative feedback, physical and social pressures conspiring to push deviant individuals back into moving with the rest. The more the deviation, the stronger the pressure. I asked Jens Krause, the supervisor of the original experimental study with the students in the circular room, whether he knew of any real-life examples of leading from within. He gave me one from his own experience. “Recently I got off the plane in Rome at midnight,” he said, and the airline stewards provided no help in directing us to the terminal. It was dark, the passengers didn’t know each other, nobody talked and most people looked utterly clueless. But sud-

The Locusts and the Bees

denly two of them walked off purposefully in a particular direction and the group self-organized into following them [in a chain reaction where a few followed the first two, and then a few more followed each of them, and so on]. Sure enough, they guided us to the right terminal. When the [experimental] study was published, we were contacted by historians of warfare who pointed out that the leadership of small groups can engage a whole army. Police officers pointed out that they try to remove a small proportion of troublemakers during demonstrations or town fights to control whole crowds. At conferences it often happens that scientists stand in small groups talking to each other and not realizing that it is time for the next thing on the agenda. However, it is enough for a few people to start walking, and once they initiate a direction, most people follow (often while still talking), not anticipating where they are going or what the next activity will be—they realize this only after they arrive.

These processes sound obvious, but the ways in which individual knowledge and behaviors can influence a group depend on a subtle dynamic interplay of positive feedback, negative feedback, and cascading chain reactions. Locust and bee logics provide important clues as to how these processes can interact to produce swarm intelligence, but these are just the first pieces in the puzzle. As I show in the next chapter, many more pieces have been discovered through the study of another social insect—the ant.

THREE

Ant Logic

“Go to the ant, thou sluggard; consider her ways, and be wise.” So says the biblical proverb, and modern scientists have been following its sage advice to learn from the ants. In doing so they have learned many lessons about the evolution of complexity and have even been able to produce a computerized version of the ants’ approach to problem solving. Ants face difficult decisions in their lives, not least in their choice of routes to a food source, where it is important to establish the shortest route so that they waste as little energy as possible when carrying the food back to the nest. Judging by the ant trails in my garden, they seem to do it pretty efficiently. The trails are invariably straight, representing the shortest distance between two points. Ants can distinguish objects up to three feet away (depending on the size of the ant), but their trails can be many feet long, with the food source hidden from view by intervening rocks, leaves, and sticks. How do the ants establish such wonderfully straight trails? Experiments on a laboratory colony of Argentine ants (Iridomyrex humilis) provided the answer. Researchers at the Unit of Behavioral Ecology in the University of Brussels set up a bridge between the 37

THE PERFECT SWARM

colony and the food source. The bridge was split in the middle to offer a choice between two curved routes, one twice as long as the other. The first ants to leave the colony in search of food chose one or the other branch at random, but within a few minutes practically the whole colony had discovered the shortest route, just as we rapidly discover a shortcut that provides a quicker way to get from home to the office. “Finding the shortest route,” said the scientists, “is extremely important, not only for Roman road builders, thirsty rugbymen, and applied mathematicians working on this very problem, but also for any animal [including humans] that must move regularly between different points.” They discovered that the ants were finding the shortest routes not by looking at their watches to check the time but by using chemical signaling compounds, pheromones, which they laid down as they traveled so that other ants could follow their trail. But how could these signaling compounds help them to find the shortest route? The reason, as Sherlock Holmes once said, is very obvious—once you think of it. The first ants to return to the nest after foraging are those that happen to have chosen the shortest path. They will have laid down pheromones to mark the trail, and this trail will be followed by other ants, who are genetically programmed to “follow the pheromone.” By the time that ants using the longer trail have returned, there will be more ants using the shorter trail, and laying down yet more pheromone. Furthermore, ants that went out by the longer trail but returned by the shorter trail will also add pheromone to the shorter trail. In the end, the shorter trail will have an overwhelmingly higher concentration of pheromone, so it will be preferred by most of the ants. The ants use the selective reinforcement of a good solution to find the most efficient route to their food source. We use a similar process to find shortcuts when we’re driving. When someone finds a short-

Ant Logic

cut, a few of other people might notice others leaving or rejoining the main route and might follow them on the suspicion that they have found a shortcut. Each of those might be noticed by a few more people, and so on—and the cascade amplifies. This process of positive feedback ensures that soon everyone knows about the route. We don’t even need pheromones to give us the message—just observation. In a computerized version of ant logic known as “ant colony optimization,” positive feedback is similarly beneficial. Let’s say that a programmer is faced with the problem of planning a bus tour between a number of cities. How can she determine the shortest route, or the quickest one if roads with different speed limits are involved? It sounds easy to solve, but an exact mathematical solution to this class of problem (known as the “traveling salesman problem”) has eluded mathematicians for centuries—and it continues to elude them. The problem is so important, both theoretically and for practical applications, that it even has its own website (http://www.tsp.gatech.edu/), which contains much information on its history and applications. One way to tackle the problem, with the aid of modern computers, is simply to measure the length of travel time for all of the possible routes and then choose the best from the list. This might work when there are only a few cities involved, but the calculations can get out of hand rapidly. To calculate the optimal route that Ulysses might have taken between the sixteen cities mentioned in The Odyssey, for example, requires the evaluation of 653,837,184,000 possible routes by which he might have gone out and returned home. That adds up to something like ten thousand billion calculations, which takes some doing, even for a modern computer. Ants do it a different way, using the positive feedback principle to obtain a good approximate solution. The virtual insects in computer simulations are let loose in an imaginary world of the sixteen cities with instructions to visit every city before returning home. The cities

THE PERFECT SWARM

are joined by imaginary lines (called “links”) in such a way that each city is connected to every other one, with the distance between each pair of cities being represented by the length of the line. Now comes the clever bit. When a virtual ant returns home, it remembers how far it has traveled and attaches a number (the equivalent of a pheromone) to each link that reflects the total length of the journey. Each link gets the same number, and the shorter the journey, the higher the number. As more and more “ants” travel through the network, those links that belong to the shortest journeys accumulate higher and higher total numbers (equivalent to higher concentrations of pheromone). The numbers increase even more because the following ants are instructed, when confronted with a choice of links, to show some preference for the one that already has the highest number associated with it. And now for the really clever bit. The size of the numbers gradually diminishes with time in a programmed countdown that corresponds to the slow evaporation of the pheromone trail, which is what happens in the real world. The effect of this is a disproportionate reduction of the numbers for inefficient links (equivalent to the gradual de-emphasis of trails that are used less because of pheromone evaporation), so that the most efficient links stand out more clearly. Soon, the most efficient route (or one that is very close to the most efficient) stands out for all to see. Ant colony optimization has done its job. It is doing many such jobs now, especially in the telecommunications industry, where the traveling salesman problem refers to routing messages through complicated networks in the most efficient manner. The messages themselves become the ants, recording their progress and marking the trail accordingly. Could we use a similar procedure to resolve traveling and networking problems in our own lives? Robert J. Dillon, one of the original Central Park commissioners, had one idea when he suggested

Ant Logic

in 1856 that the planning of pathways in the park should be postponed until New York City pedestrians had established them by habit, with the more deeply marked paths corresponding to those that were most used and therefore most efficient. Dillon did not get his way, but recent research by German traffic engineer Dirk Helbing and his colleagues has shown that Dillon’s solution, a neat example of ant colony optimization as practiced in human society, would have been a good one. Helbing and his colleagues have photographed and analyzed many such paths, and when I asked him about the validity of Dillon’s approach he replied, If people use trails frequently enough, direct connections between origins and destinations are likely to result. If the frequency of usage would not be high enough to maintain direct trails in competition with the regeneration of the natural ground, however, pedestrians with different destinations form commonly used trail segments and are ready to make compromises. In that case they accept detours of up to 25 percent, but the resulting trail system will usually provide an efficient and fair solution that optimizes the walking comfort and minimizes detours within the limits of how many trails can be maintained by the usage frequency.

When ant colony optimization is available to us, it seems that we use it spontaneously, producing solutions that are reasonably close to optimum. Getting the best solutions, however, requires very carefully setting conditions, as has happened with the communal website Digg.com, which allows its users to submit news stories they find as they browse the Internet. New submissions appear on a repository page called “Upcoming Stories.” When other members read the story, they can add a digg point to it if they find it interesting. If a submission fails to receive

THE PERFECT SWARM

enough diggs within a certain time, it is removed. If it earns a critical number of diggs quickly enough, it jumps to the front page, where it may continue to receive more diggs. This process of positive feedback is counterbalanced by the fact that the novelty of stories decays with time. The effect is similar to pheromone trail evaporation and link number decline: the stories receive less and less notice and fewer and fewer diggs. Eventually they disappear off the front page, to be replaced by newer and more interesting stories that have come to the fore. The “Letters to the Editor” section of my local newspaper follows a similar pattern. If a topic attracts enough letters, more letters are likely to pour in, and editors seem to be more inclined to print letters on that topic. With time, boredom sets in, the flood of letters slows to a trickle, and editors may announce a stop to the correspondence on that issue. Positive feedback can be very useful for bringing attention to an issue and keeping it on the public agenda. This just needs a bit more thought and planning than many community action groups seem to give. Taking a lesson from ant colony optimization, the best strategy is not to fire off a whole slew of letters at once and then leave it at that, but to plan for members of a group to keep up a steady flow of letters on different aspects of the issue. This strategy is equivalent to continually adding to the concentration of a pheromone before it has time to evaporate, and it suggests another rule: When trying to bring an issue to the notice of a group or the public as a whole, don’t be a one-hit wonder; plan to bring different aspects to the fore in succession over time. Ant colony optimization is useful for suggesting analogous strategies, but its lessons are not the only ones we can learn from ant logic. Closer to our everyday experience is a modification of ant colony optimization called “ant colony routing” In this process the antlike agents that inhabit a virtual computer world learn from experience

Ant Logic

what the shortest and fastest routes are. The next time they are called upon to perform, they use their memory of the routes rather than relying on signs left by previous users. If the routes are those in a communications network, for example, the agents remember the parts of the network that are most likely to become congested and seek new routes, just as we do when we have a choice of a number of routes between home and work. The new routes eventually become congested in their turn, but ant colony routing can cope with these dynamic changes in a way that standard ant colony optimization does not. The ultimate application of ant logic to problem solving has come in the form particle swarm optimization, a combination of locust, bee, and ant logics that no insect could have come up with, which emerged from the fertile minds of Russell Eberhart and Jim Kennedy (from the Purdue School of Engineering and Technology at Indiana University and the U.S. Bureau of Labor Statistics respectively) as the culmination of a search for a form of computerized swarm intelligence with the broadest possible problem-solving ability. The way it works is a bit like sitting for an exam in which cheating is allowed. Each candidate writes down his best answer but is allowed to look over the shoulders of those near him and to modify his answer if he thinks that someone else’s is better. That’s not the end of it, though, because the student next door may come up with still a better answer after looking at the answers of the students near her. Then the first student can produce a further improvement by copying that answer. Over time, the whole class will eventually converge on the real best answer through this process of positive feedback. The lesson is this: if you see people doing something a better way, then copy them. It is one that I was reminded of when my adult son was helping me drag dead branches along our lengthy driveway to be put out on the street for mulching. I noticed that he was gradually

THE PERFECT SWARM

getting ahead of me as the pile grew, and I couldn’t figure out why until I realized that the pile was on the left, and he was dragging the branches under his left arm (and so could drop them straight on the pile) while I was carrying them under my stronger right, but had to then put them down, walk round them, and pick them up again to place them on the pile. The lesson is especially effective in larger groups, in which best practices can rapidly spread throughout by way of positive feedback and repeated learning from those nearby. Eberhart and Kennedy produced a computer analog of this process by replacing the virtual ants of ant colony optimization with particles, which in this case were guesses at the solution to a problem. (So a particle could be an equation, for example, or a set of instructions, depending on the problem.) A swarm of such particles is then allowed to fly through the space of the problem, remembering how well they have done but also noticing how well nearby particles are doing. The particles follow similar rules to those that Reynolds used for his boids. A particle’s movement is governed by the balance of two forces, one attracting it toward the fittest location that it has so far discovered for itself, and the other attracting it to the best location uncovered by its neighbors. It’s easier than it looks, as can be seen by viewing one of the many computer visualizations that are available, such as that at Project Computing (http://www.projectcomputing .com/resources/psovis/index.html), where a swarm of particles attempts to find the highest peak in a virtual mountain landscape. Particle swarms are particularly good at detecting abrupt changes in their environment, such as peaks, troughs, edges, or sudden movements. The ability to detect peaks and troughs has even made particle swarm optimization a useful adjunct for investment decision making. It has been adopted as a tool for the analysis of MRI scans and satellite images and for automatic cropping of digital photographs because of its ability to detect edges, and its facility for move-

Ant Logic

ment perception makes it useful for detecting intruders, tracking elephant migrations, and analyzing tremors in the diagnosis of Parkinson’s disease. All of these applications require the use of powerful computers, but this does not mean that computerized swarm intelligence has taken over our decision making. Rather, it provides new and exciting opportunities for innovative approaches to problems, such as that taken by UPS when it combined years of accumulated company know-how with the route-planning capabilities of new software to plan as many right turns into its routes as possible. The reasoning was obvious. Almost every left turn involves crossing an oncoming lane of traffic, which means a potential wait and consequent loss of time, as well as a greater risk of an accident. It was the accumulated experience, though, and the process of drivers learning from each other in a way similar to ant colony routing, that convinced the firm that more right turns really would save time. This communication among agents, together with new package flow procedures (also designed with the help of ant colony logic!), resulted in substantial fuel savings: three million gallons in 2006 alone. If UPS can do it, so can we. UPS used the swarm intelligence of their drivers to come up with the strategy. We can take advantage of that swarm intelligence by copying it. So here’s another rule: In complicated journeys across a city, choose a route that incorporates a high proportion of right turns. The human-computer interface permits a whole new approach to the development of human swarm intelligence. Astronomers are using it to coordinate their activities in the hunt for supernovae. eBay shoppers are unconsciously using it to maintain quality control over transactions through the rating system, which works through a combination of positive, negative, and neutral feedbacks. Hybrid systems, consisting of hardware, software, and humans, are also beginning to emerge. They allow simple local interactions between neighboring

THE PERFECT SWARM

individuals (human and computer) to produce complex swarm intelligence, and an improved performance for the group as a whole. Tests on volunteer swarms in the U.S. Navy have shown that this sort of system can work well when it comes to cargo movement on navy ships, and there are many other potential applications in the pipeline. One of the most dramatic uses of the human-computer interface is in the production of a “smart mob.” Smart mobs (also known as “flash mobs”) are groups of people who use cell phones or other modern communications media to coordinate their activities. Such communication leads to swarm intelligence because communication on a one-to-one basis enhances the performance of the group as a whole, with no obvious leader. The communication is particularly efficient because of exponentially increasing network links between members of the group over time. Smart mobs can be very frightening, especially to authorities. Protestors in the 2001 demonstrations that helped to overthrow President Joseph Estrada of the Philippines became a self-organized group linked by text messages. The 2005 civil unrest in France, the 2006 student protests in Chile, and the 2008 Wild Strawberry student movement in Taiwan (where a group 500-strong materialized in front of the National Parliament building overnight to protest curbs to freedom of expression) were similarly self-organized. More recently, the Twitter network has been used to some effect to coordinate protests in Iran against the results of presidential election. It should be noted, however, that this sort of networking has the weakness that it is “chaotic, subjective, and totally unverifiable,” and it is also impossible to authenticate sources. Smart mobs do not always have to be concerned with protest, however. The technology that empowers them, which can lead to “smart mob rule,” has the potential to help all of us to enhance our day-to-day group performance. The Twitter social networking and microblogging service, for example, is now used regularly by politi-

Ant Logic

cians and celebrities as well as families, teenagers, and other social groups to keep in real-time contact with what everyone in the group is doing. It need not stop at person-to-person communication. These days we can even communicate with our refrigerators and washing machines. Evolutionary biologist Simon Garnier, a specialist in insect swarm intelligence, has waxed lyrical about the potential for this sort of communication in the evolution of human uses for swarm intelligence: “We have no doubt that more practical applications of Swarm Intelligence will continue to emerge. In a world where a chip will soon be embedded into every object, from envelopes to trash cans to heads of lettuce, control algorithms will have to be invented to let all these ‘dumb’ pieces of silicon communicate [with each other and with us].” We don’t have to become cyborgs to use aspects of swarm intelligence in our daily lives, though. Nor do we have to become ants, although our brains themselves use the distributed logic of the ant colony. As I show in the next chapter, most of the time, all we need for human swarm intelligence to emerge is an ability to use straightforward human logic, and in some cases also use the much rarer ability to recognize the limitations of that logic.

FOUR

A Force in a Crowd

A friend of mine who uses a wheelchair has an unusual solution for coping with crowded conditions. He is a well-respected scientist with a Ph.D., but when he gets sufficiently irritated with people blocking his path on a crowded street, he has been known to adopt a goony expression and tap on their backs, or even tug at their clothing. Most of them back away hastily and apologetically, and as he goes past he turns to them with a normal expression and says politely, “Thank you very much.” The English humorist Jerome K. Jerome discovered another way to make space in a crowd when he boarded a packed train carrying a bag full of moldy cheese. His compartment emptied quickly after a fellow passenger who looked like an undertaker said that the smell reminded him of a mortuary! Putting such extreme solutions aside, what are the best strategies to adopt in crowd situations? Should we take our lessons from locusts, bees, and ants, or does our ability to foresee the consequences of our decisions make a difference in how we should act as individuals in a human swarm?

THE PERFECT SWARM

Scientists have made remarkable progress in the past ten years in coming up with answers to these questions. That progress has largely arisen from the recognition that crowd movements involve a combination of involuntary and voluntary forces. The scientists who study crowd dynamics call the involuntary forces “physical forces.” They reserve the term for those forces that we experience when we are pushed from behind, when we bump into other people, and when we find ourselves trapped against a wall or other immovable object. The forces that we generate voluntarily to help us achieve our goals for movement in the crowd are called “social forces.” If we want to head in a certain direction, for example, we use our legs to generate a social force that pushes us in that direction. If we want to stay close to family and friends, we continuously vary the magnitude and direction of the chosen force so as to keep near them. If we want to avoid bumping into other people, we move sideways to dodge the encounter. The term “social force” can be misleading. It does not mean the sort of emotional reaction that caused people to jump aside when confronted by my friend in the wheelchair. It means the actual force that they exert when they push on the ground with their feet to make the move. That force came from a physical response to a social interaction, which is why it is called a “social force.” When conditions are not too crowded, the combination of forces that lead to staying together yet avoiding each other can be interpreted in accordance with Craig Reynolds’ three rules for boids: avoidance, alignment, and attraction. We avoid colliding with others by aligning our movement with theirs, and we stay with the crowd because it occupies the space through which we want to travel. The cumulative effect of both types of forces on our movements can be described mathematically by adapting the three laws of motion that Sir Isaac Newton famously propounded hundreds of years ago. Put briefly, they are:

A Force in a Crowd

1. Everything keeps moving in a straight line at a constant speed (which may be zero) unless it is acted on by an external force. 2. When a force does act on a body, the body is accelerated (which can mean a change in speed or direction or both) in proportion to the force, and in inverse proportion to the body’s mass. 3. For every action there is an equal and opposite reaction.

Scientists still use these laws to predict the movement of spacecraft, the acceleration of race cars, and the trajectories of balls on a pool table. Now they are also using them to calculate our movements when we are pushed and buffeted as we make our way through a crowd. The beauty of describing our involuntary and voluntary movements as the results of physical and social forces, respectively, is that we can use simple mathematics to add up all the forces and then calculate their net effect on our movements directly using Newton’s Second Law of Motion. Newton’s Second Law says that an external force accelerates us in the direction of the force and that our acceleration is proportional to the force. Put mathematically: force = mass x acceleration So a force that is twice as strong will accelerate us by twice as much. A closer look at the equation reveals a second fact that we also know from experience: the heavier a body, the harder it is to accelerate. So if we need to push our way through a crowd, our best strategy is to push past only the lighter people. (As we will see, though, there are often other ways to achieve the same objective, and a pushing strategy should be used only in exceptional circ*mstances. By avoiding that strategy we also avoid the likelihood of a punch in the eye.)

THE PERFECT SWARM

The first people to think of bundling physical and social forces together in this way were pioneering mathematical sociologist Dirk Helbing and his team of international colleagues. Their subjects for a computer simulation were members of a virtual soccer crowd— cylindrical individuals weighing in at around 180 pounds with shoulder widths between 20 and 28 inches, which seems pretty reasonable for a group of adults in heavy coats attending a soccer game on a cold winter’s day. This group of individuals didn’t get as far as watching the match, however. Instead, the scientists marched them down corridors, around pillars, and through narrow exits—just to see what would happen. Being the inhabitants of a virtual computer world, they were in no position to object; they were compelled to behave according to the social forces that they were endowed with and the physical forces that they experienced. Their behavior in response to these forces told the scientists a great deal about what happens in real crowds, and it conformed to what has subsequently been observed in video records of such crowds. These studies have led to huge improvement in the way that many potentially troublesome crowd situations are now handled.

The Individual in a Crowd When it comes to our individual behavior in crowds, the computer models revealed that it does not pay to try to weave one’s way through a large group of people. Under normal circ*mstances, both model and experience show that pedestrian crowds self-organize. No one tells the people in them to do it; their actions are just manifestations of complexity theory, which reveals how simple local rules lead to complex overall patterns. The pattern in this case is of pedestrian streams through standing crowds, which form in a way that is analogous to the way running water forms riverbeds.

A Force in a Crowd

Army ants do something similar when they organize themselves neatly into three-lane highways as they travel between their nest and a food source that may be some distance away. The ants leaving the nest occupy the margins of the highway, and those returning carry their prey down the center. The ants are practically blind, but they manage to organize themselves by following pheromone trails laid down by ants that have previously traveled along the route, and by using two additional social forces. One of those, an addition to the basic avoidance rule, concerns what happens when two ants traveling in opposite directions meet head-on: both ants turn, but the ant traveling away from the nest turns faster. The other rule is for individual ants to keep on going in the direction that they were already going after an encounter with another ant. Computer models have shown that these two rules are sufficient to account for the way in which ants that can hardly see each other form their highways. We, too, form spontaneous lanes when we are walking in opposite directions in high pedestrian densities. The flow of people above a critical density of approximately one pedestrian for every two square feet of space breaks into interpenetrating streams moving in opposite directions. The main difference between us and ants is that ants are programmed to follow the rules that lead to the formation of their highways while we have individual goals. Yet the outcomes are strikingly similar, and computer models have shown that the combined social forces of wanting to get to a goal and not wanting to bump into others cause us to form streams just as effectively as do the social forces experienced by army ants. Flowing streams of pedestrians are obviously more efficient than individuals trying to work their way around each other as they walk in opposite directions. But can the computer models of this process

THE PERFECT SWARM

help us personally to work out the best rules to follow when we are walking on a crowded street? If we could use it to make streams of pedestrians form for our benefit, that would be a good start. The obvious way to do this is to increase the local pedestrian density. I decided to give it a try with a group of about twenty friends. We went out into a moderately crowded street and started walking. At first we spread out, but gradually we worked our way toward each other so as to box others in and thus increase the local density of pedestrians. As we did so—bingo! Just as the model predicted, the other pedestrians joined with us to form a column that thrust its way through the mob of pedestrians coming in the opposite direction in the manner of Moses parting the waters of the Red Sea. What the experiment suggested was that when you are working your way through a crowd with a group of friends, stick together closely and try to get as many strangers as possible to stick with you to promote the formation of a river flowing your way. The next time you’re walking down a crowded street, you’re probably going to notice such rivers of pedestrians. In bidirectional flows, these tend to separate into two unidirectional flows (so-called lanes). This segregation effect supports a smooth and efficient flow by minimizing friction through minimizing the number of interactions that cause deceleration. But when the density is too high or the crowd is too impatient, these lanes are destroyed and the crowd locks up. In one of the few experiments that I really regret having done, a group of friends and I deliberately tried to make this happen on a very crowded street. We were too successful by half, and I definitely don’t recommend anyone else initiating a similar experiment. A crowd that had been moving fairly smoothly suddenly stopped dead. Fifteen minutes passed before it started moving again. In less extreme circ*mstances, pedestrians who weave through a crowd have a similar effect to that of drivers who weave in and out of lanes of traffic. As shown by many traffic studies, the net effect of

A Force in a Crowd

such driving behavior is to slow the traffic down without gaining any real individual advantage. Similarly, simulations of crowds show that if everyone in a crowd tries to move twice as fast, the net effect is to halve the rate of flow of the crowd. I once had a competition with a friend on New Year’s Eve in Sydney, Australia, when dense crowds of people were streaming toward vantage points to see fireworks. One favorite place to watch the display is the concourse of the Opera House at Bennelong Point, which has a view across the harbor toward the bridge from which the fireworks are launched. We set off to walk the three hundred yards from the ferry terminal; my friend tried to push through the crowd while I kept pace with the people instead. So that we wouldn’t influence each other, we started off five minutes apart. I have to admit that he beat my time—by three seconds—hardly worth the extra effort. The best way to navigate effectively in a crowd is to be aware of spontaneous crowd dynamics, going along with those dynamics rather than disrupting them. But what should we do when we come to a bottleneck, with people wanting to get through in both directions? Go with the flow, but be aware that the flow will now oscillate.

Crowd Self-Organization Crowd self-organization is yet another example of complexity theory, self-organization, and collective intelligence. Crowds of pedestrians, who come from multiple directions and interact only with others nearby, self-organize so that they can pass through a bottleneck in the most efficient manner. How do they do it? According to Helbing, it’s all a matter of social forces: Once a pedestrian is able to pass through the narrowing, pedestrians with the same walking direction can easily follow.

THE PERFECT SWARM

Schematic representation of intersecting pedestrian streams, with selforganized stripes forming perpendicular to the sum of the directional vectors of both streams. Redrawn from Dirk Helbing et al., “SelfOrganized Pedestrian Crowd Dynamics: Experiments, Simulations, and Design Solutions,” Transportation Science 39 (2005): 1–24.

Hence, the number and “pressure” of waiting, “pushy” pedestrians on one side of the bottleneck becomes less than on the other side. This eventually decreases their chance to occupy the passage. Finally, the “pressure difference” is large enough to stop the flow and turn the passing direction at the bottleneck. This reverses the situation, and eventually the flow direction changes again, giving rise to oscillatory flow.

Self-organization can even occur when pedestrian flows cross each other. Just two social forces are involved, driven by the desire to keep moving forward and to avoid bumping into others. The net result is stripe formation (see figure above), in which pedestrians move forward with the stripes (so as to progress toward their destination) and sideways within the stripes (so as to avoid colliding with crossing pedestrians). It’s a more complex version of the lanes into which

A Force in a Crowd

ants spontaneously organize themselves, and, to my scientist’s eyes, it has true inner beauty, as do so many of the complex patterns that emerge from simple rules. All of these aspects of pedestrian flow in simulated medium-density crowds have been confirmed by video recordings of real crowds. In fact, the videos have been used to fine-tune the simulations so that they correspond as closely as possible to reality. Self-organization allows pedestrian flow to proceed in an efficient manner. Any effort to improve our individual situation in the crowd is likely to disrupt the self-organization and slow everybody down, including ourselves. We can enhance our prospects mainly by being aware of the nature of the flow so that we can reinforce it, rather than disrupt it.

Escaping the Crush Things change when crowd densities are high, especially in confined spaces. Corridors become blocked, exits become jammed, and panic can set in. The physical analogy under these circ*mstances is not so much the flow of a liquid as it is trying to pour breakfast cereal out through a narrow opening in a box. Inevitably, a plug will form, the bits of cereal will jam the opening, and nothing will come out. If you shake the box, some bits will break free in a brief cascade, but as soon as you stop, the opening will become blocked again. Simulations show that something very similar happens with crowds of people trying to make their way through a narrow exit. Under pressure from behind, the crowd density increases to produce “an irregular succession of arch-like blockings of the exit and avalanche-like bunches of leaving pedestrians when the arches break.” The fasteris-slower effect dictates that the overall leaving speed becomes much more sluggish than it would be if everyone slowed down a bit and took their turn.

THE PERFECT SWARM

Even in the polite confines of a London theater, it appears that rushing is the order of the day. When I watched people emerging from a side entrance after a performance (knowing from previous experience that this is the route people take when they are in a hurry), they certainly emerged in bursts, rather than in a steady flow. One way to overcome this effect is to subdivide the crowd into small enough blocks, but this is not something that an individual can readily do. If you are with a sufficiently large group, though, it could benefit the group as a whole to hang back a bit and let those in front make their way through the exit without being pressured from behind. In the main, though, it is the responsibility of designers and architects to improve the geometry of entrances and exists so that passing through is easier. One of the more surprising conclusions from simulations of crowd movements through exits is that jamming can be initiated at places where an escape route widens; the crowd spreads out, leaving gaps into which other people can squeeze, so that jamming becomes really severe when the route narrows again. In 2005 Helbing and his colleagues came up with a clever solution: place columns asymmetrically in front of such exits. The existence of such columns can reduce the pressure at the bottleneck, and their asymmetrical placement can prevent an equilibrium of the forces of the flows from both sides and, thereby, mutual blockages. Simulations revealed that this solution works, and theater and stadium designers are now beginning to use it in practice. If we are unfortunate enough to be caught in a crowd attempting to escape from a dangerous situation through multiple exits, what should we do? Should we go with the crowd, or should we go it alone in our efforts to escape? Simulations of such situations suggest that we should do neither— or, rather, that we should do both. Our best chance of escape is to optimize our personal panic parameter, which is a measure of the extent

A Force in a Crowd

to which we allow ourselves to be guided by the actions of the crowd rather than by our own initiative. If our panic parameter is 0, we search for an exit without bothering about what the crowd is doing. If its value is 1, we always follow the crowd. But that can lead to an inefficient use of emergency exits and alternative escape routes. Therefore, in a situation where we don’t have any reliable information about escape routes, studies have shown that we do best when we operate with a panic parameter of 0.4—in other words, when we go with the crowd 60 percent of the time and use our own ideas and initiative 40 percent of the time. Just how we should mix it up would seem to depend on circ*mstances. My preference would be to do some personal searching first before linking up with the crowd (though still keeping an eye open for alternatives). I just hope, though, that I never find myself in a position to perform the experiment. There are rational grounds for using such a mixed strategy, especially in conditions of bad visibility. Pure individualistic behavior (in the absence of prior information and knowledge) means that each person finds an exit only accidentally; pure herding behavior means that the entire crowd tends to move to the same exit, which can mean that some exits might not be used at all. When some individuals spend some of their time searching for alternative exits, all of the exits are likely to be discovered, with parts of the crowd following the individuals that have found them, so that all exits are efficiently used. To use the theory effectively we need to face up to two basic facts about our real behavior in so-called panic situations: (1) we often fail to take the danger seriously until it is too late, and (2) even when we are aware of real danger, our response is often to seek out family and friends first before searching for exits or heading for the hills. Our tendency to hang around and wait for confirmation of danger can be particularly disastrous. When my wife and I were in Sri Lanka just after the devastating tsunami in 2004, we passed through village after village hung with the white flags of mourning. Our driver was

THE PERFECT SWARM

among those who had lost close friends. He told us that they might not have died if their natural curiosity had not impelled them to run out toward the ocean to find out what was going on when the waters receded rapidly—a signal (unbeknownst to them) that a tsunami was on its way. When we later visited the west coast of India, our Indian friends told us of a different cultural tendency, one that saved many lives. Rather that running out to investigate, the locals headed for high ground as soon as the waters started to recede. Like the Sri Lankans, they didn’t know what was coming, but they knew that the unusual movement of the water didn’t fit with the expected order of things, and they wanted no part of it. That’s not the usual reaction, though; most of us are reluctant to accept and act upon warnings of a danger we do not perceive as immediate and personal. The recognition of danger can be delayed until it is too late, as sadly happened for many people in the tsunami. Our social forces for escape often do not kick in soon enough. It seems that most of us have a built-in tendency to explain the unusual and abnormal in terms of the familiar and recognizable. The roar of an approaching tornado has often been mistaken for the sound of a passing train. Even when a tornado is apparent, it doesn’t necessarily alarm some people. Writer Bill Bryson tells the story of how his grandfather in Iowa was woken up one night by a noise that sounded like “a billion hornets.” He looked out the window, couldn’t see anything, and went back to bed. When he got up the next morning, he was surprised to find that his car was standing in the open air. The entire garage had been taken by a passing tornado! Less dramatically, but even more tragically, people have been known to die of carbon monoxide poisoning because they assigned the cause of their faintness to illness, and so failed to move away from the source of the leaking gas. James Thurber’s story of a whole town running from an imaginary flood may fit our stereotype of crowd behavior, but when the resi-

A Force in a Crowd

dents of Marysville and Yuba, California, were threatened with a major flood in 1955, 39 percent of those who received official warnings via the media “did not fully believe them.” According to the official report on the subsequent disaster, the failure of many people to respond to the warnings was due to “lack of past experience with disasters, the delusion of personal invulnerability, the inability to adopt a new frame of reference so as to expect unusual events, dependency upon protecting authorities, and the willingness to seize upon reassuring communications or to deny or disregard communications predicting disaster.” In general, we do not react decisively enough to warnings of danger, even when the danger is extreme. After the 9/11 attack on the World Trade Center “as many as 83 percent judged the situation to be very serious in the first few minutes after the strike. Yet despite seeing the flames, smoke, or falling paper, only 55 percent of the survivors evacuated immediately; another 13 percent stopped to retrieve belongings, and 20 percent secured files and searched floors before evacuating.” In another example, some residents who had escaped from a burning apartment building in Winnipeg, Manitoba, died, having gone back into the building to collect belongings, even though they could plainly see the smoke and flames. The calmness we show in such conditions of crisis extends to our care for others, which seldom fits the stereotype promoted by the media, which suggests that we “panic, trample over each other, and lose all sense of concern for our fellow human beings.” This is how the media interpreted events in Cincinnati, Ohio, in 1979 when eleven young people were killed in a crush at a concert by the Who. According to media descriptions, the tragedy occurred when crowd members stormed over others in their rush for good seats in the arena. One national columnist condemned them as barbarians who “stomped 11 persons to death [after] having numbed their brains on weeds, chemicals, and Southern Comfort.” A local editor with literary aspirations referred to the “uncaring tread of the surging crowd.”

THE PERFECT SWARM

The truth was otherwise, and it reflects well on our general behavior as human beings. An analysis of statements taken by police after the event showed that the overwhelming reaction of people to the crowd pressure was to attempt to help those nearby who seemed to be in trouble. One small teenager owed her life to four strangers who struggled to hold her off the ground after she had passed out. Thirty-seven of the thirty-eight interviews included descriptions of similar instances of altruistic behavior. Far from trampling on others, most people were much more concerned with helping them. It seems that the Good Samaritan is alive and well in most of us. Other research on crowd situations has shown that we tend to seek out family and friends as a first priority. Social scientists call this the “social attachment” model of crowd behavior. It is not always the best objective strategy. It can waste valuable time, for example, which might be better used by heading straight for the nearest exit. In some circ*mstances, such as the aftermath of earthquakes, anxious relatives who are trying to be helpful can get in the way of professional large-scale rescue operations. The forces of social attachment are very strong, but they have yet to be factored in to physical models of crowd behavior.

Inescapable Crushes When crowds are so dense that movement becomes difficult, social forces become less important. All evidence indicates that it is physical forces that do the damage. We may wish to help others, but we simply can’t move in the crush. We may stay calm, or we may panic, but the pressure from surrounding bodies remains the same. In the disaster at the Who concert, that pressure came from people on the outside of the crowd, pressing to get in, unaware of the enormous pressure they were exerting on those on the inside. Video

A Force in a Crowd

recordings of crowd disasters are now helping researchers to get to the core of the problem, and to understand what can be done about it. The most thorough study is of the terrible crowd disaster that occurred during the Stoning of the Devil ritual during the Muslim hajj in the city of Mina (just east of Mecca) on January 12, 2006. At least 346 pilgrims died and 286 were injured in a crush that had no apparent cause. Some three million pilgrims converge on Mecca each year during the hajj. The Stoning of the Devil ritual is the culmination, representing the trials experienced by Abraham while he decided whether to sacrifice his son as demanded by Allah. Pilgrims climb ramps to the multilevel Jamaraat Bridge in the city of Mina and throw pebbles they have collected from the plain of Muzdalifah at one of three large pillars, or jamaraat. On the first day, seven pebbles must be thrown at the largest pillar. On the two following days, seven pebbles must be thrown at each of the three pillars (a total of forty-nine pebbles over the three days). Before the pillars were replaced by substantial walls, it was not uncommon for pilgrims to be injured by pebbles thrown from the other side of the pillar that missed their target. The worst problems, though, have come when people have been trampled by the crowd while trying to approach the pillars. The tragic events of 2006 were caught on fixed surveillance cameras. Modern image analysis techniques have allowed scientists to scrutinize these recordings and to evaluate the movements of individual pedestrians and small groups of pedestrians. As a result, several formerly unknown features of the behavior of crowds at very high densities have been revealed. The first is that no matter what the density of people, the crowd as a whole keeps moving. When crowd densities become sufficiently high, however, pedestrians stop intermittently. These stop-and-go

THE PERFECT SWARM

waves propagate upstream (against the flow direction) and can be clearly seen when the footage is played faster than real time. Initially, such stop-and-go waves do not cause severe problems, because people still feel that they have some control over their movements. But when the density corresponds to around one and a half square feet per person, people are moved involuntarily by the crowd. In response, people try to gain space, by pushing other people away, for example. The resulting forces in the crowd add up over large distances. Force chains develop, causing large variations in the size and direction of the force acting on an individual. The video recordings show that, under these circ*mstances, the crowd mass splits up into clusters of people moving as a block, but relative to nearby clusters. Stress release can be quite unpredictable and uncontrollable, making falling down more likely. The mathematics that describes force chains in crowds is very similar to that used to describe earthquakes, and the consequences of the eventual eruption can be just as severe. There is very little we can do as individuals when caught in such situations. The best thing to do is to avoid them. Crowd organizers, though, can take some action. To the credit of the Saudi Arabian authorities, action in response to the scientific study of the 2006 hajj disaster was immediate and decisive. A new Jamaraat Bridge with a higher flow and stone-throwing capacity was built, and the design and organization of the plaza around it were modified to balance inflows and outflows. The accumulation of large crowds was prevented. The streets became unidirectional, supporting smooth and efficient flows. Time schedules and a routing plan for pilgrim groups were developed to distribute the flows of pilgrims throughout the day. Moreover, an automated counting system was installed so that crowd densities could be monitored, pilgrim flows could be rerouted, and time schedules could be adapted. Last but not least, an awareness program was implemented to inform pilgrims about the site before and after they arrived.

A Force in a Crowd

Overview* Crowds have emergent complex structures that arise from physical and social forces between individuals. The best way to behave in crowds depends very much on their density. For crowds of low to medium density, the best we can do is be aware of the emergent structures (such as rivers of pedestrians) and use them to our advantage. If we try to do better, we are likely to end up doing worse. The only exception is when we are in a crowd that is searching for an exit to escape from a dangerous situation. In this case, our best chance, in the absence of additional information, is to go with the crowd 60 percent of the time, using the other 40 percent searching for alternative exits on our own. When crowds become very dense, we lose much of our control of our own destinies. The best we can do is avoid getting into such crowds. If we are on the outside, we should back off and try to persuade others to do likewise. In this way we can make a small contribution to avoiding catastrophic pressure build-ups in the center of the crowd. Finally, if there is a warning of danger, act on it promptly; don’t wait until you are caught up in the crowd. * Professor Helbing suggests that most of the rules in this summary can be encapsulated by one simple piece of advice, which is equally applicable to walking in pedestrian crowds and driving in crowded traffic: always keep a sufficient distance from others.

FIVE

Group Intelligence: The Majority or the Average?

Our behavior and decisions in crowds are constrained by the fact that we can usually interact only with those nearby. In most other group situations, we can communicate much more freely with all group members to make joint decisions. But how can we use this enhanced communication to make the best group decisions? There are two basic options: to take a majority vote or to determine some sort of average opinion. Author James Surowiecki has produced a plethora of examples of the latter in The Wisdom of Crowds. Recent research has delineated the essential circ*mstances for this approach to work; more importantly, it has shown how it works. In this chapter I examine old and new examples in light of this research and provide an answer to a fundamental question: when should we go with the majority, and when should we take some sort of average opinion? One of my favorite childhood activities was camping in the Australian bush, the setting for one of my earliest memories of the

THE PERFECT SWARM

equal-weight approach. In the dead of night, my father asked me and a group of eight friends to use our compasses to work out where East was. We had only a dim tent light to see by, and some of the compasses were very battered indeed, with sticking needles and loose bearings. Needless to say, our answers were all over the place—they covered a range of some 90 degrees. Dad averaged the lot and drew an arrow in the dirt to show where the average pointed. When the sun came up, the arrow was pointing almost straight at it! By averaging our results we had achieved an apparently miraculous accuracy. Later that day, on a hike, he asked us whether we thought a wombat or a wallaby (a small kangaroo) was heavier. Six of us thought a wombat was heavier; three of us thought that a wallaby weighed more. This time there was no point in taking an average (what is the average of a wombat and a wallaby—a wannabe?). Instead, the question was simply whether the majority was right or not. As it turned out, the majority was right. In both cases we had used group intelligence to find an answer. Despite the diversity of our opinions, we had managed to arrive at the right answers. In fact, such diversity lies at the heart of group intelligence. It’s a matter of finding the best way to use it—and that depends on the type of problem we are trying to answer. For problems that involve figuring out the value of something (like a compass bearing or the classic case of the number of jelly beans in a jar) the best way to tackle the problem is to take an average of all the answers. Scientists call these “state estimation” problems. For problems that involve choosing the right answer from among a small number of possible alternatives, majority opinion serves us better. To take the best advantage of either, we need to fulfill just three conditions: •

The people in the group must be willing and able to think for themselves and reach diverse, independent conclusions.

Group Intelligence: The Majority or the Average?

•

The question must have a definite answer that can ultimately be checked against reality.

•

Everyone in the group must be answering the same question. (This may seem obvious, but it is often possible for people to interpret the same question in very different ways.)

When these three conditions are fulfilled, the mathematics of complexity leads us to three astounding conclusions: •

When answering a state estimation question, the group as a whole will always outperform most of its individual members. Not sometimes. Always.

•

If most of the group members are moderately well-informed about the facts surrounding a question to which there are several possible answers (but only one correct one), the majority opinion is almost always bound to be right. If each member of a group of one hundred people has a 60 percent chance of getting the right answer, for example, then a rigorous mathematical formulation proves that the answer of the majority has a better than 99 percent chance of being the correct one.

•

Even when only a few people in a group are well-informed, this is usually sufficient for the majority opinion to be the right one.

How can we use these principles in our everyday lives?

Taking an Average: The Many Wrongs Principle Two wrongs may not make a right, but many wrongs can come pretty close. That’s the amazing outcome of group intelligence when it comes to state estimation problems. It’s fairly easy to see how it works in principle. In the compass bearing problem of my youth, for example, some of the errors were

THE PERFECT SWARM

toward the north, others were toward the south; they largely canceled out when we took an average, so the average was pretty close to the true value. The pioneering statistician Francis Galton was the first to demonstrate that the bigger the group, the more accurate the guess. Galton was particularly keen on real-life applications of statistics. One famous example concerned the power of prayer. He argued that if the power of prayer is proportional to the number of people praying for a given outcome, then British royalty should live longer than the rest of the population because they were always being prayed for in church. In fact, their lifetimes turned out statistically to be slightly shorter. Galton, half-cousin of Charles Darwin and member of the upper class, was no democrat, but he was keenly interested in the democratic process, writing that “in these democratic days, any investigation into the trustworthiness and peculiarities of popular judgments is of interest.” When an opportunity to use statistics to analyze those judgments came up, he grabbed it literally with both hands. The opportunity came when he visited the West of England Fat Stock and Poultry Exhibition, held in 1906 in the English coastal town of Plymouth, the town from which Sir Francis Drake famously issued forth in 1588 to defeat the Spanish Armada. Galton was eighty-four years old at the time but as enthusiastic as ever, and fascinated by a guessing competition in which nearly eight hundred people had paid sixpence each for the privilege of guessing the weight of a huge ox after it was slaughtered and dressed. Galton saw the competition as a mirror of the democratic process because “the average competitor was probably as well fitted for making a just estimate . . . as an average voter is of judging the merits of most political issues on which he votes.” He was very interested to discover how the collective guess of the crowd compared to the guesses of its individual members, and after the competition was over he persuaded the judges to lend him the numbered cards on

Group Intelligence: The Majority or the Average?

which competitors had written their entries. He returned home triumphantly with an armful of cards that he lined up in order of the weights guessed. To analyze them, he used the same principle that he had espoused for democratic choice: “one vote, one value.” “According to the democratic principle . . . ,” he wrote, “the middlemost estimate expresses the vox populi, every other estimate being condemned as too low or too high by a majority of voters.” The middlemost estimate is what we would now call the “median” (half the guesses are below and half the guesses are above). The guesses for the weight of the ox ranged from 1,074 pounds to 1,293 pounds, and the median was 1,207 pounds; less than 1 percent away from the true weight of 1,198 pounds.* Galton was astounded by the closeness of the collective estimate to the true value: “This result is, I think, more creditable to the trustworthiness of a democratic judgment than might have been expected.” It would be another hundred years, though, before complexity scientist Scott Page came up with the correct mathematical explanation for this phenomenon—one that uses the mean rather than the median. Page had more than Galton’s results to explain. Similar experiments over the years have given very similar results. London architect

* The statistician Karl Pearson recalculated Galton’s result in 1924 and reported it as a mean (i.e., average) value, which came out at 1,197 pounds. This is the figure that Surowiecki and others quote in accounts of this incident, but a reading of Galton’s original paper shows that, although he was aware that the mean value came even closer to the true value, he still believed that the median was the right one to use because the results were not uniformly distributed. The same argument applies to the assessment of damages by a jury, because one aberrant vote can dramatically affect a mean but it will not much affect the median. Whichever calculation you use, though, it is obvious that the crowd did a lot better than most of the individuals in it.

THE PERFECT SWARM

Matt Deacon, for example, took a glass jar containing 421 pennies to an architectural conference and invited 106 participants to guess the number. The guesses were widely distributed, but their mean value was 419! Wall Street investment strategist Michael Mauboussin has tested the ability of students at the Columbia Business School to estimate the number of jelly beans in a jar for more than a decade, keeping extensive records. The results from year to year have been remarkably similar. To give an example for one year, in 2007 the number of jelly beans in the jar was 1,116. The average of the guesses was 1,151, and only two of the seventy-three students who participated came closer to the right answer than did the class average. New York Times columnist Joe Nocera compared the ability of a group to outguess its members to the situation in Shakespeare in Love where a series of muddles and mishaps during rehearsals somehow produce a first-class play on opening night. Asked to explain how this happens, the producer in the film explains, “It’s a miracle.” The wisdom of crowds is not a miracle. It is simply a matter of statistics. The important thing is that the guesses should be independent. If they are not, the crowd rapidly loses its wisdom. I tested the value of independent guesses on a small scale in my local pub by asking people to guess the number of chocolate-coated pieces of licorice in a small jar, with strict instructions not to let the others know what their guess was. The guesses ranged from 41 to 93, but the average was 60, just 1 away from the actual number of 61. No individual in the group of twenty got this close. I tried a similar experiment the following week with some mints, encouraging people to discuss their guesses. This time the scatter was much less, between 97 and 112. Unfortunately, there were 147 mints in the jar. Most of the group had been influenced by one strong character, producing an estimate that was quite off. With independent guesses, the group beats most of its members— not just some of the time, but all of the time. This startling truth even

Group Intelligence: The Majority or the Average?

applies to weather forecasting. Michigan weather forecaster John Bravender, for example, has pointed out to me in a private e-mail that: We have a number of computer models [each designed and believed in by a different forecaster] that may offer a number of different solutions for how the future weather pattern will evolve. Generally, if you average all of them together, you will get the most likely scenario.

Complexity theorist Scott Page, coincidentally from the University of Michigan, has explained why diversity of opinion is a key factor in getting the best out of a group in his diversity prediction theorem. It relates the collective error of the group as a whole to the average error of its individual members and the diversity of their predictions, or assessments. His theorem simply says: collective error = average individual error – prediction diversity The prediction diversity is the spread of the individual guesses. The average individual error is just what it says it is: the average of how far each guess is from the true value. And the collective error is the difference between the average of our individual guesses and the true value. The calculations are slightly tricky because statisticians use squared values of the errors (to get around the fact that some errors are positive and some are negative, depending on whether they are higher or lower than the mean). I give an example in the notes to this chapter to show how the calculation works, but you really don’t need to be able to do the calculation to understand the message of the equation. The message is simple. Just looking at the theorem shows that our collective error as a group has to be smaller than our average individual error because of the diversity of our answers. The crowd predicts

THE PERFECT SWARM

better than most of the people in it. The group always does better. Pages puts this neatly: when it comes to determining an average opinion in a state estimation problem, he says that “being different is as important as being good.” The best sort of diversity to have when tackling such problems is cognitive diversity. This encompasses diversity of: knowledge—specifically, a range of different areas of relevant knowledge within the group perspectives—different ways of viewing a problem interpretations—different ways of categorizing a problem or partitioning perspectives heuristics—different ways of generating solutions to problems predictive models—different ways of inferring cause and effect

With these in place, it’s a matter of taking advantage of the diversity. There is just one caveat, which is to remember that Page’s theorem proves only that the group outperforms most of its members when it comes to state estimation problems. It does not necessarily outperform all of them. If there is an identifiable expert in the group, it may be that they will do better than the group average. If your car breaks down while you are traveling with a mechanic, a poet, and a meteorologist, your best bet will be to consult the mechanic rather than to average the opinions of all three. That’s not to say that experts always do better than the average. There is an increasing body of evidence that suggests that group intelligence often beats them. Firms such as Microsoft, Best Buy, Google, and Eli Lilly have found, for example, that a diverse set of employees with appropriate knowledge can forecast product sales and profits more accurately than so-called budgeting experts can. A collection of experts can also outperform most, if not all, of the individual experts. Page gives the example of a group of football

Group Intelligence: The Majority or the Average?

journalists predicting the top dozen picks in the 2005 NFL draft. Not one of them performed nearly as well as the average of all of them. Experts are also being replaced by computers when it comes to making rule-based decisions. Computers are now being used in this manner for diagnosing medical and mechanical problems, credit scoring, traffic management, and even literary textual analysis. So what role is left for the poor expert? According to Mauboussin, experts come into their own in the area between rote rule following and probabilistic prediction—an area in which a combination of knowledge and initiative is required. Mauboussin argues that the best types of experts are those that political and business psychologist Phil Tetlock has identified as “foxes,” who have wide knowledge about many aspects of their field, as opposed to “hedgehogs,” who have a deeper but narrower knowledge. The claimed advantage is that foxes are able to make more accurate predictions because they have the advantage of diversity built in. I would modify this with the observation that some hedgehogs are actually like trees with very deep roots that spread underground and pop up in the most unexpected places. These people have a different sort of diversity: the ability to make diverse connections. One of my scientific colleagues, for example, who knows everything there is to know about a particular type of liquid flow called “extensional flow,” has contributed not just to the field of physics but also to plastics manufacturing, food production, and even knee surgery—all because his expertise enables him to see the importance of this sort of flow in different contexts. This ability has certainly enhanced his value as an expert. When we don’t have an expert available, we must fall back on the diversity of the group. Taking an average is not the only way to use such diversity. If the problem involves a choice between just a few possible answers, it is majority opinion that gets the nod.

THE PERFECT SWARM

Majority Opinion and the Jury Theorem The remarkable power of diversity reveals itself fully when it comes to using majority opinion to make decisions. Michael Mauboussin produces a neat demonstration in another experiment with his Columbia Business School students. Each year, just before the Academy Awards are announced, he gets the students to vote on who they think will win in each of twelve categories—not just popular categories like best actor but relatively obscure ones, like best film editing or best art direction. In 2007, the average score for individuals within the group was 5 out of 12. The group as a whole, though, got 11 out of 12 right! Why is the majority so often right? One reason can be illustrated by the story of the Constitution, and of two of its principle framers, Benjamin Franklin and Thomas Jefferson. Franklin and Jefferson both spent time in Paris before working on framing the Constitution, which was adopted in 1787. Both of them became involved in discussions with French intellectuals who were primarily responsible for the first French constitution, which was completed in 1789. One of those intellectuals was the Marquis de Condorcet, a corresponding member of the American Philosophical Society, founded by Franklin in 1743 (and still going strong). Condorcet had begun his career as a mathematician, but when Franklin met him he had been appointed as inspector-general of the Paris Mint at the instigation of the reforming economist Anne-RobertJacques Turgot. Turgot didn’t last long in the atmosphere of intrigue and double-dealing that characterized Louis XVI’s court, but Condorcet prospered. He also became fascinated by the idea that mathematics could be used to support arguments for human rights and moral principles. Franklin met up with Condorcet many times after he arrived in Paris and was impressed by the progress that Condorcet had made with his “social mathematics,” saying at dinners he attended that it

Group Intelligence: The Majority or the Average?

“had to be discussed.” Nothing was yet on paper, but that soon changed with the publication of Condorcet’s remarkable work Essay on the Application of Analysis to the Probability of Majority Decisions, published in 1785. There is a copy of the book, signed by Condorcet himself, in Jefferson’s library. Franklin was clearly influenced by Condorcet’s ideas, in particular by his mathematical proof of what is now known as “Condorcet’s jury theorem.” John Adams told Jefferson that Condorcet was a “mathematical charlatan,” but this was far from being the case, and Condorcet’s theorem is now regarded as a cornerstone for our understanding of democratic decision-making processes. Condorcet wanted to find a mathematical reason for a rational citizen to accept the authority of the state as expressed through democratic choice. He argued that the best reason would be if his or her individual probability of making a correct choice was less than the collective probability of making a correct choice. His theorem appears to prove that this is nearly always the case. The theorem in its simplest form says that if each member of a group has a better than 50:50 chance of getting the right answer to a question that has just two possible answers, then the chance of a majority verdict being correct rapidly becomes closer to 100 percent as the size of the group increases. Even if each individual has only a 60 percent chance of being right, the chance of the majority being right goes up to 80 percent for a group of seventeen and to 90 percent for a group of forty-five. Condorcet’s jury theorem looks like a stunning mathematical justification of the power of group intelligence in the democratic process, but it relies on five crucial assumptions, some of which are similar, though not identical, to the elements of cognitive diversity: •

the individuals in the group must be independent, which means that that they mustn’t influence each other’s opinions

THE PERFECT SWARM

•

they must be unbiased

•

they must all be trying to answer the same question

•

they must be well-informed enough to have a better than 50:50 chance of getting the right answer to the question

•

there must be a right answer

These requirements mean that the jury theorem is useful only in a very restricted range of circ*mstances—although it was (and continues to be) a concrete starting point for discussions on how democracy can best be made to work, and on the way that consensus decisions are arrived at in nature. Condorcet even used it after the French Revolution to suggest the best method of jury trial for the king, but his ideas were not taken up in an atmosphere that was more concerned with retribution than with fairness. Condorcet also invoked the jury theorem in a discussion about the structure of government under the new U.S. Constitution. A point on which all the Framers were firm was that the new government should consist of two houses—a House of Representatives, representing the people, and a Senate, representing the states. When copies of the U.S. Constitution arrived in Paris in November 1787, Condorcet wrote to Franklin, complaining that such a bicameral legislature was a waste of time and money because, according to his mathematical approach to decision making, “increasing the number of legislative bodies could never increase the probability of obtaining true decisions.” The point that Condorcet missed was that the two houses were put in place to answer slightly different questions. The U.S. Supreme Court made this clear in a 1983 judgment about the functions of the two houses when it said, “the Great Compromise [of Article I], under which one House was viewed as representing the people and the other the states, allayed the fears of both the large and the small states.” In other words, the House of Representatives is there to ask,

Group Intelligence: The Majority or the Average?

“Is X good for the people?” while the Senate’s job is to ask, “Is X best implemented by the federal government or by the states?” The fact that the two houses are answering slightly different questions negates Condorcet’s argument that one of the houses is redundant. It might appear that the jury theorem is more relevant to the functioning of juries themselves, but here again it is a matter of how juries are set up. To take maximum advantage of group intelligence, jurors need to be truly independent, which means that each would need to listen to the arguments of both sides and then make a decision without discussing it with the other jurors. The decisions would then be pooled, and the majority decision accepted. Condorcet suggested that Louis XVI’s jury be set up in this way, but his ideas were rejected, and as far as I can find there have been no tests of his proposal since, in France or elsewhere. It does seem a pity, because discussions between jury members before coming to a decision mean that one of the main foundations of group intelligence (that of independence) is lost. Discussions certainly have their value—allowing people to change their minds under the influence of reasoned argument—but other forces can also be at work. One of these is the social pressure to conform with other members of the group that goes under the name of “groupthink,” and which I discuss in the next chapter. So long as members of juries continue to thrash out the merits of a case between themselves before coming to a conclusion in the manner depicted in the film Twelve Angry Men, the jury theorem will largely be irrelevant to their deliberations. It comes into its own, however, when applied to the game show Who Wants to Be a Millionaire? although it turns out that our collective judgment is even more reliable than the theorem suggests. James Surowiecki points out that the “Ask the Audience” option consistently outperforms the “Call an Expert” option. This group of “folks with nothing better to do on a weekday afternoon” produces the correct

THE PERFECT SWARM

answer 90 percent of the time, while preselected experts can only manage 66 percent. It seems like an ideal case for the jury theorem. The selections are independent. The audience is presumably unbiased. Its members are all trying to answer the same question, and the question has a definite right answer. The assumption that all members of the audience need to have a better than 50 percent chance of getting the answer right, however, is not necessary. Close examination reveals that their group intelligence still works even if only a few people know the answer and the rest are guessing to various degrees. To see how this works, try the following question, originated by Scott Page, on your friends. Out of Peter Tork, Davy Jones, Roger Noll, and Michael Nesmith, which one was not a member of the Monkees in the 1960s? If you ask this question of 100 people, one possible scenario is that more than two-thirds (68, say) of them will have no clue, 15 will know the name of one of the Monkees, 10 will be able to pick two of them, and only 7 know all three. The non-Monkee is Roger Noll, a Stanford economist. How many votes will he get? Seventeen of the 68 will choose Noll as a random choice. Five of the 15 will select him as one choice out of three. Five of the 10 will select him as one choice out of two. And all of the 7 will choose him. This gives a total of 34 votes for Noll, compared to 22 for each of the others—a very clear majority. So group intelligence can work in this case with only a few moderately knowledgeable people in the group. It would even have a fair chance of working if 68 people had no clue and the remaining 32 only knew the name of one Monkee. One-third of these (11 to the nearest whole figure) would choose Noll as the exception, giving an average total of 28 votes for Noll and 24 for each of the others. Statistical scatter makes this prognostication less sure, but with increasing group size the difference becomes more meaningful.

Group Intelligence: The Majority or the Average?

When it reaches the millions, the majority vote can provide a very sure guide, which is why search engines such as Google, Yahoo, and Digg.com use it as an important guide in their ranking algorithms. The jury theorem is fine for the very restricted conditions under which it applies, and it is a necessary starting point for thinking about majority voting in different contexts. Modern analysis has built on it and shown that group intelligence can be even more powerful than the theorem suggests—so long as we get the conditions right. As I show in the next chapter, this is particularly important when it comes to getting groups to reach a consensus.

SIX

Consensus: A Foolish Consistency?

When I was young and idealistic (I am now old and idealistic) I was involved in the formation of a new political party in Australia. Eager to do my bit, I volunteered to be its policy coordinator. The other parties didn’t have policy coordinators, but we were determined to be as democratic as possible, and that meant taking account of the views of all of the members. Luckily there weren’t very many members, but even so it was an onerous task. I decided to make it easier and more efficient by using the Delphi technique, a tool used in business to help a group progress from consultation to consensus. The basis of the method is working the consultation in a series of steps: 1. Circulate the problem to group members. 2. Collate the responses, suggestions, and supporting arguments. 3. Send the collation back to the group members and ask them to rate the suggestions.

THE PERFECT SWARM

4. Repeat steps 2 and 3 if necessary until a consensus is reached.

I kept it simple. First I sent out questionnaires requesting suggestions for a policy on some issue. A group of us would then summarize the answers, prepare a set of options that reflected the range of opinions, and send out the options for the members to vote on. Our summaries, though, were treated with suspicion. One member even wrote that they should be compared with reports of parliamentary debates that were written by the great lexicographer Samuel Johnson in the eighteenth century. This was not as flattering as it sounded, because Johnson captured the essence of what was said in the debates between the Whigs and the Tories, but he invented the actual dialogue, claiming that he “took care that the Whig dogs did not get the best of it.” Our members were worried that we were doing something similar, writing the summaries with a bias toward our own views. We tried to assure them that we weren’t and that they were welcome to examine the voluminous pile of correspondence from which we had worked. Needless to say, that didn’t cut much ice, and the party soon collapsed under the weight of its own attempts at democracy. Our dilemma reflected one of the biggest problems facing any group: how to make the transition from diversity-based problem solving to a consensus agreement on a course of action, which inevitably reduces diversity. Whether we take an average, accept the majority opinion, or seek out the most knowledgeable person and accept their guidance, someone has to sacrifice their opinion so that the whole group can benefit from its group intelligence. There are several ways to make the transition from diversity to consensus. Here I focus on three. One is to follow the example of nature and go along with what the majority of our neighbors is doing. Another is to debate the issues and come to a reasoned agreement. The third is to use swarm intelligence.

Consensus: A Foolish Consistency?

Quorum Responses: The Example of Nature One way to achieve consensus is to follow the example of others who appear to know what they are doing. When we select a wellworn track over a less-used one on a hiking trail, for example, we are presuming that the well-worn one is the more likely to have been made by people who knew where they were going. Similarly, when ants use pheromone concentration as the criterion for choosing between routes to a food source, they use the decisions of ants that have previously used the trails to guide their own choice. Our main aim is to avoid getting lost. Ants and other animals that travel in groups have more serious purposes—to find the best food sources, to obtain the best shelter, and above all to avoid getting eaten during the search. These animals can improve their chances by copying the actions of better-informed neighbors. But how are they to know which neighbors are better informed? Their only real clue lies in how many others are also copying them. It’s a clue that the people of Moscow used during the 1980s when basic goods were in perennial short supply after the collapse of communism. If you were walking down the street and saw one or two people standing around outside a shop, you might have walked on. But three or four people was a signal that the shop had something available to sell, and others would hasten to join the line in a cascade effect that rapidly produced a longer line, although hardly anyone in the line knew what was for sale! This cascade effect is known to animal behaviorists as a quorum response. Put simply, the group arrives at a consensus in which each individual’s likelihood of choosing an option increases steeply (nonlinearly) with the number of near neighbors already committed to that option (the neurons in the human brain show a similar sort of response to the activity of neighboring neurons).

THE PERFECT SWARM

The quorum response is an example of complexity science at work. All of the basics are there—local interactions, nonlinear responses, positive feedback, and the emergence of an overall pattern. It’s just a matter of whether the pattern is a useful one or not. In nature, the quorum response can lead to very useful patterns. One example is when it ensures the cohesion of the group, which has advantages such as protection from predation and consensus in group decisions. The sharp nonlinearity of the response also ensures faster decision making and (if the individuals initiating the response are well informed) a higher rate of accuracy of the group’s decisions. Animals can use the quorum response to make a choice between speed and accuracy. Speed is preferable if time is of the essence. Computer modeling has shown that the requisite speed can be achieved by responding to the actions of a smaller number of neighbors. If there is plenty of time available, the reverse effect can be achieved by waiting until more neighbors have committed to a choice before making the same choice. Many of us use the quorum response when we are choosing a place to eat during a long drive. If there are no cars parked outside a roadside diner, we are likely to drive by. If there are one or two cars, we may pull in. If there are plenty of cars we are almost certain to pull in, on the basis that this must indicate that it is a good place to eat, or perhaps that there are no more places farther down the road. Wherever we pull in, we have added another car to the total, thus increasing the chances that others will pull in as well. We can also use the quorum response to make a trade-off between speed and accuracy. If I am feeling particularly hungry, for example, I am likely to take a risk on quality and pull in at a restaurant that has just a couple of cars outside. If I am less hungry, I am more likely to drive on, looking for one with more cars outside.

Consensus: A Foolish Consistency?

I am trusting that the car owners will have used some form of independently gathered information to choose this particular restaurant. They may have been there before, or read a review, or been told about it by their friends. If they were all using the same strategy that I was— relying on the presence of other diners as an indication of quality— we could all be in for a shock. In these circ*mstances, the first few would have chosen the restaurant at random, and the rest of us would have followed them blindly, like the lemmings filmed while cascading over a cliff in the film White Wilderness. This phenomenon of interdependence without individuals having reliable, independently gathered information is called, rather oddly, an informational cascade. (Maybe a better term would be disinformational cascade.) Some unscrupulous owners have been known to initiate such a cascade by parking a group of their own cars outside their restaurants just to encourage people to pull in. This points up a basic problem with quorum responses: for them to result in the desired outcome, we must be able to trust the truthfulness and knowledge of those whose decisions we are choosing to copy. Sometimes these qualities are obvious. If we are looking for shelter from a sudden downpour in an unfamiliar environment, for example, it’s usually a good idea to follow the crowd, which is likely to be made up of locals who know what they are doing. In general, it’s a good idea to follow the crowd if its purpose is the same as ours, whether that is looking for food, shelter, or a bargain. Animals such as co*ckroaches, ants, and spiders don’t do a lot of shopping, but they use the quorum response very successfully in their search for food and shelter. We can do the same, and there are many circ*mstances in which we can do a lot better. The quorum response has its uses, but we can often improve on it by factoring in our own judgment. It’s the interplay of independence and interdependence that gives us our best chance. When we

THE PERFECT SWARM

check out how many people are using a restaurant, for example, we can supplement that information by looking into the kitchen to see if it is clean, or just taking a look at the customers’ faces and the appearance of the food on their plates. We can supplement it further by asking someone who has just come out of the restaurant what they thought of it. They may not tell us the truth, of course, especially if they are friends of the manager. That would not matter if my wife, Wendy, were with us, because she seems to be able to detect a lie instantly through body language. She can certainly see through me all right. She may even be one of the 0.25 percent of people that scientists have found can do it almost all the time. Regardless of how we gather our information and form our opinions, when we are part of a group we must translate everyone’s information and opinions into some sort of consensus for action. If the group is under the control of a dictator (benevolent or not), achieving consensus presents no problem. When I was a child and my family went out to dinner, we would squabble endlessly about what sort of restaurant we wanted to go to until my father threw up his hands in frustration and said, “This is where we are going!” At other times, though, once we had all expressed our opinions and advanced our arguments, he would put it to a vote. Putting it to a vote is the democratic way of reaching a consensus. There are many hidden traps, however, both in choosing an appropriate voting method and in avoiding groupthink (see the section later in this chapter). Here I examine the problems with voting methods, and what (if anything) we can do about the dreaded groupthink.

Voting Methods Evidence shows that all voting methods are flawed, so we may as well choose something simple to suit the particular situation and get on with it.

Consensus: A Foolish Consistency?

The idea of getting together to vote on an issue goes back twoand-a-half thousand years, to a time when the city of Athens in ancient Greece was laying the foundations for Western civilization. The citizens of Athens had two great ideas about selecting their politicians. The first was to choose them by lottery from whoever put their name forward. The second was to get rid of the worst ones by an annual process of negative voting. Negative voting consisted of writing the name of the disfavored politician on a broken piece of pottery called an ostrakon. On the appointed day the citizens who wished to vote came to the civic center of Athens and handed in their ostrakons to be counted. So long as a quorum of 6,000 votes was cast (out of the 50,000 or so citizens who had voting rights), the politician who had the misfortune to get the most votes was barred from the city for ten years—in other words, they were ostracized. The main idea of the system was to break voting deadlocks that barred the way to consensus decisions. But it all fell apart during the wars with the city of Sparta. An Athenian politician called Nicias had brokered a fragile peace, but another one called Alcibiades wanted to resume all-out war. The population was evenly divided on the issue, and a vote was called to ostracize one or the other and open the road to a decision. Alcibiades and Nicias responded by each urging their supporters to ostracize a third politician, called Hyperbolus. Hyperbolus was exiled, and the issue remained unresolved. After this disastrous result, the population saw that ostracism could be manipulated, and although it remained in the statute books, it was never used again. Manipulation, though, is only one problem that voting systems face. Another is mathematics, in the form of the voting paradox. The paradox was discovered by the Marquis de Condorcet, who had more fun with voting systems than most politicians seem to have. Condorcet noticed that majority voting can lead to a paradoxical outcome when it comes to choosing between three or more alternatives.

THE PERFECT SWARM

Let’s call the alternatives A, B, and C. What Condorcet proved was that even though each voter has a definite order of preference, when all of the votes are put together it is still perfectly possible for A to beat B, and B to beat C, but C to beat A! The voting paradox is not the only one that arises when it comes to choosing between three or more alternatives. If majority voting is used, so that the winning alternative is the one that gains the most votes, it is not only possible but normal for the winner to be the choice of the minority, as experience in many elections has shown. Small children can be quicker than adults in catching on to these paradoxes and their consequences, as Illinois mathematician Donald Saari found when he presented them to a class of fourth graders. Saari used the example of a group of fifteen children who have to vote on which television show to watch in the evening (they are allowed only one show). He asked the fourth-graders what show they should watch if the voting went like this: Number of Children First Choice Second Choice Third Choice 6 ALF The Flash The Cosby Show 5 The Cosby Show The Flash ALF 4 The Flash The Cosby Show ALF

Majority voting suggests that ALF should get the nod, but the fourth graders vigorously disagreed. “Flash!” they cried—and they had a point. The Flash might come last in the majority stakes, but nine of the fifteen prefer it to ALF, and ten of the fifteen prefer it to The Cosby Show. This simple story points up the problem with majority voting—a majority of the voters can end up with the candidate that they don’t want. Condorcet’s voting paradox, though, points up an even more perplexing problem.

Consensus: A Foolish Consistency?

Suppose the children’s voting had gone like this: Number of Children First Choice Second Choice Third Choice 5 ALF The Cosby Show The Flash 5 The Cosby Show The Flash ALF 5 The Flash ALF The Cosby Show

Now ALF is preferred over The Cosby Show by 10 votes to 5, and The Cosby Show is preferred over The Flash by 10 votes to 5. So ALF should be preferred over The Flash, right? Wrong! As the fourth graders noticed (and pointed out vociferously), a simple count shows that The Flash is preferred over ALF by 10 votes to 5! The paradoxes that the fourth graders picked up on are not just academic puzzles. They often arise in the real world of voting, whether the vote is for electing politicians or for making a decision in a committee or other group. But it gets worse. Much worse. An unwelcome further complexity was discovered by the Nobel Prize–winning economist Kenneth Arrow, a founding father of the Santa Fe Institute. Arrow showed in 1950 that Condorcet’s paradox is not just an exception to the rule; it is part of a wider set of paradoxes that are the rule. Arrow first looked at what we might want from an ideal voting system. His full list of criteria (paraphrased here from his more technical descriptions) was: 1. Completeness: If there are two alternatives, the voting system should always let us choose one in preference to the other. 2. Unanimity : If every individual prefers one alternative to another, then their aggregated votes should reflect this choice. 3. Non-Dictatorship : Societal preferences cannot be based on the preferences of only one person regardless of the preferences of others.

THE PERFECT SWARM

4. Transitivity: If the aggregated votes show that society prefers choice X to choice Y, and choice Y to choice Z, then they should also produce a preference for choice X over choice Z. 5. Independence of Irrelevant Alternatives: If there are three alternatives, then the ranking order of any two of them should not be affected by the position in the order of the third. 6. Universality: Any possible individual ranking of alternatives is permissible.

Some of these criteria may look trivial. All of them look eminently reasonable for voting in a democracy. Yet Arrow proved in his impossibility theorem (otherwise known as “the paradox of social choice”) that we can’t have them all at the same time. If we have majority voting, for example, then Condorcet’s paradox shows that we can’t have transitivity. If we use my father’s dictatorial approach, then criterion 3 goes out the window. There really is no way out of it. However we wriggle, and no matter what voting system we adopt, one of Arrow’s criteria has to be abandoned. In his 1972 Nobel memorial lecture, Arrow said that the “implications of the paradox of social choice are still not clear. Certainly, there is no simple way out. I hope that others will take this paradox as a challenge rather than as a discouraging barrier.” Even before the discovery of the paradox in 1950, it was perfectly clear that democracy must involve compromise. Former British Prime Minister Winston Churchill put it best in a 1947 speech when he said, “Many forms of Government have been tried, and will be tried in this world of sin and woe. No one pretends that democracy is perfect or all-wise. Indeed, it has been said that democracy is the worst form of government except all those other forms that have been tried from time to time.” Arrow’s proof uncovered one of the difficulties for democracy. His discovery has formed the bedrock for discussions about the best com-

Consensus: A Foolish Consistency?

promises for implementing democracy ever since. From the point of view of small-scale practical democracy, the paradox means that we can never hope to achieve perfection, and the best we can do is choose a simple voting system that seems reasonable for the purpose, and stick with that. My father, at first, used a family majority vote to choose a restaurant. Later, he let each of us be a dictator in turn. Neither system was perfect, and each appeared to work equally well. When it comes to larger-scale democracy, my personal opinion is that simplicity is overrated. First past the post, for example, is obviously simpler than a preferential voting system, in which candidates are ranked in order of preference and the least favored candidate is removed from the bottom of the list and the second preferences of the people who voted for him are distributed among the other candidates, with the process being repeated until a clear winner emerges. Few would disagree that a preferential system is fairer. First-pastthe-post methods suit major parties better, though, by rendering votes for the minor parties ineffective, so it is likely to continue in countries like the United States and the United Kingdom. Even with the most perfect voting system in the world, there is still the human dimension to consider. People may vote tactically, form voting blocs, or be influenced by personalities rather than issues. All of these are the province of game theory, which is the subject of my book Rock, Paper, Scissors. There is one human issue above all, though, that consistently undermines our efforts to use group diversity to come to the best consensus. That issue is groupthink.

Groupthink Groupthink is the phenomenon where social pressures within the group push its members into “a pattern of thought that is characterized by self-deception, forced manufacture of consent, and conformity to group values and ethics.” Members of a group are drawn into agreeing to a common position and sticking to it through thick and

THE PERFECT SWARM

thin. The outcome can even be MAD (mutually assured delusion), which occurs when group members deny evidence that those outside the group can plainly see, stick to beliefs that have little or no basis in fact, and fall “prey to a collective form of overconfidence and willful blindness.” One often-quoted example is the delusion among those close to George W. Bush during his presidency that an invasion of Iraq (in March 2003) would be a short affair because American troops would be welcomed as liberators by a grateful population. According to investigative reporter Bob Woodward, the factors that encouraged groupthink on this occasion included “the support, the encouragement of [Vice President Dick] Cheney, the intelligence community saying Saddam [Hussein] has weapons of mass destruction . . . and [President George W.] Bush looked at this as an opportunity [to fulfill a private dream].” Other examples include the $50 billion scam Bernard Madoff operated, in which investors collectively deluded themselves into thinking that he must be cheating on their behalf rather than his own, and the activities of loan institutions that led to the credit crunch, in which they collectively convinced themselves that house prices would keep rising without end, so that toxic loans would ultimately lose their toxicity because of a continuously rising market. When Yale psychologist Irving Janis coined the term groupthink in 1972, he listed its main characteristics as •

Pressures for uniformity, such as the threat or actual application of sanctions that makes people feel excluded if they disagree with its way of thinking and its conclusions.

•

Closed-mindedness within the group, so that any doubt is rationalized away.

•

An overestimation of the group as strong, smart, morally superior to other groups, or even invulnerable.

Consensus: A Foolish Consistency?

The Nobel Prize–winning physicist Richard Feynman experienced all of these when he joined the committee investigating the 1986 Challenger disaster. The committee chairman, former secretary of state William Rogers, commented that “Feynman is getting to be a real pain” after Feynman decided to conduct his own investigations rather than simply sit in committee meetings. Feynman’s investigations consisted in talking to the scientists and engineers who had actually worked on the project. He soon found that there were very diverse opinions within NASA about the safety of the shuttle. In his contribution to the final report he said: “It appears that there are enormous differences of opinion as to the probability of a failure with loss of vehicle and human life. The estimates range from roughly 1 in 100 to 1 in 100,000. The higher figures come from working engineers, and the very low figures from the management.” If the management had been aware of this diversity of opinion and used it to make better decisions, it is possible that the disaster would never have happened. The management, however, was particularly subject to the second characteristic of groupthink, closedmindedness within the group. This led to a situation identified in the final report: although both NASA and its contractor, Morton Thiokol, knew that there was a design flaw, they chose to ignore its potential for disaster, which reflects the third characteristic of groupthink: an overestimation of the group’s ability. The true basis of the failure came to light only because Feynman acted independently by talking to the scientists and engineers. The credit does not all go to Feynman, though. He recognized later that the scientists and engineers with whom he had talked had led him to his conclusions. His presence and questions gave them the confidence to escape from the stultifying atmosphere and pressure of groupthink within NASA, at least to the extent of pointing Feynman in the right direction.

THE PERFECT SWARM

Feynman might never have come to his conclusions if he had not evaded the iron fist imposed by Rogers on his commission members. Rogers appears to have decided early on to focus on the NASA administration and ignore the technical details. When he found out that Feynman had been talking to the technical people, he ordered him to stop. By that time, though, Feynman had all the information he wanted. It all came to a head at a televised meeting of the commission. Feynman produced a sample of the material that was supposed to provide a flexible O-ring seal for the fuel tank from which leakage had precipitated the catastrophic explosion. He then compressed it with a clamp and dunked it in the glass of ice water that had been provided for him to drink and said: “I took this stuff that I got out of your seal and I put it in ice water, and I discovered that when you put some pressure on it for a while and then undo it, it does not stretch back. It stays the same dimension. In other words, for a few seconds at least and more seconds than that, there is no resilience in this particular material when it is at a temperature of 32 degrees.” In other words, a material that was supposed to provide a flexible seal became hard and brittle when it was just dunked in ice water, let alone exposed to the temperatures of deep space, where it would be expected to become even more brittle. Feynman’s actions appeared to demolish the groupthink that was driving the commission, which had been focused only on administrative issues such as poor communication and underlying procedures. The difficulty of truly annihilating groupthink, though, was demonstrated by the fact that the commission’s final report still focused on these issues. Only after Feynman threatened to remove his name from the report was he permitted to add an appendix that addressed the technical issues. The appendix had a belated effect when the U.S. House Committee on Science and Technology reviewed the Rogers report and con-

Consensus: A Foolish Consistency?

cluded that “the underlying problem which led to the Challenger accident was not poor communication or underlying procedures as implied by the Rogers Commission conclusion. Rather, the fundamental problem was poor technical decision making over a period of several years by top NASA and contractor personnel, who failed to act decisively to solve the increasingly serious anomalies in the Solid Rocket Booster joints.” Subsequent events demonstrated the insidious hold that groupthink exerts further. Despite the damning House committee report, the groupthink inherent in the NASA management culture continued. This time the technical issue was the heat-insulating foam on the shuttle’s tanks. Pieces kept coming off. According to NASA’s regulations, the issue needed to be resolved before a launch was cleared, but launches were often given the go-ahead anyway. The sickeningly inevitable result was the space shuttle Columbia disaster of February 1, 2003, when a piece of foam came off during the launch and struck the leading edge of the left wing, damaging the wing’s thermal protection system. As a result, during reentry the wing overheated and the shuttle disintegrated. The subsequent accident investigation board was again highly critical of NASA’s decision-making and risk-assessment processes. It concluded that the organization’s structure and processes were sufficiently flawed that compromise of safety was to be expected no matter who was in the key decision-making positions. Not much seemed to have changed in the culture at NASA, least of all the overconfidence that stems from groupthink. Feynman’s example shows, however, that although it is difficult to escape from groupthink, it is not always impossible. The key elements (apart from an iron will) are (1) getting out of the group environment for a while, (2) doing some independent thinking, and (3) committing oneself to the conclusions of that thinking before returning to the group to share your conclusions.

THE PERFECT SWARM

That’s not to say that the effects are always what one wants them to be. Feynman wrote a number of short scientific reports during his investigations, sending them to the commission secretary for distribution. When he asked what his fellow commission members thought of them, he found that they had simply been filed. I did something similar, and with even less success, when I was invited to join a panel of scientists organized by the U.K. Office of Science and Technology to “brainstorm” possible future growth areas in research. Our deliberations, and those of similar panels, were to be taken quite seriously, with money at stake in the form of investment in particular areas. Even though I don’t believe that the path of future research can be foretold in this way (or any other way), I was flattered by the invitation and duly joined the group. Not only that, I soon found myself enthusiastically joining in with the group ethos to produce ideas about where significant research advances were likely to come from. The effect of groupthink was so strong that most of us were convinced that our performance was far superior to that of other groups. Only when I came away from the group after it had met did I collect myself and start to think how silly it all was. I immediately sat down and sent an e-mail to this effect to the organizers and my fellow group members, but the only obvious outcome was that I was never invited back. Groupthink might still beat me, however, in the form of a belief that permeates government and the community, and which is now starting to pervade science itself. This is the belief that useful science can come only from focusing on its potential applications right from the start. If we had always worked from that belief, and not also asked penetrating questions about how the physical and biological worlds work, we would not have X-ray machines, antibiotics, radio, television, or cars with inflatable tires, to name just a few inventions that stemmed from questions that never had these ends in view.

Consensus: A Foolish Consistency?

Janis argued that groupthink was mainly the province of “very high echelon groups,” such as those that determine government policy, in which the perks of membership are at “intoxicating levels.” He also believed that groupthink was mainly confined to circ*mstances in which (1) the group insulates itself from outside criticism, (2) there is high stress from a perceived external threat, and (3) there is very strong group cohesion. But Janis was wrong. Groupthink is everywhere, and it is especially virulent in its ability to affect our attitudes toward each other. A demonstration of how this can happen emerged from a study conducted by psychologists Donald Taylor, from McGill University, and Vaishna Yaggi, from the University of Delhi. They asked volunteer Hindu and Muslim students in southern India to read stories that showed members of each group in either a good or a bad light. They then asked the students what they thought might be the cause of the good or bad behavior of the characters in the story. If the story was about bad behavior by a member of their own social group, the failings of the characters were conveniently attributed to external causes. If it was about bad behavior by a member of the other group, the failings were seen as typical of that group. When the stories were about good behavior, the reverse was true. Students saw good behavior as a characteristic of their own group, but if a member of the other group behaved well, it was due to some external factor having nothing to do with the person themselves! Such attitudes can reflect simple pride in the traditions and attitudes of a society, or they can take us down the terrible paths of racism, strident nationalism, and religious bigotry. They are part of our complex cultures, but they can make it difficult to achieve the maximum potential of that culture, because the group attitudes they represent are the antithesis of the diversity that gives us our best chance of making good group decisions.

100

THE PERFECT SWARM

Unfortunately, societies are infested with groupthink. It exerts its menacing influence in families, local communities, and gangs. It can also make for trouble at parties, such as one that was organized over the Internet by teenager Corey Delaney, which resulted in the near destruction of his parents’ house by drunken youths. The social pressures in this case drove the teens (mostly testosterone-filled boys) to push each other to more and more senseless acts of vandalism, all in the name of fun. One curious and rather repugnant outcome of this episode was that a publicity agent sought Delaney out and set him up as a professional organizer of parties! What can we do to avoid such damaging effects of groupthink? It is a difficult question, because group decision making must always involve conditioning our choices on those of others. Yet the moment we do this we lose some of our independence, and the group loses some of its diversity. How can collective decisions preserve independence but still come to a final consensus? As pointed out by consensus specialist David Sumpter, this is the paradox that lies at the heart of groupthink, as it does in all group decision-making processes. Can the paradox be overcome? Irving Janis thought so, and he argued for a process reminiscent of the Delphi technique. His idea was simple: get the individuals in a group to collect information independently (as Feynman did) and work out a recommended course of action, then present that course of action and the reasons for it to a smaller number of centralized evaluators. It sounds ideal in theory, but would you trust a central evaluator to truly represent your views? The members of my political party certainly didn’t. Neither did Feynman after he found that his scientific reports had been filed away instead of being distributed. Both cases illustrate a problem that has been understood by game theorists for years: an independent arbiter can sometimes help to resolve a problem, but in many cases the arbiter becomes part of the problem instead.

Consensus: A Foolish Consistency?

101

If we can’t rely on leaders or independent arbiters to take proper account of our diversity, what else can we do to smooth the path from diversity to consensus? One option is to use swarm intelligence.

The Business of Swarm Intelligence Swarm intelligence is subtly different from group intelligence. The latter, as we’ve seen, is an approach to problem solving that takes advantage of the diversity within a group. Swarm intelligence is a spontaneous phenomenon that emerges from local interactions between individuals in a group. Swarm businesses that take advantage of both are now emerging. The archetypal example is the Web. According to researchers Peter Gloor and Scott Cooper from the MIT Sloan School of Management, who specialize in the use of collective intelligence in business, the explosive expansion of the Web in its early days arose for the following reason: If someone wanted to pursue a useful idea for extending the project—for instance, to include Web browsers and servers— the swarm embraced and supported the effort. There was no managerial hierarchy or proprietary ownership of ideas. Everyone cared deeply about the cause, not about rank, salary, status or money. They just wanted to get the job done, and in the end they changed the world with their innovation.

Diversity solved the problems and provided the innovations. The principles of complexity science did the rest, with a wonderfully intricate network emerging spontaneously from local rules of behavior and interaction. Traditional businesses are following suit. BMW, for example, posts engineering challenges on its website, so customers and company

102

THE PERFECT SWARM

designers collaborate to arrive at innovative solutions. Businesses such as Ford, Boeing, Procter & Gamble, Beiersdorf, and ChevronTexaco have taken advantage of advances in complexity science to design new swarm business approaches for some of their operations. The list of traditional companies that are taking up this innovative approach continues to grow. Cooperatives are another example of the combined effects of group intelligence and swarm intelligence. These are businesses that are jointly owned and democratically controlled by their members. According to the website of the worldwide cooperative movement, Co-operatives are trading enterprises, providing goods and services and generating profits, but these profits are not taken by outside shareholders as with many investor owned businesses— they are under the control of the members, who decide democratically how the profits should be used. Co-operatives use their profits [both in industrial and developing countries] for investing in the business, in social purposes, in the education of members, in the sustainable development of the community or the environment, or for the welfare of the wider community.

Today the cooperative movement is “a global force and employs approximately 1 billion people across the world. The UN estimates that the livelihoods of half the world’s population are made secure by co-operative enterprise.” In Switzerland, the two million customers of the giant Migros supermarket chain are members of its cooperative enterprise (out of a total population of seven million!) and have helped it develop through a process of self-organization. In all of these cases the individuals who make up the swarm see themselves as stakeholders rather than shareholders. A stakeholder is defined as any party that can affect or is affected by the innovation.

Consensus: A Foolish Consistency?

103

In a traditional business, this includes not only shareholders but also employees, customers, suppliers, partners, and even competitors. Changing the view changes the balance. Shareholders see themselves as outsiders. Stakeholders see themselves as insiders. A shareholder will sell shares or agree to a takeover if she sees that there is a profit to be made. A stakeholder will be less willing to do either, because he would thereby be changing something or losing something of which he is a part. Stakeholders see themselves as members of a swarm, which may be akin to a sense of family. Whatever the reason for people wanting to belong to a swarm, however, businesses that take advantage of its power are taking off. Some of these, such as Wikipedia and Project Gutenberg (a venture to make classic literature available on the Web) are not-for-profit entities. Others, such as Amazon.com, aim to make a profit by using swarm principles that include allowing people to post product reviews for other customers to read and making suggestions about what customers might like that are based on what individuals with similar buying patterns have bought. Swarm ideas are not confined to single businesses. They can equally be applied to groups of businesses, as is the case with Amazon.com. The main prerequisite is that favorable conditions should be set up to provide a platform on which those businesses can interact in a swarmlike manner. According to Gloor and Cooper, on whose ideas the above discussion is based, the principles of a swarm business are radically different from those of a traditional business in three important ways: •

Swarm businesses gain power by giving it away. In other words, the power resides with the stakeholders, not with the business itself. This is how businesses like Amazon and eBay make their money. Both offer a business platform—that is, an environment in which more traditional retailers can develop a market. They

104

THE PERFECT SWARM

provide the facilities for the establishment of a network and then let buyers and sellers get on with it while facilitating functions like product representation, regulatory compliance, risk management, and conflict resolution. •

Swarm businesses are willing to share with and support the swarm. One example is open source software. IBM, for example, spends $100 million per year to support the development of the freely available Linux operating system. The long-term outcome is that IBM gets more sales for its products that use Linux.

•

Swarm businesses put the welfare of members of the swarm ahead of making money. For example, the giant life sciences company Novartis AG, which was created by a merger between Sandoz and Ciba-Geigy, set up a venture fund in 1996 to encourage researchers and engineers who had lost their jobs as a result of the merger to start their own companies. The resultant swarm of companies ultimately paid real dividends back to the fund, but this wouldn’t have happened without the philanthropically based gamble in the first place.

Could we use the principles of a swarm business in our personal lives? One way would be to make more effective use of the various networks to which we all belong, from family to Facebook, and from close friends to our wider circles of acquaintances and contacts. In the next chapter I examine how such networks operate as complex adaptive systems, and how we can use their guiding principles most effectively.

SEVEN

Crazes, Contagion, and Communication: The Science of Networks

When I was barely into my teens, my father took me aside for a manto-man talk. Embarrassed, I followed him into his study, expecting a talk about the birds and the bees, a subject, I was convinced, that I already knew more about than he did. But he wanted to talk about my future. His talk was full of good advice, some of which I have followed. The piece of advice that I remember most was that the secret of success in life is to have good connections. What he was trying to tell me was that if you don’t know someone who can help with a problem, it’s worth knowing someone who might know someone else who could help. He built our first house this way, doing the bulk of the work himself but able to call on friends and friends of friends in the network he had built up over the years. The network was a strong one because the help was two-way. Everyone knew that he could repair

105

106

THE PERFECT SWARM

their watches and jewelry; they in turn were willing to offer their services as carpenters, plumbers, and electricians. It wasn’t a particularly big network as networks go—hardly a hundred people, most of them living within a couple of miles of each other, most of them knowing each other. The network of scientific friends and collaborators that I have built up during my career isn’t much bigger, and most of us also know each other. If I have a math problem that I can’t solve on my own (this applies to most math problems) I know where to go for help. If someone wants to use one of the instruments I have developed, they in turn know where to come. The fact that many of us are in different countries doesn’t matter. We are still a small, tightly interconnected network. We are also connected to a number of larger networks. The World Wide Web, and the Internet of which it is a part, are the biggest, but we also use telephone networks, delivery networks, airline networks, and road networks to pass information and material goods around. Such networks are self-organized to various degrees—“a web without a spider” to use network science pioneer Albert-László Barabási’s apt description. Some, like the Internet and the Web, seem to have taken on a life of their own. All have properties that belong to the network as a whole rather than to its individual members, such as the famous six degrees of separation by which everyone in the world is supposed to be connected. It is only since the late 1990s that such properties, and their relevance to our everyday lives, have begun to be properly understood.

What Is a Network? Samuel Johnson defined network in his famous dictionary as “any thing reticulated or decussated, at equal distances, with interstices between the intersections.” Johnson’s definition of a network reflects

Crazes, Contagion, and Communication: The Science of Networks

107

a genuine struggle with a complex concept; in fact, one that lies at the heart of complexity. If you cut out the bit about equal distances, it doesn’t look like a bad definition. According to the Oxford English Dictionary, decussated just means “formed with crossing lines.” Reticulated means “constructed or arranged like a net.” And interstices are gaps or holes. Johnson’s definition, though, misses the real point of a network. It’s not the holes that matter; it’s the links between the points of intersection that make the whole thing work. If it’s a road network, the links are the roads themselves, and the intersections are where they cross. If it’s an airline network, the links are the airline routes, and the intersections (technically called nodes) are airports. If it’s people, then the people themselves are the nodes. If they know each other personally, we say that there is a link between them. Sociologists draw this set of connections as a sociogram, which permits immediate visualization of personal relationships— who is linked with who, whether someone is socially isolated, and whether there are subnetworks within the overall network. What travels along the links depends on the composition of the network. In the days when international telephone calls required operators, one of the things that traveled along the links between operators was jokes. At least that was what I was told by a friend who used to be an international operator. I was relating an experience that I had before the days of direct dialing and the Internet. I was in England, due to get on a plane to fly to Australia, and the person sending me off told me a new joke. When I reached Sydney, I was greeted by someone who told me the same joke. I couldn’t work out how the joke got there so fast until my telephone operator friend told me that she and the other operators used to fill in slack periods by calling each other and sharing jokes (and other tasty tidbits) that they had overheard in people’s conversations.

108

THE PERFECT SWARM

Whether it’s jokes, packages, information, or even a viral disease like influenza that’s traveling through a network, mathematicians represent the network in the same way—as a pattern of dots (the nodes) connected by lines (the links). By doing so, they have been able to identify what different networks have in common, and how these features affect the performance of those networks. Networks, in the eyes of a complexity scientist, are complex systems. (If nodes or links can be added or removed, they can often become complex adaptive systems.) The links between the nodes represent local interactions, but the emergent properties of the network as a whole are somehow greater than the sum of those local interactions. A city, for example, may be thought of as a very complex network of local interactions between its residents. Some of them run boutiques, others clean the streets, work in offices, and buy their lunch from the local deli. Still others deliver supplies to the deli; provide transportation for those who work in the offices; arrange for the availability of water, power, and sewage treatment; manufacture clothing; and educate children. No one specifically directs all of this activity. The city network is a self-organizing system, and usually a successful one. How does it all work? Mathematicians using line and dot models have come up with some basic rules we can use to understand how the networks we belong to function, and they are beginning to work out how we can use that understanding to help solve everyday problems more effectively.

Connectivity There are two extreme forms of network. One is laid out in a prescribed pattern, like the wiring diagram of a computer or the hierarchical structure of an army. Such networks are totally ordered, like the regularly positioned atoms in a crystal lattice or the precise patterning of a spider’s web.

Crazes, Contagion, and Communication: The Science of Networks

109

The other extreme form of network is one in which the links are formed at random, like the crisscrossing streaks of paint in a piece of art created by Jackson Pollock. Random networks are a mathematician’s joy. One of the most famous findings in the mathematics of networks is that if you keep adding random links to a sparsely connected network, there comes a point when suddenly the whole network is interconnected. A Pollock painting might have some disconnected areas, for example, but one more streak of paint could mean that you could suddenly get from one crossing point to any other along the streaks that link the crossing points. Likewise, if you have a group of people who mostly don’t know each other, it is impossible to get a rumor spreading within the group. If all those in the group know and talk to just one other person, though, a rumor can spread like wildfire. This is because the point at which interconnectedness suddenly occurs is when there is an average of exactly one link per node. Random networks may be a mathematician’s joy, but there are comparatively few real-world examples. One is the highway system; cities are the nodes, and highways are the links, and the mathematical distribution of links follows the classic bell curve that is characteristic of a random distribution. One might also think that a random network of disease is initiated when someone sneezes in public and passes the disease on to a group of strangers, who in turn pass it on to other strangers through their own sneezing. In practice, though, the resultant network of infection is far from random, as I’ll show later. Most networks in real life are somewhere in between completely ordered and totally random. It was the idea of a totally random network, though, that sparked psychologist Stanley Milgram’s famous small world experiment, which involved trying to send a letter to a perfect stranger. Milgram wanted to know how many links there might be in a chain of connectedness between two random strangers. To test his

110

THE PERFECT SWARM

idea, he performed a series of experiments that have been copied and adapted many times since, in which a document is sent by a stranger to a target person via a chain of acquaintances who are on first-name terms with those on either side of them in the chain. In his best-known experiment, Milgram asked a random group of 196 people in Nebraska and 100 people in Boston to try to get a letter to a stockbroker in Boston by sending the letter to someone they knew by their first name and who might be closer to the target, with a request to send it on to someone that they knew by their first name and who might be closer still, and so on. Sixty-four chains reached their target, with an average number of just 5.5 steps for those originating in Nebraska and 4.4 for those originating in Boston. Milgram’s experiment provided the inspiration for John Guare’s 1990 play Six Degrees of Separation, which explores the idea that we are “bound to everyone on this planet by a trail of six people” and popularized the phrase, providing the stimulus for the many plays, books, films, and TV shows that have since been based on the same theme. The simple statistics of a random network provide a plausible reason for why short chains of connection might be the norm. Let’s suppose that each of us knows 100 people fairly well, and that each of them knows 100 people fairly well. So in just two links, any of them will be connected to every other one of them. That’s 10,000 people within just two links of each other. If each of them knows 100 people, that’s 100 x 10,000 = 1 million people within three links of each other. Keep carrying the argument forward, and by the time you get to six links that’s a thousand billion people, which is considerably larger than our estimated world population of around 7 billion. Other networks have similarly short chains, although the numbers are slightly different. The Web, for example, has nineteen degrees of separation, which means that any website is an average of nineteen clicks from any other. This may seem like a lot, but it’s a small

Crazes, Contagion, and Communication: The Science of Networks

111

number compared to the billion plus pages now on offer. If the links between pages were random, the figure could be accounted for by an average of just three links per website, since one billion is approximately equal to 319. The six degrees experiment and the analysis of the Web demonstrate the reality of the small world hypothesis, although some scholars have discovered serious flaws in Milgram’s work that make his conclusions more tenuous. In neither case, though, do the results prove that our global social network or the network of websites are interconnected in a purely random way. There could be many other ways of connection that would lead to similarly short chains. When Duncan Watts and Steve Strogatz from Cornell University took a closer look at the problem of connectedness, for example, they discovered that our social worlds could consist of tight, intimate networks yet still display the small world phenomenon of six degrees of separation. The trick was to use a different sort of connectivity. Instead of many random links in an otherwise unstructured network, just a few random links between members of different social networks created the same effect. My father took advantage of this effect when he was searching for a special tool for his jeweler’s lathe. No one in his local group had one, but one of them suggested contacting a friend in England who might. That friend didn’t have one either, but through his local network he found someone who did, and the tool was duly shipped to Australia. Problem solved, thanks to one long-range link between two otherwise isolated social groups. What’s surprising is how few such links are needed to shrink a world of otherwise largely isolated social networks. They provide shortcuts between nodes that would otherwise be many links apart. In mathematical terms, in the realm between completely ordered networks and completely random networks, there is a large region

112

THE PERFECT SWARM

Schematic representation of a set of seven small world networks (black dots linked by full lines, with circles emphasizing each network). Some nodes are separated by as many as twelve links. With the addition of just three long-range links (dotted lines), the number of links separating any two nodes is dramatically reduced.

where groups of nodes form tight clusters, but where there is only a small number of steps between nodes in different clusters, thanks to the presence of a few random links between the clusters. This is a region that is characteristic of the real world. Friendship networks, for example, can exhibit tight clustering—friends of any particular person are also likely to be friends of each other, yet the average number of friendships in a chain connecting two people in different networks can still be very small. My wife discovered just how small the number could be when she went to lunch with some friends from our neighborhood. The conver-

Crazes, Contagion, and Communication: The Science of Networks

113

sation turned to mutual acquaintances, and Wendy mentioned how surprised she had been when we moved to the area and learned that one of our neighbors had been the headmistress at her English boarding school. She wasn’t half as surprised as her host, though, who revealed that his first wife had been to the same school. Then it was Wendy’s turn to be surprised again when the wife’s name was mentioned—she had not only been to the same school but had also been in Wendy’s class! The rest of the conversation was taken up with close inquiries by the present wife as to just what the first one had been like. This sort of connectivity is not so exceptional, and many of us can doubtlessly recall similar stories. When Watts and Strogatz tested their ideas on different networks—which included those of Hollywood actor Kevin Bacon, the neural network of the nematode worm Caenorhabditis elegans, and the electrical power grid of the western United States—they found in all cases that there were clusters of strong connectivity but that the network as a whole still constituted a small world in which any node was just a few steps away from any other. That’s not to say that it is easy to discover the shortest path. Just as this book was going to press, Duncan Watts advised me of recent work in which he and his colleagues have distinguished between two versions of the small world hypothesis. What they call the “topological” version holds that for a randomly chosen pair of individuals in a population there is a high probability that there will be a short chain of connection between them. The stronger “algorithmic” version claims that ordinary individuals can navigate these chains themselves. There is strong evidence for the former, but the evidence for the latter is much more tenuous. It is especially important to distinguish between the two because each is relevant to different social processes. “The spread of a sexuallytransmitted disease along networks of sexual relations,” they say, “does not require that the participants have any awareness of the

114

THE PERFECT SWARM

disease, or any intention to spread it [and they] need only be connected in the topological sense. . . . Individuals attempting to ‘network’ [however] must actively traverse chains of referrals [and] thus must be connected in the algorithmic sense.” In other words, networking can involve a lot more than six degrees of separation because we may enter many blind alleys in the process and have to backtrack or try again, but the spread of diseases such as swine flu does indeed happen via six degrees of separation. Most of the letters in Milgram’s experiment never reached their target because someone, somewhere along a chain, simply couldn’t be bothered to send it on. When Watts and his coworkers adapted Milgram’s experiment to the world of the Internet, they found something similar. Of the 24,163 chains that were started, only 384 reached their target—just 1.5 percent. When participants didn’t send the message on, it wasn’t because they couldn’t think of anyone to send it to. When asked, most of them gave lack of interest or lack of incentive as the reason. So that’s a lesson for anyone wanting to establish a chain of contacts to someone they don’t know. There will be little chance of success unless the person starting the chain is able to provide an incentive that can be passed along the chain. Of course, chain letters (nowadays largely superseded by chain emails) often provide the incentive of monetary rewards or the threat of bad luck to persuade people to keep the chain going. One notorious example of a chain letter from the 1930s was the “Prosperity Club” (also known as “Send-a-dime”). Here is the wording of the actual letter, which tells it all: Prosperity Club. We are sending you a membership to the prosperity club so be sure and send five letters, as this chain has never been broken.

116

THE PERFECT SWARM

In God we Trust. Mrs. D.O. Ostby, Sheyenne N.D. Miss Mary Borthwick, Warwick N.D. Miss Alice E. Kennedy, Sheyenne, N.D. Miss Bertha A Jacobson, Sheyenne N.D. Miss Magnhild Larson, Sheyenne N.D. Oscar Hasum, Madock N.D. This chain was started in hope of bringing prosperity within 3 days. Make 5 copies of this letter leaving off the top name and adding your name at the bottom and mail to 5 of your friends to whom you wish prosperity to come in. Omitting the top name send that person 10 ct (wrapped in paper) as a charity donation. In turn as your name leaves the top you will receive 15,625 letters with donations amounting to $1562.50. Is this worth a dime of your money? Have the faith your friends have had and this chain will never be broken.

The effectiveness of this particular letter, which started in Denver, Colorado, in 1935, was such that the Denver Post Office was swamped with hundreds of thousands of letters, which spilled over to St. Louis, Missouri, and other cities. Modern equivalents include chain e-mails on platforms like Facebook, YouTube, and MySpace, and even chain text messages on cell phones. Chain letters and their successors offer substantial rewards to keep a chain going or severe punishments for breaking it. Watts and his coworkers found that it doesn’t take much incentive to keep a chain going but that it does usually take some. Just the feeling that an email has a chance of reaching its target is sometimes sufficient, as shown by the fact that 44 percent of all completed chains were to a professor at a well-known university who was an “obvious” target

Crazes, Contagion, and Communication: The Science of Networks

117

to the largely middle-class participants in the United States, compared to the other seventeen mainly overseas targets. Even when there is an incentive to extend a chain, we still need to find some efficient way of making it manageably short. One mathematically based method relies on forwarding the message to the contact who is closest to the target in terms of lattice distance (separated from the target by the smallest number of links). This is all very well if one has an Olympian perspective of the whole network, but it is hardly practicable in real social terms. A more realistic method, which combines knowledge of network ties and social identity, relies on the fact that we are all members of numerous networks. We can take advantage of this fact by sending our message to the contact who seems closest to the target in terms of social distance—in other words, whose social position means that they have the highest chance of knowing the target, or knowing someone who does. This is what the participants in Milgram’s experiment and its successors seem to have done intuitively. It was also what happened when my father was searching for a lathe tool. He found it very quickly because his friend identified the right person to contact— one who was the closest in social distance to the problem. When someone in my family wanted to make contact with a distant relative, though, they used a different approach—they asked Auntie Lilla. Auntie Lilla had links with just about everyone in our family network. When she didn’t have a direct link, she certainly knew someone who did. By asking Auntie Lilla, you could save yourself a lot of hassle in searching for relatives. Usually it just took one telephone call. In network terms, Auntie Lilla was a hub—a node with links to many other nodes. The importance of hubs in large-scale networks has only been realized since the early 2000s.

118

THE PERFECT SWARM

Hubs Hubs emerge as a consequence of network self-organization. When mathematician Albert-László Barabási and his bright student Réka Albert looked at the distribution of links in the actor networks and the power grid that Watts and Strogatz studied, they expected to find a classical bell curve. Such curves have the shape of a hanging church bell, the peak corresponding to some average number of links. Instead, they found that a few nodes had many connections (far more than could be reconciled with the Watts and Strogatz model) while the rest had many fewer connections. There are a number of ways to express a power law. It is akin to Murphy’s Law of Management, which states that 80 percent of the work is done by 20 percent of the employees, or the customer service law, which says that 80 percent of the customer service problems are created by 20 percent of the customers. These can be visualized by drawing them as a graph (see upper figure page 119). When this is done for the distribution of global wealth, the graph takes the shape of half of the bowl of a champagne or daiquiri glass, and the 80:20 rule becomes what is known as the “champagne glass” effect. The graph reveals more detail than the stated rule. Produced as part of a U.N. Human Development Report, it shows that the richest 20 percent of humanity hoards 83 percent of the wealth, while the poorest 60 percent subsists on 6 percent of the wealth. The champagne glass shape of the graph denotes a power law, rather like Newton’s Law of Gravity, which states that the force of gravity F between two objects is inversely proportional to the square of the distance d between them. This is written as F :d –2. When Barabási and Albert looked at the distribution of links in Kevin Bac

The Perfect Swarm: The Science of Complexity in Everyday Life - PDF Free Download (2024)