Fooled by Randomness Read online

  FOOLED BY

  RANDOMNESS

  •

  CONTENTS

  Title Page

  Dedication

  Preface

  Acknowledgments for the Updated Second Edition

  Chapter Summaries

  Prologue

  PART I: SOLON’S WARNING

  •

  Skewness, Asymmetry, Induction

  One IF YOU’RE SO RICH, WHY AREN’T YOU SO SMART?

  NERO TULIP

  Hit by Lightning

  Temporary Sanity

  Modus Operandi

  No Work Ethics

  There Are Always Secrets

  JOHN THE HIGH-YIELD TRADER

  An Overpaid Hick

  THE RED-HOT SUMMER

  Serotonin and Randomness

  YOUR DENTIST IS RICH, VERY RICH

  Two A BIZARRE ACCOUNTING METHOD

  ALTERNATIVE HISTORY

  Russian Roulette

  Possible Worlds

  An Even More Vicious Roulette

  SMOOTH PEER RELATIONS

  Salvation via Aeroflot

  Solon Visits Regine’s Nightclub

  GEORGE WILL IS NO SOLON: ON COUNTERINTUITIVE TRUTHS

  Humiliated in Debates

  A Different Kind of Earthquake

  Proverbs Galore

  Risk Managers

  Epiphenomena

  Three A MATHEMATICAL MEDITATION ON HISTORY

  Europlayboy Mathematics

  The Tools

  Monte Carlo Mathematics

  FUN IN MY ATTIC

  Making History

  Zorglubs Crowding the Attic

  Denigration of History

  The Stove Is Hot

  Skills in Predicting Past History

  My Solon

  DISTILLED THINKING ON YOUR PALMPILOT

  Breaking News

  Shiller Redux

  Gerontocracy

  PHILOSTRATUS IN MONTE CARLO : ON THE DIFFERENCE BETWEEN NOISE AND INFORMATION

  Four RANDOMNESS, NONSENSE, AND THE SCIENTIFIC INTELLECTUAL

  RANDOMNESS AND THE VERB

  Reverse Turing Test

  The Father of All Pseudothinkers

  MONTE CARLO POETRY

  Five SURVIVAL OF THE LEAST FIT–CAN EVOLUTION BE FOOLED BY RANDOMNESS?

  CARLOS THE EMERGING-MARKETS WIZARD

  The Good Years

  Averaging Down

  Lines in the Sand

  JOHN THE HIGH-YIELD TRADER

  The Quant Who Knew Computers and Equations

  The Traits They Shared

  A REVIEW OF MARKET FOOLS OF RANDOMNESS CONSTANTS

  NAIVE EVOLUTIONARY THEORIES

  Can Evolution Be Fooled by Randomness?

  Six SKEWNESS AND ASYMMETRY

  THE MEDIAN IS NOT THE MESSAGE

  BULL AND BEAR ZOOLOGY

  An Arrogant Twenty-nine-year-old Son

  Rare Events

  Symmetry and Science

  ALMOST EVERYBODY IS ABOVE AVERAGE

  THE RARE-EVENT FALLACY

  The Mother of All Deceptions

  Why Don’t Statisticians Detect Rare Events?

  A Mischievous Child Replaces the Black Balls

  Seven THE PROBLEM OF INDUCTION

  FROM BACON TO HUME

  Cygnus Atratus

  Niederhoffer

  SIR KARL’S PROMOTING AGENT

  Location, Location

  Popper’s Answer

  Open Society

  Nobody Is Perfect

  Induction and Memory

  Pascal’s Wager

  THANK YOU, SOLON

  PART II: MONKEYS ON TYPEWRITERS

  •

  Survivorship and Other Biases

  IT DEPENDS ON THE NUMBER OF MONKEYS

  VICIOUS REAL LIFE

  THIS SECTION

  Eight TOO MANY MILLIONAIRES NEXT DOOR

  HOW TO STOP THE STING OF FAILURE

  Somewhat Happy

  Too Much Work

  You’re a Failure

  DOUBLE SURVIVORSHIP BIASES

  More Experts

  Visibility Winners

  It’s a Bull Market

  A GURU’S OPINION

  Nine IT IS EASIER TO BUY AND SELL THAN FRY AN EGG

  FOOLED BY NUMBERS

  Placebo Investors

  Nobody Has to Be Competent

  Regression to the Mean

  Ergodicity

  LIFE IS COINCIDENTAL

  The Mysterious Letter

  An Interrupted Tennis Game

  Reverse Survivors

  The Birthday Paradox

  It’s a Small World!

  Data Mining, Statistics, and Charlatanism

  The Best Book I Have Ever Read!

  The Backtester

  A More Unsettling Extension

  The Earnings Season: Fooled by the Results

  COMPARATIVE LUCK

  Cancer Cures

  Professor Pearson Goes to Monte Carlo (Literally): Randomness Does Not Look Random!

  The Dog That Did Not Bark: On Biases in Scientific Knowledge

  I HAVE NO CONCLUSION

  Ten LOSER TAKES ALL—ON THE NONLINEARITIES OF LIFE

  THE SANDPILE EFFECT

  Enter Randomness

  Learning to Type

  MATHEMATICS INSIDE AND OUTSIDE THE REAL WORLD

  The Science of Networks

  Our Brain

  Buridan’s Donkey or the Good Side of Randomness

  WHEN IT RAINS, IT POURS

  Eleven RANDOMNESS AND OUR MIND: WE ARE PROBABILITY BLIND

  PARIS OR THE BAHAMAS?

  SOME ARCHITECTURAL CONSIDERATIONS

  BEWARE THE PHILOSOPHER BUREAUCRAT

  Satisficing

  FLAWED, NOT JUST IMPERFECT

  Kahneman and Tversky

  WHERE IS NAPOLEON WHEN WE NEED HIM?

  “I’m As Good As My Last Trade” and Other Heuristics

  Degree in a Fortune Cookie

  Two Systems of Reasoning

  WHY WE DON’T MARRY THE FIRST DATE

  Our Natural Habitat

  Fast and Frugal

  Neurobiologists Too

  Kafka in a Courtroom

  An Absurd World

  Examples of Biases in Understanding Probability

  We Are Option Blind

  PROBABILITIES AND THE MEDIA (MORE JOURNALISTS)

  CNBC at Lunchtime

  You Should Be Dead by Now

  The Bloomberg Explanations

  Filtering Methods

  We Do Not Understand Confidence Levels

  An Admission

  PART III: WAX IN MY EARS

  •

  Living with Randomitis

  I AM NOT SO INTELLIGENT

  WITTGENSTEIN’S RULER

  THE ODYSSEAN MUTE COMMAND

  Twelve GAMBLERS’ TICKS AND PIGEONS IN A BOX

  TAXI-CAB ENGLISH AND CAUSALITY

  THE SKINNER PIGEON EXPERIMENT

  PHILOSTRATUS REDUX

  Thirteen CARNEADES COMES TO ROME: ON PROBABILITY AND SKEPTICISM

  CARNEADES COMES TO ROME

  Probability, the Child of Skepticism

  MONSIEUR DE NORPOIS’ OPINIONS

  Path Dependence of Beliefs

  COMPUTING INSTEAD OF THINKING

  FROM FUNERAL TO FUNERAL

  Fourteen BACCHUS ABANDONS ANTONY

  NOTES ON JACKIE O. ’S FUNERAL

  RANDOMNESS AND PERSONAL ELEGANCE

  Epilogue SOLON TOLD YOU SO

  Beware the London Traffic Jams

  Postscript THREE AFTERTHOUGHTS IN THE SHOWER

  FIRST THOUGHT: THE INVERSE SKILLS PROBLEM

  SECOND THOUGHT: ON SOME ADDITIONAL BENEFITS OF RA
NDOMNESS

  Uncertainty and Happiness

  The Scrambling of Messages

  THIRD THOUGHT: STANDING ON ONE LEG

  Acknowledgments for the First Edition

  A Trip to the Library: Notes and Reading Recommendations

  Notes

  References

  About the Author

  Also by Nassim Nicholas Taleb

  Copyright

  To my mother,

  Minerva Ghosn Taleb

  PREFACE

  TAKING KNOWLEDGE

  LESS SERIOUSLY

  This book is the synthesis of, on one hand, the no-nonsense practitioner of uncertainty whospen this professional life trying to resist being fooled by randomness and trick the emotions associated with probabilistic outcomes and, on the other, the aesthetically obsessed, literature-loving human being willing to be fooled by any form of nonsense that is polished, refined, original, and tasteful. I am not capable of avoiding being the fool of randomness; what I can do is confine it to where it brings some aesthetic gratification.

  This comes straight from the gut; it is a personal essay primarily discussing its author’s thoughts, struggles, and observations connected to the practice of risk taking, not exactly a treatise, and certainly, god forbid, not a piece of scientific reporting. It was written for fun and it aims to be read (principally) for, and with, pleasure. Much has been written about our biases (acquired or genetic) in dealing with randomness over the past decade. The rules while writing the first edition of this book had been to avoid discussing (a) anything that I did not either personally witness on the topic or develop independently, and (b) anything that I have not distilled well enough to be able to write on the subject with only the slightest effort. Everything that remotely felt like work was out. I had to purge from the text passages that seemed to come from a visit to the library, including the scientific name dropping. I tried to use no quote that did not naturally spring from my memory and did not come from a writer whom I had intimately frequented over the years (I detest the practice of random use of borrowed wisdom—much on that later). Aut tace aut loquere meliora silencio (only when the words outperform silence).

  These rules remain intact. But sometimes life requires compromises: Under pressure from friends and readers I have added to the present edition a series of nonintrusive endnotes referring to the related literature. I have also added new material to most chapters, most notably in Chapter 11, which altogether has resulted in an expansion of the book by more than a third.

  Adding to the Winner

  I hope to make this book organic—by, to use traders’ lingo, “adding to the winner”—and let it reflect my personal evolution instead of holding on to these new ideas and putting them into a new book altogether. Strangely, I gave considerably more thought to some sections of this book after the publication than I had before, particularly in two separate areas: (a) the mechanisms by which our brain sees the world as less, far less, random that it actually is, and (b) the “fat tails,” that wild brand of uncertainty that causes large deviations (rare events explain more and more of the world we live in, but at the same time remain as counterintuitive to us as they were to our ancestors). The second version of this book reflects this author’s drift into becoming a little less of a student of uncertainty (we can learn so little about randomness) and more of a researcher into how people are fooled by it.

  Another phenomenon: the transformation of the author by his own book. As I increasingly started living this book after the initial composition, I found luck in the most unexpected of places. It is as if there were two planets: the one in which we actually live and the one, considerably more deterministic, on which people are convinced we live. It is as simple as that: Past events will always look less random than they were (it is called the hindsight bias). I would listen to someone’s discussion of his own past realizing that much of what he was saying was just backfit explanations concocted ex post by his deluded mind. This became at times unbearable: I could feel myself looking at people in the social sciences (particularly conventional economics) and the investment world as if they were deranged subjects. Living in the real world may be painful particularly if one finds statements more informative about the people making them than the intended message: I picked up Newsweek this morning at the dentist’s office and read a journalist’s discussion of a prominent business figure, particularly his ability in “timing moves” and realized how I was making a list of the biases in the journalist’s mind rather than getting the intended information in the article itself, which I could not possibly take seriously. (Why don’t most journalists end up figuring out that they know much less than they think they know? Scientists investigated half a century ago the phenomena of “experts” not learning about their past failings. You can mispredict everything for all your life yet think that you will get it right next time.)

  Insecurity and Probability

  I believe that the principal asset I need to protect and cultivate is my deep-seated intellectual insecurity. My motto is “my principal activity is to tease those who take themselves and the quality of their knowledge too seriously.” Cultivating such insecurity in place of intellectual confidence may be a strange aim—and one that is not easy to implement. To do so we need to purge our minds of the recent tradition of intellectual certainties. A reader turned pen pal made me rediscover the sixteenth-century French essayist and professional introspector Montaigne. I got sucked into the implications of the difference between Montaigne and Descartes—and how we strayed by following the latter’s quest for certitudes. We surely closed our minds by following Descartes’ model of formal thinking rather than Montaigne’s brand of vague and informal (but critical) judgment. Half a millennium later the severely introspecting and insecure Montaigne stands tall as a role model for the modern thinker. In addition, the man had exceptional courage: It certainly takes bravery to remain skeptical; it takes inordinate courage to introspect, to confront oneself, to accept one’s limitations—scientists are seeing more and more evidence that we are specifically designed by mother nature to fool ourselves.

  There are many intellectual approaches to probability and risk—“probability” means slightly different things to people in different disciplines. In this book it is tenaciously qualitative and literary as opposed to quantitative and “scientific” (which explains the warnings against economists and finance professors as they tend to firmly believe that they know something, and something useful at that). It is presented as flowing from Hume’s Problem of Induction (or Aristotle’s inference to the general) as opposed to the paradigm of the gambling literature. In this book probability is principally a branch of applied skepticism, not an engineering discipline (in spite of all the self-important mathematical treatment of the subject matter, problems related to the calculus of probability rarely merit to transcend the footnote).

  How? Probability is not a mere computation of odds on the dice or more complicated variants; it is the acceptance of the lack of certainty in our knowledge and the development of methods for dealing with our ignorance. Outside of textbooks and casinos, probability almost never presents itself as a mathematical problem or a brain teaser. Mother nature does not tell you how many holes there are on the roulette table, nor does she deliver problems in a textbook way (in the real world one has to guess the problem more than the solution). In this book, considering that alternative outcomes could have taken place, that the world could have been different, is the core of probabilistic thinking. As a matter of fact, I spent all my career attacking the quantitative use of probability. While Chapters13 and 14 (dealing with skepticism and stoicism) are to me the central ideas of the book, most people focused on the examples of miscomputation of probability in Chapter 11 (clearly and by far the least original chapter of the book, one in which I compressed all the literature on probability biases). In addition, while we may have some understanding of the probabilities in the hard sciences, particularly in physics, we don’t have much of a clu
e in the social “sciences” like economics, in spite of the fanfares of experts.

  Vindicating (Some) Readers

  I have tried to make the minimum out of my occupation of mathematical trader. The fact that I operate in the markets serves only as an inspiration—it does not make this book (as many thought it was) a guide to market randomness any more than the Iliad should be interpreted as a military instruction manual. Only three out of fourteen chapters have a financial setting. Markets are a mere special case of randomness traps—but they are by far the most interesting as luck plays a very large role in them (this book would have been considerably shorter if I were a taxidermist or a translator of chocolate labels). Furthermore, the kind of luck in finance is of the kind that nobody understands but most operators think they understand, which provides us a magnification of the biases. I have tried to use my market analogies in an illustrative way as I would in a dinner conversation with, say, a cardiologist with intellectual curiosity (I used as a model my second-generation friend Jacques Merab).

  I received large quantities of electronic mail on the first version of the book, which can be an essayist’s dream as such dialectic provides ideal conditions for the rewriting of the second version. I expressed my gratitude by answering (once) each one of them. Some of the answers have been inserted back into the text in the different chapters. Being often seen as an iconoclast I was looking forward to getting the angry letters of the type “who are you to judge Warren Buffett” or “you are envious of his success”; instead it was disappointing to see most of the trashing going anonymously to amazon.com (there is no such thing as bad publicity: Some people manage to promote your work by insulting it).

  The consolation for the lack of attacks was in the form of letters from people who felt vindicated by the book. The most rewarding letters were the ones from people who did not fare well in life, through no fault of their own, who used the book as an argument with their spouse to explain that they were less lucky (not less skilled) than their brother-in-law. The most touching letter came from a man in Virginia who within a period of a few months lost his job, his wife, his fortune, was put under investigation by the redoubtable Securities and Exchange Commission, and progressively felt good for acting stoically. A correspondence with a reader who was hit with a black swan, the unexpected large-impact random event (the loss of a baby) caused me to spend some time dipping into the literature on adaptation after a severe random event (not coincidentally also dominated by Daniel Kahneman, the pioneer of the ideas on irrational behavior under uncertainty). I have to confess that I never felt really particularly directly of service to anyone being a trader (except myself); it felt elevating and useful being an essayist.