We are often surprised by events that seem very unlikely to occur by chance. We may dream about a person the same night that the person dies. Is this proof of a psychic link with the dead person? With identical twins separated at birth, it may be found that they "both married women named Jennifer, both drive Chevrolets, and both dislike mustard." Is this proof of genetic dominance in determining behavior? Or are both just coincidences?
Two statisticians, Perci Diaconis and Fred Mosteller (1989) defined coincidence as, "a surprising occurrence of events, perceived as meaningfully related, with no apparent causal connection." Coincidences occur all around us all the time. However, we only notice them when the coincidence is meaningful to us. A problem occurs when we try to use the coincidence to "prove a point."
This is what happens with numerology, which attempts to find meanings in numbers. One example is used by numerology to claim that William Shakespeare translated the King James Bible. Their "proof" is that, in Psalm 46 of the King James Bible, the 46th word is "shake" and the 46th word from the end is "spear". Why is the number 46 so significant? According to numerologists, it is because Shakespeare was 46 years old when that version of the bible was published. However, numerologists never specify in advance exactly where or what the coincidence will be.
Another example is using any combination of the number "666" to indicate that someone is the Beast of Revelation. Some said that our ex-president, Ronald Wilson Reagan, was the Beast. Why? Each of his names has six letters, therefore "666". However, if you look long enough, you can probably find some arrangement of 666 with almost anyone.
It seems that we humans try to find meaning in the things that occur around us. With the Arizona lottery, out of 45 different numbers, the chance of picking all six numbers is about one in 8 million. In other words, no matter what these numbers are, the chance of any particular combination of six numbers occurring in the next drawing is about one 8 million. Of course, if you don't win any money, you don't consider that specific combination of numbers as significant. If you only buy one ticket, only one combination of numbers will any meaning for you. Each of the millions of other combinations is just as unlikely as the one you picked. However, they are all lumped together with one meaning — "I didn't win."
The method used by numerologists can be applied to psychological data. Their method is called data dredging or data snooping. If you look at any set of data in enough different ways, you can always come up with something interesting. Suppose you have a large amount of random data — data that occurs solely by chance. In this random data, if psychologists find something that would occur just by chance less than 5% of the time, that information is considered significant. However, this also means that — even if the data is completely random — 5% of the time significance may be assumed when none is there.
If more independent tests of significance are done on the same random data, the probability goes up that you will find something that seems significant. If you do 14 such tests, you will have better than even odds of finding something "significant". If you do 20 such tests, the probability is almost two out of three times.
Data dredging is like
"shooting the arrow first
and drawing the bull's eye later."
Both psychologists and numerologists can use data dredging. However, this is like shooting an arrow first and drawing the bull's eye after it lands. This is why it is necessary to distinguish between exploratory and confirmatory data analysis. Exploratory analysis is done without predicting exactly what is expected. You are looking through information to see if you can find something interesting. However, if psychologists find something interesting with exploratory analysis, they typically follow it up with confirmatory analysis — they specify in advance exactly what they are seeking. Most psychologists only consider the exploratory results truly significant, if the same results show up in repeated independent analyses as predicted.
In contrast, many people just accept the relationship as "proof" of something they want to believe. They assume a "cause-and-effect" relationship. Thus dreaming of a person when they die is accepted as proof of a psychic link. Similarities among identical twins are taken as evidence of the dominance of genetics. However, such similarities have also been found between unrelated people.
So if you find some spectacular meaning in the relationship of two or more factors, we are not asking you to reject it. However, you should be critical. Can it be confirmed by other information, or is it just a coincidence?
* Adapted from Donald H. McBurney's How to Think Like a Psychologist: Critical Thinking in Psychology, Prentice Hall Publishers, 1996, pages 73-78.
Go back to "A Line on Life" page.