Science in Christian Perspective

 

 

  Statistical Problems in Extra Seensory PerceptionR. P. DILWORTH, Ph.D.

From JASA 9 (June 1957): 8-10.

Critics of ESP frequently point to the statistical nature of the evidence for ESP in questioning the validity of the experiments. However; if the use of statistical methods, per se, casts doubt upon the validity of the conclusions, then nearly all of experimental science is suspect. For it is clear that even the most carefully designed experimental procedure is subject to some error. If this error is neglected, then exact methods have been used where they do not apply and the conclusions are accordingly approximate; or else the statistical nature of the error is taken into account and the conclusions are stated in statistical terms. Thus the use of statistical techniques in ESP experiments should no more affect the evaluation of the results than in any other area of modern science. Indeed, the fact that they have been used successfully in a wide variety of scientific investigations indicates the necessity for a careful and rigorous use of statistical analysis in the study of ESP.

In a recent article, P. W. Bridgman goes even further and suggests that perhaps all is not right with the fundamental techniques of probability and statistics. He writes:

"It has long been apparent that there is something 'funny' about the probability situation. Probability rigorously applies to no concrete happening-. Yet the phenomena to which the probability calculations justifying ESP are applied are concrete actual happenings, many of them a matter of record in black and white.

"These considerations, I think, make it particularly clear that the locus of chance is in ourselves with strong involvements of 'expectation' and 'surprise', and that there is little that is 'objective' about it.

"The situation covered by the word probability is a desperately complex situation, mostly of our own making and in our own minds with a fragile and fleeting dependence on time and never coherently connected with 'objective' events."

At another point in the article he hints that there are probably some basic defects in our understanding of probability. It-may be appropriate to comment at this point that this is certainly true as far as Mr. Bridgman is concerned, for in a fairly detailed article on probability theory, he not once mentions the concept of a population. Yet this concept is essential for the understanding of probability theory and its applications. Without the notion of population the subject does appear mysterious and the problem of separating the subjective and objective elements becomes difficult indeed.

As a branch of mathematics, probability theory is concerned with a collection of elements over which a probability measure is defined. The collection is called the "population" (or sample space) ; the elements (or more generally, certain distinguished subcollections) are called "events" and the theory consists of the consequences which can be deduced from the measure axioms. Clearly from this point of view, the theory of probability rests on the same foundations as any other branch of mathematics. Now when this probability model is applied there must be a population of concrete events which corresponds to the abstract population in such a way that the probability of the abstract event represents the relative frequency of occurrence of the concrete event. The accuracy of this representation will determine the extent to which the model can be used with confidence in the study of the concrete situation. Indeed, there may be varying opinions among individuals concerning the reliability of a given model. This is always the situation in any area of applied mathematics. A mathematical formulation is an idealization of the concrete situation and though the mathematical work is entirely correct, the validity of the application may be a matter of opinion. Futhermore, just as a highly idealized mathematical analysis may be most useful in understanding the given physical process, so even a rough probability model may be very helpful in understanding a physical process in which chance effects occur. Now in the physical sciences, a mathematical model is checked by making theoretical studies and comparing the results with experiment. This comparison will involve the use of probability since experimental procedures are subject to error. The laws of probability have been checked in countless concrete instances in exactly the same way. The overwhelming consistency of these many tests constitute the most impressive evidence for the applicability of probabilistic methods. It is true that probability itself is used in making the experimental verification of the theory. However, if only the broad, general, outline of the theory is assumed to apply, it still suffices for exceedingly precise verification of the laws of probability.

Since the applications of probability have been so thoroughly tested, it would seem unlikely that the subject would be plagued with paradoxes and problems, yet this is indeed the case, as Bridgman's article indicates. Occasionally the difficulty is simply a misuse of the laws of probability, but the usual source of confusion is the misidentification of the population. It is a common misconception that probability is in some sense a number which is once and for all attached to an event and is moreover independent of other events. But as pointed out above, before probability can even be defined, the collection of events which make up the population must be specified and probability is then defined relative to this population. Now in some cases, the population in which the event occurs may not be particularly significant; thus if the population is the collection of possible tosses of a given coin, it does not make much difference which particular coin is used since all common coins have heads on one side and tails on the other. Thus a model which assigns equal probabilities of one-half to the two events is a satisfactory representation of the coin tossing experiment. On the other hand, if the coin was part of a magician's equipment, then the model might no longer be applicable. As another example consider Russell's license plate problem, which is mentioned by Bridgman. If before starting for work I ask what is the probability that the first license plate which I observe will be RGL 749, then the appropriate population is the observation of the various license plates in the city. These events will not be equally probable since I shall be much less likely to first observe a license plate on a car locked in a garage than a license plate on a taxi cruising the streets of the city. However, if there are a great many automobiles in the city, the probability of observing the particular license plate RGL 749, will be exceedingly small. If I then set out to work and the first license plate which I observe is indeed RGL749, then I have every right to be surprised for, in fact, an unusual event has occurred. However, if without specifying a particular license plate, I set out to work and observe the first license plate to be RGL 749, then I have no right to judge that an unusual event has taken place, since now the appropriate population consists of one event and is the event of first observing RGL 749. Its probability is indeed one. If one still wishes the population to consist of all license plates in the city, then a new probability measure must be assigned, namely, RGL 749 with probability one and all other license plates with probability zero. Clearly there is no mystery if the populations are carefully distinguished.  

Let us consider one further example. Player A deals five cards from a shuffled pack of playing cards to Player B who has the privilege of looking at his cards. Now if A draws a card at random from the cards remaining in the deck, let us suppose that the players are interested in determining the probability that the card will be the ace of spades. Note that the situation is different for the two players. For Player A the appropriate population is the set of all distributions of five cards to an opponent and then the selection of a card for himself. Since the elementary events are equally likely, this is equivalent to the selection by A of a card at random from the entire deck. Hence for A the probability that the card drawn is an ace of spades is 1/52. Now what is the appropriate population for Player B ? He has a specific set of five cards in his hand; hence the appropriate population for him is the set of selections of a card from a deck with the specific five cards removed. If he holds the ace of spades, the probability that A will draw that particular card is thus zero; if he does not hold the ace of spades, then the probability is 1/47.

This example shows clearly that there is not an absolute probability associated with an event, but that the probability depends upon the population under consideration. Since in the example the choice of the population for the second player is dependent upon his knowledge, this would seem to support Bridgman's contention that probability is highly subjective in character. Insofar as subjective elements may affect the choice of the population, this is indeed the case. Furthermore, this subjective element may enter in any area of experimental science. For example, if an investigator is studying the motion of a particle in a gravitational field and is unaware that the particle is charged and that an electromagnetic field is present, the mathematical model which he uses will lead to incorrect results. On the other hand, another scientist, aware of additional factor, will choose another model and will interpret the behavior of the particle correctly. Note that if the two researchers had shared their knowledge, they would likely have chosen the same model. It is in this sense that the use of mathematics in scientific pursuits is "objective." Likewise two statisticians who share the same information concerning a chance event will generally agree on the choice of the probability model to represent the phenomena. Hence the application of probability is just as "objective" as the applications of mathematics in any domain of science.

Finally let us consider the specific implications of these considerations for ESP. Clearly the evidence for ESP must be statistical in nature, for even if a sub ject were able to guess cards with complete accuracy, the number of correct guesses necessary for signifi cance would be obtained by comparison with random guessing. But, as we have seen, in order to apply a probabilistic model, a population must be specified. Now a desirable population would be a collection of a large number of guesses from a given subject at a given time. But this is impossible since the experi mental procedure takes time. Such a model might still be applicable if it can be assumed that the responses
of the subject are independent of time. However, this point of view presents difficulties, since it is readily
acknowledged in ESP work that the "ability" of sub jects vary greatly from day to day. An alternative model would be a population of guesses by a given subject over a specified period of time. Note that if such a model is adopted than the experimental pro cedure must be adopted to the model. Thus it would be inappropriate, for example, to stop the experiments whenever the subject felt that he was losing his "ability." An alternative model which allows for such"optional stopping" can be devised. However, the probability calculations will be different from those of
the previous model. 

In interpreting the results of a series of experiments by means of a given probability model, it is highly important that all of the results be included in the analysis. This means that unfavorable series must be given the same consideration as the favorable series. It is not enough for the investigator to decide that the subject had an "off-day" and to discard the data on this basis, for by a little judicious selection, a purely random series can be made to appear significant. In this connection it should be pointed out that the practice of throwing out unfavorable data occurs occasionally in nearly every field of scientific research. If the procedure cannot be justified upon sound statistical grounds, the practice is always to be deplored. This is particularly true in the case of ESP where the question at issue is so highly controversial.

In conclusion it should be observed that one of the fundamental features of the scientific method is the mutual checking of experimental work by several investigators. The ability of another research worker to reproduce a given piece of work is an important consideration in determining its validity. Thus the investigation of ESP poses particular problems in this regard, since psychic ability, if it exists, is conceded to be transient and variable. Nevertheless, in order to have the status of scientific fact ' the criterion of reproducibility must be satisfied. On the other hand, it must be acknowledged that if an ESP phenomena is produced which can be checked to be significant by any competent investigator wishing to duplicate the experiment, then in spite of the magicians, conjurers, and crackpots, the phenomena will have been established just as surely as any other portion of our scientific knowledge.