Abstract

It is obvious that the results of many psi experiments vary intra- and inter-individually and also intra- and inter-experimentally to a statistically significant degree. Under these circumstances the simple addition of hits, carried out over all experimental segments and Ss, is an inefficient method of statistical evaluation. The increase of variance and the corresponding decrease of statistical power will become particularly strong, when the psi effect varies bi-directionally between hitting and missing. In this case it is even possible that positive and negative partial effects cancel each other out and the overall deviation drops to zero. Usually, however, the tendency towards hitting may prevail so that for the partial results a small shift of the mean together with an increased variance is to be expected. Hence, for most psi experiments a method of aggregation is recommendable which simultaneously is sensitive to alternations of mean and variance. Such a method can be called a variability-related aggregation.

In contrast to this, the conventional evaluation should lead to many experiments in which no overall significance results, although real psi effects may have occurred in them. In fact this prediction is fulfilled in practice. Many parapsychologists report that in an experiment no "overall effect could be detected, but at least one partial psi effect would be verified since some partial results would be clearly significant. Unfortunately, their selective significance tests are invalid due to a systematic underestimation of the alpha error. In many previous publications, Timm has pointed out that in every psi experiment the superordinated null hypothesis, that in the whole experiment no psi effect has occurred and all partial results are caused by chance, must be rejected. Consequently a global significance test must be successful before the partial results can be tested separately with the usual test. More generally, Timm has proposed a hierarchical test procedure, according to which a partial result may be declared significant only when, besides itself, all superordinated results are significant.

In order to increase the power of such global significance tests, the variability-related aggregation proves to be amazingly successful. The principle of this technique is to transform the original z-scores on any experimental level (e.g. runs, Ss) into scores with a skewed chi²distribution (df=1). These can be summed up and evaluated as simply as the original z-scores. By means of this transformation the extreme scores get a relatively larger weight so that one can speak of a weighted summation (WS). Several modifications of the WS are possible depending on, whether an one- or a bi-directional variability is assumed. Also the traditional calculation of the so-called run score variance is one of these methods.In 1997 Timm extended the WS to the hierarchical weighted summation (HWS), in which - in a cumulative manner - the results of WSs on lower experimental levels (e.g. of runs) undergo a new WS on a higher level (e.g. of Ss), until a single overall result has been reached. This technique has additional statistical advantages. In 2000 the authors applied it to a series of S REG-PK experiments: The overall result increased from p= .34 to p=.013.