From: wsadjw@rw7.urc.tue.nl (Jan Willem Nienhuys)
Newsgroups: sci.skeptic
Subject: Ertel's Error
Date: 12 Oct 92 13:26:11 GMT
Reply-To: wsadjw@urc.tue.nl
Organization: Eindhoven University of Technology, The Netherlands



I'm sorry that this post is so long, but I have to include  almost all
of Ertel's remarks.  Moreover, the explanation of why something is wrong
takes up more space than the committing of the error itself. 
There goes my lunch break....

Ertel writes:
#
#My reply is short. 

only 5530 characters, compared to JWN's long exposition of 5300 characters :-)

#
#The main question to be kept in mind is this:
#
#            IS THERE A MARS EFFECT?
#

I prefer: is there convincing evidence in favor of the Mars effect?

#The Belgian skeptics (Comite Para) tried to refute Gauquelin's claim
#by collecting N=535 new athletes data (published in 1976).
#
#Apparently, they couldn't refute G's claim. Otherwise U.S.
#skepticts (Paul Kurtz et al.) would hardly have tried the
#laborious Zelen test consisting of collecting birth data of
#N=16,000  French ordinary people (control group) (1977).

No one contends that the Para test came out favorable for G.
The Zelen test was a test to examine one particular naturalistic
explanation for the test result.  It was a rather superfluous
exercise, but at the time Zelen and others were not convinced that
the Gauquelins had adequately corrected for demographic/astronomical
factors (but he had).

The disconcerting thing about the Para test for me is that the 535
athletes contained 203 athletes whose Mars sectors were known already
by Gauquelin. It is not clear how many causal links there are between
Gauquelin's knowledge of the Mars sectors of these 203 athletes and
the choice of proficiency levels by the comit\'e Para.  It seems clear
(again from Ertel's own researches) that Gauquelin knew very well that
Belgian soccer players with less than 20 international games would give
a poorer result than the case of 20 games as minimum level.

#
#The results of the Zelen test apparently did not shake
#Gauquelin's claim either, otherwise the same researchers would
#hardly have tried another test consisting of collecting birth
#data of N=408  U.S. athletes (1979/80).
#

#Again, the U.S. athletes test was apparently unconvincing.
#Otherwise the Dutch skeptics would hardly have dived into that
#matter again, they did it twice. 

I think I may speak for Dutch skeptics (D.S.).  Of course the U.S. test
is quite convincing - as far as any single test can be convincing.
The D.S. tried to find a naturalistic explanation for the findings
before the U.S. test.  Especially because the U.S. test was negative,
the problem remained: what was the explanation for the other results?

#                                 First their idea was: Birth
#excess of Gauquelin athletes with Mars in G-zones exists, but it
#can be explained by diurnal and/or seasonal association (published
#in 1991). On closer look (and computation) they abandoned this
#idea. Next they explained Mars birth excess for Gauquelin
#athletes by Gauquelin's selection bias (in print) - they had
#become aware of my detailed account of the Gauquelin bias effect
#in the Journal of Scientific Exploration (1988).

One skeptic (Koppeschaar) gave up also because F. Gauquelin would not
give him the data he asked for.  He had received from Ertel many data
just after he asked.  F.G. blamed Ertel for this, and refused to give
more data, unless Koppeschaar would sign all kinds of documents  declaring
his intent and purpose (and one might suspect that F.G. would be fully
prepared start legal battles when K. would say something she didn't like.)
#
#There is another skeptics group, the French CFEPP, which
#apparently remained unconvinced by the Belgian and the two
#American approaches. They started collecting new French athletes
#birth data in 1983 and gave a report (unpublished) about their
#procedure in 1990 adding an appendix providing birth data of
#1076 athletes  (Dr. Benski).  Results of an analysis of CFEPP's
#data have not been published until today. Therefore, Arno
#Mueller and myself did an analysis of CFEPP's data (we informed
#Dr. Benski in advance). We computed, first, the main indicator:
#G percentage, i.e., the percentage of French athletes born with
#Mars in key sectors), the result is 25.19%.

This is probably the sum over sectors 36, 1, 2, 3, 9, 10, 11, 12
in the 36-sector division.  I don't know what is exactly the 
expectation for these sectors, I guess something like 22.9%, taking
into account astro/demographic factors. So expected: 247 +/- 14,
and actually found 271.  
From the table below, it seems that 23.6% is a better estimate for
the "expected average", so that would mean 254 +/- 14, and hence
the actual 271 is 1.23 Standard Deviations away from the mean, which is
not too impressive.  The important question is again: how 
independent of Gauquelin's knowledge of the athletes's sectors
is this result?  I know for certain that Benski conferred extensively
with M. Gauquelin.  Were all the 1076 athletes "new", or are there
again a lot of old ones from previous researches mixed in?

#
#Then we shifted the birth dates by one year. An athlete born on,
#say, Jan 25 1938, at 3 a.m., was attributed the birth date and
#time Jan 25 1939, at 3 a.m. We calculated G% for shifted birth
#data. Then we shifted by two years and calculated G% again. The
#same procedure was repeated by stepwise yearly shifts up to 25 years.
#The same shifts were applied in the opposite direction (-1, -2
#.. -25 years).
#
#(Stepwise yearly shifts are applied here for the
#first time as an improved test for a planetary
#effect. The improvement consists of relating the experimental
#group of genuine individuals to control groups of dummy
#individuals whose "births" occurred under exactly the
#same diurnal and seasonal conditions as those present at
#the births of the experimental individuals.)
#
#What might Dr. Benski and Dr. Nienhuis (JWN) hypothesize here?

Nienhuys if you please.

#I guess they might expect both that the genuine
#G% value of 25.19% does not deviate significantly from the
#distribution of 50 G% values obtained from the dummy controls.
#
#The result, however, does not confirm this hypothesis. I am
#appending a table showing 51 G% values in descending rank order.
#The value on top is G% obtained from the genuine birth data. The
#error probability of finding the genuine value on top is p=.01
#which is generally regarded as very significant (allowing for a
#one-tailed test which is here called for).
#

#Now I would like to ask JWN how confident he is, on inspecting
#these results, that the Mars effect does not exist.

This would be all a lot more convincing when  (1) I knew more 
about the degree to which the criteria for inclusion and exclusion
in the 1076 athletes has been independent of Gauquelin's 
knowledge of their birth times/Mars sectors, (this I cannot answer,
only Benski can,.... possibly)  and (2) this were not so much 
post-hoc. We cannot know the number of analogous tests that 
there is nothing special about these results.  If I *assume* that
he would have given up only after trying 20 different tests, then
finding a single test at the p=0.01 level is not that striking.
I definitely got the impression this summer in M\"unchen, that he
had tried shifting by multiples of half an hour.

It should not be necessary that scientists have to speculate about
what other scientists have been playing around with the data when
nobody else was looking.  This gives rise to all kinds of unpleasant
insinuations. THAT is the reason why I think post-hoc analysis is
distasteful.

There is also a statistical error in the above argumentation of
Ertel. He apparently assumes that computing a one-year shift
will result in a sample that can be considered a random sample.
But different shifts are not independent. To understand that I reason 
as follows. Shifting the time back by an integral number of years will 
give you for the same time almost exactly the same configuration
of fixed stars at birth. The sky with fixed stars (and hence the 
ecliptica) will have shifted by at most a degree (because of leap
days). A degree is little, compared to the width of 40 degrees of
the Ertel G-zones (36+1+2+3 and 9+10+11+12).  The fifty years
represent therefore more or less randomly distributed positions of
Mars on that same ecliptic. On average, these positions will be
about 7 degrees apart. But positions 7 degrees apart will give 
highly correlated answers. Only when you shift by 40 degrees you get
something approaching a mimick of an independent sample.
(I've pointed out something similar to Ertel related to his half hour
time shifts.) Moreover, a shift of 100 degrees will bring an overlap of 
zone 36+1+2+3 with 9+10+11+12, so again a correlation.
In other words, what looks like 50 independent results only represents 
a much smaller number of independent results. Say about 8, then
the Benski result again has a one-sided p-value of 0.12.  Not
significant, especially not because of the unknown bias introduced
in the manner I described.  

At a previous occasion (see latest
issue of GWUP's Skeptiker) Ertel has also found a "result"  by a 
combination of a complicated statistical detour, combined with a
rather elementary error.  In that case too, if he had avoided the error,
he would have found about the same result as the simple argument without
the detour. In that case the detour consisted of a complicated and
artificial way of computing an average, here it is a tricky way to
artificially inflate the number of independent samples.

Even for somebody who could not think of the above argument him/herself,
the data that Ertel presents below (his own data) should have warned 
him that they cannot represent independent draws from random variable:
the distribution of percentages is nearly uniform (between 21, 22,
23, 24 and 25 there are respectively 10, 13, 13, 14 year shifts).
Moreover, the fact that in the interval from 21 to 25 there are 3 values
that are repeated 4 or more times with an accuracy of <=0.01 should
also have rung a warning bell. The chance that of 50 random numbers
in that interval there should be 4 that round up to two decimals to
the same value is roughly 0.003, and for 5 it's a tiny fraction of that.
That there should be 3 such values is very improbable (10^-9) IF these 
values represented independent draws from some distribution.


#
#Second - assuming he is not yet confident enough about
#the nonexistence of a Mars effect - how many additional yearly
#shifts he wants us to calculate in order to improve his
#confidence that the Mars effect actually does not exist. Or
#else, what he would suggest should be done now to put his
#conclusion and that of other critics of the Mars effect on firmer
#ground.

[ironic remark.  What about just doing all shifts again, ten times.
This will give 500 shifts, and a p-value of 0.002 ! There is hardly
any difference between  adding in the results of 450 more shifts,
or just repeating the same 50 shifts over and over again.]

I don't want any more post-hoc calculations.   I want data that are
collected (in this case birth data of eminent sportsmen and -women)
in such a way that it can be PROVED that the decision to include or
exclude CANNOT be related to knowledge of that person's Mars sector.

This means: any athlete whose birth time has ever been known to
Gauquelin should not be in the sample, because that birth time may
have contributed to setting a criterion for inclusion or exclusion.

The only test in which this condition was (almost) fulfilled was
the U.S. test.  Even that test was not perfect, because the decision
to go on collecting data was based on knowledge of the data of the
first 128 athletes.  And the CSICOP probably has come to regret that
error.

#
#Suitbert Ertel
#
#---------- APPENDIX ----------------------------------------------
#
#    Results of testing
#    for a Mars effect
#    using the stepwise yearly
#    shift procedure
#
#      Data: CFEPP
#French athletes (N=1,076)
#-------------------------
#    rank      shift
#     by       by
#    size      years  G%
#-------------------------
#      1      0     25.19   genuine
#-------------------------
#      2    -10     25.00   dummyes
#      3    -25     24.54   dummy
#      4    -14     24.54    ...
#      5     -6     24.54
#      6      6     24.54
#      7      8     24.54
#      8     17     24.44
#      9    -12     24.35
#     10    -11     24.35
#     11     21     24.35
#     12     22     24.26
#     13     11     24.07
#     14     20     24.07
#     15     -9     24.07
#     16     -8     23.98
#     17    -13     23.88
#     18      2     23.88
#     19      4     23.88
#     20    -20     23.88
#     21     -3     23.79
#     22      3     23.79
#     23     18     23.70
#     24      5     23.70
#     25     23     23.70
#     26    -23     23.61
#     27    -21     23.42
#     28     24     23.23
#     29     25     22.86
#     30     -4     22.86
#     31    -18     22.77
#     32     14     22.68
#     33     10     22.58
#     34     -7     22.40
#     35    -17     22.30
#     36    -22     22.30
#     37    -19     22.30
#     38    -24     22.30
#     39     15     22.21
#     40     12     22.12
#     41     -5     22.03
#     42      7     21.93
#     43      1     21.93
#     44     13     21.84
#     45    -15     21.75
#     46     19     21.56
#     47     16     21.47
#     48      9     21.47
#     49     -2     21.38
#     50    -16     21.28
#     51     -1     21.10
#------------------------------------------
#
#


I hope I have not been boring my audience with these technical discussions,
but it's better that this dubious type of post-hoc statistics gets nipped 
in the bud. 


J.W. Nienhuys,
Research Group Discrete Mathematics
Dept. of Mathematics and Computing Science
Eindhoven University of Technology
P.O. BOX 513, 5600 MB Eindhoven
The Netherlands

e-mail: wsadjw@urc.tue.nl

Xref: sparky sci.skeptic:17599 alt.sources:2158