The Astrotest

A tough match for astrologers

by Rob Nanninga – Correlation, Journal of Research into Astrology, 15(2), 1996/97

Introduction

Astrology text books contain many testable statements. To take an example: it is said that people with the Sun in an Air-sign (Gemini, Libra and Aquarius) are more thinking types than those with the Sun in a Water-sign (Cancer, Scorpio, Pisces). Thus we might expect that among skeptics Air signs are more numerous than Water signs. To my knowledge this hypothesis has not yet been tested. Nevertheless, it seems likely that it can be disproved because many similar tests have failed to show any relationship between given Sun signs and certain personality traits or vocation (e.g. Bastedo, R.W. 1978; Culver & Ianna, 1988; Gauquelin, 1982, 1988; Hentschel, 1985; McGervey, 1977; Startup, M. 1984; Tyson, 1980, 1984; Van Rooij, 1993).

Astrologers strongly object to these kind of tests. They emphasize that one should always look at the whole chart. The Sun sign is only one factor that is influenced by numerous other astrological factors. The system as a whole does not equal the sum of its parts, because these parts are often in conflict with one another. The influence of the Sun-sign can easily be cancelled out by the position of the Moon or anything else. Therefore, one should never isolate one particular factor, but try to interpret the whole chart.

Apparently, this implies that a group of a thousand Scorpions do not have more in common then a random group with mixed Sun signs. If this is true, we may wonder why astrological textbooks continue to tell us that Scorpions are secretive, passionate, resentful and vindictive. We may also wonder how astrologers have ever found out the meaning of the different Sun signs.

Fortunately, we don’t have to argue about this issue because there are tests in which astrologers can use the whole chart. I am referring to the so-called matching test. One of the best examples was conducted by the Australian researcher Dr. Geoffrey Dean (1987). Using the Eysenck Personality Inventory, Dean selected 60 people with a very high introversion score and 60 people with a very high extraversion score. Next, he supplied 45 astrologers with the birth charts of these 120 subjects. By analysing the charts the astrologers tried to identify the extroverts from the introverts. The results were very disappointing. It was as if the astrologers had tossed coins to determine their choices. Their average success rate was only 50.2 percent.

As might be expected, astrologers do not like this type of test either. They argue that they need more information on the subjects so as to successfully accomplish their matching task. An extreme extraversion score on the EPI is apparently not enough. Many astrologers doubt the value of personality tests. How can we overcome this hurdle? The best way to go about this is to ask astrologers what kind of information they require. This strategy was used by John McGrew and Richard McFall (1990), two psychologists of Indiana University. The ‘Astrotest’ that I am about to discuss is similar to their approach.

Devising the Astrotest

In May 1994 the Algemeen Dagblad (General Daily), a Dutch national newspaper, published an article by Martin Boot, a former astrologer who now works as a psychotherapist. Boot argued that astrologers cannot predict. In response, the astrologer Rene Jelsma wrote a rebuttal that was published under the heading “Astrologer can really predict”.

I decided to offer an experimental solution to this difference of opinion by inviting astrologers to take part in the Astrotest of the Skepsis Foundation. My proposal was as follows: All participants will receive the birth data (date, time and place) of seven anonymous test subjects. They will also receive seven questionnaires filled out by these subjects. The questions will be devised by the participants. They have to match each birth chart with the corresponding questionnaire. To encourage participation, I offered 5000 guilders (2500 dollars) to those who were able to match all seven charts.

The response was beyond expectation. More than 70 people showed willingness to participate. I sent them a brief questionnaire to determine how much experience they had in astrology. I also asked them to supply me with a number of questions that I could put to my test subjects. Later on, more astrologers volunteered. Some of them replaced those who had not returned my questionnaire. It was decided, however, that the maximum number of astrologers should be 50, because we wanted to limit our risk of loosing the 5000 guilders. Even without any astrological knowledge, each participant would have a chance of 1 in 5040 to obtain the money.

It was not easy to find suitable test subjects. Initially, I collected people who had all been born in 1948. The president of the NGPA (Dutch Society of Practising Astrologers), however, pointed out that in those days the registered birthtimes were often rounded to the full or half hour. Therefore, I decided to use subjects that were born around 1958. After I had obtained their birth certificates, I used a computer program to calculate their charts. Then I noticed that four of my subjects had an ascendent that was near the cusp between two signs. If they were born five minutes later or earlier than the time that was registered in their birth certificate, they would have had a different ascendending sign. Because I could not be sure that the birth times were correct to the minute, I felt obliged to replaced these subjects with others with a less ambiguous ascendent.

On the average the participants sent me ten questions to be answered by the test subjects, leaving aside those that did not formulate any questions. I synthesized the questions into a list of 25 that covered several aspects of the subjects life: their education, vocation, hobbies, interests, main goals, personality, relationships, health, religion, etc. In addition, I asked them for the dates of some important events in their life, because many participants had shown an interest in specific dates. Finally, I added 24 multiple choice questions that I had taken from the Berkeley Personality Profile.

Eight experienced astrologers were asked for their opinion about the questionnaire. They had no major objections against it. At the suggestion of one of them I added three multiple choice questions covering the family background of the subjects. Finally, the questionnaires and the list of birth data were sent to a few skeptics who tried to find the matching pairs. Although one of them scored 3 hits, there was no reason to suspect that any of the pairs could be identified by using hidden clues.

Expectations vs results

In the beginning of December 1994 all data were sent to the 50 participants of the Astrotest. They had ten weeks to complete the test. I allowed them to formulate an additional question after studying the charts, but only one person did.

Eventually, 44 astrologers completed the test. Many of them had much experience. Half of the participants had read at least fifty books on astrology. Three quarters had taken a course in astrology and one quarter was training others. At least half of the participants had analyzed over a hundred astrological charts and one-third was frequently paid for its services. One quarter was member of the Dutch Society of Practising Astrologers (NGPA).

The astrologers were asked to indicate how many correct matches they would have expected. There were 36 participants who revealed their expectations. Half of them predicted that they had matched all subjects with the correct charts. Only six astrologers expected less than four hits.

In fact, the most successful astrologer achieved only three correct matches, whereas half of the participants (22) did not score a single hit. The average number of hits was 0.75. This is 0.25 below the mean change expectation (MCE), a deviation that is not significant. Moreover, there was no evidence that the most experienced astrologers did any better than beginners.

It is interesting to compare the entries of the participants with each other. Because they all had received the same information, one would expect many similar responses. Actually, the lack of agreement was striking. Each of the seven charts could be paired with seven questionnaires. Of these 49 possible combinations, none was selected more than twelve times. It was as if each astrologer had used a random generator to determine the correct matches. There were only two astrologers who had independently arrived at the same solution (p = 0.18). Two other entries were also identical, but in this case the participants had joined forces.

Unconvincing excuses

Afterwards, I sent all participants a short questionnaire to find out more about the impact of their negative results. Nine of the 22 respondents confessed that they were surprised by the lack of agreement between the participants. Four admitted that the possibilities of astrology were more limited then they had thought, and seven came to the conclusion that astrology only works in actual practice. Nevertheless, sixteen respondents still believed that science can prove astrology right.

I asked the astrologers which factors might be responsible for the disappointing results. Ten respondents found the horoscopes too much alike. They pointed out that Saturn, Uranus, Neptune and Pluto were nearly always located in the same astrological sign. Uranus, for instance, was always in Leo. This is not surprising because Uranus needs seven years to move from one sign to the next. The age differences between the test subjects were necessarily much smaller, because the questionnaires contained much information that could be used to estimate their age. Anyhow, the horoscopes showed numerous differences that were hard to overlook.

Ten respondents complained that the answers of the subjects were often similar. For instance, in reply to a multiple choice question about their job, all subjects claimed to be reliable workers. But again, there were many more differences than similarities. Moreover, even in those cases when most subjects expressed a similar opinion, there were usually dissenters. For example, although four subjects had a vivid imagination and were fond of aesthetic experiences, there was one subject who showed not the slightest appreciation for such experiences. Why didn’t his chart stand out from the others?

Nine respondents suspected that the questions were not always answered truthfully. Perhaps the subjects had a lack of self-knowledge or tried to present a flattering image of themselves. However, in spite of this potential bias, most questions concerned verifiable facts. There is no reason to assume that the subjects lied about their hobbies or the date of their wedding. Even if all facts were false (including the birth dates), this would not explain why the astrologers failed to demonstrate mutual agreement.

The majority of the respondents (13) asserted that the subjects had not supplied enough information. We cannot rule out the possibility that the chance results were due to a lack of relevant details. It should be remarked, however, that nearly all participants had asked for less information than they actually received. One might just as well argue that they could not see the wood for the trees.

Is there a way out?

If astrology really works, it should be possible to design a test that satisfies both scientists and astrologers. Unfortunately, astrologers never specify what kind of test would be acceptable to them. They argue that astrological research is very difficult because scientific methods are too crude, intrusive or mechanical. Apparently, the alleged astrological effects are so subtle and hard to detect that we may wonder how astrologers had ever been able to identify them.

In order to explain the lack of objective evidence, some leading Dutch astrologers (Van Assem, 1993) resorted to the paranormal. They believe that astrologers can only arrive at the correct interpretation of a chart by using their higher intuition or by tuning in with the cosmic order. This intuition can solely be used during authentic consultations. As soon as a scientist interferes by selecting clients or posing questions, it disappears. Presumably, scientists are not part of the cosmic order.

According to Geoffrey Dean and Arthur Mather (1994) scientists and astrologers are in conflict over whether astrology works because they tend to look at different things: scientists are concerned with accuracy whereas astrologers look for client satisfaction. Astrologers tend to believe that their clients are satisfied because astrology is accurate. This implies that clients would be less satisfied if the astrologer had inadvertently used an incorrect birth date. Astrology is in conflict with science as long as its claims are testable. An increasing number of astrologers manage to escape from scientific scrutiny by confining themselves to statements that can not be falsified. They assert that a horoscope can only provide information about our inner life, basic nature, true reality, hidden potential, deepest aspirations, unconscious fears and motives, forgotten trauma’s and possibilities for future development. Astrology, they say, gives insight into the meaning and quality of our subjective experiences by locating them within a cosmic frame of reference. It tells us why we are the way we are, it can clarify our problems and help us to find solutions – but it can not predict our behaviour. Astrological interpretations do not even necessarily fit with the introspection by the person involved, so nothing can prove them wrong.

The most “progressive” astrologers regard astrology as a counselling skill. To them the horoscope is only a therapeutic tool and not a source of reliable information.

References

Bastedo, R.W. (1978). An emperical test of popular astrology. Skeptical Inquirer, 3(1), 17-38.
Culver, R.B. & Ianna, P.A. (1984). Astrology: true or false? Buffalo: Prometheus.
Dean, G. (1987). Does astrology need to be true? Scientific Inquirer, 11(3), 257-273.
Dean, G. & Mather, A. (1994). Is the scientific approach relevant to astrology? Correlation, 13(1), 11-18.
Gauquelin, M. (1982). Zodiac and personality: an emperical study. Skeptical Inquirer, 6(3), 57-65.
Gauquelin, M. (1988). Written in the stars. Wellingborough: Aquarian Press, p. 252.
Hentschel, U. & Kiessling, M. (1985). Season of birth and personality: another instance of noncorrespondence. Journal of Social Psychology, 125(5), 577-585.
McGervey, J.D. (1977). A statistical test of sun-sign astrology. The Zetetic, 1(2), 49-54.
McGrew, J.H. & McFall, R.M. (1990). A scientific inquiry into the validity of astrology. Journal of Scientific Exploration, 4(1), 75-83.
Startup, M. (1984). Personality and planetary positions at birth: an attempted replication with ordinary people. Correlation, 4(2), 4-13.
Tyson, G.A. (1980). Occupation and astrology or season of birth: a myth? The Journal of Social Psychology, 110, 73-78. Tyson, G.A. (1984). An emperical test of the astrological theory of personality. Personality and Individual Differences, 5(2), 247-250.
Van Assem, L. (1993). The astrologer’s philosophy of life. Correlation, 12(1), 52-54.
Van Rooij, J.J.F. (1993). Jungian typology and astrology: an emperical test. Correlation, 12(1), 28-32.

Source: Correlation, Journal of Research into Astrology, Northern Winter 1996/97, 15(2), p. 14-20.