From: skh@draper.com (Kjeld Hvatum)
Newsgroups: rec.org.mensa
Subject: Re: Reply to previous comments on Mega Test
Date: 28 Sep 92 05:31:51 GMT
Organization: Draper Laboratory

>From: hollombe@polymath.tti.com (The Polymath)
>Subject: Re: Reply to previous comments on Mega Test
>Message-ID: <1992Sep25.011216.18898@ttinews.tti.com>
>
>}Not sure what your point is, really.  The test says you *should*
>}look things up, if that's what you mean by "look for them [answers]".
>}I'm curious, which non-verbal problems do you think are easier to
>}look up than to simply do?
>
>Any of the ones that appear in rec.puzzles, for a start.
 
The maintainer of the rec.puzzles FAQ has informed me that only a couple of
Mega Test answers have ever been posted to rec.puzzles.  In the mid-range of
the Mega Test, each problem is worth only about 1 IQ point, so the answers you
could obtain by "cheating" may not amount to much more than the standard error
on the test, if even that. 
 
>To anyone willing to cheat, the
>answers are available to cheat with.
 
Can you support your generalization?  Name the specific problems you
think are available by "lookup", besides the single example which
appears to have prompted your observation.  I can think of perhaps just
one or two others.  Personally, I'd admire anyone who could look up
answers to many of the Mega Test problems - I found it easier to just do
them, and I'm not sure where I'd find answers to many of them.
 
>The SAT is not an intelligence test, nor does ETS claim it to be. (In
>fact, they've gone to considerable trouble to distance themselves from
>intelligence testing).
 
The distinction is nowhere near as clear as you imply.  Many IQ tests
have a heavy verbal loading, and some are even entirely verbal.  The
vocabulary section of the Wechsler is regarded as one of the best single
indicators on that test.  The Princeton Review (though not a scholarly
organization, they're no dummies at picking apart the SAT) states
bluntly on page vi of their 1992 edition of "Cracking the ACT", "The
SAT, on the other hand, is an IQ test."
 
Public relations has played no small part in the ETS's considerable
trouble to distance themselves from the term, IQ.  The "College Board
Technical Handbook" (1984) doesn't even mention IQ or intelligence (that
I've found - certainly not in their lengthy index), but a closer look,
as on page 138, reveals such statements as, "Hunt et al. concluded that
high verbal persons manipulate information in short-term memory more
rapidly than persons with low verbal ability."  Sounds a bit like some
experts' description of intelligence.
 
>The fact that it correlates positively with intelligence test scores at
>its high end is a statistical artifact which has been discussed here
>before. 
 
What do you mean by "high end"?  Above 1000, 1500, or what?
In the range 1000-1600, for example, you'd be on thin ice claiming
the correlation is merely an artifact.
 
>Accepting only one answer is
>appropriate _for their purposes_, but not for general intelligence
>testing.
 
Oh, so many of the group-administered, paper-and-pencil, IQ tests
are not appropriate?  Perhaps you simply mean to say "not as
good"?
 
>I'm not dogmatic about testing procedure.  I'm just saying that revealing
>the answer to a Mega Test question in this group won't compromise the test
>any more than it already is.
 
Only a few questions have been "compromised", and most of the Mega Test
takers I've known still don't know about the Usenet.  I completed the
test before I'd ever heard of rec.puzzles.  The resourcefulness you
assume could also obtain answers to Wechsler test items, in spite of its
"secureness", though the methods might not be entirely honest.

On the other hand, you're right to some degree - the answers are slowly
leaking out.  But I'd like to see the life of the test extended.  It
has certainly produced a lot of interesting data. 
 
>started taking it, I offered my _opinion_ and _impression_ in response.
>Others have disagreed with that opinion.  Well and good.  I make no claim
>to absolute knowledge.
 
I thought you might enjoy hearing how my opinions differed.  I've
enjoyed your rebuttals.
 
>...because I don't know whether they worked the answers out
>themselves or copied them from someone else or hired a research staff to
>figure it out for them.
 
I know what you're saying, but you can't have that assurance with the
SAT either, or many of the tests you might consider legitimate.  There
have been many cases (though a small percentage, hopefully) of high
SAT scorers taking the test for others.  However, since I know the items
on the Mega Test, and since the items on many SATs are secure, and
change from test to test, I could more quickly determine if a candidate
had cheated on the Mega test by simply asking a question or two which
would required having had the proper insight on a Mega Test problem.
 
>Why do you think Mensa requires results from _proctored_ exams?
 
Because their by-law requires it, according to their supervisory
psychologist, and that was probably put into place before the
wide-spread use high-end tests like the LAIT, Mega, Titan, etc.  And, by
the way, at Mensa's cutoff level, I agree with their policy.  It's too
close to the bottom of the Mega test, requiring relatively few correct
answers, which might be too easily obtained.
 
Incidentally, if you quote Mensa as an authority, why do they
accept the SAT while they've stopped accepting the ACT?  As
the Princeton Review, they too have come to the conclusion that
the new ACT is not sufficiently similar to an intelligence test,
while the SAT *still* is.
 
>IMHO, the Mega Test is what it began as:  A self administered test in a
>popular news stand magazine.  Granted, it's more difficult than most
>and you have to send it in to get the "official" score.  Put that
>score on your resume and you'll be laughed at.
 
And, for the same reason, I wouldn't put a Wechsler IQ score on my resume,
at least not for the jobs that would interest me. 
 
Kjeld Hvatum <skh@draper.com>