From: rcd@raven.eklektix.com (Dick Dunn)
Subject: Re: New Employee Test
Summary: you want test-takers, or programmers?
Organization: eklektix - Boulder, Colorado
Date: Sun, 22 Nov 1992 07:05:45 GMT

markmc@halcyon.com (Mark McWiggins) writes:
>A few years back when I was helping interview candidates at the small
>company where I work, I came up with a real-worldish test: a C program
>that compiles but doesn't work correctly. We also included an element
>of time pressure, an instruction that "the customer doesn't have a modem
>and the Federal Express pickup is in 45 minutes."

That's a great test, but the rest of your response indicates that you
didn't recognize the correct answer.  The simplest correct response is to
stand up, politely apologize for having mistaken the job opening as one for
a software person, and walk out.  A complete answer to the question would
point out the following problems with the test:
  - Since job interviews generally try to show a company's good side, this
    suggests that desperate last-minute hackery is a way of doing business
    and somehow important to your company, rather than the once-in-several-
    blue-moons event that it should be.
  - More than that, you're selecting for ability to work under extreme
    pressure (the job interview process imposes enough stress by itself),
    not for ability to produce good code under normal conditions.
  - If a program needs a last-minute fix, and that fix can be found in a
    short amount of time by someone who's never seen the code, it means
    you've got trashy code in critical spots.
  - There must be management and/or customer-relations problems, because
    the stage has been set for a lose-lose situation:  If the problem is
    really of such earth-shattering importance and the account is so
    critical, making a last-minute fix without adequate time for checking
    and testing the fix is foolhardy at best, suicidal at worst.  (You may
    *think* there's just one error...but maybe it masks another, more
    critical one.)  If the problem isn't so earth-shattering, the situation
    should be handled to deal with the customer to buy enough time to "do
    it right".
  - If you buy the idea that the fix and the timing are critical, then the
    situation where the fix has to be made by someone who's never seen the
    code is ludicrous; it says you've got some sort of resource problem,
    not having critical code covered by people who know it.
  - You're selecting for people who are good at debugging and making last-
    minute fixes.  People get that skill in one of two ways: by being
    generally good at working with code; or by being unable to code, test,
    or both, thereby getting a lot of practice at debugging and last-
    minute hacks.
  - There's a fair element of luck in finding a bug quickly.  On average,
    good debuggers find them faster than bad ones, but there's a wide
    variance on any individual bug.  Sometimes the problem leaps out at
    you; sometimes you look right past it a dozen times.  Testing someone's
    debugging abilities with a sample size of one is lousy science and bad
    statistics.
    
...
>The first time, we didn't tell the candidates that they'd be taking a
>test, which brought up a whole extra layer of dysfunctional test-taking
>behavior.

Assuming that this sentence wasn't a complete non-sequitur, what does it
have to do with selecting among the candidates for a job?

And what, please, is "dysfunctional test-taking behavior"?  Would it be,
perhaps, a candidate's inability to pretend that an irrelevant test under
unrealistic, stressful circumstances bears some relationship to the job
for which one is being interviewed?
-- 
Dick Dunn    rcd@raven.eklektix.com   -or-   raven!rcd    Boulder, Colorado
	...Simpler is better.