From: hes@unity.ncsu.edu (Henry E. Schaffer)
Newsgroups: rec.guns
Subject: Re: Handgun Ammunition Debate
Date: 27 Sep 92 21:07:36 GMT
Organization: North Carolina State University

In article <1992Sep26.194530.10787@henson.cc.wwu.edu> n9020351@henson.cc.wwu.edu (James Douglas Del-Vecchio) writes:
#nsc!jrr@decwrl.dec.com (Jerry Roe) writes:
#
##*in those cases where there's only one torso hit*, what percentage
##of them are stoppers?  In other words, if there's only one torso hit,
#
#No, what Marshall's stats measure can more accurately be summarized
#as: 
#
#*In those cases where only one torso hit is required*, how does everthing
#turn out?
# ...

  Jim makes an extremely good point which is worth considering further,
and in more generality.  In statistics when one is doing statistical
inference or test of hypotheses from data, it is very important to know
the characteristics of the data.  I'll ignore the times when one can
use all of the data and only discuss the relevant situation in which a
sample is used.

  If a sample is "random" then it will tend to follow the statistical
distribution of the "real" data, except for statistical sampling
fluctuations - and we can deal with these fluctuations through the 
usual statistical methods.  While there is a lot more which can be
said on this topic, I want to discuss the particularly nasty situation
in which we {think|hope|pretend} that we have a random sample and yet
the sample is non-random.

  One way in which this can happen is that the data has been "selected"
somehow.  This can happen inadvertantly by use of an incorrect sampling
method, or by dropping out data according to some criterion which
results in a bias, or ...  One can easily end up with a "biased sample",
i.e., one which points towards an incorrect conclusion.  One of the
classic examples of this is the infamous poll of voters in a
Presidential election (phooey, I can't remember the details -something
like the Literary Digest poll, sometimes during the Depression years)
in which the sampling was done by pulling names from telephone
directories and automobile registrations.  The sample strongly indicated
one candidate as the winner, but the other candidate won.  In
retrospect it was realized that ownership of a car or having a
telephone listing during the Depression was a selective factor in 
making the sample be biased towards wealthier people.

  Could there be selection (and a resulting bias in the sample data) 
resulting from the one-shot criterion?  If all shootings were one-shot
only then there could be no bias.  But with some shootings being
one-shot and some being more than one shot (which I'll call multiple-
shot) then some bias could enter if the choice of when to shoot more
than once was related in some way to the effectiveness.

  Let me give an example which popped into my mind when reading Jim's
comments.  Imagine a low-power round which can stop with one shot only
if it inflicts a CNS (central nervous system) injury and can't
otherwise.  Further imagine that it is often or mostly used by shooting
once, seeing if it stops, and then shooting further times if it hasn't
stopped.  Let us specify the percentage of times this first shot leads
to a stop (by CNS injury) as L%.  Note that this L% is the figure of
interest to us - it is the true one-shot stop % for this low-power
round.  How would this round show up in the "one shot stop %" type of 
calculation we're discussing.  It would show up as 100% (or close to 
it) regardless of the value of L%.  Even if L% = 1%, there would be
enough data selection (by the discarding of the multiple-shot
incidents, when these incidents preferentially contained the failure-
to-stop-with-one-shot cases) so that those incidents which we retained
(in this cases it would be the CNS injuries) were about 100% one-shot-
stops.

  OK - this is an extreme case - but a) do you agree that it would
give these *very* biased and misleading results? and, if so, b) can
something like this, or any other type of biased data selection be
entering into the one-shot-stop statistics?  I think that Jim is
implying a yes answer to question a) and answering yes to question b
in his following statement.

#i.e. the "97% stopper" figure for Fed 125 gr JHP .357 can only be arrived 
#at by discounting all the cases where the first, the second, and the third
#Fed 125 gr JHP .357 to go through the perp failed to make any impression
#on him.

  To go to the other extreme, imagine a high-powered round which
causes such a large effect that it often stops with one shot,
regardless of where the impact is.  Let us further imagine that
people using this round tend to shoot once and then hold fire
because even if one shot doesn't stop it causes enough effect that
non-firearms methods to stop are often sufficient.  So for this 
imaginary scenario there never is more than one shot.  Let the one-
shot-stop % for this round be called H%.  How large would H% be.  It
might be well below 100%, perhaps as low as 75% or even lower.  Could
something like this happen?  (Remember that it doesn't have to be
this extreme.)  If so, then it could bias the comparisons between
different rounds.

  In serious statistical analyses it is considered essential to
formulate a "model" of all the "statistical effects" which comprise
the data.  It is in doing this that many of the possibilities which
I've covered above can become evident, and one can then try to
account for them in formulating the analyses and tests of hypotheses.
(This is why it is recommended that one should start working with a
statistician *before* designing and experiment and/or collecting
data.)

  I haven't read any of the literature on this one-shot-stop topic,
but haven't noticed anyone mention anything about a model or avoiding
bias in data selection, etc.  Has it been overlooked in the postings,
or isn't it there?

--henry schaffer  n c state univ