From: tloughin@IASTATE.EDU (Thomas M Loughin)
Subject: Re: Plexiglass Cracks
Reply-To: tloughin@IASTATE.EDU (Thomas M Loughin)
Organization: Iowa State University
Date: Fri, 22 Jan 1993 15:03:47 GMT

Dear r.s.b.-ers and Roy,

Below is a copy of an e-mail message I tried to send to the original poster 
in this thread, Roy Cabaniss.  Unfortunately, Roy, it keeps bouncing.  But,
since you suggested that I post the whole thing anyway, I will.

In rereading my response to Roy's initial post, I realize I was a little 
harsh.  I apologise to him and to those who wish this to be an information 
exchange rather than a flame war.  However, I continue to assert that his 
analysis has done nothing to provide evidence for or against the 
"plexiglass principle."  It appears that he realizes this as well; I'm 
glad.  

Warning:  what's below is, um, a tad wordy.  It was, as I said, not 
originally tailored for posting.  Sorry...



To: "Roy F. Cabaniss"  <cabanrf@wkuvx1.bitnet>    
Subject: Re: Plexiglass Cracks
<00966F27.F60F3D80.4
525@WKUVX1.BITNET>
--------
Date: Thu, 21 Jan 93 20:34:27 CST

>>Get out the earplugs, folks, she's about to blow...
>>
>>In article <1993Jan19.170007.5082@wkuvx1.bitnet>, cabanrf@wkuvx1.bitnet
writes
:
>>> While scanning the net, I started thinking about the Plexiglass
>>> Principle, as promulgated by Bill James.  I asked if anyone had
>>> done more work with this and never got a response so I designed
>>> the little study below to provide a test of this principle.
>>>
>>> In his 1982 Abstract James says "all things in baseball have a
>>> powerful tendency to return to the form they previously
>>> held.......As applied to teams note how consistently an up is
>>> followed by a down and vice versa:"  What Bill appears to be
>>> implying is that at time T, the performance at time T minus 2 is
>>> a better predictor of performance than other years.
>>>
>>No one, not even someone misinterpreting James' intimations, should
>>sensibly think that is true.  What he is referring to, albeit perhaps
>>obtusely, is the simple statistical principle of "regression toward the
>>mean."  This is not in any way to be confused with the notions of
>>improvement or decline.
>
>Woah!!!  Why do you think I quoted him?  What he said was what I was looking
>at.  Not the various interpretations that have come after. ;-)  But ok I don't
>really consider a regression toward the mean to be that simple a concept.

Perhaps "simple" is not to be confused with "oft-discussed...

The term "regression" was coined by Sir Francis Galton in 1885 to describe
precisely this phenomenon.  The term has since mushroomed into reference of
any procedure that relates some explanatory variables with some other variable.

>>
>>Regression toward the mean suggests that observations following one
>>which has a large deviation from the mean (average) tend to be closer
>>to the mean that their predecessor.  Sons of 6'8" fathers tend _on
>>average_ to be shorter than their fathers, just as those of 5'1"
>>fathers tend to be taller.  This is not to say that _every_ son of a
>>tall father will be shorter, just that an even-money wager is
>>well-placed on "shorter."
>>
>Stop here please ;-)  What is large?  1 Std dev?  2 ?  This here is just what
I
>don't like.  the word suggests (to me ;-)  ) means that a probability
statement
>is coming shortly thereafter.  But we NEVER see them.  I've only got 5 years
of
>the Abstracts so I don't have all he wrote on the subject.  But the only place
>where I saw ANY numbers attached to the PP was in the 83? issue.  Four years
of
>data and he doesn't even say that it was tested.  It just "looks like this".
>
In Galton's study (which is essentially the example I gave above), ANYTHING
above average tended, on average (key words), to result in something a bit 
smaller, and conversely for ANYTHING below average.  Picture a 45-degree 
line and another at something less than 45, intersecting at their middles.  
The 45-er is "expected height" if children are the same size as their 
parents; the other represents reality--children's height "regresses" back toward
the mean.

The point of the word "large" is for guys like us who would go trying to
debunk this theory and not find anything because the difference in "small"
changes is too small to observe, given the roughness of the data.

>>In baseball, this manifests itself as sudden changes in the observed
>>level of a team or player.  When a team goes from last to first
>>_without having made substantive changes in that direction_, they are a
>>good bet to drop back down to the previously-established level of
>>performance, assuming that no further changes are made in the next
>>year.  Nobody with a brain and a calculator expects Brady Anderson to
>>repeat his numbers of last year.  He just hasn't changed himself _that_
>>much.  However, that doesn't necessarily imply that he hasn't
>>established some new level of performance for himself.  It just isn't
>>likely to be as radically different from his career numbers as last
>>year was.
>>
>So you are saying then that a team will return to their own mean.

Essentially.  What makes it so darn slippery is that each team's mean keeps 
changing, since teams don't remain constant.  It's a wrench in the works.
:-(
Galton did not have that problem.  Evolution is a slow thing...

>Good, thats
>a nice starting place.  In the 14 year period that I was looking at here 9
>teams had an average won/loss record that was significantly different than 500
>(BOS, CLE, YANKS, SEA, KC, MIN, MON, LA, ATL).  tO TAKE THIS ARGUEMENT FURTHER
>THEN, using your example of the 6'8" father are you saying then that the
father
>is an anolomy and that the son is a "normal" member of the populaition?

Close.  Not quite, but close.  And here the example loses its potency.

Think about the genetic makeup of a 6-8 man.  There's some pretty tall genes
in him (and some pretty tall jeans _on_ him...), so his "true" genetic height
is something big.  However, what we have observed is that "true" height, plus
or minus some "error" or "noise," if you will.  The key point is that it's
less likely, given knowledge of the human race, that he was "truly" a 6-11
man with a -3 random error than it is that he was "truly" a 6-5 man with a +3
random error.  Acceptable?  (it only stands to reason:  there are more 6-5-ers
than 6-11-ers, and this random stuff has mean zero)

Now, that being said, the distribution of errors for a 6-8 person, by this
logic, consists of lots of plusses and somewhat fewer minuses.  Another way of
saying this is that most of these people got to be 6-8 by random chance of
adding some positive error to their true height.

Now, what about their kids?  A new draw from the error distribution will have
a _mean_ of zero.  Each individual draw may vary, but _on the average_, the
next generation will better represent the "true height" of the fathers.  In
other words, they will tend to be somewhat shorter.

Extend this to a team that has undergone a drastic change in winning
percentage.  _In absence of any substantive change_ (key words), the odds are
that they got there by having a large positive error added to their "true
ability"  (i.e. they got lucky).  Hence, one would expect, on the average,
that the next year's performance will more resemble the "true" ability than it
will the big jump.  Now, how good a measure of that "true" ability is the year
before all of this happened?  That's open to debate.  Perhaps the mean of the
preceding two years would be a better measure of the (assumed _unchanging_)
mean.

In baseball, unfortunately for statistics, but fortunately for Seattle fans,
things change.  It's difficult to measure that change, so it's difficult to
measure the amount to which large changes represent substance or error.

>>> Taking the 1978-1990 seasons I constructed a dataset where each
>>> year had the last 5 years won/loss percentages.
>>
>>Excellent! Attack the problem with numbers.  Set up a study to prove
>>(or disprove) a hypothesis.
>>
>>> Using a multiple regression with performance at time T as the
>>> dependent variable and the teams performance at times T minus one
>>> through five as the independent variables gave the following
>>> results:
>>>
>>Well, it was a nice idea, anyway.  This isn't going to help show much
>>of anything that we didn't already know, and it certainly isn't likely
>>to turn up anything contrary to the regression toward the mean
>>principle.
>>
>>Your individual observations are not going to exhibit anything near the
>>independence required for this type of analysis.  To wit:  in one
>>observation you use Boston's 1990 results as Y, believing that 1989,
>>1988, and so forth might be good predictors.  In the next observation,
>>you use that same 1989 result as Y, and in the next, 1988.  In order to
>>get ANY kind of reliable analysis results, you'll need INDEPENDENCE
>>among your Y-values.  You just don't have it here.  Instead, you're
>>predicting essentially the same things over and over again.
>>
>Au contraire!  And what if I did this using factor scores?  Hmmmmm?
>
No, my friend, I do not refer to your five explanatory variables when I bemoan
the lack of independence.  I refer to the "dependent" variables, the ones
you're trying to predict.  What you would really need for this is some sort of
time series method, which, unfortunately, is not an area in which I'm adept.
I'd expect that some sort of ARMA model (Auto-Regressive Moving Average) might 
fit well.  

>Actually,
>to a degree I concur.  ;-)  Surprise, surprise!!  I expected
multicollineariaty
>out the wazoo.  What's interesting is that I didn't get it.  Notice that only
>t-1 and t-2 are significant.  Why?  All 5 past years should have been in at
the
>finish.
>
Well, you've got enough of it.  Correlations of around .4, as I recall.  Not
severe (I, too, would have expected more), but it's still a potential problem.
Not nearly as bad as the lack of independence, though.  That's enough to
invalidate a regression analysis, right there.  Massive biases in the
parameter estimates and their standard errors often result when the
independence assumption is badly violated.  I'm sorry, but them's facts.

>
>>> First, the best predictor, among these variables is THE PREVIOUS
>>> YEARS PERFORMANCE!! By itself it explains about 12% of the
>>> variance in won/loss percentage.  If you add two years ago, you
>>> explain another 2% of the variance.
>>>
>>What this means is that there are more teams that show small changes
>>in performance than there are those that change drastically from year
>>to year.  I'm not a bit surprised; it does nothing to refute James'
>>argument.
>>
>I don't this this follows from what I was looking at.  In fact, I'm sure it
>doesn't.  If the analysis was done on a team by team basisi, you'd be right
>But it wasn't.
>
Well, you're right in saying that the results of your analysis do not imply
my contention.  That's because my contention is not worded properly.  I claim
that the results you obtained were themselves the result of the structure of
the data that I describe.  In other words, there were, without a doubt, more
small changes than there were drastic changes, and therefore you should expect
that bulk of the data to indicate a greater importance of 1-year trends than
any other trends.

I stand adjusted.

>
>>> Third, neither one is any great shakes as a predictor of
>>> performance (r-square of .14)
>>>
>>Makes ya wonder what _would_ work, don't it?
>
>Yes, but that wasn't what I was investigating.  One plrblem at a time PLEASE!!
>
>;-) ;-)
>
I'm a Scientist (tm).  I want to know EVERYTHING!

>>
>>> To conclude:  We have statistical evidence that the plexiglass
>>> principle does not hold true for teams (which is what the
>>> original analysis was based upon.)  We have anectdotal evidence
>>> that the plexiglass principle does not hold true.  I think the
>>> plexiglass principle needs a HARD look.
>>  
>>>
>>That's true.
>>
>>What I'd like to see is the following:
>>
>>Go through the books and find teams whose winning percentages took
>>"big" jumps (up or down) from one year to the next.  Then look at their
>
>I have a real problem here.  You're taking lovely ratio type data and imposing
>catagories on it.  Why??? ;-)

Because, as I've outlined above, that's the definition of the plexiglass
principle.  Drastic changes are most often followed by a rebound effect.
Let's go out and find some drastic changes and measure the rebound effect.

Think of it mathematically:  if y2-y1 > c, then on average (those bloody words
again) y3 < y2.  Similarly , if y2-y1 < -c, then an average, y3 > y2.  That's
presicely what I propopse measuring below.

>Multivariate techniques are a much better way of
>looking at this kind of stuff.  Rather than imposing my opinion as to what a
>big jump was, lets look at the actual movement.
>
Again, read on below...(we will eventually let the data decide how big is big)

>Extreem movements are, by
>definition, deviates.  A principle (I will reluctantly remind you) requires
>that the evidence corroborating it be overwhelming (Hunt 91)  ;-)
>
IS IT OVERWHELMING IF I SHOUT IT?  :-}
>
>Rather than say that the data has to jump through a hoop before I analyze it,
>why not just analyze it.
>
No hoops.  A clearly defined analysis plan, aimed at a specific analysis.
If we find that there's no relationship between the definition of "big" and
the corresponding rebound, then we have evidence against the conjecture.  If,
on the other hand, there IS a relationship, then we have evidence in its
favor.

>>next year's results.  THIS is>_really_ about.  Define "big" in some suitable
w
a
>y that you aren't
>>including practically everyone in the analysis.  Maybe use several
>>definitions, so we can find out how much effect there is (if any) for
>>different-sized jumps.
>>
>>Take differences (Y2-Y1) and (Y3-Y2), where the Y's are the yearly
>>winning percentages.  Let's see what that tells us.  Anyone care to try
>>it?
>
>If you're going to do it, then I like this concept here.  But it seems to me,
>as if this were the infamous "SEARCH FOR SIGNIFICANCE".  ;-)
>
If there weren't that smiley there, I'd be insulted. :-}

Again, it's no fishing trip.  It's a clearly-defined analysis plan.  It only
turns into a fishing trip if the interpretation gets shady...
>
>>
>Yep, here tis.
>
>First things first.  I just did an  analysis using "exploding" and
>"imploding" as two created catagories.  (Nominal type.)  Exploding was a
>movement away from the mean, imploding was a movement towards.  Looking at the
>gaps (as suggested above, only I did it before I read this) if I've got the PP
>right, there should be a relationship.  There was not.  (Chi square, Prob =
>.78)
>
>I also created a variable ((t-1)-(t-2)) and called it diff.  Used diff in a
>regression on performance at time T.  If it's just diff, no significant
>relationship (p^=.76).  If I add (t-1) to the regression, we are now
>significant, but the beta for diff is 1/3 that of (t-1) and negative.  Other
>things scream multicollinearity at me.  The only real way to have at this is
>probably going to be factor scores.
>
No, listen closely:

Take a single team.  Take the differences in percentages at each year.

Now let's pick some cutoffs.  What's the biggest difference, about +/- 1.5?  
Let's go something like

-1.5 - -1.0
-1.0 - -0.6
-0.6 - -0.2
-0.2 -  0.2
 0.2 -  0.6
 0.6 -  1.0
 1.0 -  1.5

Change these a little as you see fit to allow adequate sample sizes in each
group, but try to keep it symmetric about 0.  

Now, each time one year's difference falls into a category, enter the next
year's difference as a data point in that category.  Then skip to the next 
difference and record it's following change, and so on.

EX:  a string of .531, .606, .588, .550, .600
  gives differences of .75, -.18, -.38, .50
So in the 0.6 - 1.0 group enter a -.18.
Then go to the -.06 - -.02 group and enter a .50.  

Do this with all your teams, all your years.

Take means in each category.

Plot these against their midpoints.  That'll be pretty.

Then you can fit a regression line of means vs. category midpoints, or do
one-sample t-tests in each category to test for differences from mean zero.

While you're at it, do a "Galton" plot:  Plot winning percentage at year I+1
against percentage at year I.  That may tell you what we want to know right
there.

I repeat...

>>C'mon, it'll be fun!
>>Let me make it clear:  I'm all in favor of this effort.  The
>>implementation and interpretation just missed the mark.  The answer's
>>out there somewhere.  We'll find it eventually.
>>
>
>Let's see.  If I remember right, what I said was that the implication from
Bill
>James' work (which I quoted!!), was that the record two years ago was a better
>predictor than last years.  To me, given what he wrote, that is still an
>implication which is present in his writing.

As I suggested in the post, I'm sure this interpretation is more due to poor
expression of ideas on James' part than to intention of what he was trying to
say. Early on, he talked of a "Johnson Effect," which is essentially this
principle we've been discussing.  If his words lend themselves to loose
interpretation, then I guess that's his fault.

>I concur with your interpretation
>of the PP. ;-)  Your interpretation is also testable.  I tested whether the
>performance two years ago was better.  The answer I got was nope. ;-)  In
other
>words, what I tested for, I found evidence of.  For the other stuff, give me
>time, I'm new to this gig;-)  ;-) ;-).
>
Do the analyses I suggest, and tell me what you get.  I'll help you draw
conclusions for a post, if you like.

And quit winking so much.  You'll hurt yourself!

-Tom.