From: tlilley@jarthur.claremont.edu (Thomas (Ted) Lilley)
Subject: benchmarks
Organization: Harvey Mudd College, Claremont, CA 91711
Date: Tue, 15 Dec 1992 02:28:46 GMT


[Brian says]
>I did not say that running the benchmark was FUD, I asked for a copy of
>the benchmark so I could try it on other hardware.


[Here's what he said originally]
>I also resent the fact you cross-posted to the programmers groups and
>c.o.o.misc.
>Your message was clearly a biased advocacy message, but you expect
>people that followup to only post to the advocacy group.
>This basically means that there are people (especially in c.o.o.misc)
>that have the wrong impression of the whole matter.
>
>First that "benchmark" and now this message... I'm a bit surprised...
            ^^^^^^^^^^^^
>it's not like you to spread FUD, Tim.

Sounded to me like you were illegitemizing the benchmark, and calling it
FUD.

I'll let everyone else judge for themselves.

>I also said that the 40% figure was incorrect, that my usage experience
>says otherwise.  

Actually, no you didn't.  At least not in the letter I responded to.
Besides, here is the quote from the benchmark:

(begin quote)
Exact numbers are not available to me yet, but on a stock IBM PS/2
Model 57 with 16 MB of RAM and standard VGA, the relative Winmarks
were approximately as follows:

Windows 3.1 on DOS 5.02					100%
OS/2 2.0x, FS Win-OS/2, Not "Optimized"			95%
Windows NT Beta						40%

(end quote)

Evidently, your subjective experience differs with the approximate numbers.
The answer is simple:  bear out your observations by supplying us with hard
fact.

>Win32 discontinued one form of Windows drawing mode, it
>is my guess that the benchmarks Tim ran used this mode, which is currently
>emulated very poorly, and that this wildly skewed the benchmark results.
>This should be evident from looking at breakout figures from the benchmark.
>There are perhaps other such minor problems.  The NT beta currently
>feels like running under Windows 3.0.

Well, considering the following quote from the benchmark, you can't accuse
Tim of skewing anything:

(begin quote)
All system defaults at install time were used.  (Exception: OS/2 2.0x
had PRIORITY_DISK_IO set to NO, FILES=100, and PRINTMONBUF=402,0,0.
These settings are disk and printing related and do not impact Winbench.
Also, DOS was loaded HIGH for the OS/2 2.0x DOS sessions, but every other
default was retained for the Win-OS/2 sessions.  The DOS HIGH setting
also should not impact Winbench.  DOS HIGH was the default on the other
two.)  Winbench was the only task running on each system when run.

Windows 3.1 on DOS 5.02 was installed with SMARTDRV using all default
install time settings.  Windows NT was installed with all default settings.
All used 1024x768 in 256 colors for their desktops.
(end quote)

If anything, MS skewed itself, if you get my meaning.

>As to "I can get it too"... how?  I don't even know the version he is
>using.  Nor do I know where to get that version.

Perhaps you should have read the benchmark, before you responded to it:

(begin quote)
The following is a summary of results of the PC Magazine Winbench
Version 2.51 benchmarking software when run under Windows 3.1 on
DOS 5.02, Windows 3.1 NT (October public beta, CD-ROM), and OS/2 2.0x
(November Professional Developer's Kit CD-ROM, the third public Win-OS/2
3.1 beta).
(end quote)

>I'm not overreacting, Tim appears to be on a campaign to slur NT.

Actually, I'd say you're on a campaign  to promote NT:
"
>C. Brian Sturgill                          NeXTStep 486 is for _YOU_!!!!
>University of Utah                    Microsoft needs competition, but I want
>Center for Software Science          to be one of the many running Windows NT!
>brian@cs.utah.edu; CIS: 70363,1373    :-) :-) :-)  NT SDK: $69; (800) 227-4679
"

>The very fact he is benchmarking a _first_ beta drop against a "beta" that
>is due out in a couple of weeks in my view qualifies as FUD, especially
>since he did not point out that fact.

(begin quote)
The following is a summary of results of the PC Magazine Winbench
Version 2.51 benchmarking software when run under Windows 3.1 on
DOS 5.02, Windows 3.1 NT (October public beta, CD-ROM), and OS/2 2.0x
(November Professional Developer's Kit CD-ROM, the third public Win-OS/2
3.1 beta).

Note that Microsoft does not recommend using NT currently for performance
benchmarking.  Also please note that these are a first set of
benchmark figures for one particular benchmark.  I hope to have numbers
on a mixed sample Excel macro and on a disk I/O test.  However, these
numbers may be instructive.
(end quote)

I think that speaks for itself, and I think you owe Tim an apology.

>  Running the benchmark itself
>is not FUD, but to not point out the relative statuses of the OSes being
>tested is. 

He did, as quoted above.

> Also, if what I believe holds true, that NT is very
>poor in a few minor cases but reasonable in most, I think posting just
>the overall number is FUD.

If NT is *SO* poor that it's results in a few areas negate it's superior
performance in most others, then it deserves to be relegated the scores
it got.  I can't understand why you would try to say overall results
are unimportant, or in some way irrelevant.  Moreover, I would say
you use the word FUD a little liberally.  You are being paranoid and
injurious.  

>The post he made about booting and NT and OS/2 was unjustified, and thus
>I believe constitutes FUD. Though perhaps Tim did not know that
>MS actively tells you how to get around the problem.
>  After all, anybody
>with the NT SDK is supposed to contact MS through CompuServe, which is where
>the information is.  I don't see IBM explaining how to dual boot NT and
>OS/2, now do I?

I suppose if they were the cause of the problem, they would be.  As it
is, it will probably be assimilated into the store of OS/2 tips, but
it is most certainly MS's responsibility to their customers to fix the
problem they caused.

>  Tim said to consider the behavior of the companies when
>choosing between the two OS's. MS helps you use OS/2 2.0, IBM doesn't help
>you use NT.  This however was not Tim's message, that message made it
>appear that MS had some evil plot to not work with OS/2.

I wouldn't say it's unjustified, although it is definitely a matter of opinion.
MS has a terrible record with concern to OS/2.  It is obvious they are trying
their utmost to combat it's use, and they do not shirk hardball tactics.
Whether the boot manager problem is an intentional attempt to conflict with
OS/2, or just a result of incompetent programming is a question that is up
in the air.  "Who knows what evil lurks in the heart's of men?"

I  leave you with a couple thoughts:

Needlessly and intentionally removing OS/2 functionality from compilers.
Dumping OS/2 support from all of their app lines.
Leaving OS/2 1.x developers out in the cold, after selling them a $2600
developers kit.
NT refusing to install with OS/2 boot manager present on disk.

All this with an operating system they were jointly responsible for creating,
and which they hailed as the future of the desktop.

I call Bill Gates and his et. al. assholes with no shred of remorse.

Ted
Newsgroups: comp.os.os2.advocacy