From: els@icf.hrb.com (Eric L. Schott)
Newsgroups: comp.os.vxworks
Subject: SUMMARY: RE: Evaluating VME CPUs
Date: 7 Oct 92 12:12:46 EST
Organization: HRB Systems, Inc.

Most of the responses I received to my post were requests to
reveal who was the vendor with the best performance in my tests. 
I am not at liberty to reveal the actual manufactures.  This is because:
  1.  The evaluation was done under a contract.
  2.  Products change.  What is currently the best board may
      not be best next year.
  3.  My best may not be your best.
  4.  I want manufactures to build better boards.  By telling how
      the failed, they can redesign their products.
However, when evaluating any single board computer, here are some
guidelines:
  1.  Select a good basic CPU chip.  Most current chips squeeze extra
      performance by use of cache.  This means the chip needs the bus 
      signals to drive/invalidate cache.
  2.  Make sure ALL peripheral chips allow the CPU to run with cache
      enabled.  This is what caused poor performance on most of the
      boards we evaluated.  Because the cache could not be invalidated,
      the boards had to run with cache disabled.  For a 68040, this
      reduces the dhrystone performance from 23K dhyrs/s to 10K dhyrs/s.
      If all peripheral interfaces do not invalidate cache for the CPU, 
      then cache for selective regions of memory must enabled/disabled.
      The "Guide to VxWorks/68040" contains a discussion of this.  
      Also refer to the target documentation.  
  3.  Make sure the board has an EXCELLENT VMEbus interface ASIC or
      logic.  Most board failed here because their performance peaked
      around 5 Megabytes/second.  We found only two boards which 
      supported D32 BLT and D64 BLT with reasonable rates.  One used
      a DMA hardware for BLT and the other used the 68040 block move 
      support for BLT.
Some of the responses are below.

> From: whorner@vlsi12.gsfc.nasa.gov (ward horner)

> By the way, it is interesting that board B had a higher Net benchmark
> in the combined test! (re: puzzling).  It is also surprising how low
> the board A Dhry benchmark in the combined test dropped.

You were correct with the net tests.  The board B standalone test
was to a SparcEngine.  The combined test was to a SparcStation 2.
I suspect board A's CPU performance suffered due to its poor net
interface: no DMA with net interace and/or inability to run with
cache enabled (TCP copies a lot of data from interface layer to 
layer).


> From: tcarey@p2tacs.Lanl.gov (Tom Carey)

> Just between you and me (I bet you've heard that 1000 times already!),
> what was Board B and what was Board A (manufacturer, model #, special
> features, ...)?  Also, is your "benchmark application" available for
> general consumption?

The software for the tests was developed on a computer not attached
to the Internet.  However the applications are straightforward
to develop.

> From: christ@tekig1.pen.tek.com (Chris Tilt)

>   I am interested in knowing which board was "board B.", the one
> that so clearly won your test.

Yes, it won MY test, but it might not win YOUR test.

> From:	stan@rti.com (Stan Schneider)

> I realize there are political issues in naming names on the net, but are you
> willing to provide specifics to individuals?  I'd be curious to see which
> boards you found were best.

> >> (See if you can determine which entry is "puzzling.")
> I'll take a crack: Why did the net speed go up when the board was loaded?
> Could it be your host SparcStation couldn't keep up?  That would throw your
> numbers off.  My experience is that VxWorks running socket tests to
> SparcStations isn't very informative.

You are on track.  See above response.
-- 

Eric L. Schott,    HRB Systems, Inc.    814/238-4311     els@icf.hrb.com
  "As we acquire more knowledge, things do not become more comprehensible
   but more mysterious."                  Albert Schweitzer, "Paris Notes"