From: backstrom@ranger.enet.dec.com (ranger::backstrom)
Subject: Re: SUMMARY: Wanted: PATHWORKS site configurations [eXcursion]
Organization: Digital Equipment Corporation
Date: Wed, 11 Nov 1992 05:00:11 GMT
X-Posted-Via: NEWS-NOTES gateway (a hack!)

>                   vhalkka@klaava.Helsinki.FI (Vesa Halkka)
>              Re: SUMMARY: Wanted: PathWorks site configurations
>
>The DEC Win3.x X-server didn't do very well on a Digital review test.

    Fwiw, below's the rebuttal our eXcursion engineering group wrote for DN&R
    (I believe it was also published in a later issue (October?)).
    
    ...petri (petri.backstrom@ljo.mts.dec.com)
       Personal Computing Integration Engineering
       Digital Equipment Corporation

	             Rebuttal to Digital News and Review
	          Analysis of eXcursion v 1.0 Capabilities
                              

	      The  August 24, 1992 issue of Digital News and  Review
	contained an article comparing and contrasting the  features
	and  performance of five PC/Windows based X servers.   Based
	upon  that review, the responsible analyst decided that  "We
	cannot  recommend Excursion (sic). It showed  irregularities
	during  our  testing...". A careful  review  of  the  setup,
	configuration  and execution of eXcursion in the  reviewer's
	testing  environment lead us to believe that several serious
	flaws  in  the  testing environment led to this unfortunate,
	and, clearly erroneous, decision.
	
	
	1.Network Setup and Environment
	
	      The reviewer did not describe the network installation
	procedures employed. We are unable to form a judgment  since
	the underlying configuration is unknown. DN&R Labs installed
	FTP's  PC/TCP  of  an  unknown  version.  eXcursion  is  not
	compatible  with  versions of PC/TCP prior  to  v  2.05.  It
	remains unclear whether an NDIS, native mode, or dual  stack
	configuration was employed.
	
	
	2.eXcursion Setup and Configuration
	
	      It  would  appear that eXcursion installed  correctly,
	based  upon the display of the available icons appearing  on
	page  83.  There is no indication of how the font  path  was
	configured.  NOTE: There is a known bug in v 1.0  concerning
	font path length, described in the release notes. It appears
	from  the results obtained when attempting to run Wingz that
	the  reviewer  was  unaware of this condition  and  did  not
	attempt  a  workaround. The problem is a limitation  on  the
	number  of fonts to 450. Any attempt to request a font  list
	when the number exceeds this value will cause precisely  the
	"Sequence  lost" error noted by the reviewer.  There  is  no
	need  to configure both the 75 and 100 dpi fonts at the same
	time. It would appear from the data that this was done.  The
	reviewer was made aware of this limitation by DEC developers
	but  appears not to have corrected it. eXcursion v  1.1  has
	fixed this problem.
	
	      The  review  indicates that problems were  encountered
	starting  up  applications from eXcursion installed  Windows
	icons.  We  are unable to duplicate this problem,  nor  were
	several  third party reviewers and testers (1). The reviewer
	describes    "talks   with   several   DEC   tech    support
	representatives" in an attempt to resolve this issue. We are
	aware of no such discussions.
	
	
	
	3.eXcursion Performance
	
	      The  performance  results  obtained  by  DN&R  are  so
	consistently  at  odds with those obtained  by  every  other
	tester   to  date  that  clearly  something  was  configured
	incorrectly.  Attempts  to  duplicate  similar   performance
	numbers  succeed  only  when  the  Request  Message  Logging
	feature was activated. Message logging reads and writes to a
	file  on  the hard disk every time an X request is processed
	by eXcursion. An independent analysis of eXcursion (2)showed 
	a reduction in time required  to  complete a  graphics  task
	from 62 hours to 2 minutes when logging was turned off. This
	is an extreme example of a graphics and X request  intensive
	application.  X11PERF  does  not  send  a  large  number  of
	requests, so the performance numbers will be skewed by  less
	than  or  equal to one order of magnitude - consistent  with
	the  observed results. We believe that the entire  range  of
	X11PERF  tests  in the study were run with  message  logging
	turned  on. Note that eXcursion v 1.1 will include a feature
	which  automatically turns off request message logging  when
	the  eXcursion session is exited to ensure that  performance
	is not accidently compromised.
	
	      Readers  are  advised to refer to the OpenSystemsToday
	review  article for a completely different conclusion.  That
	review  was conducted with FTP's PC/TCP transport  as  well,
	and concluded that eXcursion's performance was equal to that
	of any of the other server tested.(3) That review appears to
	be  much  more representative of the values we have obtained
	in our own testing.
	
	         Relative Performance of X Servers Reviewed
	
	NOTE:   All  values  are normalized to  eXcursion.  A  value
	greater   than  1.00  indicates  performance   better   than
	eXcursion,   a   value  less  than  1.00  indicates   poorer
	performance.


REVIEWER    TEST    eXcursion XoftWare    HC:-     X11/AT      PC-    PC-Xview
                                        eXceed/W             Xvision
                                                                          
DN&R    General       1.000     0.787     1.543    failed     1.455   untested
        Term. emulate 1.000     6.232     6.634    14.413     6.563   untested
        Win manager   1.000     6.740     7.264     9.272     7.218   untested
        X specific    1.000    21.216    23.825    10.064    19.898   untested
          
        500 pixel     1.000     2.198     0.558    failed     0.708   untested
                                                                          
Open    General       1.000     0.449     1.217    failed     1.101     0.565
Systems Term. emulate 1.000     0.452     0.959    failed     1.017     0.203
Today   Win manager   1.000     0.377     1.123    failed     1.033     0.328
        X specific    1.000     0.636     1.188    failed     1.340     0.509
          
                                                                          
Intel   Rect outline  1.000   untested    1.063   untested    1.417     1.417
        Opaq stipple  1.000   untested    1.250   untested    1.333     0.363
        Lines         1.000   untested    1.031   untested    1.029     0.461
        Circles       1.000   untested    1.400   untested    0.040     0.045
        Complex poly  1.000   untested    1.000   untested    1.000     0.375
        Scroll        1.000   untested    1.000   untested    1.000     0.950
        Copy area     1.000   untested    1.000   untested    1.000     0.980





	      It  is  quite obvious to even a casual observer  after
	studying  the  results  that there  is  consistency  in  the
	numbers  obtained  both  in the OpenSystemsToday  and  Intel
	tests,  and  that there is wild fluctuation evident  in  the
	results  of  the Digital Review tests. In fact, the  results
	are so inconsistent that it leaves the reader asking why the
	tester  did  not  immediately question them and  attempt  to
	resolve the issue before printing ANY of the results  -  not
	just the results for eXcursion.
	
	
	4.Conclusion
	
	     It is our firm belief that the negative impression left
	by the Digital Review tests is entirely unwarranted, and not
	supported  by  any  other  available  data.  eXcursion   has
	consistently  won  high  marks  both  for  performance   and
	useability  from  users and reviewers  alike.  We  have  now
	completed  two  external field test cycles,  with  15  third
	party  testing environments each, and have never  once  seen
	results  reported which even remotely compare to the Digital
	Review Labs findings.
	_______________________________

	1 OpenSystemToday - "Windows, X Servers Collide" March 2,
	  1992
	  Boeing Computer Services PC X-Server review June 16, 1992
	  15 External Field Test sites for eXcursion v 1.0

	2 eXcursion v 1.0 running a Swanson stress analysis program
	  from a CONVEX UNIX host on top of FTP PC/TCP V 2.05

	3 XoftWare v 1.01, HCL-eXceed/W v 3.0, X11/AT V 3.2.1, PC-
	  Xview V 3.0, XVision v 4.0