From: xappeal@xtreme.sublink.org (X Appeal Info)
Newsgroups: fj.mail-lists.x-window
Subject: Re: benchmark for xserver
Date: 21 Jul 92 05:30:20 GMT
Distribution: fj
Organization: Xtreme s.a.s. - Livorno, Italy

Date: 17 Jul 92 23:44:58 GMT
Message-Id: <m0B3NB1w164w@xtreme.sublink.org>
Newsgroups: comp.windows.x

paul@cgh.cgh.com (Paul Homchick) writes:

> fmc@uhura.neoucom.edu (Fu-Min Chen) writes:
> >Can someone tell me what are the available benchmark for testing xserver?
> >What is xstone? Where can I get it?
> 
> Along the same line, has anyone ever seen an article showing xengine
> or Xstone performance reports for X-terms vs PCs running different
> types of Xterm emulation software?  I have tested a few PCs on a wide
> area network with Xengine clients on HP 9000s.  The 9000/730 runs at
> 1600 for the xengine, while an IBM PS/2 56SLC with HCL Exceed/W
> software is 15!  A Gateway 486DX2/50 (at another point on the network
> runs at 55 xengine rpms.  It certainly is hard to figure out where the
> bottlenecks are and a few other examples would certainly help!

Never seen an article like the one you are looking for. There are some
benchmarks for X terminals floating around, but almost nothing on PC X
servers. Probably the xstone ratings (summary from the xbench program,
to answer the original question) are generally poor, and vendors do not
consider them useful to promote their product.

You should however do some considerations about what you are comparing.
A high-performance workstation and PCs running MS Windows X servers are
probably near the two opposite ends of the performance spectrum. And the
test you are running (xengine) is nice and gives a qualitative feeling
about the X server speed, but it is far from being a comprehensive test
of the X server ability to meet the user needs in a common application mix.
Xengine uses a lot of floating point computation on the server side (arcs)
and is mostly CPU-bound on the X server. In contrast, one of the usual
bottlenecks on PC X servers is VGA video bandwidth, especially for BitBlts.
The well-known xbench gives results that in my opinion are far closer to
the real behaviour of the server in an application environment.

Someone can say no single number can show the real differences between
completely different implementations. That's obviously true, but it can
be a starting point, better than nothing. And xbench gives a lot of output,
which can be converted in a summary giving performances for operation
classes (fillStones, blitStones etc.) and the grand total (XStones).
By comparing the different classes, the real bottlenecks are easily found,
like the differences between products; a more detailed analysis can be done
using x11perf.

Both xbench and xengine come with some (unofficial) test results, that can
be used as a reference point. I would suggest these ones:

[xengine]

SparcStation 1, X11R4 PL14, 256 color  ==>   186 rpm
Sun 3/60, X11R4 PL14, 16 color         ==>    90 rpm

[xbench]

SparcStation 1, X11R3, 256 color       ==>  3424 xstones
DECstation 310, X11R3, 256 color       ==> 15261 xstones
(these ones are rather old, but I found no X11R4 result with xbench).

As for other examples about PC X servers, I can give you some results we
obtained with our PC X server. Please don't see any marketing purpose in
that, since everyone can verify these figures using our freely available
demo version. I sincerely hope other vendors will publish their results,
so that you (and anyone interested) can have some hard facts to support
product comparisons.

Hardware:  386DX 33MHz, 4MB, ET4000 1MB 1024x768 SVGA
Software:  X Appeal V1.2 (X11R5 PL12, 256 colors)

[xengine]   ==>    64 rpm
[xbench]    ==>  6550 xstones

Hope that helps.



Best Regards

Giovanni Novelli
Xtreme s.a.s. - Livorno, Italy
xappeal@xtreme.sublink.org