MTRioom June 1987 

JSC 22603 

Jerry K. Richardson MIPS: The Good, 

The Bad, and The 

Useful 

388-24184 

UncZas 

63/63 0345920 

CONTRACT SPONSOR NASA 

CONTRACT NO, Fl%28-86-C&1 T-8114M 

PROJECTNO. 852E 

DEPT D115 

The MITRE Corporation Houston, Texaa 
Department Approvat 

W. P. Kincy,/Jr. 

Edwis S. Herndon 

ii 
ABSTRACT 

Many authors are critical of the use of (MIPS) Million 

of Instructions per Second as a measure of computer power. 

Some authors say that MIPS are meaningless. While there is 

justification for some of the criticism of MIPS, sometimes 

the criticism is carried too far. HIPS can be a useful 

number for planning and estimating purposes when used in a 

homogenous computer environment. 

Comparisons between published MIPS ratings and benchmark 

results reveal that there does exist a high positive correlation between MIPS and tested performance, given a homogenous computer environment. 

MIPS should be understood so as not to be misused. It 

is not correct that the use of MIPS is always inappropriate 

or inaccurate. 

iii 
ACKNOWLEDGEMENTS 

I wish to express my appreciation to several individuals 

who provided help and information for this document. 

Bill Kincy and Stuart Bell provided much needed document 

review and publishing suggestions. Walter Bays and Danny 

Labasse provided articles and references related to MIPS and 

benchmarks. Jeff Lorentz provided editorial help and hints 

on how to structure various sections of the document. 

Don Simanton (NASA sponsor) furnished the initiative for 

the selection of the topic, and also suggested the publication of this paper. 

Special thanks goes to Jerry Trust who functioned as my 

principal sounding-board, technical advisor, and helpful 

critic. Without his help, this document would not have been 

completed. 

iv 
TABLE OF CONTENTS 

TITLE ...... APPROVAL ..... ABSTRACT ..... ACKNOWLEDGEMENTS 

TABLE OF CONTENTS 

LIST OF TABLES . . LIST OF ACRONYMS . 

.............. i 

.............. ii 

.............. iii 

.............. iv 

.............. V 

.............. vi 

.............. vfi 

1 INTRODUCTION ................. 1 

1.1 GENERAL ................. 1 

1.2 PURPOSE. ................ 2 

1.3 SCOPE.. ................ 3 

2 MIPS, INTERNAL THROUGHPUT RATE, AND POWER . . 5 

2.1 USE OF MIPS .............. 5 

2.2 INTERNAL THROUGHPUT RATE ....... 6 

2.3 PROBLEMS WITH THE USE OF MIPS ...... 8 

TEST RESULTS ................. 17 

3.1 MITRE NOMAD2 BENCHMARK RESULTS ..... 17 

3.2 ANALYSIS OF REPORTED BENCHMARK DATA ... 23 

4 CONCLUSIONS ................. 33 

REFERENCES ........ ......... 35 

DISTRIBUTION LIST .............. 37 

V 
n 

P : 

LIST OF TABLES 

1 ITR RATIOS . . . . . . . . . . . . . . . . . . 18 

2 PROJECTED ITR AND MIPS RATIOS . . . . . . . . 19 

3 MEASURED ITR . . . . . . . . . . . . . . . . . . 20 

4 PROCESSOR COMPARISONS USING ITR 

AND MIPS RATIOS . . . . . . . . . . . . . . 21 

5 MEASURED ITR RATIOS VS PROJECTED ITR RATIOS . . 22 

6 INSTRUCTION TIMES (MICRO-SECONDS) 

AND PUBLISHED MIPS . . . . . . . . . . . 23 

7 INSTRUCTION RATES (MIPS) AND PUBLISHED MIPS . . 24 

8 RESULTS OF MULTIPLE REGRESSION 

CORRELATION FOR 8 PROCESSORS . . . . . . . . . 28 

9 WEIGHTED INSTRUCTION RATES VS 

PUBLISHED MIPS WITHOUT DEC PROCESSORS . . . . . 30 

10 RESULTS OF MULTIPLE REGRESSION CORRELATION 

FOR 11 PROCESSORS . . . . . . . . . . . . . . 31 

11 WEIGHTED INSTRUCTION RATES VS 

PUBLISHED MIPS WITH DEC PROCESSORS . . . . . 32 

vi 
LIST OF ACRONYMS 

CI sc 

CP 

CPU 

DASD 

DP 

DPSD 

ETR 

I TR 

JSC 

MFLOPS 

MIPS 

MP 

MTR 

NASA 

PROFS 

RISC 

RPMs 

Complex Instruction Set Computer 

Control Program 

Central Processing Unit 

Direct Access Storage Device 

Data Processing 

Data Processing Systems Division 

External Throughput Rate 

Internal Throughput Rate 

Johnson Space Center 

Millions of Floating Point Operations per Second 

Millions of Instructions per Second 

Multi-processor 

MITRE Technical Report 

National Aeronautics and Space Administration 

Professional Office Systems 

Reduced Instruction Set Computer 

Revolutions per Minute 

vii 
This page left blank intentionally. 

viii 
SECTION 1 

INTRODUCTION 

This paper was written in response to a request (May 

1987) from our NASA sponsor that MITRE prepare a paper 

discussing quantitative analysis of MIPS, along with any 

usefulness, or non-usefulness, of MIPS in comparing large 

and small computers, and computers with like and unlike 

architectures. The initial response was simply the kernel 

of this paper and was written as a four page PROFS note and 

mailed to the various interested individuals. 

DPSD Division Chief, Don Simanton, subsequently suggested that I "clean-uptt the PROFS note and make it into a 

publishable paper. This MTR is the result of that suggestion 

1.1 GENERAL 

Many articles and presentations have pointed out the 

inappropriateness of using MIPS (Millons of Instructions per 

Second) as a measure of performance or power of a modern 

large scale computer system. The following is typical of 

many such discussions 

Performance rating implies a measure. Traditionally, the 

performance measures used by the industry have been either cycle 

times or instruction execution rates (in millions of 

instructions per second HIPS), the instructions here being 

machine level instructions. These measures today are not only 

misleading, but downright irrelevant as the measures of the 

performance power of today's data processing systems. [11 

Very few articles are objective and balanced in their 

discussion of the utility of MIPS as well as the dangers and 

difficulties with the use of MIPS. Many authors make 

stereotyped and senseless statements concerning MIPS and 

then proceed to use them extensively , and with good effect. 

HIPS (meaningless indicator of processor speed) i;, of course, 

not a good measure of relative CPU power, but for our purposes, 

it will work fine, since we are only interested in relative 

performance and not actual numbers. 121 

1 
Most discussions on MIPS fail to point out, or adequatetreat the following: 

The inevitable, wide variations in MIPS across 

non-homogenous architectures or non-homogenous workloads 

does not mean that MIPS cannot be used with relative 

confidence in homogenous computer environments 

There are at least two kinds of useful MIPS available. 

The most common notion for MIPS is in the context of an 

aggregate or composite rate of instruction execution. 

But, one may also be interested in MIPS numbers (instruction execution rates) for individual machine level 

functions (load/add, move, etc.). 

MIPS does not have to be a measure of system power 

(total throughput capacity) in order to be a useful 

estimator of relative CPU power. 

Obtaining performance estimates that are superior to 

MIPS is often expensive and time consuming. And the 

difference in accuracy may not justify the cost. 

1.2 PURPOSE 

The primary purpose of this paper is to demonstrate that 

MIPS is a good indicator of relative CPU performance in a 

homogenous computing environment. Homogenous means similar 

architectures and equivalent workloads. 

This paper will assist the reader's suspicions of an 

aggregate MIPS number as a measure of CPU power across an 

undefined range of computers. At the same time this paper 

will encourage the reader to realize that there is an 

equally legitimate use of aggregate HIPS numbers when they 

span a range of homogenous comquters. 

This paper will discuss both single-function MIPS 

numbers and aggregate MIPS numbers. Normally a weighting 

process is used to form an aggregate MIPS number; or weighting occurs due to the statistical composition of the workload if MIPS is measured rather than calculated. In other 

words, the rates for individual instructions such as loads 

and adds are combined in a weighted average fashion to form 

an aggregate MIPS number. 

2 
The terms 'aggregate MIPS' will be used synonymously 

with 'MIPS' and both will refer to an aggregate instruction 

rate comprised of different single-function instruction 

rates. The term 'single-function' MIPS will refer to an 

instruction rate for an individual machine level function 

such as a load or an add. 

1.3 SCOPE 

This paper will be concerned with MIPS as it pertains to 

digital, scalar computers. For the comparison of vector or 

array processors we would use Millions of Floating Point 

Operations per Second (MFLOPS). 

Although a discussion of MFLOPS has considerable similarity to a discussion of MIPS; MFLOPS as a measure has not 

yet drawn the widespread opprobrium that MIPS has drawn even 

though it is rather obvious, upon examination, that MFLOPS 

alone is not sufficient to compare the performance of two 

vector processing computers of dissimilar architecture or 

with dissimilar workloads. 

Note that some authors classify both MIPS and MFLOPS as 

useless numbers. 

Measuring the performance of a machine is a complex issue. The 

inaccuracy and danger of using a simplistic measure like HIPS is 

widely known. Unfortunately it is still used, even though more 

accurate measures such as Internal Throughput Rate (ITR) are 

available. In the world of vector processing, the situation is 

no better. 

The most common measurement used to compare vector processing 

speeds is HFMPS, pronounced nHegaFLOPS,tt which means "millions 

of floating-point operations per second." Many modern vector 

machines can also do integer and logical operations on vectors, 

but it is the floating-point operations that consume the most 

time in scalar mode, and they are used to measure performance. 

This is still true, although even 

that "to quote HIPS is dangerous; 

t31 

computer salesmen recognize 

to quote MFLOPS is suicide. 

There are many scholars who 

type of blanket; unqualified 

R.W. Hockney and C.R. Jesshope 

PARALLEL COMPUTERS [41, use MFLOPS as one of two parameters 

for comparing computer performance. 

would not agree with the 

criticisms offered above. 

in their excellent book, 

3 
4 

The two parameters [the half-performance length and the maximum 

or asymptotic performance] ... completely describe the hardware 

performance of the idealized generic computer and give a 

first-order description of any real computer. 151 

The two parameters ...p rovide us with a quantitative means of 

comparing the parallelism and maximum performance of all computers. [61 

The operative phrases in the quotes above, for our 

discussion, are "first-order description of any real computer" and '!quantitative means of comparingff. This is the 

reason for wanting to use a number like MFLOPS or MIPS in 

the first place. 

It is instructive to note that Hockney and Jesshope 

imply at least three parameters in comparing the performance 

of vector processors. One of the parameters [the 

half-performance length1 is a measure of parallelism in a 

computer; the second parameter [maximum or asymptotic 

performance] measures the theoretical maximum computational 

speed In MFLOPS. The third (implied) parameter is workload 

for both the CYBER 205 and the CRAY1. The comparison of 

these machines is difficult because their performance is 

strongly dependent on the problem being solved and the manner in 

which it is organized and programmed. [71 

Without assumptions concerning parallelism and workload, 

MFLOPS alone is not sufficient to compare the power of two 

computers However, if one does know or make assumptions 

concerning the amount of parallelism and the workload, then 

MFLOPS is a reasonable number to use to compare computer 

power. Similar assumptions are needed when MIPS is used to 

compare computer power. 

The recurring themes that are always interwoven in any 

realistic comparison of computer performance are 1) speed, 

2) architecture, and 3) workload characteristics. Whenever 

one of these themes is selected as a basis of comparison, 

the stated or underlying assumption is that the other two 

are comparable. 

If the reader is interested in relationships between 

MIPS and MFLOPS for a given computer running a given workload. Literature on the DENELCOR HEP supercomputer t81 

shows some tested relationships between MIPS and MFLOPS on 

the DENELCOR HEP. 

4 
SECTION 2 

MIPS, INTERNAL THROUGHPUT RATE, AND POWER 

2.1 Use of MIPS 

The desire by planners to use MIPS as a single, simple 

measure of computer performance is as reasonable as the 

desire to know the maximum possible speed of a racing car, 

or perhaps the rated horsepower of a racing car's engine. 

In that sense, it is not unreasonable to expect that one 

could derive or measure a single number that would express 

the power delivery capability of a machine, even if that 

machine happens to be as complex as a modern digital computer 

On the other hand, to expect that a single number could 

capture how well a racing car would start, steer, and stop 

or how well it would fare in the Indianapolis 500, is simply 

expecting too much from a single measurement. The same is 

true for computers. 

There is no perfect, single measure for the performance 

of a complex digital computer However, single number 

estimates of power are needed and used Capacity planners, 

computer performance analysts, system modelers, and those 

concerned with procuring new systems or upgrades to existing 

systems need (and will find, guess, or manufacture) planning numbers. These numbers certainly can be, and usually 

are, ballpark estimates with a precision sufficiently 

appropriate for planning and modeling. 

The Wadsworth's article 121, referenced earlier, is an 

example of a well written article that refers to MIPS more 

than a dozen times in the course of developing a practical 

methodology that "uses readily available numbers to determine the effect of adding various pieces of hardware to the 

DP center to reduce on-line response times". And what was 

one of the most important of those readily available numbers 

used? MIPS! 

5 
Still, before Mr. Wadsworth would use MIPS, he made the 

disclaimer quoted earlier. How can MIPS be Itnot a good 

measure of relative CPU power, but for our purposes, it will 

work fine, since we are only interested in relative performance.. . If ? In other words, MIPS is not a good measure, but 

it will work fine. That's a reasonable statement of the 

argument this paper is making. 

2 2 INTERNAL THROUGHPUT RATE 

Internal Throughput Rate (ITR), which can be interpreted 

as a measure of CPU power (for a given workload) is the sort 

of performance measure that we would like to be able to 

estimate with a MIPS number. Although they are not perfect, 

benchmark results are preferable to MIPS. Whenever possible, users should conduct their own benchmark runs on target 

computers in order to insure that the test environments map 

the proposed real environments . Benchmarking is, however, 

time consuming and expensive; also some users may not have 

access to various vendor's test facilities. 

A timed test represents the completion of some sort of 

actual end user defined work; hence, a meaningful measure of 

power can be derived from such a test. IBM, along with 

numerous other vendors and CPU testers, uses the concept of 

Internal Throughput Rate which is defined as the number of 

completed jobs or transactions (commands) per processor busy 

second. The count of completed transactions in unit elapsed 

time (wall clock time) is usually referred to as the 

EXTERNAL THROUGHPUT RATE (see EQUATION 1). 

EQUATION 1: 

(EXTERNAL THROUGHPUT RATE)*(lOO) 

(% TOTAL SYSTEM (CPU) BUSY) 

INTERNAL THROUGHPUT RATE IITR) = 

To determine ITR, throughput is measured in a context 

which prevents 1/0 or memory configurations from being a 

constraint If non-constrained measurement conditions are 

not provided, then the time that the CPU spends waiting on 

1/0 or memory is factored out by the use of the above ITR 

formula. 

6 
ITR can be very useful and accurate as a measure of CPU 

power when comparing computers with similar workloads and 

measurement conditions. As with any other measure however, 

the misuse of ITR may result in faulty conclusions, 

Internal throughput is a theoretical measure of processor 

capacity. An assumption is made that if the processor were 

driven to 100% busy in an unconstrained environmentp the 

projected ITR numbers could [bel delivered as ETR [external 

throughput ratel.. .. 

Here we see a situation system reached the 

throughput indicated by the ITR numbers. There are three 

factors causing this: 

in which neither HP 

o DASD Contention 

o Increasing CP busy time per transaction 

o Main Storage Constraint. ... 

Here we see a case where we were unable to achieve the ITR rates 

when the processor was driven to capacity. The measured ITR of 

the 5880 processor, even at 74% utilization, still projected a 

capacity about 11% greater than the maximum possible There was 

no way to accurately predict effect except by 

measuring it. The point here is that if we had planned our 

capacity based on ITR projections, we would be out of capacity 

far sooner than expected. (91 

the size of this 

ITR may be invalid unless the CPU is actually tested at 

or near 100 percent busy in order to see whether or not any 

unsuspected constraints exist. If ITR is calculated with 

the CPU busy considerably less than 100 percent, there is no 

guarantee that the theoretical ITR can, in fact, be accomplished. The major problem, of course, is that in many cases 

it is difficult to construct a benchmark and secure the 

necessary DASD and memory to cause a given processor to stay 

close to 100 percent busy during the entire run (excluding 

tight looped routines that consume only CPU resources). 

7 
2.3 PROBLEMS WITH THE USE OF MIPS 

2.3.1 The Proble m of Aaareaa te Measure ments 

One of the most, if not the most, serious problems with 

MIPS is the fact that it is supposed to be an aggregate 

number. This problem of MIPS being an aggregate, single 

number descriptor is perhaps not obvious enough. ITR, 

though a better measure, also suffers from being an aggregate, single number descriptor of computer performance. 

The best way to describe the performance of a computer 

is with a vector of numbers, each of which measures a 

different aspect of computer performance. However, a 

descriptor vector must be eventually converted into a scalar 

(single number) if one is interested in making an overall 

evaluation or making a GomDet itive comDarison among two or 

more machines. An aggregate MIPS number can vary depending 

upon the instruction mix used to generate the number, or 

(what amounts to the same thing) an aggregate MIPS number 

can vary depending upon the weight assigned to the individual instructions that comprise the MIPS number. 

The issue of rational aggregation is non-trivial. 

Suppose one wishes to perform an overall comparison (ranking) among three or more processors based upon their execution times for a multiple set of machine level instructions; 

it turns out that there are some common perceptions of 

rationality that cannot simultaneously be preserved. Nobel 

Laureate, Dr. Kenneth Arrow formally proved this proposition 

in what is now called Arrow's Theorem (also called Arrow's 

Impossibility Theorem) 1101. A slightly modified version of 

Arrow's rationality perceptions stated in computer processor 

terms are the following: 

Any number of processors is allowed and any number of 

machine level functions may be specified in a composite 

evaluation scheme designed to produce an overall ranking. 

If processor A ranks higher than processor B in every 

single machine level function used in the evaluation, 

then processor A must rank higher than processor B in 

overall ranking. 

There shall be no dictatorial function in which processor A outranks processor B while A is being outranked by 

B in all other functions. That guarantees that processor A will outrank processor B in the overall ranking 

8 
The somewhat shocking results of Arrow's Theorem is that 

there is NO SCORING SYSTEM POSSIBLE that will preserve the 

above stated rationality perceptions. That is, if you wish 

all of the above rationality perceptions to function jointly 

as rationality requirements in an evaluation scheme, you are 

out of luck. It can't be done. Which does one give up in a 

ranking or scoring scheme if MIPS are being used as an 

aggregate measure? Rationality perception number three is 

the one that is sacrificed with the concept of aggregate 

MIPS. 

In other words, it is unavoidable that an aggregate MIPS 

number be so influenced by one particular machine 

function (because of the instruction mix or assigned 

weights) that it will not matter how all the competing 

processors actually performed (ranked) an all other functions used in the evaluation. Also, it is conceivable that a 

given machine's architecture could be so constructed that a 

particular processor would make such a tremendous score in 

one machine level function that it would not matter that it 

ranked dead last in all other functional categories. 

Requirement three cannot be preserved without giving up 

one of the other rationality requirements. Which one do you 

give up? There are no good choices from the standpoint of 

equitable evaluation. 

We have to live with the fact that an aggregate MIPS 

score can be unacceptably driven by one of the functional 

parts that make up the aggregate. This is potentially one 

of the most serious problems with the use of aggregate MIPS 

numbers. Within the bounds where MIPS is useful, it will 

seldom be a problem. However, it is one of the reasons why 

it would be helpful if vendors would publish aggregate MIPS 

numbers for their processors along with the single-function 

MIPS numbers (or times) that make up the aggregates. But 

most vendors have no interest in doing this; they are more 

interested in castigating the concept of MIPS and preserving 

the mystical nature of the performance of their machines 

(nobody can beat us if we keep our performance definitions 

secret, or sufficiently vague). 

The aggregate measurement problem is not unique with 

aggregate MIPS. The same problem exists, in a hidden form, 

with the use of ITR. Aggregate MIPS implies a set of 

instructions executed in certain proportions; so does a 

benchmark that generates an ITR measure. 

9 
Furthermore, with a high level language benchmark, one 

has to worry about the instruction mix at two levels: the 

higher language level and the compiled object code level. 

These are often different due primarily to differences in 

compiler efficiencies. One also has to worry about any 

differences in the operating environment, since many modern 

languages are not just a simple compiler and a run stream. 

In Ada, for example, one is benchmarking more than just a 

compiler; a total operating environment is benchmarked. 

A recent report from Intermetrics Corporation concerning 

benchmark results on their Ada compiler versus IBM'S Ada 

compiler and DEC's Ada compiler states the following for 

execution time performance t113: 

o Intermetrics' MVS Ada COMPILERS VERSUS IBM's Ada COMPILER 

o All tests were run on an IBM 3084 

o Of the 42 test compiled 37 ran successfully for 

o On the 37 successful tests 

both compilers 

- IBM was faster on 2 tests (by 30%) 

- Intermetrics was faster on 35 tests 

- Average ratio was 2.47:l [In favor of 

- Intermetrics 3 times faster or 

Intermetricsl 

more on 8 out of 37 tests 

o Intermetrics' Ada (running on IBM 4341) VERSUS DEC's Ada 

(running on VAX 11/780) 

o Intermetrics was faster on 39 of 52 tests (75%) 

o Average ratio was 3.7:l (Favoring Intermetrics) 

The results from this report serve to demonstrate the 

difference in performance on the same machine that can exist 

given the same set of high language instructions (in this 

case Ada) and a different set of machine language instructions (due to the difference in compiler efficiencies). 

The report states that the Intermetrics to DEC ratio of 

3.7:l is "Much more than the variation in MIPS" t121. The 

report does not specify what model IBM 4341 was used; 

however, assuming that the 4341 used was a model 12, the 

aggregate MIPS ratio would be 1.4 (1.5/1.06). It is possible, but not likely, that the difference between the 

performance ratio (3.7) and the MIPS ratio (1.4) is due to 

the Intermetrics compiler being that much more efficient 

than the DEC compiler. 

10 
2.3.2 The Definition Problem 

One of the major problems with the use of MIPS could be 

quite easily solved by vendor cooperation. As has already 

been pointed out MIPS implies an instruction mix. Hence, 

for a precise comparison using HIPS one needs to know 

exactly what instruction mix went into the making of a 

quoted MIPS estimate. 

Vendors often complain about the imprecision of MIPS; 

yet they will not take simple and effective steps toward 

increasing the precision of MIPS estimates. If vendors 

would publish MIPS estimates for their processors along with 

the exact instruction mix used in the estimate, a planner 

could use MIPS with as much precision as he could estimate 

the instruction content of his projected workload, and this 

could be estimated with considerable accuracy in many 

cases. Why will most vendors not do this? Partly because 

of a HIDDEN-AGENDA problem which will be discussed in 

section 2.3.4. 

It is easier to understand the use and significance of 

MIPS if it is interpreted as a measure of CPU speed and not 

as a measure of CPU power. Power is the ability to do work. 

More precisely, work performed in unit time equals power. 

To illustrate why MIPS should be considered a measure of CPU 

speed and not of CPU power consider the following equation 

(EQUATION 2) published in an article discussing MIPS t171. 

EQUATION 2: 

HIPS = 

1 

(Cycles per average instruction)*(Cycle time in microseconds) 

An examination of the above formula reveals that it is 

composed of two variables. MIPS is functionally (inversely) 

dependent upon 11, the cycle time of the processor and upon 

21, the number of cycles it takes to execute the average 

instruction. 

The cycle time of the CPU is usually fixed for a given 

processor. Cycles per average instruction depend to a large 

extent upon the architecture of the computer. The important 

thing to keep in mind is that a computer instruction is the 

entity that performs work for a user. Individual Computer 

instructions vary in the amount of work which they are 

11 
capable of producing. And they may exhibit large variance 

across computer architectures. Hence one machine may be 

turning 15 Millions Instruction per Second (MIPS) and doing 

the same 'work' for the user that another machine would 

require 30 Million instruction per second. MIPS then are 

best thought of as speed of the CPU, and not power. We are 

likely to see more of this sort of variance in instruction 

efficiency in the near future as the concept of the Reduced 

Instruction Set Computer (RISC) matures. RISC computer 

systems are based on the set 

of instructions that are frequently used, and avoiding 

instructions that are not frequently used. This approach 

has the goal of reducing hardware complexity and increasing hardware speed. 

Complex Instruction Set Computer (CISC) processor has three commonly 

accepted characteristics: 

1) a RISC machine must execute one instruction in a 

single clock cycle 

a RISC machine must use a fixed format for instructions 

concept of optimizing a small 

A RISC processor as opposed to the conventional 

2) 

31 a RISC machine must use only a load/store architecAn examination of EQUATION 2 will reveal that for a 

processor with a given clock speed (cycle time in microseconds), one can increase the MIPS rate by reducing the 

average number of cycles per instruction. Given current 

technology, RISC computers can be expected to run at higher MIPS rates than comparable CISC computers. However, the 

power of the individual RISC instruction may not be as great 

in some cases as the power of the individual CISC instructions; hence, MIPS comparisons between RISC and CISC machines may be subject to large discrepancies. It should be 

noted that 

use of una djusted MIPS susDect. 

ture for interacting with memory 

2.3.3 The P ower Problem 

There is a real problem and a hidden-agenda problem with 

the use of Millions of Instructions Per Second (MIPS) as a 

measure of power for a central processing unit (CPU). 

However, MIPS numbers are useful for planning and estimating 

purposes if a few simple caveats are assumed. 

12 
The real problem with MIPS as a measure of relative CPU 

power is the fact that it does not actually measure power 

(i.e. work done in unit time), even though people still use 

it that way. It seems reasonable, therefore that one would 

like to understand some concept of computer "Powerff measurement and where MIPS would fit into this concept. 

In order to legitimately measure power, one must first 

define work. Units of software work can be defined as that 

which takes place when a byte of data is transferred from a 

processor to a storage device. In other words, the central 

processor does a unit of work on main storage for every byte 

transferred into real storage. This type of definition of 

Software Work is the basis for what is often called Software 

Physics 

The definition of software work proposed by Kolence in the 

context of his software physics [see Kolence (197211 is very 

similar from a pragmatic viewpoint to the one given by 

Rozwadowshi: 'a processor performs one unit of software work on 

some storage media when one byte of that media is altered.' (181 

In Software Physics, a workload or workload component is 

characterized by the work done by the CPU, DASD, terminals, 

printers, and other devices as they process and move data 

about the system. The resulting workload description is 

usually called a Software Work Vector. 

To understand where MIPS fits into the concept, consider the 

operation of a sawmill in the following analogy: 

The purpose of a sawmill from the viewpoint of the owner is 

to process as many logs through the mill during hours of 

operation as he possibly can. The power of the sawmill can 

be measured by the quantity of lumber that the mill will 

output in some given unit of time (e.g.- one day). The 

maximum rate at which a sawmill could turn out lumber 

(throughput) would correspond to the notion of computer 

power in the computer. 

The speed at which the saw spins (Revolutions Per Minute 

or RPMs) would correspond roughly to MIPS in a computer 

system. Obviously, the speed of the saw is important. In 

fact, if one assumes the speed of the feeder belts, the 

diligence of those placing and removing logs from the belts, 

the diameter and sharpness of the saw are all equal, then 

the speed of the saw becomes the determining factor in the 

ffpowerff of this uni-sawmil. 

13 
The REAL PROBLEM, then, with the use of MIPS is that 

SOME people may forget that they are talking about "the 

speed of the saw". and that the other factors in sawmills 

being compared may not be close enough to allow using "the 

speed of the saw" as a good comparison tool. 

This is exactly the case when trying to use MIPS to 

compare the performance of a Microprocessor to that of a 

well equipped mini or mainframe. Although the micro may 

have a very fast saw, the feeder mechanisms, in comparison 

to a mainframe, simply are not enough to give it comparable 

power in terms of the ability to achieve work in a throughput sense. 

How does the 80386 compare to other processors, small and large? 

Clearly, the 80386 is a microprocessor because of its 

implementation on a single chip. But do its characteristics 

qualify it as a mainframe? Certainly its speed is more than 

adequate for that classification. But while the architecture is 

as complex as some mainframes, it is incomplete in the area of 

I/O paths .... 

Given a proper 1/0 design and adequate memory size and speed, 

the 80386 would compare closely in performance to an 

IBM Hodel 4341-1 or 4341-2, depending on a number of factors. 

The higher instruction execution rate of the 80386 would be 

offset by the more powerful instructions of the 4341. Memory 

and 1/0 bandwidth would also have to be evaluated for specific 

designs. In capability, however, the 80386 could perform at the 

level of an IBH 4341. The major differences in performance 

would depend on how the 80386's 1/0 subsystem compared to the 

4341's 1/0 subsystem and the use of special 4341 instructions. 

191 

You do not use "the speed of the sawft to compare the 

power of two sawmills UNLESS it is reasonable to assume that 

the other essential mechanisms are similar in performance 

capability. The same statement is true with MIPS. 

14 
- 2 3.4 De Hidden AaUa Problpm 

Let us now discuss a HIDDEN-AGENDA PROBLEM. To continue 

our sawmill analogy, the fact that some people may incorrectly use "the speed of the saw" to compare two sawmills, 

does not mean that everyone does, or that "the speed of the 

saw" is a useless measure. It is time that we stopped being 

amused whenever some vendor defines MIPS as standing for 

Meaningless Indicator of Processor Speed. The simple fact 

that some people do not understand the use or limitation of 

MIPS as a way to estimate comparative power between computers does not mean that all people do not Understand. 

Many vendors are in opposition to the use of MIPS for 

rating their computers (fearing that those who do not 

understand the limitations may misuse it). However that 

reason (fear) does not necessarily invalidate the use of 

MIPS by other people. Aspirin and penicillin can be misused 

if their limitations are not understood but certainly no one 

would argue their use be discontinued for that reason. 

Similar objections can be raised to single evaluation 

scores in any environment. The fact is, we are often faced 

with either using a single, numerical estimate; or not 

quantifying at all. For example, in no justifiable sense 

can a single I Q. score measure the total potential of a 

human being. Yet, for those who understand the limitations, 

a tested 1.0. score may be useful for some academic predictions. 

Should educators stop using 1.0. scores because some 

people may not understand their limitations? Should we stop 

using MIPS simply because the uninitiated may not understand 

that the usefulness of MIPS as an estimator of power becomes 

very tentative when you try to use it across non-homogenous 

computers or with non-homogenous workloads? 

Most of those who use MIPS to estimate relative computer 

power are VERY, VERY aware that when you cross vendor lines 

the usefulness of MIPS decreases drastically. However, if 

"othertf factors are reasonably close, MIPS can at least put 

you in the right ballpark, which is usually where you want 

to get when you use MIPS - in the ballpark! 

15 
A vendor's objection to the use of MIPS as a rough 

measurement of relative CPU power is8 it would seem, a 

HIDDEN-AGENDA PROBLEM. The vendor often does not want the 

user to independently estimate the power of his computer. 

Why? There cannot be much market differentiation based upon 

power or MIPS, unless your processor is definitely superior 

in power to its competitors. The vendor wants you to buy, 

based upon the features he has selected to market. For the 

most part, he does not market raw MIPS. He markets a 

differentiated, positi.oned product. MIPS is an indignity; 

it reduces a vendor's product to a single number. Ouch! A 

MIPS number puts everyone on a similar basis. Let's face 

it, if a MIPS number was the most accurate and dependable 

single number measurement in the world, most vendors would 

not like it; and they cannot be faulted for feeling this 

way. But that does not mean that the number should not be 

used; especially if one understands the usefulness and 

1 imitations. 

16 
SECTION 3 

TEST RESULTS 

3.1 MITRE NOMAD2 BENCHMARK RESULTS 

In 1985, as preparation for support of upcomiag NASA 

procurements, the author tested an initial version of a 

sizing benchmark on a series of IBM compatible mainframes. 

Among the mainframes tested were IBM's 3083EX, 3083JX, 

3081KX, and 3090/200; Amdahl's 5840, 5850, 5860, 5867, 5870 

AND 5880; and NAS's 9050 and 9060 processors. 

The benchmark used was coded using NOMAD2 (data base 

management system and registered trademark of D&B Computing) 

and was implemented by running concurrent Virtual Machines 

under IBM's VM/CMS operating system. The rules for successful implementation of the benchmark favored an environment 

with minimum constrained I/O. This meant that the vendor 

was encouraged to supply enough DASD and memory so that I/O 

was never a bottleneck. Thus a simple way to eliminate 1/0 

wait time was chosen so that the processors would always 

closely approach 100% busy. 

The NOMAD2 benchmark performed four NOMAD2 routines 

sequentially: 1) a database dump, 2) a database load, 3) a 

series of change requests against the database, and 4) a 

list request (report) against the database. The database 

consisted of about 3000 records with each VM/CMS user 

possessing and exercising against his own copy of the 

database. The benchmark provided heavy loading. One VM/CMS 

virtual machine running the NOMAD benchmark on a dedicated 

IBM 3083EX processor required 80 seconds of CPU time, 100 

seconds of DASD service time (3380 non-cache), and issued 

7288 1/0 requests. 

Using the NOMAD2 benchmark and the chosen workload, the 

Internal Throughput power of the tested processors was 

measured and tables of ITR values for the processors tested 

were compiled. 

17 
TABLE 1 shows ratios of ITRs for the various processors 

compared to the IBM 3083EX. The data was based on single 

CPU service times for one user. The ratios would have to be 

multiplied by an appropriate constant to project the ITR for 

the dyadic processors involved. A projection multiplier of 

1.8 was used as a conservative estimate based upon previous 

benchmarking experiences. 

TABLE 1 

ITR RATIOS 

PUBLISHED 

NO. MIPS FROM 

AHDAHL 

PROCESS 

AMDAHL 

5840 

OR 

5850 

- 

AHDAHL 5860 

AMDAHL 5867 

AMDAHL 5870 

AMDAHL 5880 

NAS 9050 

NAS 9060 

IBM 3083 EX 

IBM 3083 JX 

IBM 3081 KX 

3090/200 

1 

1 

1 

2 

2 

2 

1 

1 

1 

1 

2 

2 

1.78 

2.41 

3.12 

4.25 

5.47 

5.36 

2.17 

2.73 

1.00 

2.36 

4.16 

7.00 

7.4 

9.8 

12.40 

15.40 

21.00 

21.70 

8.10 

10.20 

4.06 

8.12 

15.40 

27.70 

STATISTICS FOR THE ABOVE DATA: 

Correlation between MIPS and ITR is (0.996) Regression equation: [HIPS=(3.93)*(ITR)-0.2541 Standard error of the estimate is (0.62) 

NOTE: A dyadic rocessor has two CPUs that share memory and one set of 

1/0 channels. 

The AMDAHL 5870 ran as a dyadic processor, while the AMDAHL 5hO ran as 

a dual processor. 

A tual processor has two separate sets of 1/0 rocessors 

All other two CPU systems listed above are dyadic. 

Linear regression analysis was performed on the data in 

TABLES 1 through 5. In TABLE 1, published MIPS from CMI 

Corporation were correlated with projected ITR ratios from 

the tested computers. The correlation was high positive 

(0.996) and demonstrates that published MIPS are a useful 

predictor of projected ITR across the span of computers 

tested given the test workload. 

TABLE 2 presents all possible ratios between MIPS and 

ITR for each processor listed in TABLE 3. The purpose of 

this table is to show that the MIPS ratio between two 

individual machines in the test group is a useful predictor 

of the projected ITR between the same two machines. For 

18 
example, if one were planning to upgrade from an IBM 3083EX 

to an AMDAHL 5870, one would look at the MIPS (Y line) ratio 

and find 5.17. One would use 5.17 to estimate the CPU power 

ratio between the 5870 and the 3083EX. The projected ITR 

between the two machines is 5.47. In this case our estimate 

(5.17) is 5.5% under the projected ITR (5.47). 

TABLE 2 

3090 

ITR X 1.78 2.41 3.12 4.25 5.47 5.36 2.17 2.73 1.00 2.36 4.16 7.00 

HIPS Y 7.40 9.80 12.40 15.40 21.00 21.70 8.10 10.20 4.06 8.12 15.40 27.70 

W?&seso 58he !&u 5870 fiaae9050 eQde83EXw Btlsx L2M 

5840 X 1.36 

Y 1.32 

5850 X 0.74 

Y 0.76 

5860 X 0.57 0.77 

Y 0 60 0.79 

5867 X 0.42 0.57 

Y 0.48 0.64 

5870 X 0.32 0.44 

Y 0,35 0.47 

5880 X 0.33 0.45 

Y 0.34 0.45 

9050 X 0.82 1.11 

Y 0 91 1.21 

9060 X 0.65 0.88 

Y 0.73 0.96 

83EX X 1.78 2.41 

Y 1.82 2 41 

83JX X 0.75 1.02 

Y 0.91 1.21 

81KX X 0.43 0.58 

Y 0 48 0.64 

3090 X 0.25 0.34 

J200 Y 0.27 0 35 

1.76 

1.68 

1.29 

1.27 

0.73 

0.81 

0.57 

0.59 

0.58 

0.57 

1.44 

1 53 

1.14 

1.22 

3.12 

3.05 

1.32 

1.53 

0.75 

0.81 

0.45 

0.45 

2.39 

2.08 

1.76 

1.57 

1.36 

1.24 

0.78 

0.73 

0.79 

0.71 

1.96 

1.90 

1.56 

1.51 

4.25 

3.79 

1.80 

1.90 

1.02 

1.00 

0.61 

0.56 

3.08 

2.84 

2.27 

2.14 

1.75 

1.69 

1.29 

1.36 

1.02 

0.97 

2.52 

2.59 

2.01 

2.06 

5.47 

5.17 

2.32 

2.59 

1.32 

1.36 

0.78 

0.76 

3.02 1.22 

2.93 1.09 

2.23 0.90 

2.21 0.83 

1.72 0.70 

1.75 0.65 

1.26 0.51 

1.41 0.53 

0.98 0.40 

1.03 0.39 

0.40 

0.37 

2.47 

2.68 

1.97 0.80 

2.13 0.79 

5.36 2.17 

5.34 2.00 

2.27 0.92 

2.67 1.00 

1.29 0.52 

1.41 0.53 

0.77 0.31 

0.78 0.29 

1.54 0.56 1.33 

1.38 0.55 1.10 

1.13 0.41 0.98 

1.04 0.41 0.83 

0.87 0.32 0.76 

0.82 0.33 0.65 

0.64 0.24 0.56 

0.66 0.26 0.53 

0.50 0.18 0.43 

0.49 0.19 0 39 

0.51 0.19 0.44 

0.47 0.19 0.37 

1.26 0.46 1.09 

1.26 0.50 1.00 

0.37 0.87 

0.40 0.80 

2.73 2.36 

2 51 2.00 

1.16 0.42 

1.26 0.50 

0.66 0.24 0.57 

0.66 0.26 0.53 

0.39 0.14 0.34 

0.37 0.15 0.29 

2.34 

2.08 

1.73 

1.57 

1.33 

1.24 

0.98 

1.00 

0.76 

0.73 

0.78 

0.71 

1.92 

1.90 

1.52 

1.51 

4.16 

3.79 

1.76 

1.90 

0.59 

0.56 

3.94 

3.74 

2.91 

2.83 

2.24 

2 23 

1.65 

1.80 

1.28 

1.32 

1.31 

1.28 

3.23 

3.42 

2.57 

2.72 

7.00 

6.82 

2.97 

3.41 

1.68 

1.80 

The correlation between all ratio pairs of 0.9938, a 

standard error of the estimate of 0.1215, and an average 

percent absolute difference of 6.7% between the MIPS ratios 

and the ITR ratios verifies that published MIPS figures are 

a useful. predictor of projected ITR for the tested processors running the test workload (the benchmark). 

19 
TABLE 3 shows ITR ratios for the tested processors based 

upon the results of running the NOMAD2 benchmark on 20 

concurrent VM/CMS virtual machines. In all cases the 20 

concurrent users pushed the target machines to at least an 

average 95% CPU busy during the total benchmark run. Hence 

this table is an estimate of ITR that takes into consideration the concerns expressed by Andrew Lockey of AMDAHL t91. 

Published MIPS from CMI Corporation were correlated with 

measured ITR from the tested machines where the tested 

machines were actually operated very close to saturation. 

The correlation was again high positive (0.9631 and demonstrates that published MIPS are a useful predictor of measured ITR across the test group running the test workload. 

TABLE 3 

HEASURED ITR 

NO PUBLISHED HIPS 

pROCESSOR cpi]s JTR RATIOg. FROM CHI CORP. 

AHDAHL 5840 

AHDAHL 5850 

AHDAHL 5860 

AHDAHL 5867 

AHDAHL 5870 

AHDAHL 5880 

NAS 9050 

NAS 9060 

IBH 3083 EX 

IBH 3083 JX 

IBH 3081 KX 

3090/ 200 

1 

1 

1 

2 

2 

2 

1 

1 

1 

1 

2 

2 

1.76 

2.34 

2.87 

4.55 

5.09 

4.81 

2.11 

2.67 

1.00 

2.38 

4.47 

5.81 

7.40 

9.80 

12 40 

15.40 

21.00 

21.70 

8.10 

10.20 

4.06 

15.40 

27.70 

a. 12 

STATISTICS FOR THE ABOVE DATA: 

Correlation between HIPS and ITR is (0.963) Regression equation: [HIPS=(4.39)*(ITRf-1.131 Standard error of the estimate is (1.81) 

NOTE: A dyadic rocessor has two CPUs that share memory and one set of 

1/0 channels. 

The AHDAHL 5870 ran as a dyadic processor, while the AHDAHL 5g80 ran as 

a dual processor. 

A Sua1 processor has two separate sets of 1/0 rocessors 

All other two CPU systems listed above are dyadic. 

The purpose of TABLE 4 is to show that the MIPS ratio 

between two individual machines in the test group is a 

useful predictor of the measured ITR between the same two 

machines. For example, if one were planning to upgrade from 

an IBM 3083EX to an AMDAHL 5870, one would look at the MIPS 

(Y line) ratio and find 5.17. One would use 5.17 to estimate the measured ITR ratio between the two machines. The 

20 
measured ITR ratio between the 5870 and the 3083EX is 5.09. 

In this case our estimate is 2% under the measured ITR 

ratio. 

TABLE 4 

PROCESSOR COHPARISONS 

USING 

ITR AND HIPS RATIOS 

3090 

5840 5850 5860 5867 5870 5880 9050 9060 83EX 83Jy /200 

ITR X 1.76 2.34 2.87 4.55 5.09 4.81 2.11 2.67 1.00 2.38 4.47 5.81 

HIPS Y 7.40 9.80 12.40 15.40 21.00 21.70 8.10 10 20 4.06 8.12 15.40 27.70 

5840 X 1.32 

Y 1.32 

5850 X 0.76 

Y 0.76 

5860 X 0.61 0.81 

Y 0.60 0.79 

5867 X 0.39 0.51 

Y 0.48 0 64 

5870 X 0.35 0.46 

Y 0.35 0.47 

5880 X 8.37 0.49 

Y 0.34 0 45 

9050 X 0.84 1.11 

Y 0 91 1 21 

9060 X 0.66 0.87 

Y 0 73 0.96 

83EX X 1.76 2.34 

Y 1.82 2.41 

83JX X 0.74 0.98 

Y 0 91 1.21 

81KX X 0.39 0.52 

Y 0 48 0.64 

3090 X 0.30 0.40 

1200 Y 0.27 0 35 

1.63 

1.68 

1.23 

1.27 

0.63 

0 81 

0.56 

0.59 

0.60 

0.57 

1.36 

1 53 

1.07 

1.22 

2.87 

3 05 

1.21 

1.53 

0.64 

0.81 

0.49 

0.45 

2.58 

2 08 

1.94 

1.57 

1.58 

1 24 

0.89 

0.73 

0.94 

0.71 

2.16 

1 90 

1.70 

1.51 

4.55 

3.79 

1.91 

1.90 

1.02 

1 00 

0.78 

0.56 

2.89 

2.84 

2.18 

2 14 

1.77 

1.69 

1.12 

1.36 

1.06 

0.97 

2.42 

2 59 

1.90 

2.06 

5.09 

5 17 

2.14 

2.59 

1.14 

1.36 

0.88 

0.76 

2.73 1.19 

2.93 1.09 

2.06 0.90 

2 21 0.83 

1.68 0.73 

1 75 0 65 

1.06 0.46 

1 41 0.53 

0.94 0.41 

1.03 0 39 

0.44 

0 37 

2.28 

2 68 

1.80 0.79 

2 13 0.79 

4.81 2 11 

5.34 2 00 

2.02 0.89 

2.67 1 00 

1.08 0.47 

1.41 0 53 

0.83 0.36 

0.78 0 29 

1 52 0.57 1.35 

1.38 0.55 1.10 

1 14 0.43 1 02 

1 04 0.41 0 83 

0.93 0.35 0.83 

0 82 0 33 0.65 

0.59 0.22 0.52 

0.66 0 26 0.53 

0 53 0.20 0.47 

0 49 0.19 0 39 

0 56 0.21 0.49 

0.47 0.19 0.37 

1.27 0.47 1.13 

1 26 0.50 1.00 

0.37 0.89 

0 40 0.80 

2 67 2.38 

2 51 2 00 

1.12 0.42 

1 26 0.50 

0.60 0.22 0.53 

0 66 0.26 0 53 

0.46 0.17 0.41 

0 37 0.15 0 29 

2.53 

2 08 

1.91 

1 57 

1.56 

1 24 

0.98 

1 00 

0.88 

0 73 

0.93 

0.71 

2.12 

1.90 

1.67 

1 51 

4.47 

3.79 

1.88 

1 90 

0.77 

0 56 

3.29 

3.74 

2.49 

2.83 

2.02 

2 23 

1.28 

1.80 

1 14 

1 32 

1.21 

1.28 

2.76 

3.42 

2.17 

2 72 

5.81 

2.44 

3.41 

1.30 

1.80 

6.82 

STATISTICS FOR THE ABOVE DATA: 

Correlation between MIPS RATIOs and ITR RATIOs is (0.974) Regression equation: [MIPS RATIO=(l.O396)*(ITR RATIO)-0.02291 Standard error of the estimate is (0.2505) 

AVERAGE (Y) DIFFERENCE IN MIPS AND ITR RATIOs is 13.1% 

HAXIMUH (Y) DIFFERENCE IN MIPS AND ITR RATIOs is 40.7% 

The correlation of 0.9735 with a standard error of the 

estimate of 0.2505 and the average percent absolute difference of 13% between the MIPS ratios and the tested ITR 

ratios verifies that published MIPS figures are a useful 

predictor of tested ITR for the target processors given the 

test workload (benchmark). 

21 
TABLE 5 compares the results of TABLES 1 and 3. TABLE 1 

presents projected ITR, based upon the CPU required for one 

VM/CMS machine running on a dedicated processor. TABLE 3 

presents an actual measure of ITR ratios (3083 Ex = 1) when 

the tested processors where pushed to the limit. The 

purpose of this table is to show that there is a high 

positive correlation between projected ITRs and measured 

ITRs (0.978). However, in most cases, the projected ITR is 

larger than the measured ITR. 

TABLE 5 

HEASURED ITR RATIOS 

vs 

PROJECTED-ITR RATIOS 

PROJECTED HEASURED 

RATIO OF 

HEASURED 

PROCESSORk IJJuAma l2lumQa & 

1 

1 

1 

2 

2 

2 

1 

1 

1 

1 

2 

2 

1.78 

2.41 

3.12 

4.25 

5.47 

5.36 

2.17 

2.73 

1.00 

2.36 

4.16 

7.00 

1.76 

2.34 

2.87 

4.55 

5.09 

4.81 

2.11 

2.67 

1.00 

2.38 

4.47 

5.81 

STATISTICS FOR THE ABOVE DATA: 

Correlation between: 

PROJECTED ITR RATIO and HEASURED ITR RATIO is (0.978) Regression equation: [TESTED ITR=(0,843)*(HAX ITRlt0.3961 

Standard error of the estimate is (0.3024) 

NOTE: rocessor has two CPUs that share memory and one set of 

1/0 channels. 

The AHDAHL 5870 ran as a dyadic processor, while the AHDAHL 5080 ran as 

a dual processor. 

A dyadic A ha1 processor has two separate sets of 1/0 rocessors 

All other two CPU systems listed above are dyadic. 

A good conservative rule of thumb for using MIPS to 

predict ITR for sizing purposes (where you wish to upgrade 

from a smaller to a larger machine) is to take the MIPS 

ratio between the target and current machines and reduce it 

by 20 percent. The 20 percent reduction will cover most of 

the cases where your estimate is too small, and will add a 

comfortable buffer in others cases. 

22 
3.2 ANALYSIS OF REPORTED BENCHMARK DATA 

3.2.1 S_insle - FunctiuPS Nwrs 

Let's examine some single-function MIPS numbers. Dr. 

David S. Lindsay of National Advanced Systems (NASI has 

developed a set of benchmarks to measure CPU speed. Dr. 

Lindsay has run his selected set of 117 tests on various 

CPUs [131,[141. Table 6 contains results from some of Dr. 

Lindsay's tests (IBM plug-compatible machines plus DEC). 

The results show the variability of various machine level 

operations across several different vendor's products and 

contains comparative performance figures in microseconds. 

TABLE 6 

INSTRUCTION TIMES (MICRO-SECONDS) 

AND PUBLISHED MIPS 

COLUMN 1 2 3 4 5 6 7 8 9 10 

AMDAHL 5860 0.07 0.45 0.82 0.33 1.80 0.05 0.17 0.17 1.34 12.40 

AHDAHL 5890 0.05 0.40 0.40 0.22 0.93 0.02 0.10 0.05 0.10 17.22 

IBH 3081-KX 0.13 0.90 1.18 0.49 2.19 0.06 0.33 0.17 0.39 8.12 

IBH 30901200 0.09 0.72 0.59 0 24 1.20 0.03 0.14 0.06 0 11 15.40 

IBM 4341-12 0.52 2.72 8.48 2.18 10.07 0.43 2.30 0.87 4.23 1.50 

IBH 4361-5 0.52 4.10 8.94 2.05 9.39 0.35 3.20 2.60 2.05 1.33 

IBH 4381-2 0.32 2.65 4.00 1.29 6.24 0.25 0.99 0.51 0.37 2.70 

NAS XL80 0.04 0.22 0.39 0.29 1.12 0.02 0.05 0.03 0 04 25.00 

VAX 111780 1.23 7.52 7.66 2.89 15.80 0.94 2.53 2.93 4.98 1.06 

VAX 8600 0.33 1.64 2.12 0.99 3.68 0.21 0.71 0.45 1.11 4.40 

VAX 111785 1.45 2.67 5.13 2.03 13.25 0.62 1.54 1.81 3.22 1.70 

Dr. Lindsay developed his benchmarks sole1 .to measure CPU speed. And in. those cases where the processors were mulgi-processors, the speed of a single CPU was measured. Hence MIPS estimates shown are for the 

base (single) processor on those system that are dual-processor system. 

1-4 LARGE 3-D INNER SUBR 1-4 1-4 R-8 R78 CHI 

cpu JsuBBRAYARRAYLOOPC-RADPS1_HULTADDSmnIPS 

TABLE 7 is based upon the data of TABLE 6, the times 

have been inverted and expressed as rates in single-function 

MIPS. I have labeled the columns from 1 to 10 for ease of 

reference. The columns contain performance times for 

1) assignment or movement of data in memory (14 ASGN), 

2) loading a large array that will not fit in most cache 

memories, hence a "cache buster activity" (LARGE ARRAY); 

3) loading a 3-dimensional array (3-0 ARR); 4) branching and 

23 
loop 

call 

and 

PlY 

ing from DO 

s and returns 

Add (I4 ADDS) 

(1-4 MULT); 8 

Loops (INNER LOOP); 5) making subroutine 

(SUBR C-R); 6) performing an integer Load 

; 7) performing in integer load and multi1 performing a real number Load and adds 

(R8 ADDS); 9) performing a real number load and multiply 

(R8 MULT); 10) MIPS number taken from the latest chart 

published by CMI Corporation. In the case of dual processors, the MIPS rating of the base processor was used. 

TABLE 7 

INSTRUCTION RATES (HIPS) AND PUBLISHED HIPS 

COLUHN 1 2 3 4 5 6 7 8 9 10 

CPUASSNARRAPARBBYLOOPC-RADDSnOLTADPSnULTHIPS 

1-4 LARGE 3-D INNER SUBR 1-4 1-4 R8 R-8 CHI 

AHDAHL 5860 14.29 2.22 1.22 3.03 0.56 20.00 5.88 5.88 0.75 12 40 

AHDAHL 5890 20.00 2.50 2.50 4.55 1.08 50.00 10.00 20.00 10.00 17.22 

IBH 3080-KX 7 69 1.11 0.85 2.04 0.46 16.67 3.03 5.88 2.56 8.12 

IBH 3090/200 11.11 1.39 1.69 4.17 0.83 33.33 7.14 16.67 9.09 15.40 

IBH 4341-12 1.92 0.37 0.12 0.46 0.10 2.33 0.43 1.15 0.24 1,50 

IBH 4361-5 1.92 0.24 0.11 0.49 0.11 2.86 0.31 0.38 0.49 1.33 

IBU 4381-2 3.13 0.38 0.25 0.78 0.16 4.00 1.01 1.96 2.70 2.70 

NAS XL80 25.00 4.55 2.56 3.45 0.89 50.00 20.00 33.33 25.00 25.00 

VAX 11/780 0.81 0.13 0.13 0.35 0.06 1.06 0.40 0.34 0.20 1.06 

VAX 8600 3.03 0.61 0.47 1.01 0.27 4.76 1.41 2.22 0.90 4.40 

VAX 11/785 0.69 0.37 0.19 0.49 0.08 1.61 0.65 0.55 0.31 1.70 

Or. Lindsay emphasizes the anomalies that are exhibited 

by the test data. 

... We have pointed out several performance anomalies of the 

machines we have tested, anomalies that could easily dominate 

CPU consumption for some benchmarks or applications. We hope 

that publishing these data will help analysts avoid such 

pitfalls in the future. 

Because of these performance anomalies, we have avoided 

averaging the test results and deriving some kind of "overall 

MIPS" or "power* figure. As was pointed out in the introduction, and as has become clear from the data presented above, 

relative performance varies over a wide range depending on the 

instruction mix. Even among machines with similar architecture, 

large differences occurred. (151 

24 
MIPS as generally used implies an instruction mix. 

Benchmarks also involve the use of instruction mixes, often 

at two levels, a higher language level and the compiled 

(machine code) level. If one wishes to avoid the use of 

instruction mix instruments, one must not only avoid the use 

of MIPS, but also the use of benchmarks which attempt to 

map a real Workload, for real workloads are comprised of 

instruction mixes. 

That there is variance in the test data is understandable. Variance in the performance of machines with different speeds is absolutely to be expected. The sort of 

variance that Dr. Lindsay points out is the variance in the 

ratio of the execution times for different functions across 

different machines. Dr. Lindsay uses as one of his illustrations the performance of the NAS XL-80 versus the 

VAX 11/785. Dr. Lindsay states that: 

... thus the assignment test measures cache speed. The cache 

speeds of the machines tested varied considerably. The fastest 

was the NAS XL-80, and the slowest of those tested was the 

DEC VAX 11/785; they differed by a factor of 36 (a much larger 

spread than the supposed ''HIPS" ratings of the two machines 

would suggest). [161 

An examination of TABLE 7 will reveal what Dr. Lindsay 

is talking about. The ASSN rate for the NAS XL-80 is 25 and 

the ASSN rate for the VAX 11/785 is 0.69. Hence the ratios 

between ASSNs would be 36 (25/0.69). The MIPS ratio between 

the XL-80 (one of its processors) and the VAX 11/785 would 

be 24 (25/1.06). 

But this sort of isolated comparison does not tell the 

whole story. Take a few minutes and examine the results for 

the VAX computers in TABLE 7. One can see that the 1.7 MIPS 

rating for the VAX 11/785 is suspect in view of the test 

data. No single measurement for the VAX 11/785 expressed as 

a MIPS number is equal to or greater than 1.7. The closest 

number to the 1.7 is the value 1.61 in column 6 which 

represents integer load and add performance. In the case of 

the other two VAX entries, only one entry in each line is 

equal to or greater than the MIPS estimate. In each case, 

it is the value in column 6 (integer load and Add). 

25 
On the other hand, four measurements for the XL-80 

(assignment, integer load and add, real time load and add, 

and real time multiply) are equal to or greater than the 

MIPS number. If one is going to measure MIPS, then speed of 

assignment (not just load and add) should probably weigh 

heavily in the estimates. Apparently Dr. Lindsay would 

agree : 

One important set of tests in this benchmark measures assignment 

time, the time required to move data from one location to 

another--clearly a measure of great importance. In fact, 

HacDougall (Ref. 5) has found that fully 50% of the instructions 

executed by production COBOL jobs on IBH systems merely move 

data. This result is probably not unique to either COBOL or IBH 

systems. If the single most important CPU performance measure 

were to be selected, it would probably be assignment time. The 

assignment statements that we timed were the four-byte integers 

(called INTEGER*4 in FORTRAN). 

Because assignment time is so important, we have used it as a 

basis to compare other CPU functions. [161 

It appears that in the case of the VAX processors, the 

load add values would have had to be used almost exclusively 

to derive a MIPS number equal to the estimates used. It is 

not surprising, then, that the ratio between the MIPS 

estimate for the XL-50 and the VAX 11/785 is less than the 

ratio between their corresponding ASSNs. 

When you ask which of two machines is faster, the answer 

may depend upon whether or not you wish to compare the speed 

of specific machine functions or whether you wish to compare 

an aggregate speed of some sort. Look again at TABLE 7 and 

examine the rates for the VAX 11/785 and the IBH 4361-5. 

You will see that the 4361-5 is 2 8 (1.92/0.69) times 

faster on ASSNs than is the 11/785; however, the 11/785 is 

2.1 (0.65/0.31) times faster on 1-4 MULT than is the 4361-5. 

Asking which machine is faster is much like asking which of 

two track teams is faster. Depends upon which race they are 

running. We do not however, hesitate to pick a winner at a 

track meet using a composite result of speeds (finishes) 

from their individual races. 

26 
To argue that we cannot pick the faster of two computers, or the faster among several computers, would be 

analogous to arguing that we cannot pick a winner at a track 

meet simply because different teams are faster in different 

events. We can and must make choices between entities based 

upon composite differences. We do it every day. If Dr. 

Lindsay were to manage a track meet perhaps he would announce at the conclusion of the day's events that "We have 

avoided adding the race results and hence have not selected 

an f'Overall winner" for the track meet." I realize full 

well that computing is not track. There are cases where we 

need to select a computer for a very specialized performance; however, there are many more cases where we do, 

indeed, need to selected a computer for a general (hence 

composite) performance. Knowing all the individual instruction rates in the world will be of no help in the overall 

evaluation of competing processors without the method and 

the resolve to construct an appropriate aggregate evaluation 

device. 

Sometimes, of course, one computer is simply faster in 

every category than another. Look in TABLE 7 and compare 

the entries between the AMDAHL 5890/200 and the 

IBM 3090/200. You will see that the AMDAHL 5890/200 is 

faster in every category than the IBM 3090/200. This, 

however, represents an easy case and in such situations 

performance comparisons are relatively easy. 

3.2.2 baression hlvsis and Weighted Instruction Rates 

Multiple regression analysis was performed on the data 

presented in TABLE 7. The first multiple regression that 

was performed included the eight IBM plug-compatible processors listed in TABLE 7, in addition to four selected 

instruction rates along with the published MIPS numbers. 

The instruction rates that were selected were: 

1) assignment of data in memory (column 1: 1-4 ASSN) 

2) branching and looping from DO loops (column 4:INNER LOOP) 

3) integer load and add (column 6: 1-4 ADDS) 

4) real number load and adds (column 8: R-8 ADDS) 

27 
TABLE 8 shows the results of the multiple regression 

analysis performed upon these variables (8 processors, 4 

instruction rates, and the published MIPS number). 

The coefficient of multiple correlation of 0.9986 

(TABLE 8) establishes quite well that when the four variables considered are taken as a set, they form a useful 

predictor of the corresponding MIPS number. However, the 

question that needs to be addressed is how well the variables would predict MIPS if they were given a 'realistic' 

workload weight. 

TABLE 8 

RESULTS OF XULTIPLE REGRESSION CORRELATION 

FOR 8 PROCESSORS 

Actual VS Predicted 

Observation 

Number Actual 

12.40000 

17.22000 

8 12000 

15.40000 

1.50000 

1.33000 

2.70000 

25.00000 

Predicted 

12.49815 

17 51085 

7,13145 

15.45532 

1.74559 

1.19428 

3.21678 

24.91757 

D i f f er ence 

-0 09815 

-0.29085 

0 98855 

-0 05532 

-0 * 24559 

0.13572 

-0.51678 

0.08243 

%Difference 

-0 a 79156 

-1 68904 

12,17427 

-0.35923 

-16.37288 

10.20449 

-19.14005 

0 32973 

Regression Equation: 

Y=-0 488l229tO.564O717*X1+2.867562*X2t-O.3588757*X3tO.58O6958*X~ 

Coefficient of Determination is (0.9972268) 

Coefficient of Multiple Correlation is (0 9986) 

STD Deviation of Estimate is (0 6897138) 

28 
To generate TABLE 9, postulated instructions weights 

were assigned to each of the four instruction categories 

used in TABLE 8. TABLE 9 shows the aggregated instructions 

rates calculated from the separate instruction rates and 

corresponding weights; in addition, all possible ratios 

between the aggregated rates for each of the processors and 

the corresponding published MIPS numbers are shown. The 

weights selected for each of the categories were: 

1) 50% for assignment (1-4 ASSN) 

2) 15% for branching and looping (INNER LOOPS) 

3) 20% for integer load and add (1-4 ADDS) 

41 15% for real number load and adds (R-8 ADDS). 

TABLE 9 shows that across the eight IBM plug-compatible 

computers, published MIPS numbers are a useful predictor of 

the performance of the machines executing a 'realistic' 

workload instruction mix. The Correlation for all the pairs 

of ratios is 0.9855 (TABLE 9). One of the other pertinent 

statistics is the average percent difference in the ratios 

across all the pairs. This average was based upon the 

average absolute difference. The average difference of 13% 

(TABLE 9) with a maximum difference of 38% (TABLE 9) shows 

that the MIPS ratio between two machines (across the eight 

compared) is a useful predictor of the performance ratio of 

the same two machines (given the postulated instruction 

mix). 

Obviously other weights (workloads] could be justified; 

however, Dr. Lindsay is probably correct in his opinion of 

the importance of the assignment function (161. 

29 
TABLE 9 

WEIGHTED INSTRUCTION RATES 

vs 

PUBLISHED HIPS 

WITHOUT DEC PROCESSORS 

COHPUTER 5860 5899.sl-Kx 439L m 4;iBf.XL-80 

RATE 1: 14.29 20.00 7.69 11.11 1.92 1.92 3.13 25.00 

WEIGHT 1: 0.50 0.50 0 50 0.50 0.50 0.50 0.50 0.50 

RATE 2: 20.00 50.00 16.67 33.33 2.33 2.86 4.00 50.00 

WEIGHT 2: 0.20 0 20 0.20 0.20 0.20 0.20 0.20 0.20 

RATE 3: 5.88 20.00 5.88 16.67 1.15 0.38 1.96 33.33 

WEIGHT 3: 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 

RATE 4: 3.03 4.55 2.04 4.17 0.46 0.49 0.78 3.45 

WEIGHT 4: 0.15 0 15 0.15 0.15 0 15 0.15 0.15 0.15 

AGGREGATE 

RATE: 12.48 23.68 8.37 15.35 1.67 1.66 2.78 28.02 

HIPS Y 12.40 17.22 8.12 15.40 1.50 1.33 2.70 25.00 

5860 X 

Y 

5890 X 

Y 

3081 X 

Y 

3090 X 

Y 

4 34 1 1 12X 

Y 

434115 X 

Y 

438112 X 

Y 

XL180 X 

Y 

0.53 

0.72 

1.49 

1.53 

0.81 

0.81 

7.49 

8.27 

7.51 

9.32 

4.50 

4.59 

0.45 

0.50 

1.90 

1.39 

2.83 

2.12 

1.54 

1.12 

14.20 

11.48 

14.25 

12.95 

8.53 

6.38 

0.85 

0.69 

0.67 

0.65 

0.35 

0.47 

0.55 

0.53 

5.62 

5.41 

5.03 

6.11 

3.01 

3.01 

0.30 

0.32 

1.23 

1.24 

0.65 

0.89 

1.83 

1.90 

9.20 

10.27 

9.23 

11.58 

5.53 

5.70 

0.55 

0.62 

0.13 

0.12 

0.07 

0.09 

0.20 

0.18 

0.11 

0.10 

1.00 

1.13 

0.60 

0.56 

0.06 

0.06 

0.13 

0.11 

0.07 

0 08 

0.20 

0 16 

0.11 

0.09 

1.00 

0.89 

0.60 

0.49 

0.06 

0.05 

0.22 

0.22 

0.12 

0.16 

0.33 

0.33 

0.18 

0.18 

1.66 

1.80 

1.67 

2.03 

0.10 

0.11 

2.24 

2.02 

1.18 

1 45 

3.35 

3 08 

1.83 

1.62 

16,80 

16.67 

16.85 

18 80 

10.09 

9.26 

Correlation for all XY ratios (r) is (0.9855) 

Regression equation for XY RATIOS: Y=(O.9982*X)t(0.0300) Standard error of the estimate is (0.7513) 

AVG X =3.0534 

AVG Y =3.0780 

SD (X)=4.3720 

SD (Y)-4.4284 

AVG ABS DIFFERENCE=0.3850 AVG (Y) DIFFERENCE=13.22% 

SD ABS DIFFERENCE=0.6457 SD (Y) DIFFERENCE=10.00% 

HAX ABS DIFFERENCE=2.7224 HAX (Y) DIFFERENCE=38.00% 

The second multiple regression that was performed on the 

data from TABLE 7 included all eleven processors listed, and 

the four selected separate instruction rates along with the 

30 
MIPS estimate. TABLE 10 shows the results of the multiple 

regression analysis performed upon these variables (eleven 

processors, four instruction rates, and the published MIPS 

number 1. 

TABLE 10 

RESULTS OF MULTIPLE REGRESSION CORRELATION 

FOR 11 PROCESSORS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12.40000 

17.22000 

8.12000 

15.40000 

1.50000 

1.33000 

2.70000 

25.00000 

1 06000 

4 40000 

1.70000 

12.48550 

17.42226 

7.26150 

15.56545 

1.99833 

1.43508 

3.44757 

24.90757 

1.05906 

3.92895 

1.31873 

-0.08550 

-0.20226 

0 85850 

-0.16545 

-0.49833 

-0.10508 

-0.74757 

0.09243 

0.00094 

0.47105 

0.38127 

-0.68953 

-1.17457 

10.57264 

-1.07438 

-33.22220 

-7.90088 

-27.6876 1 

0.36971 

0.08903 

10.70579 

22.42785 

Regression Equation: 

Coefficient of Determination is (0.9969736) 

Coefficient of Hultiple Correlation is (0.9985) 

STD Deviation of Estimate is (0.5783098) 

Y~~0.2006487+0.5457429*X1+2.860474*X~t~0.3632949*X~t0~592882*X~ 

The coefficient of multiple correlation of 0.9985 

(TABLE 10) establishes quite well, in this case also, that 

when the four variables considered are taken as a set, they 

form a useful predictor of the corresponding MIPS number 

across IBM and DEC processors. Again, the question that 

needs to be addressed is how well the variables would 

predict MIPS if they were given a 'realistic' workload 

weight. 

TABLE 11 shows that across the eleven listed computers 

(eight IBM plug-compatible machines plus three DEC machines) 

published MIPS numbers are nefL a very useful predictor of 

the performance of the machines executing a 'realistic' 

workload instruction mix. The Correlation for all the pairs 

of ratios is down to 0.9182 (TABLE 11). The other pertinent 

statistic is the average percent difference in the ratios 

across all the pairs. 

31 
TABLE 11 

VEIGHTED INSTRUCTION RATES VS PUBLISHED HIPS 

91 I WITH DEC PROCESSORS 

WEIGHT 1: 14.29 20.00 7.69 11.11 1.92 1.92 3.13 25.00 8.81 3.03 0.69 

RATE 1: 0.50 0 50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.50 

UEIGHT 2: 20.00 50.00 16.67 33.33 2.33 2.86 4.00 50.00 1.06 4.76 1.61 

RATE 2: 0.20 0.20 0.20 0.20 0.20 0.20 0.20 0.20 0.20 0.20 0.20 

UEIGHT 3: 5.88 20.00 5.88 16.67 1.15 0.38 1.96 33.33 0.34 2.22 0.55 

RATE 3: 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 

UEIGHT 4: 3.03 4.55 2.04 4.17 0.46 0.49 0.78 3.45 0.35 1.01 0.49 

RATE 4: 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 0.15 

AGR RATE: 12.48 23.68 8.37 15.35 1.67 1.66 2.78 28.02 0.72 2.95 0.82 

AGR HIPS: 12.40 17.22 8.12 15.40 1.50 1.33 2.70 25.00 1.06 4.40 1.70 

0.24 0.07 

0.35 0.14 

0.12 0.03 

0.26 0.10 

0.35 0.10 

0.54 0 21 

0.19 0.05 

0.29 0.11 

1.77 0.49 

2.93 1.13 

1.78 0.50 

3.31 1.28 

1.06 0.30 

1.63 0 63 

0.11 0.03 

0.18 0,07 4.10 1.14 

4.15 1.60 

0.28 

0.39 

3.59 

2.59 

The average difference of 37% with a maximum value of 

184% shows that the MIPS ratio would not be a useful predictor of the performance ratio of two machines (given the 

postulated instruction mix) when you span IBM, AMDAHL, NAS, 

and DEC. If one examines the aggregate rate row one will see 

why the inclusion of the DEC results brought the correlation 

down (from 0.9855 to 0.91821. If the aggregate rates are 

taken as reasonable estimates of MIPS, one can see that the 

VAX processors are seriously overstated. 

32 
SECTION 4 

CONCLUSIONS 

The results of the analyzed test data clearly show (with 

correlations above 0.97) that for a given instruction set 

such as IBM compatible, there is an excellent correlation 

between published MIPS and benchmark data in a complex data 

base environment such as NOMAD2 running under VM/CMS. This 

is a counter example to the generally held belief that MIPS 

are meaningless. 

If other words, if one wishes to estimate the internal 

throughput performance of an IBM compatible mainframe where 

the 1/0 environment is not constrained, then published MIPS 

numbers are a very useful predictor of internal throughput 

performance. 

The results of the analysis also show that published 

MIPS correlate well (0.986) with aggregated instruction 

rates. The average percent difference in the corresponding 

ratios of MIPS and the aggregated instructions rate (13.22%) 

is approximately equal to the average percent difference in 

the ratios of MIPS to measured ITR (13.1%). The analysis 

also showed, however, that when DEC processors were considered along with the IBM plug compatibles, the average 

percent difference in MIPS and aggregate instruction ratios 

became quite large (36.7%). 

These results simply serve to verify the accuracy and 

sensibility of using MIPS as a capacity planning number when 

your environment is homogenous (in this case IBM compatible) 

and when you are aware that you will have to engineer 

suitable 1/0 and memory subsystems to support whatever 

processing power you put into place. 

One must not place undue confidence in a MIPS number 

across vendor lines. Also, one must not place undue confidence in a MIPS number if one is trying to predict delivered 

power in support of non-homogenous workloads. This would 

include the case of comparing two identically configured 

processors from the same vendor running different operating 

systems (such as IBM'S VM/CMS and MVS). One faces the 

probability that two different operating systems will result 

in the generation of different machine level instruction 

mixes even if the higher level load is identical. 

33 
Comparing published MIPS figures for microprocessors, 

minis, and mainframes may serve no useful purpose because a 

micro, a mini, and a mainframe do not have comparable 1/0 

and memory support mechanisms. If one can not feed the 

sawmill, or get the cut lumber off the receiver tray, then 

it does no good to make the blade turn faster. This paper 

has presented evidence that published MIPS for IBM 

plug-compatibles and published MIPS for DEC are inadequate 

for predicting the actual performance differences of the two 

classes of machines. 

The use of MIPS as an estimator becomes dangerous when 

one forgets what he is trying to estimate. With MIPS, the 

sensible planner is trying to get a ballpark estimate of 

internal throughput power of a processor operating in a non 

I/O constrained environment across reasonably similar 

processors with postulated similar workloads. If a planner 

remembers these caveats, then a MIPS estimate can be and 

usually is very useful. 

34 
REFERENCES 

[ll 

L31 

[81 

193 

[lo1 

Dzelzgalvis, I., "Performance Comparison Measures for 

Computer Systems, '' muter Performance Evaluat ion 

Users GrouD ; ed. by James E. Weatherbee,National Bureau 

of Standards Special Publication 500-528 pp. 163-164 

(1979). 

WadSWOrth, E., "A Systematic Approach to Obtaining Better Online Response Time," CMG '86 Int ernat ional 

conference on Manaaement an d Performanc e Evaluation of 

ComDuter Svstems, pp. 831 (December 1986). 

Smith, R.D., "Vector Processing and the Performance 

on 

Panacremen mouter 

Analyst, If CMG 86. International Conference 

t and Performance E valuation of Co 

Svstems. pp. 740 (1986). 

HOCkney, R. w., Jesshope, c. R.8 Par allel C om~u ter s I 

Adam Hilger Ltd, Bristol, Great Britain (1981). 

Hockney, R. W., Jesshope, C. R., Parallel C omouters, 

Adam Hilger Ltd, Bristol, Great Britain, pp. 51 (1981) 

HOCkney, R. w., Jesshope, c. R., Parallel ComDuters, 

Adam Hilger Ltd, Bristol, Great Britain, pp. 56 (1981) 

Hockney, R. W., Jesshope, C. R., Parallel ComDuters, 

Adam Hilger Ltd, Bristol, Great Britain, pp. 123 

(1981). 

Jordan, H.F, "HEP Architecture, Programming and 

ComDutation: HEP 

ercomnuter and its ADDliCatlOnS, ed. by J.S. 

Performanceff, Parallel MIMD 

Ealik; MIT Press, Cambridge, Massachusetts, pp. 1-40 

(1985). 

Lockey, A. 8 "Large Systems Performance Studies in 

on Manaae Comoute r 

VM/CMS Environments, If CMG 

ment and Performance Evaluation of 

Svstems. pp. 217-219 (1986). 

f 86 ternational Conference 

MacKay, A. F., Frro w's Theorem, The Paradox of 

Social Ch oice, Yale University Press, New Haven, 

Connecticut (1980). 

35 
REFERENCES (Concluded) 

1111 

f121 

[ 131 

[141 

1151 

t 161 

(171 

I181 

I191 

03. Generated Code Perfarmance: BeTest Results for IntermeWics I IBM - e Ada 

Comoilers, Intermetrics Inc. (May 1987). 

. ce: BeTest Results for - s' IBM Real - T A& 

wilers, PIWG RESULTS (INTERMETRICS VS. DEC), 

Intermetrics Inc. (May 1987). 

Lindsay, D. S., ffCPU Performance Comparisons Among 

Vendors," CMG '86. Inter- Conference on 

Panaaement and Performance Evaluation of ComDuter 

Systems. pp. 179-186 (1986). 

Lindsay, D. S., Bell, T. E., "Directed Benchmarks 

for CPU Architecture Evaluation," CMG '86- 

Snterna tional Conferen ce on Manauement and P er formance 

Evaluation of C omDuter Svs temg A pp. 379-384 ( 1986 1. 

Lindsay, D, S., "CPU Performance Comparisons Among 

Vendors, If CMG '86, Int ernat ional Confe rence 04 

Manawment and Performance Evaluation of ComDuter 

Sys t ems, pp. 183,184 (1986). 

Lindsay, D. S., "CPU Performance Comparisons Among 

Vendors," CMG '86, In ternational Confer ence on 

Panauemen t and Per formance F: valuation of Co mouter 

Svstems, pp. 179 (1986). 

Serlin, 0. , "MIPS, DHRYSTONES, and Other Tales", 

DATAMATION, pp. 112-118 (June 1986). 

Ferrari, D. 8 CamDuter Svstw Performance 

Evaluatioq 8 Prentice-Hall, Inc. Englewood Cliffs, New 

Jersey, pp. 237 (1978). 

Well, P., "Inside the IBM PCs, Intel's 80386 

Architecture,", -8 Volume 11, Number ll, pp. 98 

(1986). 

36 
P 

DISTRIBUTION LIST 

MITRE 

B;Le 

E. L. Key 

A. J. Tachmindji 

C. A. Zraket 

B. M. Horowitz 

pL2 

J. J. Fearnsides 

S. W. Gouse 

e22 

Technical Report Center (2) 

DllO 

E. S. Herndon 

D. A. MacQueen 

RUa 

W. N. Bays 

S. A. Bell 

K. B. Hennigan 

W. P. Kincy 

T. R. Mitchell 

F. M. Richards 

J. K. Richardson (25) 

Systems Engineering (15) 

D_116 

M. A. 

D. M. 

E. E. 

s. I. 

L. w. 

P. A. 

J. V. 

J. F. 

R. W. 

Allen (Gren) 

Er b 

Hill (NAHQ) 

Linde 

Morgan 

Nussman (NAHO) 

Pietras (NAHQ) 

Spitzer 

Zears 

J. S. Brown 

P. M. Brown 

P. J. Gregor 

R. M. Jackson 

W. B. Wood 

w12e 

A. H. Ghovanlou 

L. L. Holmes 

H. M. Strong 

u121 

A. A. Elsaway 

K. H. Miller 

T. R. Mitchell 

A. 0. Zeitlin 

y122 

F. L. Willingham 

37 
DISTRIBUTION LIST (Concluded) 

Records Resources (2) 

F. Abernethy 

J. Arnold 

D. Bell 

W. Bennett 

S. Berthiaume 

B. L. Brady 

J. Broadfoot 

J. Cools 

R, Dorman 

J. Garman 

D. Ingram 

J. Jue 

S. Milo Keathley 

L. Kirbie 

C. Krpec 

S. Leathers 

E. McHenry 

C. Mains 

D. Simanton 

J. Smith 

W. Stewart 

B. Stuckey 

P. Stull 

R, Voiqt 

C. Vowel1 

6, Walker 

D. L. Ward - FD4 

Technical Library (3) 

38