From: andrewr@highland.oz.au (Andrew Rothwell)
Subject: FS & IO Sub-system Performance, Tuning & Benchmarking.
Organization: Highland Logic
Date: Thu, 16 Apr 92 04:05:09 GMT

OK. I can't stand it anymore!

It concerns me that there appear to be many people who are using
ill equipped or incorrect test programs to characterize I/O throughput
rates of various file and I/O sub-systems. While programs such as IOSTONE
(Princeton) and IOZONE (Tandem) are valid, they have LIMITED APPLICATION.
These programs for example, tell you little about physical level I/O
performance. (Predominantly they exercise the disk block cache manager
(except in sequential operation).) They will tell you almost nothing about
the operation of the disk controller or device. Really, one must develop
test rigs for each level of the File-I/O sub-system.

With this in mind, I have developed two utilities to aid in file system
and physical disk I/O characterization. The file headers I've listed below
give a concise description of their functionality and capabilities. [Even
these tools don't tell the whole picture!]

Isn't it about time the industry got serious about developing and/or
standardizing file system (logical) and disk (physical) I/O benchmarks?
[Yeah right! At least as standard as the number of Unix implementations
in existance... :-( ].

Comments?

Andrew.
P.S: No replies about TPC benchmarks etc; we're talking about benchmarks
at the system level. TPC et al, are a level(s) above this.

------------------------------------------------------------------------------

/* PIO_RW.C,  IO/PIO_RW/SRC,  OS Group,  10/03/92.
/* PIO_RW - Physical disk I/O performance evaluation tool.
/* Copyright (c), 1992. Chainsaw Software, Inc. No fixed address.
/*                      All rights reserved.
/*
/* Description:
/* This program is designed to perform "raw I/O" to the disk. It can
/* perform I/O to various entities, ie track, cylinder, partition, or
/* physical spindle. Read and write paths are tested. Sequential and
/* random I/O are possible. Support for (a)synchronous, multi-block
/* (scatter/gather) and ordered I/O operations is available. Record
/* interleaving is handled. Additionally, a feedback algorithm is
/* implemented to set and dynamically retune the interleaving gap for
/* optimum performance.
/*
/* Some operating systems have an intermediate layer between the file
/* system and actual disk driver, which provide (for example) the
/* capabilities to support MVPs (multi-volume-partitions), record
/* stripe-ing, and disk mirroring. If this layer exists, PIO_RW can
/* be easily made to exercise and test this too.
/*
/* Total elapsed (wall clock) time and CPU time consumed are captured
/* and displayed. These times allow us to characterize various aspects
/* of device drivers, controllers and drives; and to compute physical I/O
/* bandwidths.
/*
/* Running another daemon which can report percentage CPU utilisation
/* for each process should be run simultaneously too.
/*
/* Implementation:
/* PIO_RW devicename -DISK_ENTITY <TRACK [n] | CYLINDER [n] | PARTITION>
/*                   -OPERATION <read | write | write_verify>
/*                   -PATTERN <sequential | random>
/*                   <-INTER_RECORD_GAP records | -DYNAMIC_TUNE>
/*                   [-MEMORY_BLOCK_SIZE page_size_to_xfer]
/*                   [-MULTI_BLOCK number_of_blocks]
/*                   <-ASYNCHRONOUS | -SYNCHRONOUS>
/*                   [-ORDERED]
/*                   [-SYNCHRONIZE_PROCESS sync_id]
/*
/* For scalability v's performance tests; ie where I/O is being initiated
/* to multiple controllers and/or devices, this program should be
/* replicated in a new process for each controller/device instance.
/* All the processes can be made to wait on a synchronizer just prior
/* to the I/O beginning. A small synchronizer notification master program is
/* used to kick all the waiting processes into life.
/*
/* Abnormal Conditions:
/* None.
/*
/* Modification History:
/*   Date    Programmer    Description of modification.
/* 10/03/92  AndrewR       Original coding.
/*
/**/

-------------------------------------------------------------------------------

/* LIO_RW.C,  IO/LIO_RW/SRC,  OS Group,  10/03/92.
/* LIO_RW - Logical (file) I/O performance evaluation tool.
/* Copyright (c), 1992. Chainsaw Software, Inc. No fixed address.
/*                      All rights reserved.
/*
/* Description:
/* This program is designed to exercise the file system I/O paths,
/* notably read and write. It can perform sequential and random I/O.
/* It can operate on user specified file sizes, and if the operating
/* system supports them - multiple file types (eg indexed versus extent
/* or contiguous based files). It can perform multi-block (scatter/
/* gather), (a)synchronous and ordered I/O if requested. Under PR1MOS
/* it used to be able to perform buffered mode I/O AND block mode I/O
/* (ie the file system disk block cache manager is bypassed - I/O goes
/* direct to disk, however file system semantics are still obeyed).
/* Takes extent length (ie number of contiguous disk blocks) as a command
/* line option. Can do pre-allocation of file disk blocks.
/*
/* Total elapsed (wall clock) time and CPU time consumed are captured
/* and displayed. These times allow us to characterize various paths and
/* functional components of the file system; and to compute logical/file
/* I/O bandwidths.
/*
/* Running another daemon which can report percentage CPU utilisation
/* for each process should be run simultaneously too.
/*
/* When used in conjunction with a process wait address monitor, it is
/* possible to uncover bottlenecks due to (file system) resource locking.
/*
/* Could be used with a purpose built disk block cache monitor. This
/* would require probe points being inserted into the cache management
/* code. Good for investigating different page replacement strategies.
/*
/* Implementation:
/* LIO_RW pathname -FILE_SIZE size [-FILE_TYPE type]
/*                 -MODE <block | buffered>
/*                 -OPERATION <read | write>
/*                 -PATTERN <sequential | random>
/*                 [-PRE_ALLOCATE]
/*                 [-EXTENT_LENGTH size_in_records]
/*                 [-MEMORY_BLOCK_SIZE page_size_to_xfer]
/*                 [-MULTI_BLOCK number_of_blocks]
/*                 <-ASYNCHRONOUS | -SYNCHRONOUS>
/*                 [-ORDERED]
/*                 [-SYNCHRONIZE_PROCESS sync_id]
/*
/* For scalability v's performance tests; ie where I/O is being initiated
/* to multiple controllers and/or devices, this program should be
/* replicated in a new process for each controller/device instance.
/* All the processes can be made to wait on a synchronizer just prior
/* to the I/O beginning. A small synchronizer notification master program is
/* used to kick all the waiting processes into life. The disk block cache
/* is purged (note: sync'ed and invalidated) before I/O is initiated.
/* (This means building a new system call into the file system.)
/*
/* Abnormal Conditions:
/* None.
/*
/* Modification History:
/*   Date    Programmer    Description of modification.
/* 10/03/92  AndrewR       Original coding.
/*
/**/

-------------------------------------------------------------------------------




Newsgroups: comp.benchmarks