From: alan@nabeth.enet.dec.com (Alan Rollow - Alan's Home for Wayward Tumbleweeds.)
Subject: Re: Some buffer cache questions
Reply-To: alan@nabeth.enet.dec.com (Alan Rollow - Alan's Home for Wayward Tumbleweeds.)
Organization: Digital Equipment Corporation
Date: Wed, 23 Dec 1992 23:37:13 GMT


In article <1992Dec23.022724.27844@pony.Ingres.COM>, rog@Ingres.COM (Roger Taranto) writes:
>
>2) Can anyone explain the various stats displayed in the bufstats command?
>   I assume that readhit is this number of cache hits, and that readahit is
>   the number of hits for some sort of read ahead algorithm.  Is this true,
>   and what is the read ahead algorithm (or at least enough information on
>   how it works so that I can tune the system to take maximum advantage of it).
>
>   Any explanation of the rest of the stats would be useful.

The ULTRIX buffer cache code (V4+) maintains a large group of 
counters in a data structure called bufstats.  This data structure
is declared in /usr/include/sys/buf.h.  Because there are so many
counters it is impracticle to go through them all.  In fact more
than half of them are bflush() statistics.  For this discussion
I'm going to limit myself to the clear read and write counters.

Other groups of counters deal with buffer allocations, blkflush()
operations, binval() operations, binvalfree() operations a couple
of miscellaneous counter and the host of bflush() counters.

There are four read I/O counters.  Two for direct reads and two
for read-ahead I/Os.  The file system code does read-ahead when
it decides that the user is doing sequential I/O.  For ULTRIX
V4.2 this appears simply be when the current LBN to be read
is the previous LBN + 1.  I'm not sure what the unit size of
LBN is, but I'd hope that it was the file system block size.
This has probably been changed with the V4.3 I/O enhancements.

The routine doing the read-ahead (breada) duplicates part of the
work done in bread() to see if the block is in the cache or to
start the I/O for the block if not.  If the block wasn't in the
cache then it calls bread() to get it.  If there is a read-ahead
block, it also starts an read for it, unless it was in the cache.
This way, the odds are good that the bread() (or breada()) for 
the next block will find it in the cache and not have to wait 
for I/O.

The two read counters are for cache hits and misses.  If the block
was in the cache then the readhit counter is incremented.  Otherwise
the readmiss counter is incremented.  The read-ahead counters also
count hits and misses, for just the read-ahead attempts.  This lets
you compare how much read-ahead is being done and within that how
often the read-ahead was in the cache.

For example; using crash(8) to examine the bufstats structure on
a V4.2A system running most file system exercisers, the read
stats I get are:

readhit            1319716 (99.3%)      readmiss              9597 ( 0.7%)
readahit             29552 (97.7%)      readamiss              701 ( 2.3%)

For this I/O load the buffer cache is very effective for both regular
reads and read-ahead reads, though very few of the reads caused read-
ahead (~2%).

The next group of interesting counters are the write and error counters.
There are three counters related to writes in the structure but only
two of them count the actual writes.  The 3rd I'll discuss a bit later.
The two writes counters are incremented in bwrite() depending on whether
the B_ASYNC bit is set in the buffer flags.  It if is set, then the
I/O is done asychronously and the async_write counter is incremented.
If not set, then bwrite waits for the write to complete and increments
the sync_write counter.  All cache writes go through bwrite() (as far
as I can tell), so the sum of these two counters is all the writes.

Using the two read counters and these write counters you can quickly
get an idea of the I/O balance seen by the buffer cache.

The 3rd write counter in this group counts the number of times a
buffer had to be written because the B_DELWRI bit was set in the
buffer flags.  This will only happen for asychronous writes.  Among
the reason it can happen are dirty buffers that need to be flushed 
before being read and the cache running out of buffers.

The last counter in this group is the biodone_errs counter.  This
counts the number of times the B_ERROR bit is set when biodone has
been called for a buffer.

Another write counter in the structure, but not with the previous
three is forcewrite.  This is a subset of delayed writes that
happen when getnewbuf() flushes dirty buffers.  The delwrite
counter also gets incremented in this case.

If I manage to find more spare time I'll try to quickly go through
some of the others.

>
>Thanks,
>-Roger
>rog@ingres.com
>
--
Alan Rollow				alan@nabeth.cxo.dec.com

Newsgroups: comp.unix.ultrix