From: ketil@edb.tih.no (Ketil Albertsen,TIH)
Subject: Re: What does buffering do in smartdrv ?
Organization: T I H / T I S I P
Posting-Front-End: Winix Conference v 92.04.09  1.20 (running under MS-Windows)
Date: Fri, 24 Jul 1992 15:07:19 GMT

In article <3726@svin02.info.win.tue.nl>, luit2@wsinfo11.info.win.tue.nl (Studenten Erik Luit) writes:

>If you start smartdrv.exe, it displays cache yes/no AND buffering yes/no.
>What does buffering do?
>How many buffers should I install?
   
There is some confusion of terms here. The smartdrv has nothing at all to
do with the BUFFERS= in you config.sys; changig that value has no effect
on the 'buffering' setting of smartdrv.

Even if you read MS documentation you may be confused: They repeatedly use
the term "double buffering", which traditionally is interpreted as a way
to use asynchronous I/O: When you have filled one buffer you may give it
to the file system to write, and while waiting for the disk, you may fill
a second buffer. When disk transfer is complete and you have filled another
buffer, you swap buffers with the file system and repeat. You can get some
throughput increase that way, if you are lucky. (Obviously it works with
reads too: You ask the file system to fill the next buffer while your
program consumes data in the preceeding one)

But this is NOT the new MS meaning of the term (although there are a few
details that look similar). The MS thing has been called "windowing" (in
the old days when even teletypes where scarce): It has to do with the
CPU and the disk controller seeing the same data, in a common memory 
"window". The problem is this: If your program requests data transfer to,
say, address 23000, that is NOT the physical address going out on the
memory bus, but a "logical" one that is translated to a physical address, 
say 31100, by the paging/segmentation hardware in the CPU.
But the disk controller sees physical addresses only, so you cannot pass
23000 to it - it would put data in the wrong place, at *physical* 23000.

Now, the driver could inspect page/tables to find the proper physical
address to pass to the the controller. But many high performance
controllers, notably most SCSI ones, prefer/require that integral disk pages
are transferred at a time, and your request might be for a smaller amount
of data, or an area crossing page limits (hence at two different physical
address ranges). Reading entire pages will overwrite neighbouring data in 
your program. Rather, the controller is given the physical address of 
another buffer, where it can safely put entire pages. The CPU and 
opsys/Windows obviously knows the logical and physical address of this 
buffer, which is the "memory window" for passing data.

When transfer is complete, the opsys copies from the window to your
destination address the amount of data you wanted. This is of course done
by the CPU, so the virtual addresses of your buffer and the disk buffer
are used. Two buffers are used, so MS terms it double buffering (but while
traditionally double buffering speeds up, in this case it slows down...).

This is not particular to neither MS Windows nor the Intel CPU; similar
mechanisms are used in almost all machines with memory management hardware
and DMA devices. Actually, the unusual part is that in PCs, you can 
sometimes work around it - but at a noticable cost. The standard "DMA" on
a PC is a very primitive one, compared to typical superminis/mainframes,
and a lot of the details are still controlled by the CPU. The DMA
controller can be seen more or less as a CPU extension; it contains 
registers, counters etc. directly controlled by the CPU, which may
set up the exact number of bytes to transfer, the physical address
to which data goes etc. This simplifies the controllers a lot - they are
not at all concerned about the memory addressing etc. - they just delivers
data a word at a time to the DMA controller, along with a "channel ID"
which the DMA controller uses to find the right memory address and counters.
(One DMA controller can handle a number of controllers.) The DMA controller
can stop sending words to memory at any time, when the counter reaches zero.
These simple interfaces do not intepret logical addresses as physical - 
because they interpret no addresses at all!

But in particular in older PCs with no separate local memory bus, the cost
was high: A data word crossed the bus *twice*: Once from memory to DMA
controller, once from DMA controller to disk controller. This put a practical
limit on less than 500 Kbytes transferred on the ISA bus. Newer PCs, where
the DMA controller directly speaks to memory across the local bus, does
not suffer as much; the local bus usually have abundant capacity. But there
still are two transfers, and the CPU does a lot of details in the planning
of a DMA transfer.

More advanced disk devices use their own DMA logic rather than the PCs 
navtive one, and take over a lot of detail work to offload the CPU. They
calculate memory addresses, and data goes straight from the interface to
memory. But then: The CPU looses the control it had over the DMA operation!
So problems occur that can be solved only by memory windowing.

Well, that is not entirely true: If the disk controller's DMA could
interpret a more detailed order, eg. the exact number of bytes wanted,
you could do a direct transfer. Many controllers can - each to its own way,
and Windows doesn't know them all - only the standard, native PC DMA. It
might come in some later version of the disk driver (but then probably
also according to the "32 bit disk access" protocol that came with Win3.1).

Things are not really that simple: With logical addresses you might also
experience a page fault mid-transfer! This may require one or more 
transfers to the same disk controller that caused it. So, the controller
must be capable of halting the first transfer temorarily to serve the
page fault transfers before it can complete the first one.
So, if you are not really simple, you must be rather sophisticated!

I found a couple of MS tech notes at cica; I think it was in the misc
directory, with alphanumerical codenames like ww0531.zip or thereabouts.
The one discussing smartdrv and 32 bit disk access is worth reading!

Ketil Albertsen



Newsgroups: comp.os.ms-windows.setup