From: tody@noao.edu (Doug Tody)
Subject: Re: Yet more about 'tc' (blocking issues)
Reply-To: tody@noao.edu
Organization: National Optical Astronomy Observatories
Date: Wed, 11 Nov 1992 03:27:12 GMT

In article <1228@jagubox.gsfc.nasa.gov>, jim@jagubox.gsfc.nasa.gov (Jim Jagielski) writes:
> Is the fact that 'tc' needs to be blocked at 8k a big problem (or pain)
> for most people? So far I've heard from only one person who says that it is,
> but the only reason is because it makes it tough (impossible) to read tapes
> from other workstations that write at 512/1024 and not because of any
> A/UX-world problems.

The 8k limitation is a minor nusiance for A/UX-only usage, e.g., file backups,
but is a MAJOR problem if the tape drive is to be used to transfer data to or
from an external, non-A/UX system.

For example, at NOAO we have a data analysis package (IRAF) which can run on
A/UX and is used to process astronomical image data.  This data is
voluminous, hundreds of megabytes, and is written to DAT or Exabyte tapes by
the systems used to acquire the data, typically Suns.  On these systems the
standard way to write data transfer tapes is with a variable record size.
For example we might write FITS tapes (an astronomy tape data format) using
a block size of 28800 bytes per tape block.  Given the large amount of data
involved, tape is about the only practical way to transfer this data.  Network
transfer is also feasible, but is not an option for some users, and does not
provide a backup capability.

The blocking issue can also be a serious problem for other types of data
interchange, for example tar tapes used to export software.  We cannot
currently write a DAT or Exabyte tar tape on any workstation class machine
at NOAO (Sun, DECstation, VAXstation, IBM RS/6000, MIPS, SGI, HP, ...) which
can be read on an A/UX system, due to the 8k blocking requirement.

> I for one don't mind the 8k blocking at all. It makes the driver more
> efficient (and therefore quicker) but you have to recall to block at
> 8k. Of course, if 'tc' used the tape drive's actual physical block size
> as it's blocking factor, then you'd have to block at either 512 or 1024
> or 8192, depending on your tape drive so you'd still have to concern
> yourself about blocking...

A variable record size is preferred over a fixed block size since for some
tape formats the record size may have some meaning, and since variable size
records cause fewer transport problems.  If fixed size blocks are used, one
system may use, e.g., 1k blocks and the other 8k blocks, and it may not be
possible to read the tape.  By using variable size records it is possible to
make an exact copy of a 9tk tape on a DAT or Exabyte.

There is very little difference in terms of efficiency between fixed blocks
and variable sized records for devices like DAT and Exabyte that use a large
FIFO buffer internally to buffer the data.  So long as this buffer is kept
full during a write the tape will stream, writing data onto tape at the
maximum rate.  For cartridge drives like the DC2000 or QIC with no internal
buffering a large block size is needed to transfer data efficiently, but
this is better done in the application than in the driver.  Rather than
requiring 8k blocks, a good approach would be for the driver for a cartridge
tape device to require that data transfers be a multiple of the physical
device block size, e.g., 512 bytes, and leave it up to the application to
buffer data to cause the tape to stream.

Cartridge drives support only fixed size blocks, but DAT and Exabyte are
capable of a variable record size mode as well as a fixed block mode
(variable record size mode is enabled by setting the device block size to 1
byte; in variable record mode each read returns one record).  9 track reel
tape drives support only variable size records.  For DAT and Exabyte a good
driver will provide different device entries for fixed block mode i/o and
variable size record i/o, so that the user can decide which to use.  If the
device is capable of compression, there should be different device entries
for compression mode and non-compression mode as well.

-- 
Doug Tody	National Optical Astronomy Observatories
tody@noao.edu   P.O. Box 26732, Tucson, Arizona, 85726