Newsgroups: comp.bbs.waffle
Subject: Re: A Poor Stacker application
From: mike@batpad.lgb.ca.us (mike batchelor)
Date: Sun, 30 Aug 92 03:14:01 PDT
Distribution: world               
Organization: The Batchelor Pad

gettys@regent.enet.dec.com (Bob Gettys) writes:
>
> 	As for DOS - the most interesting thing is just how STacker handles
> storage internalto its own file (stacvol.dsk). It seems that the 8k cluster
> that stacker says that it uses is a fiction presented to DOS so that things
> will work for normal programs in  a transparent manner. It has absolutely
> nothing to do with its internals. Internaly, stacker uses 512 byte chunks for
> its storage. i.e. - the smallest portion of the stacvol file that will be used
> is 512 bytes. so the maximium wasted space per file is 511 bytes (assuming a
> one byte file or overflow). 
> 
> 	He did say that they are working on redoing some of the documentation
> to make all this clearer so that there won't be the massive misconceptions that
> are rampant at this time. 

This is correct, according to the information I have.  To put this
thread to rest, I now present the Stac Electronics Techincal Note
about "Slack Space":

_________________________________________________________________________
STACKER NOTE                      Stac Electronics Technical Note

Subject: Stacker and "Slack" Space.

Tec031- 3/24/92
Page 1 of 2
__________________________________________________________________________
Definitions:

Sector: The smallest unit of storage that a disk controller is
        physically able to read or write.  Sectors have a fixed 
        size of 512 bytes in a Stacker drive.

Cluster (Allocation Unit):  A multi-sector unit; the smallest
        unit of storage used by DOS.  DOS varies the allocation unit size
        to suit the storage medium, however, the size is set during
        logical formatting and does not change thereafter.  Called a
        cluster prior to MS-DOS 4.0, the words are interchangeable.

The DOS 16-bit FAT is limited to 64K, therefore the maximum drive
size with 2K clusters is 128Mb (64K x 2K).  When the drive being
formatted is larger than 128Mb, DOS merely makes the allocation 
units bigger, because the limitation on drive size is the number 
of clusters, not the number of sectors.  This would suggest to 
those of you who write many letters and other small documents, 
or who use relatively small database and spreadsheets, that you 
should probably not partition a drive over 128Mb, because from 
128Mb to 256Mb, DOS uses 4K allocation units.  From 256Mb to 512Mb, 
the allocation units are 8K.  As the drive gets bigger, the allocation 
units get proportionally larger also.

Slack Space:The space lost on your hard disk when DOS allocates
      an entire cluster to save an amount of data which is less than
      the size of that cluster.
          e.g.: Your file is 612 bytes, so normally DOS uses one
      allocation unit (cluster) to save the 612 bytes.    Because an
      allocation unit is 2048 bytes, the lost space (slack space) is
      1436 bytes (2048 - 612).  
      
      The larger the drive, the larger the allocation unit, and the 
      greater the amount of slack space (waste).  If your drive has 8K 
      allocation units, and you save a 178 byte batch file (filename.bat), 
      the wasted space is 8014 bytes.  A normal letter would waste from 5K 
      to 7K.  This space is lost forever, unless we change the size of the 
      allocation units on this particular drive.

Stacker and flexible allocation units:

Stac had a better idea, we decided to have a flexible
allocation unit.  When Stacker builds a new drive, the default
cluster size is 8K.  What this really means is that the largest
possible cluster size is 8K, not that all clusters are 8K.  This
allows Stacker to build a drive as large as 512Mb logical (64K x
8K).  Where Stacker saves space is in the use of sectors.  When
Stacker saves a file which is less than a sector in size, it only
allocates one real sector to the cluster which is keeping track
of that data.  Therefore, if the compressed data is 6 bytes,
Stacker only uses one sector to save that file, whereas DOS would
use 4 sectors, so the savings in lost space is 1536 bytes (2042 -
506).

More simply stated; Stacker only uses as many sectors as
necessary for a given file.  Sectors are not tied permanently to
a specific allocation unit until they are used, and even then,
only enough are used to do the job.  This means that slack space
is greatly reduced in a Stacker drive.

Of course, there is a slight catch: too many small files only use
one or two sectors each, but each one also uses an allocation
unit.  This means that the FAT can run out of entries even while
much of the space is unused because the sectors were not
allocated to clusters.

In Stacker version 1.x, the only fix for this problem was to
change the cluster size to 4K.  This allows twice as many entries
in the FAT.  This method can also be used with Stacker version
2.0 by users of DOS versions which are limited to 32Mb drives.

In Stacker version 2.0, Stac improved the product by allowing the
user to logically "grow" an existing Stacker drive with a high
compression ratio, by increasing the number of allocation units.  
When the drive is getting close to full, and the compression ratio 
is appreciably higher than the original ratio, the user may choose to 
"grow" the drive by running SDEFRAG /G.  This can increase the number
of allocation units up to twice the original number, thereby increasing
the logical capacity.

For the user of a database file which is approximately 400Mb, and
which compresses at 8:1, we need to do a little arithmetic before
building the original Stacker file.  The maximum size of a
Stacker drive is 512Mb.  Divide this by the anticipated maximum
compression ratio of 8, and we get 64Mb.  Therefore we can create
a new Stacker drive, choose a compression ratio of 8:1, and stack
an existing drive with its data.  If the data really compresses
at 8:1, we will have used 64Mb of physical space to create a
drive of 512Mb.  If the file is very large, there will be no
slack space, as the file is many clusters long.  The FAT is still
only 64K, but is now handling a very large drive in a relatively 
small space.

To prove this, you can conduct a fairly simple test.
First, run SCHECK on a Stacker drive, and record the results
which should look like this:

No errors found
Stacker Drive Statistics:
             Stacker Drive               STACVOL File
                Drive C:                 D:\STACVOL.DSK
             ______________              ______________

Total Bytes:     155,435,008               77,718,016
Bytes Used:       80,887,808  ( 52.0% )    44,773,888  ( 57.6% )
Bytes Free:       74,547,200  ( 48.0% )    32,944,128  ( 42.4% )

Stacker Drive Compression Ratio = 1.8:1
Projected Bytes Free           = 59,473,920

     The important number to observe here is the bytes used in
the STACVOL file (44,773,888 bytes).

Second, create a very small file, or copy a small file from
anywhere.  Make sure that the file is less than 512 bytes.  Now
run SCHECK again, and look at the number of bytes used in the
STACVOL file again.  This time, the number should be 512 bytes
less.  Record the number again, and now add another file which is
somewhat larger, and then look at the results again.  This time
the bytes free may decrement by 512 or 1024 or 1536 or 2048 or
some other multiple of 512.  This should demonstrate that Stacker
only uses as many sectors as it needs, no more.





[] ---
[] Mike Batchelor -- mike@batpad.lgb.ca.us
[] Long Beach, California