Newsgroups: bit.listserv.vm-util
Date:         Tue, 1 Sep 1992 08:37:23 EDT
From:         "Robert L. King" <BOBKING@GDLVM7.VNET.IBM.COM>
Subject:      Re: Disk full threshold
Comments: To: vm-util%ubvm.bitnet@cunyvm.cuny.edu

 <PEPMNT@CFAAMP.BITNET>

John F. Chandler asked:
> Does anyone know how CMS decides when a disk is full?  Obviously,
> there must be some provision for writing the alternate directory, but
> there seems to be more to it.  ...               Then, at 133 files
> (142 blocks used), the disk was deemed full, despite the existence of
> 8 free blocks.  Providing for the alternate directory requires only
> 3 blocks, so what's with the other 5?  I'm sure there is an algorithm
> somewhere in the file system code, but it would be nice not to have to
> dig for it.  By the way, this experiment used CMS 4, but I have a
> feeling the algorithm hasn't changed any in later releases.

The attached is from Kim Worm.

Bob King                        InterNet: bobking@gdlvm7.vnet.ibm.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Essentially, you always need space to rewrite a second copy of the entire
directory and allocation map.  Note that since the first directory data block
has its shadow permanently allocated (CMS blocks 4 and 5 flip-flop back and
forth and both are always allocated), the number of reserved blocks which
are unavailable for file usage is therefore one less than the sum of the
number of blocks in the directory (including pointer blocks) and the number
of blocks in the allocation map (including pointer blocks).

The rest of the unused blocks can be attributed to "fat" added into the
space checking formula to ensure safety.  Note that overestimating the amount
of space left is far worse than underestimating it.  If the file system allows
too many blocks to get allocated to files and then cannot get all the blocks
it needs for the allocation map or directory, it crashes CMS.  On the other
hand, being a little bit conservative merely means you get disk full a few
blocks too soon.  The old algorithms in CMS release 4 are not very precise,
and one can certainly do better. In fact, there were long standing problems
where it actually was not conservative enough and the aforementioned crash
could occur -- this could happen if you had numerous opened files causing
multiple new directory blocks to get allocated all at once on a FINIS * * fm.

These algorithms have been substantially improved starting in VM/SP Release
6.  It is now possible to actually get to the point where all blocks on the
disk (except for those actually needed for the directory and allocation map)
can be used.  There are some circumstances where the new algorithms will
overestimate how much space is needed, but I believe the inaccuracy is
limited roughly to the number of pointer levels in the file being written.
Thus it is possible to try and write a record to a large file and get disk
full, and yet be able to write the same record to a brand new file.  Mostly,
this is due to the difficulty in establishing just how many blocks are needed
when you are writing a record to a file.  For the first record written to a new
file, we can calculate exactly how many blocks are needed.  However, for an
existing file, it depends on how many of the blocks in the file structure
need to be reallocated (some may have been touched already and are already
reallocated). Since the space checking is done up front, before any
retrieval or navigation of the file pointer structure is done, the approach
is to err on the side of safety.  Therefore, even on VM/ESA releases, you may
not always be able to completely fill the disk and may find that some blocks
are reported left even when you get disk full.

Also note that QUERY DISK was changed in VM/ESA Release 1.1 to report those
reserved blocks as used (and not to include them in blocks left).  This way
the numbers more closely represent what the user can actually use for files.

Cheers,
Kim J. Worm, CMS File System Development
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU