From: leichter@lrw.com (Jerry Leichter)
Newsgroups: comp.os.vms
Subject: re: Re: Please, any comments on these outputs..... [high-water marking]
Date: 19 Dec 92 14:13:44 GMT
Distribution: world
Organization: The Internet


	[Commenting on a configuration displayed by another user, Mr. Bandy
	writes:]
	Everything looks good to me -- except...

	If you're not running in a high security environment, you should
	consider turing off high-water marking.

Incorrect advice.

					         What it does is automatically
	fill file extensions with zeroes.  It's goal is to keep users from
	"disk scavaging" by using the following to read files that others
	have deleted:
	   $ copy login.com foo.foo /allocate=100000
	   $ set file /end foo.foo
	   $ dump foo.foo

This is a correct description of the REASON high-water marking exists, and the
"external interface".  It is an INCORRECT description of the implementation.

	This will pick up other user's deleted mail messages, program
	listings, memos and other perhaps non-public things.  It takes
	additional I/Os  to fill the extensions with zeroes and if security is
	not an issue,  then it will return you a little disk performance.

Turning off high-water marking will have essentially no effect on system
performance.  When it was first implemented, high-water marking used the
obvious, naive implementation of physically zeroing every block allocated to
a file.  This is obviously expensive - and also unnecessary.

The way high-water marking is actually implemented is different.  Stored with
the file is the "high-water mark":  The block number of the highest block
ever written to the file.  The rules for accessing a file at block B, when
the high water mark is H, are as follows:

	1.  If B <= H, either read or write access is permitted.
	2.  If B = H+1 and this is a WRITE operation, simply do it and
		set H to B. (There's no point zeroing a block that is about
		to be written to.
	3.  If B > H+1 and this is a WRITE operation, first simulate writes
		for blocks H+1, H+2, H+3, ... B-1, writing all zeroes.
		Set H=B-1, and then continue as in step 2.
	4.  Finally, if B>H and this is a READ operation, we have two
		choices:  Either zero the buffer without even looking at
		the disk, or "advance" H as in step 3.  I'm not sure which
		of these VMS does - probably the former, as there are no
		advantages to the latter.

Assuming the "cheaper" implementation in step 4, reads cost exactly the same
whether you use high-water marking or not.  Writes cost exactly the same
except in the special case that you write to a block past the current end
of the file.

All sequential access to a file (that doesn't attempt to read data that was
never written - i.e., to scavange) requires NO additional disk accesses.  I
believe RMS has been organized so that indexed files also incur few, if any,
additional disk operations.  (The only time there might be an additional disk
operation would be when the file was growing, in any case.)

In the case of a relative file, while you might write to records near the end
without filling in records near the beginning, RMS will have to pre-format all
the blocks anyway or it will find spurious "full" records.  I'm sure it's
careful to fill them from low to high, thus avoiding any high-water marking
costs.

In summary:  There is little reason not to enable high-water marking today.

	I hope usenet readers will not take offense to me publishing a way for
	non-priviledged users to "abuse" your systems.  Better that everybody
	knows the tricks, IMHO.

This particular technique was old in the 1970's.  However, the general philo-
sophy - I might was well tell everyone, the bad guys all know anyway (which is
often not true until you helped tell them) - is foolish.

							-- Jerry