From: YOUNG@tattoo.cs.widener.edu (Rob Young)
Newsgroups: comp.os.vms
Subject: Extending Indexf.sys
Date: 1 Oct 1992 01:57:44 GMT
Organization: Widener University Department of Computer Science, Chester PA
Distribution: world
NNTP-Posting-Host: tattoo.cs.widener.edu
X-News-Reader: VMS NEWS 1.20

Ray Kaplan's post of "where is Nick" prompted me to search around for his
e-mail address and lo and behold here is his method to extend indexf.sys (or 
any file for that matter).


From: NICK@ncdlab.ulcc.ac.uk (Nick de Smith)
Newsgroups: comp.os.vms
Subject: RE: INDEXF.SYS and the state of your disks...
Date: 27 Aug 91 09:32:00 GMT
Distribution: world
Organization: The Internet

Hi,		(long message including code)

The original question, summed up, was "Why doesn't BACKUP acknowledge the
/HEADERS=nnn qualifier from the INITIALIZE command for a disk when doing a
BACKUP /IMAGE /NOINIT".

The answer is that BACKUP only uses information in the home block of the disk
when re-initialising it with a BACKUP /IMAGE /NOINIT. The initial value of the
/HEADERS=n qualifier is not saved in the home block, so BACKUP just pre-extends
the INDEXF.SYS file to hold the number of headers required for the files on the
source disk (as well some other stuff).

DEC know that users regard this as a problem, and have said that this behaviour
will "change in a future..." etc etc etc.

Detecting the problem:
----------------------
The problem is to avoid the "SYSTEM-F-HEADERFULL" messages that occur when the
INDEXF.SYS file cannot be extended any more. As far as I know, no ON-LINE disk
defragmenter can handle this problem. DEC say "Do an IMAGE backup and restore".

To see how far you have to go, type the DCL command:

	$ DUMP /HEADER /NOFORMAT /BLOCK=COUNT=0 destdev:[000000]INDEXF.SYS

The last word of the displayed "File Header" section is the checksum. If you
count back the number of WORDS (not bytes) until the first non-zero value,
divide this value by 3 (there are 6 bytes in each type 2 retrieval pointer) and
then multiply the value by 1000 (rounded up to the next cluster value), this
will give you APPROXIMATELY the number of NEW headers that can be allocated
(does not include any that are already free within the existing extents). eg.

Dump of file DKB100:[000000]INDEXF.SYS;1 on 27-AUG-1991 10:08:06.52
File ID (1,1,0)   End of file block 5406 / Allocated 5406

                             File Header

 151F0000 151E0000 02000001 00000000 00000000 00010001 02010000 FFFF6428
 00010001 0009FE00 00000020 00000000 00000000 00000000 00000200 00000000
 20202020 313B5359 532E4658 45444E49 0000151F 00000000 00000004 0004FA00
 57000000 00000000 00000094 CC68395C 6A400093 C41D9C2C 21A0003E 20202020
 20202020 20202020 20202020 20202020 20202020 20202020 20200094 CD7DE6CD
 20202020 20202020 20202020 20202020 20202020 20202020 20202020 20202020
 00000000 00000014 012A9511 163C5402 05014002 00004005 20202020 20202020
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 7CC60000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

This has 146 blank words. The cluster size of the disk is 3, so 1000 rounded up
to the next clustersize is 1002. Therefore, APPROXIMATELY (146/3)*1002= 48764
NEW file headers can be allocated before the index file is full. The program
INDEX_HEADER.MAR included at the end of this message will tell you how many
header blocks you have free BEFORE any new allocation takes place. The number of
free headers if therefore the number displayed by INDEX_HEADER.MAR plus the
number from the above calculation.

One day I'll get round to doing the program that integrates this into a simple
utility.

Workrounds:
-----------
There are basically two, both of which can be deduced from the source listing
CD. The index file is extended by exactly the same mechanism as any other file,
with the exception that a flag is set that stops extra header blocks being
added. The implication is that you can extend the index file just like any other
file, and you can do it by a very large amount if you want to. So...

Method 1.
---------
This is the method for me.

$ INIT destdev: name/MAXIMUM_FILES=bignum
$ MOUNT destdev: /FOREIGN
$ BACKUP/IMAGE/NOINIT sourcedev: destdev:
$ DISMOUNT destdev:
$ MOUNT destdev: name			! Mount disk privately
$ extend = "$sys$disk:[]extend_file"	! See below
$ extend destdev:[000000]INDEXF.SYS another-bignum ! Extend the index file by
$						! "another-bignum" headers
$ ANALYZE/DISK/REPAIR destdev:		! Fix up the secondary header block
$ DISMOUNT destdev:
$ MOUNT destdev: name /SYSTEM /CLUSTER /whatever ! All done, make disk available

Method 2.
---------
Do a non-image backup. This is rather too slow for me, and not an option if you
are restoring from tape. The following notes are from a DEC article. Note the
caveat about synonym directories.

$ INIT destdev: name/MAXIMUM_FILES=bignum/HEADERS=bignum ! Pre-extend INDEXF.SYS
$ BACKUP device:save/SAVE_SET device:[*...]*.*;*/BY_OWNER=ORIGINAL
$!
$! For V4.n systems, the syntax is '/OWNER=ORIGINAL'
$! Since a non-IMAGE restore cannot be performed from
$! STANDALONE BACKUP, online BACKUP must be used.
$!
$! To do this for a system disk, VMS must be booted from another disk.
$! Since non-IMAGE BACKUP restore operations do NOT preserve synonym
$! directory structures, the following additional steps are necessary
$! (once the non-IMAGE restore completes):
$!
$ DELETE disk:[000000]VMS$COMMON.DIR;1  !Should be empty
$ SET FILE/ENTER=disk:[000000]VMS$COMMON.DIR disk:[SYSn]SYSCOMMON.DIR
$!
$! SYS0 is the most likely candidate, otherwise use the
$! lowest numbered root.
$!
$! Do the next two steps for each root on the volume
$! except for the lowest numbered root.
$!
$ DELETE disk:[SYSn]SYSCOMMON.DIR;1
$ SET FILE/ENTER=disk:[SYSn]SYSCOMMON.DIR disk:[000000]VMS$COMMON.DIR
$!
$! The above procedure is NOT supported by DIGITAL.  We
$! recommend that only IMAGE restores be done for the system
$! disk.

I would like to thank the following for their ideas, shared experiences,
re-posted articles, general statements of interest and code fragments...

Pat Rankin		rankin@edu.caltech.eql
Micheal De Buitlear	micheal_de_buitlear@eurokom.ie
Richard B. Gilbert	dragon@nscvax.princeton.edu
Ehud Gavron		gavron@vesta.sunquest.com
Pasquale F. Scopelliti	SCOPELLITI+_P%ROVAX%MRGATE%Corning_Incorporated@com.mcimail
Dean Sheets		sheets@inland.com
Michael K. Ferris	mkf@bitnet.swtexas
(European Space Agency)	phall@earn.esoc
Greg			npri6!greg_a@com.psi.uu

Regards,

Nick de Smith
NICK@NCDLAB.ULCC.AC.UK	PSI%234213300154::NICK

	.title	HEADERCNT
;++
;
;	HEADCNT	-- Count the number of free headers in the INDEXF.SYS
;
; Author:	David L. Cathey
; Date:		23-SEP-1990
; Copyright:	1990, Montagar Software Concepts
;
;	Note:  This software is intended as an example only.  No warranties
;	as to its usability are implied.  It's free and you get what you pay
;	for!  This software may be freely distributed as long as it is
;	accompanied by the copyright notice.
;--

	.library	/SYS$LIBRARY:LIB/
	$HM2DEF			; Home Block (ODS-2) Definitions
	;
	; Some home grown macros...  
	;
	.MACRO	_STKDEF	SYM,ALLOC,SIZ
	.IIF	NB,ALLOC,	ALLOC	SIZ
	.IIF	NB,SYM,	SYM = - .
	.ENDM	_STKDEF
	.MACRO	_STKEND
	STACK_K_LENGTH = - .
	.CROSS
	.RESTORE
	addl2	#STACK_K_LENGTH,sp
	.ENDM	_STKEND
	.MACRO	_STKINI
	.SAVE	LOCAL_BLOCK
	.NOCROSS
	.PSECT	_ABS_,ABS
	.ENDM	_STKINI
	.macro	_ILDEF
	$DEFINI	IL
	$DEF	IL_BUFLEN	.blkw	1
	$DEF	IL_CODE		.blkw	1
	$DEF	IL_ADDR		.blkl	1
	$DEF	IL_RETLEN	.blkl	1
	$EQU	IL_LENGTH	.
	$DEFEND	IL
	.macro	_ILDEF
	.endm	_ILDEF
	.endm	_ILDEF
	_ILDEF
	.macro	ITMLST_ITEM	code,buflen,bufadr,retlen=0
	movw	code,IL_CODE+<ITEM_NUMBER*IL_LENGTH>(r0)
	movw	buflen,IL_BUFLEN+<ITEM_NUMBER*IL_LENGTH>(r0)
	moval	bufadr,IL_ADDR+<ITEM_NUMBER*IL_LENGTH>(r0)
	.if	IDN <0> <retlen>
	.iff
	moval	retlen,IL_RETLEN+<ITEM_NUMBER*IL_LENGTH>(r0)
	.endc
	ITEM_NUMBER = ITEM_NUMBER + 1
	.endm	ITMLST_ITEM
	.macro	ITMLST_START	itmlst,count
	_ILDEF
	pushr	#^M<r2,r3,r4,r5>
	movc5	#0,(sp),#0,#<count*IL_LENGTH>,itmlst
	popr	#^M<r2,r3,r4,r5>
	movab	itmlst,r0
	ITEM_NUMBER = 0
	.endm	ITMLST_START

i_fab:		$FAB	dnm=<[000000]INDEXF.SYS>,fac=<bio,get>,fop=<bio>, -
			shr=<get,put,upd,upi>
i_rab:		$RAB	fab=i_fab

sys$disk:	.ascid	/SYS$DISK:/

;++
;	Entry point
;--
	.enable	local_block
	.entry	Main,0
_STKINI
_STKDEF	FP_ITMLST	.blkb	<3*IL_LENGTH>	; 3 element item list
_STKDEF	FP_DEVNAM	.blkb	16		; full device name
_STKDEF	FP_DEVNAMLEN	.blkl	1		; full device name length
_STKEND
	ITMLST_START	FP_ITMLST(fp),count=3
	ITMLST_ITEM	code=#DVI$_FULLDEVNAM, -
			buflen=#16, -
			bufadr=FP_DEVNAM(fp), -
			retlen=FP_DEVNAMLEN(fp)	; Get SYS$DISK device name
	$GETDVIW_S	devnam=sys$disk, -
			itmlst=FP_ITMLST(fp)
	blbc	r0,999$
	$FAB_STORE	fab=i_fab, -
			fna=FP_DEVNAM(fp), -	; Fill in FAB
			fns=FP_DEVNAMLEN(fp)
	$OPEN		fab=i_fab
	blbc	r0,999$				; Access INDEXF.SYS
	$CONNECT	rab=i_rab
	blbc	r0,999$
	calls	#0,Process_INDEXF		; Process the INDEXF.SYS
999$:
	ret
	.page
;++
;	Process the INDEXF...
;--
	.enable	local_block
	.entry	Process_INDEXF,^M<r6,r7,r8,r9,r10,r11>
_STKINI
_STKDEF	FP_DESC		.blkq	1		; Output descriptor
_STKDEF	FP_BUFFER	.blkb	132		; Output Buffer
_STKDEF	FP_HOMEBLK	.blkb	512		; HOME Block Buffer
_STKEND
	$RAB_STORE	rab=i_rab, -
			ubf=FP_HOMEBLK(fp), -
			usz=#512, -
			bkt=#2			; Get HOME block
	$READ 	rab=i_rab
	blbs	r0,5$
	brw	999$
5$:
	movab	FP_HOMEBLK(fp),r6		; Get address of HM in r6
	movzwl	HM2$W_IBMAPSIZE(r6),r7
	mull2	#512,r7				; Compute stack required for BM
	subl2	r7,sp				; Free up stack space
	movab	FP_HOMEBLK(fp),r11
	subl2	r7,r11				; Calculate stack buffer address
	movzwl	HM2$W_IBMAPVBN(r6),r10		; Get index bit map VBN
	$RAB_STORE	rab=i_rab, -
			ubf=(r11), -
			usz=r7, -		; Set up RAB for trasfer
			bkt=r10
	$READ	rab=i_rab			; Get bit map...
	blbc	r0,999$

	clrl	r8				; Clear r8 (bit number)
	clrl	r9				; # of free headers
10$:	ffc	r8,#32,(r11),r8			; Find a free header
	cmpl	r8,i_fab+FAB$L_ALQ		; Out of bounds of INDEXF ALQ?
	bgtr	20$				; Yes, done!
	incl	r9				; Increment # free headers
	incl	r8				; Skip this bit, now...
	brb	10$				; Find another
20$:
	movzwl	HM2$W_IBMAPVBN(r6),r8		; Get bit map VBN,
	movzwl	HM2$W_IBMAPSIZE(r6),r1		; and bit map size,
	subl3	r8,i_fab+FAB$L_ALQ,r11		; subtract # blocks in INDEXF to
	subl2	r1,r11				; first actual header from ALQ
	movab	FP_BUFFER(fp),FP_DESC+4(fp)
	movl	#132,FP_DESC(fp)
	$FAO_S	ctrstr=1000$, -
		outlen=FP_DESC(fp), -
		outbuf=FP_DESC(fp), -		; Format output buffer
		p1=r11, -
		p2=r9
	blbc	r0,999$

	pushaq	FP_DESC(fp)			; Output results...
	calls	#1,g^LIB$PUT_OUTPUT
999$:
	ret

1000$:	.ascid	/Number of headers: !SL, Number of free headers: !SL/

	.end	Main

EXTEND_FILE.C
-------------

/*
  Program	: EXTEND_FILE.C
  Author	: Nick de Smith
  Date		: 03-Aug-91

  Program to extend any file by the specified number of blocks.

  Usage:	$ extend = "$dev:[dir]extend_file"
		$ extend dev:[dir]file.ext nuber-of-blocks

  Typically, this is used on the dev:[000000]INDEXF.SYS to allocate a large
  number of headers when the device has just been restored from an IMAGE backup.

  $ CC EXTEND_FILE
  $ LINK EXTEND_FILE,SYS$INPUT/OPT
  SYS$SHARE:VAXCRTL/SHARE
  $!

*/
#include	stdio
#include	rms
#include	ssdef
#include	stsdef

#define	ss_check( command ) { \
	unsigned long ss_status = (command);	\
	if ( (ss_status & STS$M_SUCCESS) == 0 ) { \
		return ss_status;		\
	}					\
}

main(
	int	argc		,
	char	*argv[]		)
{
	struct FAB r_fab;
	unsigned long l_status;

	if ( argc < 3 ) {
		printf( "Usage: extend <file> <number-of-new-blocks>\n" );
		return SS$_NORMAL;
	}

	r_fab = cc$rms_fab;
	r_fab.fab$b_fac	= FAB$M_PUT;
	r_fab.fab$b_shr	= FAB$M_SHRPUT;
	r_fab.fab$l_fna	= argv[ 1 ];
	r_fab.fab$b_fns	= strlen( argv[ 1 ] );

	ss_check( SYS$OPEN( &r_fab, 0, 0 ) )

	r_fab.fab$l_alq = atoi( argv[ 2 ] );

	ss_check( SYS$EXTEND( &r_fab, 0, 0 ) )

	ss_check( SYS$CLOSE( &r_fab ) )

	return SS$_NORMAL;
}

[end]

                                                     young@tattoo.cs.widener.edu
"Even if the biblical assertion is incorrect that where there is no vision,
 the people perish, it is difficult to think what could be the engine or
 stimulus for social behavior in a nihilistic system committed only to the
 certainty of the passage of time, without any energetic relationship to 
 another principle or purpose."  --Lionel Tiger