March 12, 1990 M20-4, Part II

Change 1

CONTENTS\
CHAPTER 9. SYSTEM BACKUP\
PARAGRAPH PAGE\
9.01
Purpose\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
9-1\
9.02 Backup
Policies\...\...\...\...\...\...\...\...\...\...\...\...\...\.... 9-1\
9.03 Backup of System
Files\...\...\...\...\...\...\...\...\...\...\...\... 9-1\
9.04
System-Minder\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
9-1\
9.05 Procedure for Daily System-Minder Incremental Backup\...\... 9-3\
9.06 Backup Log
Files\...\...\...\...\...\...\...\...\...\...\...\...\...\... 9-5\
9.07 System-Minder Restoration of
Files\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\.....
9-6\
9.08 Full-Volume
Backup\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...
9-7\
9.09 Reinitialization of Disk
Volumes\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\.....
9-8

94

March 12,1990 M20-4, Part II

Change 1

CHAPTER 9. SYSTEM BACKUP

9.01 PURPOSE

This chapter provides System Administrators with information on
mandatory VBA (Veterans Benefits Administration) system backup
procedures. To supplement this chapter, a copy of the VS System
Administrator\'s Learning Guide, Chapter 5; VS System Utilities
Reference, Release 7 Series, Chapter 7; VS System Operator\'s Reference,
Chapter 12; and Instructions for System-Minder Software (from Integrated
Custom Software, Inc.) should be obtained.

9.02 BACKUP POLICIES

a\. Backup is the periodic copying of information stored on disks from
one disk to another or from disk to tape. System-Minder software and the
VS BACKUP system utility will be used to perform backup functions. In
case of a disk crash or some other type of data loss, the only way to
recover the data is from backup volumes. The importance of sound backup
procedures cannot be over emphasized. Local station management is
responsible for implementing the backup procedures outlined in this
chapter.

b\. Provided in paragraphs 9.03 through 9.06 are mandatory procedures
for daily incremental backup and reorganization, as well as full volume
backup. Use of these procedures will assure that adequate backup copies
of all necessary files are available in case files are damaged or
destroyed. The System-Minder programs have been developed to
automatically accomplish the incremental backups and reorganization with
minimal inconvenience to the user environment. At a minimum, full volume
backups (manual) should be scheduled monthly or, if necessary, biweekly.
When the second incremental backup volume becomes full, it is
recommended that a full volume backup should be completed for each
volume.

9.03 BACKUP OF SYSTEM FILES

a\. Backup copies of the system volume are kept on off-line volumes so
that if on-line system files are damaged in any way, an off-line volume
can be used to retrieve an undamaged copy.

b\. The BACKUP utility will be used to store an off-line duplicate of
the system volume. This volume must be housed in a secure place away
from the CPU (Central Processing Unit). The station\'s safe could be
used.

c\. At least two copies of current system files should be maintained.
The first copy is identified in subparagraph b above. A second copy
could be kept in the room with the CPU. Each time a change is made to
the system library (@SYSTEM@), all backup copies should be updated
accordingly. Backup volumes will be named where represents the operating
system release level; e.g. 71403.

9.04 SYSTEM-MINDER

a\. System-Minder will be used for daily incremental backups. The
System-Minder software consists of a disk monitor (DISKSTAT), five
background task programs for performing on-line backup (FILEBACK),
on-line word processing and WP (word processing) PLUS document
reorganization (FILEWP and FILEWPP), on-line file reorganization
(FILEFIX), and on-line disk reorganization (FILEREOR). It also contains
a program FILEAUTO, which facilitates control of the job queue, SMCONTRL
for system fine tuning and control, SMSEARCH for detailed backup tog
analysis, and RESTORE for full volume restoration. The following
System-Minder files must be resident in the library \"ICSPROG\" on each
volume (S1, V1-V9):

9.1

M20-4, Part II March 12,1990\
Change 1\
\
FILE DESCRIPTION

FILEAUTO Queues the programs FILEBACK, FILEWP, FILEWPP, FILEFIX and
FILEREOR to

run in sequence.

FILEBACK On-line backup program

FILEFIX On-line file reorganization program

FILEREOR On-line disk reorganization program

FILEWP On-line word processing document reorganization program

FILEWPP On-line Work Processing Plus document reorganiza­tion program

b\. When the program \"FILEBACK\" runs, copies of each file modified
within the last 3 days will be copied to the designated backup volume.
(The FILEBACK program will be the only System-Minder module use during
production hours.) Open files, damaged files, documents queued to
printers, or indexed files that were not property closed will not be
copied but will be identified in the System-Minder logs.

c\. The FILEAUTO program provides for backup, reorganization of WP
files, data processing files, and disk reorganization. This is
automatically accomplished when FILEAUTO schedules the appropriate
programs in sequence. As opposed to the FILEBACK program which runs
during the work day, the program FILEAUTO will to run during
non-production hours since that system resources will be compromised if
scheduled during the work day resulting in severe system degredation.

d\. Requirements for System-Minder software are:

\(1\) All System-Minder background tasks must be scheduled by a system
Security Administrator through ICSPROG.

\(2\) A complete set of full-volume backups must be performed before
processing any data for the first time. The naming convention will be
followed for the full-volume backup packs where \"\*\*\" represents the
input volume name (S1, V1-V9) and \"####\" represents the full-volume
backup date. For example, the output volume for a full-volume backup of
S1 performed on September 29, would be \"S10929.\"

\(3\) The output volume for the incremental backups must be a blank, 288
megabyte removable disk, mounted on the system and labeled with the date
(4 digits) and the suffix \"BK\"; e.g. \"0216BK.\" This disk may contain
a paging pool and may be opened for print and work files, if desired.
This disk is not available to users and should not contain data or
program files since no backup will be provided.

\(4\) The backup volume name must be associated with every input volume
by using the SMCONTRL utility. The SMCONTRL utility can be accessed
through the main System-Minder menu (File = MENU, Library = ICSPROG,
Volume = S1), PF7, System-Minder Controller, or by running the program
\"SMCONTRL\" in Library \"ICSPROG\" on Volume \"S1.\" The first screen
lists all currently installed volumes. You should cursor to the desired
volume name (SI), press return. At the top of the next screen, you will
be able to alter the default output volume name (BACKUP) by overstriking
the word \"BACKUP\" with the name of the backup volume (e.g. 0419BK) in
the \"Output Backup Volume:\" field. If a detailed listing of every file
that is backed up by the System-Minder software is required by your
site, the \"Detailed Backup Logging (Y or N?)\" field should be changed
to \"Y.\" By pressing PF16, you will store this new information. These
steps must be repeated for the remaining input volumes (V1-V9).

9-2

March 12,1990 M20-4, Part 11

Change 1

\(5\) The backup volume disk utilization should be monitored daily using
either the DISKUSE utility or the System­Minder DISKSTAT program. When
the pack is 90 percent full, it should be replaced with a pack that has
been initialized or reformatted with the Diskinit utility. This pack
should be labeled to reflect the current date. Again, the procedures in
subparagraph 9.04d(4) should be completed to associate the new backup
volume name with the input volume by using the SMCONTRL utility.

\(6\) WP PLUS requires a library for document reorganization. The
library must be defined as \"ICSREORG,\" with public access privileges
or delete privileges given to each administrator running the tasks. The
administrator must create a document with ID #1 and a glossary ID \# in
library ICSREORG, both with password \"ICS.\" This library cannot be
used for production use and must be reserved only for System-Minder.

\(7\) System-Minder requires three (3) non-interactive tasks for
processing. These must be available to the system with a modifiable data
area of 1024, class \"A.\" (To verify their availability, enter the
Operators Console, PF7 (Non­interactive tasks), PF5 to scroll the screen.
All available tasks will appear on this screen as AVAILABLE.

\(8\) The non-interactive task entitled \"QSUBMIT\" must be running in
background in order for the automatic scheduler to initiate the above
noted programs. The QSUBMIT program must be submitted through the
System­Minder menu function entitled \"SUBMIT SINGLE JOB\" (see subpar.
9.05g).

e\. Questions concerning the System-Minder software or its functioning,
may be addressed to the Integrated Customer Software Hotline (203)
282-9787.

9.05 PROCEDURE FOR DAILY SYSTEM-MINDER INCREMENTAL BACKUP

a\. FILEAUTO. FILEAUTO will perform the following functions in the order
specified: FILEBACK, FILEWP, FILEWPP, FILEFIX, and FILEREORG. FILEAUTO
will be scheduled to run during non-production hours following
completion of the network shutdown procedures (see ch. 3). It is
imperative that all WSN (Wang Systems Networking) background tasks have
been executed and WSNREORG has been run prior to scheduling the FILEAUTO
program.

b\. FILEBACK. It is recommended that FILEBACK be scheduled to run during
the day in which the least amount of system activity is noted. In most
stations, this period of time occurs around 12 m.(noon), or lunchtime,
and will continue for approximately I hour. The best time for scheduling
is determined by the System Administrator based upon the processing
requirements of local users. Running FILEBACK during production hours
will insure that backup copies of recently created or modified files are
maintained on the backup volume.

c\. Scheduling FILEAUTO and FILEBACK Programs. To schedule the
incremental backups, perform the following steps:

\(1\) Run MENU in Library ICSPROG on Volume SI.

\(2\) Press PF3, Background Job Queue, to enter the System-Minder
Background Job Queue.

\(3\) Type FILEBACK in library ICSPROG for every volume you wish to
processed during production hours. Type FILEAUTO in library ICSPROG for
every volume processed during non-production hours. Type Date, Time
(24-hour clock), job class \"A,\" and \"I,\" in the reschedule interval
column headed \"Days.\" Figure 9-1 illustrates a sample schedule for a
site performing incremental backups (FILEBACK) at 20 minute intervals
during production hours, and incremental backups and reorganization
functions (FILEAUTO) during non-production hours at 2-hour intervals. In
this example, the WSNHALT and WSNREORG procedures would have been
executed by the System Administrator at 1800 hours.

9-3

M20-4, Part II March 12,1990 Change 1

BACKGROUND TASK AUTOMATIC SUBMITTAL QUEUE\
10:06:22\
Reschedule\
Date Time job Interval\
Program Library Volume Mo Da Yr HR MI Class Days HR MI

FILEBACK ICSPROG S1 09 14 88 12:00 A 1 0 0\
FILEBACK ICSPROG V1 09 14 88 12:20 A I 0 0\
FILEBACK ICSPROG V2 09 14 88 12:40 A 1 0 0\
FILEAUTO ICSPROG S1 09 14 88 19:00 A I 0 0\
FILEAUTO ICSPROG V1 09 14 88 21:00 A I 0 0\
FILEAUTO ICSPROG V2 09 14 88 23:00 A I 0 0

4-Previous Screen 5-Next Screen 8-Delete Item Above Cursor 16-Quit

Figure 9-1. Background Task Automatic Submital Queue

g\. QSUBMIT is a background task which activates the System-Minder
programs (FILEBACK, FILEAUTO, etc.) to run from the scheduler. QSUBMIT
must be submitted through the System-Minder menu (File = Menu, Library =

ICSPROG, on Volume = SI), Function Key 4, entitled \"SUBMIT SINGLE
JOB.\" On the Background Task Job Submittal screen, type:

Program QSUBMIT\
Library ICSPROG\
Volume S1\
job Class A

and press Return. To verify that the task has actually been submitted to
background, check the Procedure Queue, from the Operator\'s Console.
QSUBMIT should be listed with the status of \"Running.\" If the
procedure is listed but is not \"Running,\" there is good possibility
that there is no available Non-interactive task with a procedure class
of \"A.\"

h\. To adequately prepare the system every night for FILEAUTO
processing, it is recommended that all workstations be \"INHIBITED.\"

i\. Should any of the System-Minder programs be prevented from running,
remove any of the pending System­Minder background programs (FILEBACK,
FILEAUTO) from the Procedure Queue. To cancel any System-Minder programs
that have a status of \"Running\" in the procedure queue, run SMCONTRL
in ICSPROG on SI, place the cursor under the volume name that the
program is processing, and press PF2. The volume name will be
highlighted, indicating that all System-Minder tasks for that volume
will be canceled. Canceling in this manner (rather than from the
Operator Console) performs all necessary file cleanup before task
shutdown and also prevents other tasks waiting in the job queue from
starting. The shutdown condition remains in effect for 10 minutes after
which System-Minder tasks will again be allowed to run. PF3 from the
SMCONTRL screen will immediately cancel the shutdown condition.

94

March 12,1990 M20-4, Part 11

Change 1

9.06 BACKUP LOG FILES

a\. Monitoring System-Minder Backup Log File

\(1\) A log file is written for each volume whenever System-Minder
programs run. Log files are created in library ICSLOGS on the volumes on
which the programs are run and will be monitored daily. Since these log
files are in use while System-Minder programs are running, the log file
cannot be read until System-Minder programs are completed.

\(2\) The log contains the start and stop time of each program and a
listing of any files the program was unable to access or any files with
zero records (which cannot be moved). An error message will also be
written if the system cannot find the programs COPY and SORT in library
\@SYSTEM@. Generally, the review should check that each program started
and finished as expected. If any program starts but does not have a
record that it finished in the log file, the procedures described in
subparagraph b concerning system faults should be followed.

\(3\) Files that are inaccessible or cannot be moved during backup are
not being backed up. If there is inadequate space on the BACKUP volume,
a fresh pack should be mounted. If the file was inaccessible for several
backups in a row, the backup schedule should be changed to a time when
the files will be available. A full listing of error messages can be
found in the System-Minder Instruction manual, appendix B.

\(4\) To read System-Minder log files, complete the following steps:

\(a\) From the ICSMENU, press PF2 to enter the FILEFIND utility.

\(b\) In the Library field type \"ICSLOGS.\" Leave everything else blank
and press RETURN.

\(c\) The screen will then display an ICSLOGS file for each volume on
which System-Minder programs ran. For example, if System-Minder ran on
volumes S1, V1 and V2, the FILEFIND screen would display the following
files:

File Library volume\
\
S1 ICSLOGS S1\
V1 ICSLOGS V1\
V2 ICSLOGS V2

\(d\) Move the cursor to the first filename, in this case, SI, and press
RETURN.

\(e\) From this screen, press PF1 to enter Display Mode. For each
program that ran, there will be an entry for

program \"STARTED\" and an accompanying entry for program \"FINISHED.\"
If any program does not show a completion time, refer to subparagraph b
for appropriate action. The log also contains a listing of messages
addressing files that could not be accessed, files with zero records,
etc. For a detailed translation of all messages, refer to the
System-Minder Instruction manual.

\(f\) Press PF16 twice to exit Display Mode and return to the FILEFIND
screen.

\(g\) Repeat steps outlined in subparagraphs (e) and (f) for each log
file listed.

\(h\) If the System-Minder backup system has run properly, and no action
items are necessary, the log files can be deleted. To delete the log
file from the FILEFIND screen, move the cursor under the filename and
press PF8. A warning message that the file is about to be deleted will
be displayed.

\(i\) Press PF8 again to actually delete the log file.

9-5

M20-4, Part 11 March 12, 1990 Change I

\(j\) Press PF16 to return to the ICSMENU.

b\. System Faults

\(1\) File Contention. The backup, file, and disk reorganization systems
check files for availability before accessing them and will not
experience contention problems. If other tasks are running in background
which open and close files repetitively, these tasks may hang if the
background tasks attempt to open a file being accessed by the backup
system. These programs will produce a message on the Operator screen and
the programs will hang until the operator acknowledges the problem.
After System-Minder has finished with the file, press RETURN to permit
background task to continue processing. These files should be bypassed
using SMCONTRL as described previously.

\(2\) Unfinished Tasks

\(a\) If the system goes down or, for some reason, one of the programs
in the package cannot be completed, it will be evident in the log file
since an entry for program startup will be found without an accompanying
entry for program completion.

\(b\) If this occurs, a work file may be left behind and can be deleted.
The file name will begin with \"@,\" followed by the second digit of the
day number, followed by the time. This file can be found by using the
FILEFIND program and entering \"@: plus the second digit of the day the
system went down for the file name while leaving the library and volume
blank. Since the system processes files one at a time, there will
generally be no more than one of these files left behind. Next, check
the library in which the \" @ \" file is found to be sure that no files
are missing, and then the \" @ \" file can be deleted. If not all files
are present, the \" @ \" file can be left on the system for a period of
time. The \" @ \" file will not interfere with the operation of
System-Minder and generally will not interfere with other programs
provided the library does not begin with \"O\" or \"DOCMNT.\" If it
begins with either of these library names, copy the file to a holding
library and note where it came from.

\(c\) Whenever a System-Minder task does not finish properly, it is
important to determine the cause of the problem to prevent its
recurrence. Possible causes are a machine malfunction, someone canceling
the program from an operator terminal, a damaged file, or insufficient
paging or work space. Normally, the system will create a file which will
either have printed automatically or be waiting as a print file. A
search with the program FILEFIND for all files beginning with DUMP in
libraries specified as \"#?PRT\" will locate any available dump files.
The dump file will usually identify the cause of the fault and the file
that was being processed, if any. If the cause of the problem cannot be
identified by these techniques, it may help to run the program module
directly from an interactive terminal. Any error messages will then be
written to the screen.

c\. VS Backup Log File

\(1\) The Backup Log File contains information about all files copied
during the backup, including any errors that prevented a file from being
copied. The log is created at the end of the backup process from
information in the Backup Work File, which is then deleted.

\(2\) The Backup Log File can either be printed on-line or stored on
disk as a print file, according to the chosen print mode. (The print
mode depends on the default set in the user\'s Usage Constants.)
Depending on the system, it may be more convenient to print the logs
than to maintain them on disk. If maintained on disk, the logs should be
deleted as they become outdated. Logs are no longer useful when the disk
pack to which they pertain are reused.

9.07 SYSTEM-MINDER RESTORATION OF FILES

a\. General. System-Minder is designed to accommodate three levels of
data recovery: immediate on-line recovery of individual files, a
multistep process for full disk recovery following a head crash or other
disk damage, and longer term system recovery following a CPU room
disaster. All levels can be provided on-line, without ever taking the
system down or restricting users.

9-6

March 12,1990 M20-4, Part II

Change I

b\. Restoration of Individual Files. Restoration of individual files can
usually be accomplished by simply copying the file back from the on-line
backup disk. If it is not there, the program SMSEARCH will identify
which incremental volume contains the latest version of the file.
Otherwise, the incremental backup disks should be loaded in the order in
which they were last used until the file is found. If it is not
contained on any of the incremental backup packs, the last full backup
volume can be mounted.

C. Restoration of Full-Volume. The System-Minder RESTORE program is used
whenever a full volume needs to be recovered. Initially, the full-volume
backup must be loaded to a new disk, which must have the same name as
the original production volume. Each of the incremental backup volumes
is then mounted in the order in which it was created, and the RESTORE
program is run to recover the appropriate files. The program will
request the names of the input backup volume and the output production
volume. The program will then copy back only the files that came from
the original volume. The program produces a log file that will indicate
if any files that should have been restored could not be restored,
either because they were deleted from the backup volume or because there
was insufficient space on the production volume.

9.08 FUL-VOLUME BACKUP

a\. Backup Copy. As with the system files, at least one backup copy of
all files will be maintained away from the station or the building where
the CPU is located. The off-site volumes should not be more than 4 weeks
old.

b\. Leading Full-Volume Backups. When running full-volume backups, the
system should be IPLed without the communication file @ CNS COM@. This
will insure that there are no open files such as the Directory file @
DIRECT@ that will be omitted by the backup. Backup processing time can
be reduced significantly if the system is IPLed with the configuration
file \@BAKFIG@. WSNSTOP (or WSNHALT) should always be run prior to
performing any BACKUP functions and logons should be inhibited when the
BACKUP program is executing.

c\. Performing an IPL (Initial Program Lead) From a Non-system Volume

\(1\) \@SYSTEM@. Full volume backup of the system volume (S1) requires
an IPL from a non-system volume (V1-V9).To perform an IPL from a
non-system volume, the library \@SYSTEM@ be maintained on this volume.
If the library \@SYSTEM@ is not installed, a copy can be restored from
the most current backup of S1. At a minimum this library requires the
following files to support IPL and the BACKUP utility:

\@CONFIG@ \@PROC@ \@TSKMGR@ \@7OV98E\
\@FFM \@PRTTSK@ \@WSLOAD@ \@CPYDMP@\
\@GENMSG@ \@QUEVER@ BACKUP \@RSFSYNC\
\@IOCDRT@ \@SHARER@ BUILDALT \@SECFILE\
\@IOCLOD@ \@SYSOO8@ DEVLIST CLOGON\
\@IOSLOAD \@SYSCPR@ DISKINIT COPY\
\@LOGGER@ \@SYSDBG@ FILSRV LISTVTOC\
\@MC2246S \@SYSDMS@ GENEDIT VERIFY\
\@MC4250 \@SYSGEN@ GENEDITF\
\@23V97W \@SYSINIT SECURITY\
\@23V98 \@SYSSERV SORT\
\@OPER@ \@SYSTRN@ USERLIST\
\@MC2256C \@SYSTSK@ WTEFORM\
\@MC4230 \@423OMWS \@SYSO12@ (VS-8000 ONLY)

\(2\) \@ERRMSG@. WLREMSG in \@ERRMSG@ is required to provide error
messages for operating system programs.

\(3\) \@SYSFORM@. PROCFORM in \@SYSFRM@ is required to provide eff or
messages for procedures.

9-7

M20-4, Part 11 March 12,1990 Change 1

\(4\) Backup Configuration. To accelerate full-volume backup processing,
it is recommended that a separate configuration file be created for
backup. The backup configuration file can be restricted to a few
workstations and printers, but must include all disk devices. The VTOC
(Volume Table of Contents) cache option in the backup configuration file
should be set to a minimum of 64 pages. The configuration file must be
maintained in the library \@SYSTEM@ on the system volume and the
non-system volume that will be used for IPL.

9.09 REINITIALIZATION OF DISK VOLUMES

a\. The primary function of the DISKINIT utility is not only to
initialize new disks, but to reformat a previously initialized disk
volume. It is recommended that aU disk volumes be reinitialized every 3
months. It is mandatory that they be reinitialized every 6 months.
Reinitializing a disk volume will identify bad block addresses and will
replace them, if possible, with good blocks from the reserve cylinder.
If all available reserved cylinder blocks have been used, the volume
will be identified as unusable.

b\. Reinitiatlization Procedure

\(1\) Prior to running the DISKINIT utility, a full-volume backup must
be performed (see par. 9.08).

\(2\) Run the DISKINIT utility; type \"BRIEF\' in the \"PASSES field
(fig. 9-2). For information concerning page pool sizes see chapter 2.

\*\*\*\*\*\*\*\*\
\*\
\* MESSAGE 026 BY WT1M00\
\*\
\* INFORMATION REQUIRED BY PROGRAM PWDRIVER\
\* TO DEFINE INPUT\
\* ACTIVE SUBPROGRAM IS DISKINIT\
\*\
\
\*Please enter the following input parameters for Initialization:\
\*\
\
\* Specified volume is X on device 050 removable platter, VSID 0.\
\*\
\
\* WARNING: INITIALIZE function destroys the VTOC and all other data on
volume!\
\* To terminate this function. press PF1. Press ENTER to continue.

\* NEWVOI. = S1\*\*\*\* (Volume Name for initialized volume)\
\* VSID = \*\*0 (0 =Single Volume; 1-255 =Volume Set, 1 =Root)\
\* LABEL = SL, (SL - Standard Label, NL - No Label)\
\* TOLERATE = NONE\* (NONE - Smallest VTOC - No fault tolerance)\
\* (CRASH - Medium VTOC Tolerate system halt)\
(MEDIA - Largest VTOC Also tolerate bad media\
VTOCSIZE = 0005 (Minimum of 4 page-size blocks)\
\
OWNER (Owner identification for VOL\] label)\
DUMPFILE NO\' (YES/NO - Allocate Control Mode dump area)\
PAGEPOOL YES (YES/NO - Allocate a pool for paging)\
PASSES BRIEF\* (BRIEF - Only I pattern for quick initialize)\
(NORMAL - All patterns for thorough initialize)

Figure 9-2. Diskinit Program Menu

\(3\) Once the disk reinitialization is complete, run backup and restore
the volume from the backup pack.

9-8