From: op@holmes.acc.Virginia.EDU (Olaf  Pors)
Subject: Re: rootvg restores by inode
Originator: op@holmes.acc.Virginia.EDU
Organization: University of Virginia
Date: Thu, 3 Sep 1992 18:13:34 GMT

> I have never heard of this requirement before - and Ive heard many
> about AIX deficiencies from customers and support personnel.
> (at least more than I can count using my fingers and toes :-))

  The requirement referred to is to be able to completely,
accurately, and relatively automatically restore the root
volume group from daily backup tapes.  In other words, at
our site, if a rootvg disk dies, we need to be able to
restore rootvg to exactly the state it was in when the
previous day's backups were done.  On a standard system,
this is close to impossible, since it means doing a mksysb
every day.

  I have always taken this requirement for granted.  In
your words, I have never heard of this requirement NOT
being met before, except on IBM systems.  Try to find an
experienced CDC NOS system administrator who doesn't
expect the combination of the deadstart tape and daily
PFDUMP tapes to fully restore the disks.  Or find an
experienced SunOS or SGI administrator who doesn't expect
to be able to boot miniroot and restore every partition,
including the root partition, directly from daily backup
tapes.  Similarly for a Vax running BSD Unix.  Or how
about an MSDOS PC? Boot from a diskette, format your hard
disk, and use RESTORE to completely restore everything
from daily diskettes created by BACKUP.  If your PC is a
Novell Netware server using the Maynard software package
to create daily 8mm backup tapes, during a restore it fully
restores all files from tape, cleverly sitting around in
memory to be able to overwrite everything on disk.

  If customers have never expressed concerns along these
lines, you've been talking to the wrong customers!
Backup/restore software on the platforms listed above was
not developed on a whim - the developers perceived a need,
a requirement, otherwise they wouldn't have created the
software.

  The "backup gap" became very apparent when we ran VM/HPO
on our IBM 3090.  It was possible to do binary copies of
entire disk volumes, or selected files, but no capability
existed for daily incremental backups (and restores).  It
took a third party, Systems Center, Inc, to perceive the
need and develop the VMBackup software product to fill the
need, for a handsome price.  But even at VM/XA 2.1, saved
segments and spool files are not handled by VMBackup, so
restoring disks completely is an ongoing problem.

  AIX is similarly afflicted.  It is currently impossible,
without local modifications, to back up and restore your
root volume group by inode.  This means that unless you
are willing to do a mksysb every day, you can't have
complete, accurate, and automatically-restorable daily
backups of the root volume group.

> And I dont see how inode backups solves your problem.

  Unlike backups by name, backups by inode allow for
incremental backups - only those files that have changed
since the last backup, are written to tape.  This makes it
feasible to back up all disks daily.

> 11 systems programmers independently changing the rootvg of 27 RS6000's
> sounds like quite a unique environment to me.

  I need to explain this better.  Judging from my contact
with the Support Center and PTFs I've seen, IBM software
development has always been quite disciplined.  You
development folks don't just decide to generate some code
out of the clear blue sky.  Software is discussed and
approved by managers, is tested, cataloged, and documented.
IBM is a money-making outfit, and can't afford chaos.  In
our state university environment, there's no such
discipline.  We're paid the same, regardless of how well
software is organized, so there's very little incentive
for efficiency.  So our jobs are a congenial affair:
systems programmers often make changes without documenting
them for others, and if something breaks, it gets fixed.
Some amount of chaos results.  For example, someone takes
the time to fix a certain system problem on a certain
machine.  The fix doesn't get documented, or gets
documented in a confusing manner, or nobody refers to the
documentation, so that the same problem reappears later on
other machines, and somebody else has to reinvent the
wheel.  Or a certain convention is enthusiastically
adopted, only to fall by the wayside as people
enthusiastically pursue other interests.  Fortunately,
chaos is limited by the new operating systems released by
vendors such as IBM and Sun who have the necessary
discipline to incorporate fixes into new releases.  Get
the picture?  In our environment, changes are made by
independent programmers to files in rootvg such as
/usr/lib/terminfo, /etc/exports, /etc/ethers,
/etc/printcap, /etc/inittab, /etc/syslog.conf, to name a
few.  Changes over which we have no control are made to
/etc/passwd, and various files in /etc/security.  It's
pretty strange to have to lose all changes to these files
since the last mksysb if a disk dies.  It's also strange
to consider keeping some kind of a list of all possible
files that may change, and, say, tarring only these to
tape daily.  Who can guarantee that the list stays up
to date?

  I believe that our environment is not unique, but typical
for state agencies and possibly federal agencies as well.

> Bottom line is that I dont think the IBM designers are concerned about
> this because it seems to be the first time that it has come up.

  Barring just a plain oversight of the Berkeley
backup/restore by inode programs, or an incredible amount
of insulation from the real world, I suspect that $$$ are
involved.  After all, IBM is not our big warm fuzzy friend
- it is a business.  Perhaps the managers figured out that
they could sell an operating system without suitable
backups.  As long as it makes money, a business does not
need to care about occasional customer inconvenience.
This is, however, not a smart long-term sales attitude.
Just ask the Japanese.

>  At first glance, the first
> solution that comes to mind is to change volume group arrangements so
> that rootvg is more static. IE rootvg should just have AIX stuff in it.

  This is probably what we would have to do, if our
restore-by-inode local mods can no longer be maintained.
But you can only make it so static - see the list of files
above.  Thank you for the suggestion, but I'm still hoping
that the AIX designers/managers will find a motivation
to keep in step with the rest of the world.  Who does one
contact about a design change along these lines?  Or have
my recent postings accomplished that?

Olaf Pors
Academic Computing Center
University of Virginia
op@Virginia.EDU
804-924-0633
Newsgroups: comp.unix.aix