From: spencer@med.umich.edu (Spencer W. Thomas)
Subject: 4.2 to 4.8 upgrade experiences
Date: Fri, 18 Dec 92 12:04:17 EST
Organization: University of Michigan
Distribution: comp

This week, we finally decided to do the 4.8 upgrade thing.  After
hearing from the net how easy it was, we (first mistake) started early
in the afternoon.  We've got a pretty small database, so figured it
shouldn't take too long.

1. Dump current databases to DAT tape.
2. Move old software out of the way.
3. Install new software from tape.  OOPS, it's bigger than the
   partition that the old software fit in.  Reinstall without DWB.
4. (Mistake 2) Run sybconfig and ask it to upgrade. 
5. Upgrade fails because we forgot to increase the SHMEM size in the
   SunOS kernel.  Takes a while to figure this out because we're
   working from the wrong copy of the installation manual (we've got
   one with full instructions, one with just instructions for the
   toolset (APT)).
6. After finding the right manual, reload SunOS and reboot.
7. Run buildmaster, installmaster, and load database master from tape.
8. Now dbcc fails on all the user databases (overlap of something
   with object "sysgams").  Restore them.
9. Run upgrade again.  dataserver goes into an infinite loop during
   upgrade.
10. Buildmaster, installmaster load database master, load database
   (user databases).
11. Dbcc checkalloc shows an inconsistency in one of the user
   databases that apparently is present in the dump file.  Offending
   table is not used any more (this is the "test" db, so has some junk
   in it).  Drop table fails miserably because of allocation problems.
12. Restore test database again, set up directories for 4.2 server,
   and go home (almost 8pm at this point).
13. (Next morning) Using useful script from comp.databases.sybase,
   dump out the schema of the "broken" db. Use bcp -n to dump all
   tables into flat files.  Drop database, recreate, rebuild schema,
   bcp data back in.
14. Dump all databases to DAT again.
15. Tryupgrade succeeds.  All right!
16. Run sybconfig.  Upgrade fails with a memory allocation failure
   (suggestion: physical memory fragmented), it wanted 2million of
   some unknown unit of memory (if bytes, I don't understand why it
   failed, if some larger unit, why was it asking for that much??).
   This problem was repeatable at this point, but disappears later.
   Not sure why.
17. buildmaster, installmaster, load database master from dat, load
   user databases from dat.  The last load fails with a bogus logical
   page number.  Very mysterious.
18. dbcc checkalloc shows serious problems in that user database, but,
   more seriously, also in model and tempdb.  buildmaster,
   installmaster does not fix these, and they can't be dropped.
19. Rerun 4.2 sybinstall to get a totally clean db.  Load old master
   from tape.  Server now marks old user dbs as unrecoverable, so drop
   them with dbcc dbrepair and reload (except for the one that won't
   load) from tape.
20. Investigate load problem, and discover that there is a bad spot on
   the tape in the middle of the last dump.  This is why it won't load
   (see my other message where I flame on this).
21. Rebuild offending database from schema and bcp files.
22. Run tryupgrade.  It works.
23. Run real upgrade with fingers crossed.  It works, too.
24. Start 4.8 dataserver.  Runs fine.
   
Moral(s) (All obvious to the experienced practitioner, I'm sure, so I
   don't need any "I told you so" responses -- I'm already kicking
   myself enough, and I should know better after 20 years in this
   field):

A. Make sure (doubly) that your DB is absolutely clean before you
   upgrade.  Any inconsistencies will totally screw up the upgrade.
B. Make sure the 4.8 server will run before applying it to a real
   database.  We could have avoided the first round of failure if we
   had done this.  Just try to create a toy DB with it.
C. Make sure your dumps are clean and readable.  We had both
   problems: our readable dump wasn't clean, and our clean dump wasn't
   readable.
D. Consider doing an unload/reload instead of dump/load.  This is
   significantly more painful, but has less that can go wrong with it.
   Be sure to dump tables like sysusers, and be sure to get all your
   types, triggers, rules, and stored procedures in addition to the
   data.
E. Make sure you know the exact segment configuration of each
   database, because if you don't, you may not be able to load it from
   a dump if you had to drop it or dbcc dbrepair(x,dropdb) it.  (The
   installation manual tells you to do this.  In my experience, it's
   really important!)
F. Start early in the morning on a day when you have nothing else to
   do.  If all goes well, then you can regard the rest of the day as
   "won time".  If it doesn't, you reduce the amount of overtime
   you'll have to put in.

--
=Spencer W. Thomas 		|  Info Tech and Networking, B1911 CFOB, 0704
   "Genome Informatician"	|  Univ of Michigan, Ann Arbor, MI 48109
Spencer.W.Thomas@med.umich.edu	|  313-747-2778, FAX 313-764-4133
Newsgroups: comp.databases.sybase