From: drew@ophelia.cs.colorado.edu (Drew Eckhardt)
Subject: Re: Stabilizing Linux
Organization: University of Colorado at Boulder
Date: Thu, 13 Aug 1992 19:54:06 GMT

In article <1992Aug13.133313.15221@crd.ge.com> davidsen@crd.ge.com (bill davidsen) writes:
>In article <1992Aug12.173012.17552@colorado.edu>, drew@kinglear.cs.colorado.edu (Drew Eckhardt) writes:
>
>And the person trying to install on a new machine can't rebuild the
>kernel, so s/he needs another working system to get going. Bleh.
>Relocation at load time might take 1-2 sec of cpu. Big deal. Without
>loadable device drivers you must have a working system to generate a new
>kernel.

Most Unices distribute the distribution kernel as a generic kernel,
ie one with support for all device drivers compiled in.  It gets a little
bloated, but if you can boot it, you can run anything.

The Linux kernel is distributed the same way - for instance, .97
incorporates support for Adaptec 154x, Future Domain 16 bit, 
Seagate, Ultrastor 14F, and Western Digital 7000FASST SCSI 
boards.

>As for generating a new kernel in 30 sec, some Linux users don't have
>the disk to keep the kernel sources and objects online all the time (or
>even any of the time).

Currently, my Linux source tree weighs in at 1.5-2M.  That's 
for one copy of "checked out" sources under RCS, another copy 
still under RCS, plus intermediate .o files for everything else. 

A minimal system, with only driver ".o" files, Makefile template, 
automatically generated Makefile and ioconf.c, plus room for 
the kernel, should fit in under 1M.  

>For those without a 486-800 to recompile the kernal in 30 sec, here's

Linking is significantly faster than recompiling the kernel.

>how loadable dd's work. If you want to install on a new dipthong50 disk
>controller, you boot the floppy kernel, and then add the device driver
>for your HD controller by reading it off a floppy. Then install as
>usual, and write the enhanced kernel on the hard disk. No sources, no

So?  The kernel configuration method takes another 500K of disk to
hold all of the .o files.  With a minimal system having 40M of disk 
these days, that's insignificant.  Using a kernel configuration program 
instead means we don't need hooks for character devices, block devices,
and any pseudo-devices, like the low-level SCSI drivers and 
perhaps the ethernet drivers (like under BSD - I haven't looked 
at the Linux TCP/IP code yet, so I won't make a definate comment on
this).

>compile time, just snuggle some position independent code up to the
>kernel and adjust the vectors in the kernel to use it.

You also have to allocate a contiguous piece of physical memory large 
enough to hold the device driver, since drivers using DMA expect that 
kernel memory is identity mapped.

Furthurmore, you have to patch the Linux kernel so that it doesn't 
try to swap the newly allocated memory  - you can't swap kernel 
memory.  Swapping kernel memory doesn't work because you may 
get a page fault from within an interrupt handler.  This will 
get passed to a device driver, which will try to sleep - but 
you can't sleep from within an interrupt handler.

Other things that have to change include the initiation code 
for all of the subsystems (character device, block device, 
SCSI low level driver) so that new drivers can be added later, 
the memory allocation code so that the buffer cache and user 
memory pool can shrink and kmem can grow, etc.  

The changes would be non-trivial.

>Same ease of testing new device drivers, load them with the kernel in
>memory, and if the drive panics the kernel, as might ahppen now and
>again, you can just reboot. Beats rebuild kernels, booting from floppy,
>etc.

Since shoelace was ported to work with Linux .012, I have never booted 
a new kernel from floppy.  I can just spacebar it when it tries 
to load /vmunix, and substitute /usr/src/linux/Image.

If you eliminate fsck's (When I *know* I'm going to reboot, I 
touch /fastboot, which is checked by /etc/rc on startup.  If it's 
there, it assumes the system went down cleanly, and doesn't
fsck anything), reboots are quite tolerable.


>You may be able to rebuild your kernel in 30 seconds, but I doubt that
>many other linux users can. Besides, this o/s is breaking a lot of new

That's not a complete rebuild - that's essentially running gcc on 
one file, a couple of make processes, and running ld and 
tools/build on the whole mess.  If the device drivers are available 
in .o form, the only changes made to config generated ioconf.c and 
Makefiles deciding what gets linked in, ANY ONE should be able 
to relink a custom kernel, on a 386, in under a minute if it 
isn't thrashing heavily (ie, 4M of RAM, which is the Defacto Standard
for new PC's.  If not, RAM wholesales for ~$25 a meg, and you should 
be able to buy it for ~$30).

>ground, and the kernel needs virtually no extra code, since the memory
>allocator and disk io is there already. Just a bit of code to actually

Have you ever used the kernel memory allocators, either the 
get_page() function, or the kernel malloc()?  The former can 
allocate single pages, and they will be non-contigous (which 
will break things assuming that kernel memory is 
identity mapped, and use DMA), and kernel malloc() is a bucket 
allocator that is limited to  memory pieces smaller than 
page sized.

>add the driver to that vectors. If the system was limited to only what

It's not that simple - if it was, some one would have done it already.
It involves significant modifications to the Linux memory allocation 
code, device driver code, and initialization code.  Also, you'll
have to hack a user level ld program that performs the necessary 
relocations and resolves references to symbols listed in the 
kernel nlist.  Finally, if you want to dump the new kernel do disk,
you'll need to figure out some way to dump it, with the 
INITIAL values of all variables, not what they currently are.


>every single user needed, it wouldn't be very righ (or open).

You can make your own version of the kernel that has whatever options 
you want in it, dynamically loaded device drivers won't change that.

-- 
Microsoft is responsible for propogating the evils it calls DOS and Windows, 
IBM for AIX (appropriately called Aches by those having to administer it), but neither is as bad as AT&T.  Boycott AT&T, and let them know how you feel.