From: karttu@mits.mdata.fi (Antti Karttunen)
Subject: Re: WEIRD IDEA? (chroot) Some corrections.
Organization: MITS, Helsinki, Finland
Date: Mon, 28 Dec 1992 21:44:12 GMT
Followup-To: comp.unix.bsd,comp.unix.wizards
Keywords: chroot, shadow login, pirates BBS

It seems that there were some serious flaws in the chrooting-scheme
I posted here two days ago, flaws which would have resulted an
unbootable system in the worst case. I'm grateful to Gordon Burditt
for pointing them out, and with his permission I post the reply he
mailed to me.

> = Gordon.
>> = Me (Antti Karttunen).

>Date: Sun, 27 Dec 92 11:10 CST
>From: gordon@sneaky.lonestar.org (Gordon Burditt)
>To: karttu
>Subject: Re: WEIRD IDEA? (chroot)
>Newsgroups: comp.unix.bsd,comp.unix.questions
>In-Reply-To: <1992Dec26.191816.26596@prime.mdata.fi>
>Organization: Gordon Burditt
>Cc: 
>Status: RO
>
>>We are thinking about building a kind of unix BBS-system,
>>and for one reason (don't ask me why) I would want to keep certain class
>>of users segregated from the rest, and their id's in the separate
>>passwd-file.
>>
>>We probably will be using BSD 4.3, and I have thought about using the
>>chroot function for these special users, in the patched login I'm
>>about to hack.
>>
>>Now, let's assume that the root-directory for these special users
>>is called /usr/customers. To there I'll create subdirectories
>>bin, dev, usr, etc, 'emulating' the real /bin, /dev, /usr and /etc
>>directories, like is done with the most ftp-sites nowadays.
>>I want to give these users access to certain commands but not to all,
>>so I copy some commands from real /bin to our 'fake' /bin, namely
>>/usr/customers/bin. Because we probably will have / and /usr in the
>>separate filesystems I can't use hard links, and neither the symbolic links
>>will work from 'inside' /usr/customers to 'outside' of it. To the rest of
>>commands, programs and files in /usr/ucb, /usr/bin, /usr/games,
>>/usr/local/bin and /usr/lib I can make hard links with ln.
>
>Suppose your worst nightmare happens:  someone cracks root INSIDE
>the chroot environment.  You really don't want modified versions of
>binaries used outside the chroot environment - Trojan Horse programs
>could sucker root OUTSIDE the chroot environment into running them.
>You probably want COPIES of binaries instead of links, even if they
>take more disk space.
>
>Note that having the inode number of the chroot / not equal to 2
>(not the top of a mounted filesystem) is a dead giveaway to the user
>that he's in a chroot environment.
>
>>(Hmm, just realized that I could move some of the commands in /bin to
>>/usr/customers/bin and then make symbolic links to them from the real /bin
>>From 'outside' to 'inside' they will work, of course.)
>
>Sure you want to do that?
>
>Is /usr a mounted filesystem?  Consider what happens when you're single
>user or booting and and /usr isn't mounted yet.  DON'T put the only
>copy of mount on /usr !!!
>
>If your system has shared libraries, those need to be available in
>the chroot environment.  And you really don't want to have those
>only available on a non-root disk, as booting may become impossible.
>
>>About devices in /dev,  man chroot  gives the following warning:
>>
>>  BUGS
>>       One should exercise extreme caution when	referencing device
>>       files in	the new	root file system.
>>
>>and then tells nothing about why this extreme caution is needed.
>
>If someone cracks root INSIDE the chroot environment, they can make
>their own devs (they might have to download a binary of mknod)
>and do damage outside the chroot environment.  
>
>Having multiple copies of tty devs can cause problems with locking,
>especially dialin/dialout.  Your hacked-up login should change ownership
>on BOTH copies of the tty devs when someone logs into the chroot section,
>and change back both copies when someone logs out.   You'll probably
>have to maintain a chroot-version of utmp yourself.
>
>
>>With these I could probably again move the tty-devices and others
>>to /usr/customers/dev and then make symbolic links to them from /dev,
>
>Is /usr a mounted filesystem?  Consider what happens while booting,
>and you don't have /usr mounted yet.
>
>>or create devices with the same names into /usr/customers/dev
>>with /etc/mknod command. Probably it's this latter case where we
>>will run into more problems, if we have processes started by special
>>and normal users trying to access the same devices via /dev and
>>/usr/customers/dev, with differing protections and different owners
>>in these device-files.
>>
>>With mail there are probably some problems when considering the mail
>>between special and other users (separate spool-directories), but I
>>think they will be also solvable with symbolic links or weird alias-tricks.
>
>Leaving outside user's mailboxes inside the chroot environment leaves
>a possibility of mischief (spying, mail tampering, etc.).  Whether
>it's worth the risk is up to you.
>
>>1) Has anybody built this kind of system before, for _interactive_ users?
>>(I don't mean the ftp-servers).
>
>Well, I tried it briefly on System III Xenix.  The system wasn't really
>used heavily, and there weren't any seriously hostile users.  System III/
>System V login cooperates by using a shell beginning with '*' in the
>password file as an indication to do a chroot, and the home-directory
>field indicating where, and then it loops back and does its thing again
>using the new password files.  JFH's Shadow Login package does this
>nicely.  Even if you don't want to install it on your system, reading the
>source code will give you an idea of things you need to take care of.
>
>>2) Have I forgot anything essential, something which will run us
>>into deep problems later? (If we want to use some software X or thing Y
>>in the future.)
>
>I recommend initially, for testing purposes, writing a script to construct
>the /usr/customers tree, and making sure that if you do a 
>"rm -rf /usr/customers" except maybe for user-specific stuff (like
>home directories, mail, message base, and arrange for password and group
>file entries to survive the upgrade), none of the outside users will be 
>affected.  (DON'T put the only copy of binaries of standard utilities 
>inside the chroot environment!)  Have a root-privileged chroot login, 
>which you can get rid of after testing if you like, but you might find 
>it easier to use it to do maintenance like changing chroot-user's 
>passwords when they forget them.  You might need this "build" script if 
>your system software gets an upgrade, to upgrade the copies.  
>
>Keep editing the script, and tearing down and re-creating /usr/customers, 
>until you are convinced it works well enough to let real users in.  It
>might be worthwhile if you have a couple of knowledgable and hackerish
>but trustworthy users who can test out the environment and report bugs
>and security holes, don't mind having the environment destroyed 
>occasionally, and let them in during the initial testing.
>
>Make yourself an unprivileged chroot'ed login from which you can test
>things.  It's very easy to disable things by forgetting or not knowing
>that ls calls awk sometimes, or similar things.  When users complain
>that things are broken, use your test account to reproduce the problem.
>Don't give it group "sys" or any special privileges - admins with
>privileged logins are often blind to problems because their accounts
>don't have the problem due to extra privileges.  This account might also 
>be a good point-of-contact for mail from chroot customers.
>
>Lots of wierd things will happen if you forget to give users access
>to /dev/null and /dev/tty.  Don't forget to give the users a
>decent termcap file.  Someone has to maintain a pseudo-utmp file
>containing at least the chroot users (some programs respond with
>"you don't exist!" or other rude messages if an entry isn't found).
>
>						Gordon L. Burditt
>						sneaky.lonestar.org!gordon

Further points from Gordon:

>There are other programs you don't want stuck over in /usr/customers
>while you're trying to fix the system after a bad crash.  It is useful
>to be able to fix the root disk first, THEN worry about fixing the others,
>rather than having to do all of them at once.  
>
>It's nice to have, say, ls around.  Things like test, expr, and 
>echo (if not built into the shell) are handy so shell scripts work.
>Oh, yes, you need a shell, too.  rm, mkdir, rmdir, cat, and an editor
>are handy also.  All of these might be useful over in /usr/customers
>also, but you don't want the only copy there.  Naturally you also need
>restore programs, fsck, mkfs or newfs, but users probably don't need
>these.
>
>In general, I'd advise not having the only copy of any standard system
>executable over under /usr/customers.  Make copies.  You can put specially 
>crippled editors, specially configured mailers, unique versions of the 
>password and group files, user data, BBS software and a message base 
>there, but not stuff the normal system needs.  I'm not that familiar with 
>NFS, but if your system has it, you might manage to NFS-mount a bin 
>directory on your root disk onto /usr/customers/bin (preferably read-only).
>

JFH's Shadow Login package mentioned above can be downloaded at least
from the following ftp-sites:

pinus.slu.se:
/pub/usenet/comp.sources.misc/volume26/shadow  (The main code is here)
                              volume28/shadow  (These later volumes
                              volume29/shadow   contain
                              volume30/shadow   just patches).

archie.au: similarly in /usenet/comp.sources.misc/volume26-volume29
gatekeeper.dec.com: similarly in /.0/comp.sources.misc/*

And from many other sites also. Check with your local archie.

Other netters have told me about the software called Pirates BBS, which
also uses a chroot trick. It can be downloaded from the ftp-site:
src.doc.ic.ac.uk::/computing/comms/bbs/PBBS

(Note: Now I crossposted this to comp.unix.wizards also, and set the
follow-ups so that comments won't go to comp.unix.questions anymore.
I hope the topic is/remains wizardy enough.)

--
Antti Karttunen / karttu@mits.mdata.fi / $B%"%s%C%F%#!!%+%k%C%H%%%M%s(J