From: per@erix.ericsson.se (Per Hedeland)
Subject: Re: SUMMARY: Login box for openwin
Organization: Ellemtel Telecom Systems Labs, Stockholm, Sweden
Date: Wed, 12 Aug 1992 14:05:42 GMT

In article <1992Aug11.013417.15041@ccu.umanitoba.ca>, mills@ccu.umanitoba.ca (Gary Mills) writes:
|> >  There should still be a way to get to the
|> >console vt100 emulator in case the user wanted to run something else,
|> >like sunview.  A button or a magic login name might do it.  Can this be
|> >done with X11R5?
|> 
|> Thanks to all that replied.  What I would like seems to be impossible
|> with the tools available.

Well, not impossible - at least we're doing it, but it's perhaps a bit
awkward... The ideal solution would be to have something like a "shell
escape", that let the user get a login shell on the console, and then
when he logged out fired up the X server again - and *that* is not
possible without hacking xdm, as far as I know.

But there are other ways - xdm has the abort-display() action, which can
be bound to a e.g. a function key and will cause it to just kill the X
server and stop managing the display in question. Next, the login
shell: I've found that leaving the console getty enabled doesn't
conflict with xdm/server in any way, and as soon as the server is
killed, the getty is ready for a normal login (i.e. the user should call
abort-display() *before* logging in under xdm).

Finally, getting xdm to fire up the X server again is done by logging in
as a dummy user, which may or may not have a password and has the
program below as "login shell" (you get to decide whether it (now:-)
meets your definition of "tools available"). The security note might not
apply if you have installed recent SunOS patches - but I won't make any
guarantees in either case...

All this under X11R{4,5}, of course, I haven't tried it with OpenWindows
(users will have to go through the contortions above if they want to run
it:-) - but I can't see any reason why it wouldn't work, provided you
can overcome the apparent difficulties with getting it to run *at all*
under xdm.

Hope this helps...
--Per Hedeland
per@erix.ericsson.se  or
per%erix.ericsson.se@sunic.sunet.se  or
...uunet!erix.ericsson.se!per

xdmrestart.c-------------------------------------------------------------
/*
   Little program to "restart" xdm, making it remanage aborted displays.
   We require it to be started from the console (code stolen from xdmshell),
   and that argv[0] starts with a dash - the idea is to install it setuid
   to root and use as login shell for some dummy user (e.g. 'x').

   NOTE: It is important that the dummy user has a harmless userid (e.g.
   that of 'nobody') and this program is setuid, rather than the dummy
   user having userid 0, otherwise this can be exploited as a security
   hole (by playing tricks with LD_LIBRARY_PATH and running login -p).
   Static linking might be a good idea, too.
*/

#include <stdio.h>
#include <sys/file.h>
#include <sys/ioctl.h>
#include <signal.h>

/* Shouldn't hardwire this here but it's too hard to do right... */
#define PIDFILE "/etc/xdm/xdm-pid"

main (argc, argv)
    int argc;
    char *argv[];
{
    int ttyfd, consfd;
    int ttypgrp, conspgrp, pid;
    char *ttyName;
    FILE *pidfile;
    extern char *ttyname();

    if (*argv[0] != '-') {
	fprintf (stderr, "%s:  must be run as login \"shell\".\r\n",
		 argv[0]);
	exit (1);
    }

    if (argc > 1) {
	fprintf (stderr, "usage:  %s\r\n", argv[0]);
	exit (1);
    }

    ttyfd = open ("/dev/tty", O_RDWR, 0);
    if (ttyfd < 3) {			/* stdin = 0, stdout = 1, stderr = 2 */
	fprintf (stderr, 
		 "%s:  must be run directly from the console.\r\n",
		 argv[0]);
	exit (1);
    }
    if (ioctl (ttyfd, TIOCGPGRP, (char *)&ttypgrp) != 0) {
	fprintf (stderr, "%s:  unable to get process group of /dev/tty\r\n",
		 argv[0]);
	(void) close (ttyfd);
	exit (1);
    }
    (void) close (ttyfd);
    
    ttyName = ttyname (0);
    if (!ttyName || strcmp (ttyName, "/dev/console") != 0) {
	fprintf (stderr, "%s:  must login on /dev/console instead of %s\r\n",
		 argv[0], ttyName ? ttyName : "non-terminal device");
	exit (1);
    }

    consfd = open ("/dev/console", O_RDWR, 0);
    if (consfd < 3) {			/* stdin = 0, stdout = 1, stderr = 2 */
	fprintf (stderr, "%s:  unable to open /dev/console\r\n",
		 argv[0]);
	exit (1);
    }

    if (ioctl (consfd, TIOCGPGRP, (char *)&conspgrp) != 0) {
	fprintf (stderr,
		 "%s:  unable to get process group of /dev/console\r\n",
		 argv[0]);
	(void) close (consfd);
	exit (1);
    }
    (void) close (consfd);

    if (ttypgrp != conspgrp) {
	fprintf (stderr, "%s:  must be run from /dev/console\r\n", 
		 argv[0]);
	exit (1);
    }

    if ((pidfile = fopen(PIDFILE, "r")) == NULL ||
	fscanf(pidfile, "%d", &pid) != 1 || pid < 3) {
	fprintf (stderr, "%s:  unable to get PID of xdm from %s.\r\n",
		 argv[0], PIDFILE);
	exit (1);
    }

    if (kill(pid, SIGHUP) != 0) {
	fprintf (stderr, "%s:  unable to restart xdm (PID %d).\r\n",
		 argv[0], pid);
	exit (1);
    }
    
    exit(0);
}
Xref: sparky comp.sys.sun.admin:5465 comp.sys.sun.hardware:3830