From: OLIVOTTO%ITNCISCA.bitnet@icineca.cineca.it (OUT, OUT - YOU'LL NOT FEEL THE FALL-OUT...)
Newsgroups: comp.unix.wizards
Subject: (From: IN%"Unix-Wizards@BRL.MIL") (From: IN%"UNIX-WIZARDS@BRL.MIL")
Date: 11 Oct 92 00:38:05 GMT

UNIX-WIZARDS Digest          Sat, 10 Oct 1992              V16#034

Today's Topics:
            DAMN: deleted all my files; saved partition tho'
          Re: DAMN: deleted all my files; saved partition tho'
                             Disk leakage?
                           Re: Disk leakage?
                      Re: fork() and forget..How?
                          malloc() and free ()
                        Re: malloc() and free ()
                          Re: Keyboard logging
               process(es) stuck in 'D' wait (Sun OS 4.1)
                         TTL value in ip struct
                       Re: TTL value in ip struct
                         Help : Copy protection
                       Re: Splitting the connect

-----------------------------------------------------------------

From: Doc Chaos <mdr@bjorn>
Subject: DAMN: deleted all my files; saved partition tho'
Keywords: rm bonehead
Date: 7 Oct 92 18:18:43 GMT
X-Newsreader: Tin 1.1 PL4
To:       unix-wizards@sem.brl.mil

Yeah, bonehead at the wheel, typed 'rm *' into wrong window.
Unfortunately the files were never backed up - they weren't
really important, but I'd like them back, if possible.  It
would take me at least two days to get them all again, and
what the hell, it could be a useful exercise.

What I did was to immediately 'dd' the whole partition containing
the "deleted" files onto a tape; total size ~118 Meg; deleted
files about 100 taking up 30M.  The system is a SPARC running
SunOS 4.1.1.  I guess I've got to muck around with inodes or summat
but don't really know where to start.  I'm guessing it can be done
'coz people make a living out of rescuing schmucks, don't they? :-)
I'd appreciate any and all help.

AtDhVaAnNkCsE

mark

--
m.robinson@reading.ac.uk

"I really like the jacket but the sleeves are much too long" - Lemmy

-----------------------------

From: "Alan Rollow - Alan's Home for Wayward Tumbleweeds."
 <alan@nabeth.enet.dec.com>
Subject: Re: DAMN: deleted all my files; saved partition tho'
Keywords: rm bonehead
Date: 7 Oct 92 22:25:14 GMT
To:       unix-wizards@sem.brl.mil


In article <BvrK78.Hvq@csug.cs.reading.ac.uk>, mdr@bjorn (Doc Chaos) writes:
>
>Yeah, bonehead at the wheel, typed 'rm *' into wrong window.
>Unfortunately the files were never backed up - they weren't
>really important, but I'd like them back, if possible.  It
>would take me at least two days to get them all again, and
>what the hell, it could be a useful exercise.
>
>What I did was to immediately 'dd' the whole partition containing
>the "deleted" files onto a tape; total size ~118 Meg; deleted
>files about 100 taking up 30M.  The system is a SPARC running
>SunOS 4.1.1.  I guess I've got to muck around with inodes or summat
>but don't really know where to start.
>
Odds are Sun clears all the inode when they free it.  So, even if
you can find the inodes that were associated with the files there
won't be block address left.  There are some solutions.

Theme - Brute force.  Examine every block and save the ones that
look like your data.  Then put the pieces of the puzzle back
together.

Variation #1 - Optimize.  The file system is arranged in block that
are probably either 8 KB or 4 KB long.  Rather than examine disk
blocks, examine file system blocks.

Variation #2 - Optimize some more.  Since Sun is probably still using
something that looks a lot like a Berkeley fast file system, each
cylinder group should have a bit map or something to indicate which
blocks are free.  Only examine those.  The implementation of this
is left as the exercise to the reader.  Clue:

The free map appears to be the data structure cg_free in the
description of a cylinder group.  Once you can find the cylinder
group blocks for each cylinder group, this should be fairly easy
to examine.  The macro cgtod() might be of particular interest.
Also keep in mind that many things are counted in file system
fragments as opposed to disk blocks.  You might have to look
around a bit to make sure you have the cylinder group block.

Variation #3 - And more.  Once you get the point that you can examine
things a cylinder group at a time, you may be able to take advantage
of the fact that you can make a good guess where the files were.  Files
tend to get allocated in the same cylinder group as the directory
they are in.  If you still have the directory, you can find what
cylinder group it is in and start there.

>AtDhVaAnNkCsE
>
>mark
>
>--
>m.robinson@reading.ac.uk
>
>"I really like the jacket but the sleeves are much too long" - Lemmy
>
--
Alan Rollow                             alan@nabeth.cxo.dec.com

-----------------------------

From: David Steffens <das@eplunix.uucp>
Subject: Re: DAMN: deleted all my files; saved partition tho'
Date: 8 Oct 92 20:39:59 GMT
To:       unix-wizards@sem.brl.mil

In article <1992Oct7.222514.29935@nntpd2.cxo.dec.com>,
alan@nabeth.enet.dec.com (Alan Rollow) writes:
> ... each cylinder group should have a bit map or something to indicate
> which blocks are free.  Only examine those.

Eh?  If the partition was completely wiped, then _all_ the blocks should
be on the free list and the bit map should show all 1's.  How can this help
to figure out which blocks were once allocated, but have since been freed?
--
David Allan Steffens       | I believe in learning from past mistakes...
Eaton-Peabody Laboratory   | but why does a good education require so many?
Mass. Eye & Ear Infirmary, 243 Charles Street, Boston, MA 02114
{harvard,mit-eddie}!eplunix!das    Tel (617) 573-3748    Fax (617) 720-4408

-----------------------------

From: Barry Shein <bzs@ussr.std.com>
Subject: Re: DAMN: deleted all my files; saved partition tho'
Date: 10 Oct 92 02:18:58 GMT
To:       unix-wizards@sem.brl.mil


>> ... each cylinder group should have a bit map or something to indicate
>> which blocks are free.  Only examine those.
>
>Eh?  If the partition was completely wiped, then _all_ the blocks should
>be on the free list and the bit map should show all 1's.  How can this help
>to figure out which blocks were once allocated, but have since been freed?

I don't think the orginal poster said the entire partition was wiped
out, only the entire directory, in which case limiting examination to
free blocks is an optimization.

I have played with this problem (recovering deleted files from a SunOS
partition) and the original suggestions all made sense.

Two more suggestions:

Write a heuristic tool similar to the "file" command which "guesses"
whether or not a file system block appears to be human-readable text
(assuming most of what you hope to recover is human-readable text.)

One good way to do this is just look at the characters in the block
and ignore all printables and common controls (LF, CR perhaps, ^L) and
play around with a weighting for how good the block is (like add 1 for
every unlikely control, 2 for every character >0176, etc.) The gotcha
here is half-ascii files like from WYSIWYGs where they might begin
with a hunk of binary but thereafter have good hunks of readable text
worth recovering (or even pasting back together into the format), they
can fool a program like that. Tempts one to distance-weight readable
text (e.g. by run-length etc), but it can get pretty crazy pretty
fast.

Similarly, look at /etc/magic if you use certain tools like a WYSIWYG
to see if you can figure out a distinctive header for files you are
interested in, and files you aren't (eg. ignore a.out headers if
you're pretty sure you don't care about such blocks.) Remember that
other than frags files start at block boundaries so their distinctive
headers (where they exist) will be at the beginning of the block.

Hell, if you're patient you could write each block to a tmp file and
run "file" on it and see what it comes up with and grep through the
output for obvious good hits (e.g. sparc executable, GIF file,
compressed, etc.)

Also, directory blocks, if they were wiped out, will start with the
stuff described in /usr/include/sys/dir.h and the first two names will
be . and .. (ok, in theory that's not absolutely necessary, though it
will be true unless something weird is going on.) So suck back two
struct direct's of stuff from the top and check d_name[]:

        struct direct d[2];

        read(fd,d,sizeof(d));

        if((d[0].d_name[0] == '.') && !strncmp(d[1].d_name,"..",2))
           probably_a_directory.

Locality is very important, keep an eye on cylinder groups as the
Rollow mentioned, bunches of readable-text in the same cylinder group
tend to be from the same file.

It's probably worth building a little yes/no previewer front end which
shows some stuff (a block), what the program thinks it likely is, and
offers to save it away easily so you can rebuild files on the fly from
blocks with a few keystrokes.



--
        -Barry Shein

Software Tool & Die    | bzs@world.std.com          | uunet!world!bzs
Purveyors to the Trade | Voice: 617-739-0202        | Login: 617-739-WRLD

-----------------------------

From: Paul Smee <smee@bristol.ac.uk>
Subject: Disk leakage?
Date: 8 Oct 92 09:53:28 GMT
To:       unix-wizards@sem.brl.mil

We've got a mystery, in that large (in our context) chunks of disk
occasionally mysteriously vanish, to reappear if you reboot.  I believe
I understand the mechanism, question is a bit deeper than that.

My current theory is that the space is 'vanishing' owing to files which
have been removed from the directory hierarchy, but which are still
being held in existence because someone still has them open, so keeping
the inode's refcount from zeroing.  Sounds reasonable, Unix does that.

Question is, anyone have any magic which would aid in determining which
process(es) still hold the files, or (less useful, but also less
likely) even which files they were while they still were attached to
directories, from which I could probably make reasonable guesses
about who was using them?

The space never comes back before reboot, so the 'holder' is almost
certainly either one of the daemons, or a 'zombie' (though I've found
no correlation between number of Zs and disk loss), or (worst case) one
of the standard 'listeners' -- init, getty, ...

The system is small, but pivotal to our service, so taking it out (or
even selected processes out) for extended periods is not an option --
so, something which works on the fly would be best.  We don't have
another similar box which we could nab for extended test work.  And,
just to put the icing on the cake, we couldn't afford a source license
for it, so I'm flying semi-blind.

System is (if it matters, probably does) a Sun Sparc 4/330 on SunOS
4.0.3.  I'm (probably vainly) hoping someone has already seen and
cracked this.  RTFM is, as always, an acceptable answer, but if that IS
the answer, some pointers into the FM would be really appreciated.
(I've already gone thru the obvious man -k on inode, dir, and file, and
read, several times each, man dcheck, icheck, and fsck.)  Alternative
theories about possible cause would also be appreciated.  Thanks...

--
Paul Smee, Computing Service, University of Bristol, Bristol BS8 1UD, UK
 P.Smee@bristol.ac.uk - ..!uunet!uknet!bsmail!p.smee - Tel +44 272 303132

-----------------------------

From: Barry Margolin <barmar@think.com>
Subject: Re: Disk leakage?
Date: 8 Oct 92 17:30:46 GMT
NNTP-Posting-Host: telecaster.think.com
To:       unix-wizards@sem.brl.mil

In article <1992Oct8.095328.4108@bristol.ac.uk> P.Smee@bristol.ac.uk (Paul Smee)
 writes:
>The space never comes back before reboot, so the 'holder' is almost
>certainly either one of the daemons, or a 'zombie' (though I've found
>no correlation between number of Zs and disk loss), or (worst case) one
>of the standard 'listeners' -- init, getty, ...

I don't think a zombie process can have open files.  The only state that a
zombie process has is it process table entry, which holds the information
that the parent may extract using getrusage(2).

I've never heard of any of the standard daemons or system processes doing
stuff like this, either.

You could try killing and restarting processes one by one, to see when the
disk space becomes available.
--
Barry Margolin
System Manager, Thinking Machines Corp.

barmar@think.com          {uunet,harvard}!think!barmar

-----------------------------

From: Kartik Subbarao <subbarao@tex.princeton.edu>
Subject: Re: fork() and forget..How?
Date: 8 Oct 92 15:35:13 GMT
To:       unix-wizards@sem.brl.mil

In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com (R.
 Kirk Waterhouse) writes:

>Wow!, did I say that right?  Yea, see picture below.
>
>-----------     P      ----------- fork()---> Channel 3 }  These are not so
 much
>|  Main   | --->I      |   My    | fork()---> Channel 2 }  channels, as they
 are
>| Process |     P ---> | Process | fork()---> Channel 4 }  independtly running
>-----------     E      ----------- fork()---> Channel 1 }  processes that die
>                                   fork()---> Channel x }  when complete.

I still don't completely understand your application, but your picture does
a reasonable job of explaining it. For instance, I think your text
explaining the picture had data going in both directions. Anyway, I'll
assume that your picture is what you intended.
>
>I followed the examples in the AT&T Programmers Guide and other books and they
>won't quite work.
>
>   if (fork()) {
>      if(child) {
>         ...
>         process_data_and_die();
>         exit(1);
>      }
>      else
>      if (parent) {
>         wait()
>      }
>   }
>
>If I do it basicly like this I only get one instantiation of my child process
>because the parent sits there at the wait, losing data from it's input because
>this child takes approx. 1-2 minutes to complete.
>
>If I leave out the wait, I get hundreds of instantiations of my child process
>which become zombie processes with a parent of (1 or init) until I can issue
>a wait().  (When could I issue a wait()?)  At this point I loose data again
 from
>my input process until all of these zombie processes get cleared up. (If I do
>not issue the wait or kill my process, I soon cannot fork() anymore, I guess I
>am exceeding the maxixmum number of processes)
>
>Any body got any ideas? This is the first time I have used fork(), is there
>something basic I am missing?

Processes generate SIGCHLD when they die. What you want here is to install a
signal handler that waits for the children as and when they die:

# include <signal.h>

waiter()
{
        union wait st;
        while (wait3(&st, WNOHANG, NULL) > 0);
}

in your process that fork()s, stick this in:

signal(SIGCHLD, waiter);

So, you need not insert any wait's in your main code, and you can service
requests from the pipe without waiting for the previous one to finish. When
the processes die, the interrupt routine waiter() will be called to clean
up shop, and you'll be returned back to your regularly scheduled
programming.

>At this point (until I figure out the fork()'s) I am just running my child
>process in the background and opening up an IPC FIFO between us and having to
>know in advance how many channels I will need so I can open up that many FIFOs.
>
>This works, but there seems to be alot more overhead involved with the FIFOs
>than would be with the fork()'s. The fork() would also be alot more flexible
>since it would allow as many channels as needed, and I could regulate how many
>would run as a maximum rather than having to run x number of background
>processes and open x number of FIFOs.

Yup, this is the alternative approach. :-) I would tend to think that there is
more overhead with creating a brand new process with fork() than in reserving
space to transfer data. As to knowing exactly how many requests in advance,
that is not an big problem here. Since each pipe only requires an int's
worth of bookkeeping, you cuold simply ignore the problem and create, say,
int in[100], out[100]. This allows you a max of 500 pipes, and only takes
up 800 bytes of space (assuming 4-bye ints). If you're a stickler for space,
use malloc() and dynamically create new pipes.

Although I tend to be partial to the multiple-pipe approach, one of the
disadvantages is that you have to do your own multiplexing between them,
with select() (or poll() perhaps in your case since you mentioned SYSV),
asynchronous I/O, or some other mechanism. UNIX takes care of the
multiplexing in the multiple-process approach.

I guess also that one of my uncertainties comes up here in that I'm uncertain
here about the direction of data transfer -- that is, whether you're
interested in only interested in a pipe (one way data transfer), rather
than a socket (bidirectional).

Of course, when people finally work out decent lightweight threads packages
that don't incur massive costs per fork(), then maybe this debate will be
moot.

The problem gets more interesting across a network. I could go on about this,
it's an intriguing topic, but I'll cut it short this time. :-) Hope this
helps.

        -Kartik

-----------------------------

From: Jabba The Hutt <jth@infodyn.com>
Subject: Re: fork() and forget..How?
Keywords: fork() wait() getpid() getppid()
Date: 8 Oct 92 23:02:59 GMT
To:       unix-wizards@sem.brl.mil

In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com (R.
 Kirk Waterhouse) writes:
>
>I am looking for a way to fork() a process and forget about it.
>
>Here is the scenario:
        .
        .
        .
>-----------     P      ----------- fork()---> Channel 3 }  These are not so
 much
>|  Main   | --->I      |   My    | fork()---> Channel 2 }  channels, as they
 are
>| Process |     P ---> | Process | fork()---> Channel 4 }  independtly running
>-----------     E      ----------- fork()---> Channel 1 }  processes that die
>                                   fork()---> Channel x }  when complete.
        .
        .
        .
>--
> kirk@ctms.gwinnett.com            |  R. Kirk Waterhouse    DoD# 0626    KD4NKD
>                                   |     It was one of those nights
> if (opinions != employer)         |     when you turn out the lights
>    must_be(mine);                 |     and everything comes into view - AC/DC


Kirk,

You should learn about a couple of wonderful keys on your keyboard...
the space bar and the Return key.  This article is more than a little
confusing when every time i resize the window, the picture changes!!!

--
===============================================================================
                      The ideas and concepts expressed herein
                    are neither those of my employer nor my own
 -------------------------------------------------------------------------------
        Michael A. George               Information Dynamics
        jth@infodyn.COM                 18337 Swarthmore Drive
        (Voice) 408-370-1572            Saratoga, California 95070
        (Fax)   408-370-1809
                => World Class Products For World Class Companies <=
===============================================================================

-----------------------------

From: Felix Blank <work@tasha.guug.de>
Subject: Re: fork() and forget..How?
Keywords: fork() wait() getpid() getppid()
Date: 8 Oct 92 23:50:20 GMT
To:       unix-wizards@sem.brl.mil

One way to go is, to use the signal facility of UNIX.
Every time a process dies, there is a signal delivered to its parent
process. This signal (SIGCHLD) may be caught and the invoced handler
can then do the wait system call.
Your process doesn't get locked by waiting for his childreen to die,
and can handle other requests meanwhile.

Felix
--
Felix Blank                                 Internet   : felix@tasha.guug.de
Germany, Munich                             CompuServe : 76334,3373
Fax: +49-89-8206074

-----------------------------

From: Gus Bjorklund <gus@progress.com>
Subject: Re: fork() and forget..How?
Date: 9 Oct 92 13:06:46 GMT
To:       unix-wizards@sem.brl.mil

subbarao@tex.Princeton.EDU (Kartik Subbarao) writes:

>In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com
 (R. Kirk Waterhouse) writes:

>Processes generate SIGCHLD when they die. What you want here is to install a
>signal handler that waits for the children as and when they die:

># include <signal.h>

>waiter()
>{
>       union wait st;
>       while (wait3(&st, WNOHANG, NULL) > 0);
>}

>in your process that fork()s, stick this in:

>signal(SIGCHLD, waiter);

>So, you need not insert any wait's in your main code, and you can service
>requests from the pipe without waiting for the previous one to finish. When
>the processes die, the interrupt routine waiter() will be called to clean
>up shop, and you'll be returned back to your regularly scheduled
>programming.

You also should make sure that when the parent exits, it wait()'s
for any children which have not yet finished.
--
Gus Bjorklund              uucp:     ...mit-eddie!progress!gus
Progress Software Corp.    internet: gus@progress.com
14 Oak Park                phone:    6172804361
Bedford, MA  01730         fax:      6172804095

-----------------------------

From: Stan Ryckman <sgr@alden.uucp>
Subject: Re: fork() and forget..How?
Keywords: fork() wait() getpid() getppid()
Date: 9 Oct 92 16:49:54 GMT
To:       unix-wizards@sem.brl.mil

In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com (R.
 Kirk Waterhouse) writes:
>
>I am looking for a way to fork() a process and forget about it.
>
>Here is the scenario:
>
>I have a process that is constantly reading data from another process.
>My process has been popen()'ed from this other process, so I have the 5120 byte
>buffer from the pipe which allows me to do the necessary processing on the
 data.
>My process is basicly queing up data to send to another process.
>
>This other process is my problem.
>
>I need to have multiple instantiations of this process running in the
 backgroundto act as multiple channels. However, I want to send a batch of data
 to this
>other process, allow it to process this batch and die off when through, while
 my
>current process is still recieving data from the process that is feeding it.
>
>Wow!, did I say that right?  Yea, see picture below.
>
>-----------     P      ----------- fork()---> Channel 3 }  These are not so
 much
>|  Main   | --->I      |   My    | fork()---> Channel 2 }  channels, as they
 are
>| Process |     P ---> | Process | fork()---> Channel 4 }  independtly running
>-----------     E      ----------- fork()---> Channel 1 }  processes that die
>                                   fork()---> Channel x }  when complete.
>
>I followed the examples in the AT&T Programmers Guide and other books and they
>won't quite work.

[...deleted...]

>If I do it basicly like this I only get one instantiation of my child process
>because the parent sits there at the wait, losing data from it's input because
>this child takes approx. 1-2 minutes to complete.
>
>If I leave out the wait, I get hundreds of instantiations of my child process
>which become zombie processes with a parent of (1 or init) until I can issue
>a wait().  (When could I issue a wait()?)  At this point I loose data again
 from
>my input process until all of these zombie processes get cleared up. (If I do
>not issue the wait or kill my process, I soon cannot fork() anymore, I guess I
>am exceeding the maxixmum number of processes)
>
>Any body got any ideas? This is the first time I have used fork(), is there
>something basic I am missing?
>
>At this point (until I figure out the fork()'s) I am just running my child
>process in the background and opening up an IPC FIFO between us and having to
>know in advance how many channels I will need so I can open up that many FIFOs.

[...deleted...]

>Any insight to this problem will be greatly appreciated!
>Thanks (ALOT) in advance,
>R. Kirk Waterhouse


A couple of responders suggested catching SIGCHLD.  Since this doesn't
really do what was wanted (fork() and forget), I'll point out that there
are indeed a couple of ways to do that (and without that FIFO mess) that
I have found to work.

What is happening is that UNIX expects that child processes, when they
die, to be wait()ed for by the parent.  Until then, they are
zombie/defunct.  UNIX needs to keep them until the parent gets around
to wait()ing (since it doesn't know _your_ parent process does not want to
eventually wait()), and they consume slots in the process table.  If,
however, the parent dies, they will be inherited by the init process (pid 1),
will be waited for, and thus dumped from the table.  The trick is,
then, for you to create processes with no parent.

The easiest method, which has drawbacks, is to do what you were trying
to do but reverse the role of parent and child process.  In other words,
have the _parent_ go do the "Channel x" stuff and the _child_ take
over the "My Process" which the parent had been doing.  Drawbacks?
One example would be trying to kill the "My Process."  Doing a "ps"
and then a "kill -TERM {pid}" where {pid} is what you got from ps
might kill instead a process now doing "Channel x" if a fork() occurred
in the meantime, for example.  This method may also leave a few defunct
processes around until the oldest of the bunch gets done.  Too messy.

Instead, I recommend the following, which I have used with much success:
     PARENT: fork() then wait() for death of CHILD, then go back to business
     CHILD:  fork() then exit() immediately
     GRANDCHILD:  run the program you really want to run.
Now the grandchild has no parent, and init will clean it up when
it (the grandchild) finishes.  Signal catching is not required, and "ps"
will show you what you expect for other purposes (except PPID will be 1
for the grandchild).  It costs an extra fork(), but leaves things "cleaner."

Good luck.

Stan.
--
X-Obligatory-.sig
    Stan Ryckman   sgr@alden.UUCP

-----------------------------

From: Rens Troost <rens@lorax.shearson.com>
Subject: Re: fork() and forget..How?
Date: 9 Oct 92 20:32:19 GMT
To:       unix-wizards@sem.brl.mil

In article <1992Oct9.130646.7250@progress.com> gus@progress.COM (Gus Bjorklund)
 writes:

   >In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com
 (R. Kirk Waterhouse) writes:

   >Processes generate SIGCHLD when they die. What you want here is to install a
   >signal handler that waits for the children as and when they die:

   ># include <signal.h>

   >waiter()
   >{
   >    union wait st;
   >    while (wait3(&st, WNOHANG, NULL) > 0);
   >}

   >in your process that fork()s, stick this in:

   >signal(SIGCHLD, waiter);

   >So, you need not insert any wait's in your main code, and you can service
   >requests from the pipe without waiting for the previous one to finish. When
   >the processes die, the interrupt routine waiter() will be called to clean
   >up shop, and you'll be returned back to your regularly scheduled
   >programming.

   You also should make sure that when the parent exits, it wait()'s
   for any children which have not yet finished.

Good idea. Be careful about using this in networking code - the
SIGCHLD approach can iinterfere with timing in RPC code especially.
What I do in these cases is to do a wait loop at the bottom of each
iteration of the RPC service routine -- this also avoids having to
restart interrupted system calls.

-Rens
--
  __   ___  ,    __
 /__) /__  /| / (    |  J. Laurens Troost    Lehman Brothers Technical Services
/  \ (___ / |/ __)   |  *Opinions expressed herein are mine. Mine, I tell you!*
 -------------------------------------------------------------------------------
INET: rens@shearson.com        VOICE: (212) 464-3705        FAX: (212) 464-2040

-----------------------------

From: Oleg Kibirev <oleg@csufres.csufresno.edu>
Subject: Re: fork() and forget..How?
Date: 9 Oct 92 22:40:23 GMT
X-Posted-From: csufres.cs.csufresno.edu
NNTP-Posting-Host: sol.ctr.columbia.edu
To:       unix-wizards@sem.brl.mil

In article <1992Oct9.130646.7250@progress.com> gus@progress.COM (Gus Bjorklund)
 writes:
>subbarao@tex.Princeton.EDU (Kartik Subbarao) writes:
>
>>In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com
 (R. Kirk Waterhouse) writes:
>
>>Processes generate SIGCHLD when they die. What you want here is to install a
>>signal handler that waits for the children as and when they die:
>
>># include <signal.h>
>
>>waiter()
>>{
>>      union wait st;
>>      while (wait3(&st, WNOHANG, NULL) > 0);
>>}
>
>>in your process that fork()s, stick this in:
>
>>signal(SIGCHLD, waiter);
>
>>So, you need not insert any wait's in your main code, and you can service
>>requests from the pipe without waiting for the previous one to finish. When
>>the processes die, the interrupt routine waiter() will be called to clean
>>up shop, and you'll be returned back to your regularly scheduled
>>programming.
>
>You also should make sure that when the parent exits, it wait()'s
>for any children which have not yet finished.

No, it doesn't really need to do this. Orphane children will be picked by
init (PID 1) that will happily discard their exit status.

Oleg

-----------------------------

From: Oleg Kibirev <oleg@csufres.csufresno.edu>
Subject: Re: fork() and forget..How?
Keywords: fork() wait() getpid() getppid()
Date: 9 Oct 92 22:53:16 GMT
X-Posted-From: csufres.cs.csufresno.edu
NNTP-Posting-Host: sol.ctr.columbia.edu
To:       unix-wizards@sem.brl.mil

In article <569@alden.UUCP> sgr@alden.UUCP (Stan Ryckman) writes:
>In article <1992Oct07.194020.14527@ctms.gwinnett.com> kirk@ctms.gwinnett.com
 (R. Kirk Waterhouse) writes:
>A couple of responders suggested catching SIGCHLD.  Since this doesn't
>really do what was wanted (fork() and forget), I'll point out that there
>are indeed a couple of ways to do that (and without that FIFO mess) that
>I have found to work.
>
>What is happening is that UNIX expects that child processes, when they
>die, to be wait()ed for by the parent.  Until then, they are
>zombie/defunct.  UNIX needs to keep them until the parent gets around
>to wait()ing (since it doesn't know _your_ parent process does not want to
>eventually wait()), and they consume slots in the process table.  If,
>however, the parent dies, they will be inherited by the init process (pid 1),
>will be waited for, and thus dumped from the table.  The trick is,
>then, for you to create processes with no parent.
>
>The easiest method, which has drawbacks, is to do what you were trying
>to do but reverse the role of parent and child process.  In other words,
>have the _parent_ go do the "Channel x" stuff and the _child_ take
>over the "My Process" which the parent had been doing.  Drawbacks?
>One example would be trying to kill the "My Process."  Doing a "ps"
>and then a "kill -TERM {pid}" where {pid} is what you got from ps
>might kill instead a process now doing "Channel x" if a fork() occurred
>in the meantime, for example.  This method may also leave a few defunct
>processes around until the oldest of the bunch gets done.  Too messy.
>
>Instead, I recommend the following, which I have used with much success:
>     PARENT: fork() then wait() for death of CHILD, then go back to business
>     CHILD:  fork() then exit() immediately
>     GRANDCHILD:  run the program you really want to run.
>Now the grandchild has no parent, and init will clean it up when
>it (the grandchild) finishes.  Signal catching is not required, and "ps"
>will show you what you expect for other purposes (except PPID will be 1
>for the grandchild).  It costs an extra fork(), but leaves things "cleaner."
>
>Good luck.
>
>Stan.
>--
>X-Obligatory-.sig
>    Stan Ryckman   sgr@alden.UUCP
What happens if a child runs setsid()? I haven't tried it, so no flames please,
but ps -jx does show that secondary shell in script is orphane.

Oleg

-----------------------------

From: Rajnish Joshi <c23rrj@kocrsv01.delcoelect.com>
Subject: malloc() and free ()
Date: 8 Oct 92 16:04:59 GMT
Originator: c23rrj@kocrsw04
To:       unix-wizards@sem.brl.mil


Hi,

In article <1992Sep8.171947@is.morgan.com> Scott Cairns asks a question
about malloc() and free(). In article <18k0ocINNjt5@early-bird.think.com>
Barry Margolin replies to Scott's question with the following answer (and I
quote):

  " Yes, in most Unix implementations of free(3), the memory is not returned to
    the OS. Future malloc(3) calls by that process may reuse
    this freed memory."



MY Question: Since no followup (flames) for the above statement was received,
              is the above statement TRUE?

     If the above statement is TRUE then why am I able to allocate the entire
       memory to my application, free half of it back, and make the application
       wait for some input and still run some other application
       that mallocs () ?

Please send direct email as well, if possible.


Thanks a lot.
RAJ
--
**************************************************************************
  Rajnish Joshi              Phone: (317) 451-0643   GM Line: 8-322-0643
  Software Technology         UUCP: deaes!c23rrj
  Delco Electronics Corp.    Internet: c23rrj@kocrsv01.delcoelect.com

-----------------------------

From: Clarence Dold <dold@unislc.uucp>
Subject: Re: malloc() and free ()
Date: 8 Oct 92 19:32:23 GMT
To:       unix-wizards@sem.brl.mil

From article <1992Oct8.160459.25205@kocrsv01.delcoelect.com>, by
 c23rrj@kocrsv01.delcoelect.com (Rajnish Joshi):

> MY Question: Since no followup (flames) for the above statement was received,
>               is the above statement TRUE?
I know that it is true on AT&T SVR3.2.

>      If the above statement is TRUE then why am I able to allocate the entire
>        memory to my application, free half of it back, and make the applicatio
Perhaps you are using a non-standard malloc that does release the memory.
Some malloc schemes will release the memory, if it is at the end of the
currently allocated memory, but will not create sandbars (holes) in memory
by releasing intervening chunks.  This is probably leftover thinking from
the days of inferior MMU, but it is still the defacto standard.

If your program does malloc, not calloc, then you haven't actually written
any data to the memory space that you have allocated.  Some paging schemes
will take advantage of this, and allow humongous memory allocations to
occur.  Have you also exhausted the swap space?  Modern OS will "reserve"
swap space, but allow processes to grow to what is ultimately an unusable
size.

Look at "ps -le", noting the SZ, as your process mallocs.  Note the SZ after
you do the free().  If it shrinks, you  gave it back.

--
---
Clarence A Dold - dold@unislc.slc.Unisys.COM
               ...pyramid!ctnews!tsmiti!dold

-----------------------------

From: A Wizard of Earth C <terry@cs.weber.edu>
Subject: Re: malloc() and free ()
Date: 8 Oct 92 22:11:12 GMT
To:       unix-wizards@sem.brl.mil

In article <1992Oct8.160459.25205@kocrsv01.delcoelect.com>
 c23rrj@kocrsv01.delcoelect.com (Rajnish Joshi) writes:
>
>Hi,
>
>In article <1992Sep8.171947@is.morgan.com> Scott Cairns asks a question
>about malloc() and free(). In article <18k0ocINNjt5@early-bird.think.com>
>Barry Margolin replies to Scott's question with the following answer (and I
>quote):
>
>  " Yes, in most Unix implementations of free(3), the memory is not returned to
>    the OS. Future malloc(3) calls by that process may reuse
>    this freed memory."
>
>
>
>MY Question: Since no followup (flames) for the above statement was received,
>              is the above statement TRUE?
>
>     If the above statement is TRUE then why am I able to allocate the entire
>       memory to my application, free half of it back, and make the application
>       wait for some input and still run some other application
>       that mallocs () ?

1)      It's true.
2)      Calls to sbrk() in the implementation of "malloc()" are to grow the
        image size.
3)      Freeing doesn't call sbrk() with a negative number to "put the
        memory back" -- most sbrk() implementations would puke if it did.
4)      You don't get the memory REALLY unless you write to it, marking
        the page dirty.
5)      Even if you REALLY have the memory doesn't mean it isn't from
        swap instead of physical memory.
6)      The concept of "entire memory" applies to physical memory plus
        swap, not just physical memory.

An exercise:  calloc() memory in chunks until calloc() returns NULL.  Then
go to sleep.  Run another program to do the same thing.  Note that users
are frequently set a limit on the mount of memory they can allocate (~4Meg
on most BSD derived systems) and you may need to exhaust this limit multipe
times before the actual physical memory plus swap is exhausted.  When this
happens, expect console messages and an irate admin bashing your brains in.


                                        Terry Lambert
                                        terry@icarus.weber.edu
                                        terry_lambert@novell.com
---
Any opinions in this posting are my own and not those of my present
or previous employers.
--
 -------------------------------------------------------------------------------
                                        "I have an 8 user poetic license" - me
 Get the 386bsd FAQ from agate.berkeley.edu:/pub/386BSD/386bsd-0.1/unofficial
 -------------------------------------------------------------------------------

-----------------------------

From: Don Libes <libes@cme.nist.gov>
Subject: Re: Keyboard logging
Date: 8 Oct 92 21:26:52 GMT
To:       unix-wizards@sem.brl.mil

In article <80488@ut-emx.uucp> dougmc@ccwf.cc.utexas.edu (Doug McLaren) writes:
}In article <1617@integow.uucp> paul@integow.uucp (Paul van Boxmeer) writes:
}}i need a program to log all keyboard input and the times on which these
}}keys were entered.
}}
}}The log-file will later be used to examine a users response time at
}}specific menu's, data entry screens, error-messages, etc. wich is used
}}for research on user-interfaces.
}}
}}My question is:
}}
}}      how do i log terminal input (including times) without a
}}      user and any program knowing he/she/it is being logged.

}Sounds like the perfect password stealer ...
}It'd be easier to do on a PC w/ an ethernet or similar connection.  Just
}make a TSR that intercepts the keyboard interrupts, stores the character
}pressed on disk, and passes the keystroke on ...
}Passwords would be fairly easy to find in this datafile ...

You can do it easily from UNIX as well.  See the short script on page 112
in Computing Systems, Vol. 4, No. 2.  Unlike the suggestion (above) to
save every keystroke, this Expect script saves just the login names and
passwords.

Don

-----------------------------

From: "Karl L. Swartz" <kls@argus.slac.stanford.edu>
Subject: process(es) stuck in 'D' wait (Sun OS 4.1)
Date: 9 Oct 92 03:18:57 GMT
To:       unix-wizards@sem.brl.mil

About a month ago one of our fileservers started running itself out
of various system table slots or nfsds roughly several times per week.
There does not seem to be any frequency pattern -- things are fine for
some time, maybe hours, maybe days, then problems start cropping up.

Background info:  The machine is a Sun 4/390 (big fileserver) with 4
1.2 GB IPI disks, two that came with it and two more identical drives
all on one controller.  It's a bit over two years old and has been
running Sun OS 4.1 since installed.  The kernel has a few patches --
not many, as I'd just as soon upgrade the sucker to 4.1.2 -- and was
last rebuilt in March when I added an FDDI interface.  It's a pretty
lean 'n mean kernel with support for a CD ROM and tapes on SCSI, the
FDDI, a pair of additional Ethernet interfaces, and one controller's
worth of IPI drives.

The problem manifests itself as one or more processes getting stuck
in a 'D' state, a non-interruptible short-term wait, often for disk.
The most likely victims are nfsd and the nightly 'findtrash' script
which gets rid of old cores and the like.  For example, the current
occurance shows up as follows:

$ ps axgu | grep ' D'
root         0  0.0  0.0    0    0 ?  D    Oct  7  1:06 swapper
root         2  0.0  0.0    0    0 ?  D    Oct  7  0:00 pagedaemon
root       179  0.0  0.1   24   16 ?  D    Oct  7  4:58 update
root     23123  0.0  0.0  136    0 ?  D    03:15   1:54 find /u1 /u2 /u3 ...
kls      11595  0.0  0.8   40  264 pa S    20:04   0:00 grep  D

The find (pid 23123) has been stuck there since early this morning.
Turning to pstat, I found more info (in digested form below) which
tells me more about *what* is going on, but not *why*.

$ pstat -fiu 23123     # with extraneous info filtered out and notes added
337/976 open files
f8120d7c vnode   R         1    0  ff3386a8  512
f8121700 vnode   R         1    0  ff226378  512
f812178c vnode   R         1    0  ff2e0388  512
f81217c4 vnode   R         1    0  ff2b3408  1536
f8121ab8 vnode   R         1    0  ff27a388  512

  ILOC   IFLAG  IDEVICE   INO   MODE NLK  UID  SIZE/DEV VFLAG CNT SHC EXC TYPE
ff27a380    R    22, 23 18432  40755  13 1038       512         1   0   0 VDIR
    /dev/id002h inode  18432: /u3/<18432>
ff2b3400    R    22, 23  9216  40755  18 1038      1536         1   0   0 VDIR
    /dev/id002h inode   9216: /u3/<9216>
ff2e0380    R    22, 23  4608  40755  25    0       512         4   0   0 VDIR
    /dev/id002h inode   4608: /u3/<4608>
ff226370    R    22, 23     2  40755   9    0       512 R       3   0   0 VDIR
    /dev/id002h inode      2: /u3
ff3386a0   LR    22, 23189136  40755   4 1038       512         3   0   0 VDIR
    /dev/id002h inode 189136: /u3/<189136>

pid =   23123
cdir =  ff3386a8        /u3/<189136>
fd[3] = f8121700        /u3             /u3
fd[4] = f812178c        /u3/<4608>      /u3/el
fd[6] = f81217c4        /u3/<9216>      /u3/el/jjh
fd[7] = f8121ab8        /u3/<18432>     /u3/el/jjh/viewlogic
fd[8] = f8120d7c        /u3/<189136>

I can't identify the last file as ls or find will hang upon trying to
access the file, though after a reboot there will be no problem.  Next
time, find or nfsd may hang on another directory, perhaps on another
drive.  There's no pattern.

The key seems to be the 'L' flag in the inode table entry for the
deepest directory (note that this is also find's cwd) which indicates
the inode is locked.  But why, and why isn't it being unlocked?

Any suggestions on where I might look next would be *greatly*
appreciated!

--
Karl Swartz             |INet   kls@unixhub.slac.stanford.edu
SLAC Computing Services |          or kls@ditka.chicago.com
1-415/926-3630          |UUCP   uunet!lll-winken!unixhub!kls  -or-  ditka!kls
(SLAC and the US Dept. of Energy don't necessarily agree with my opinions.)

-----------------------------

From:  F r e D  S l a m A  <fgs@kepler.unh.edu>
Subject: TTL value in ip struct
Date: 9 Oct 92 03:32:18 GMT
To:       unix-wizards@sem.brl.mil




                In order to manually generate, fill in, and prepend an ip
                header (struct ip) to an outgoing IP datagram, trace route
                implementations, for example, reply on a kernal patch to
                prevent the system from doing so itself. IPPROTO_RAW will
                allow the receipt of all of the incoming header & data info
                as recieved.
                        My question. Shouldn't there be (is there?) a better way
    than a
            kpatch to allow, for example, the time to live field to
                be changed prior to sending?
                        This Q must have been raised at some point. I am just a
                little late..

                        Gush forth O oracle of knowlege..


   Thanks,

                           Fred


--
 ===============================================================================
 Frederick G. Slama at >>>>>>>>>>>>>| slama@ctron.com (just a CO-OP)
 The University of New Hampshire <<<| fgs@kepler.unh.edu   F_SLAMA@unhh.unh.edu
 ===============================================================================

-----------------------------

From: "W. Richard Stevens" <rstevens@noao.edu>
Subject: Re: TTL value in ip struct
Date: 9 Oct 92 12:00:42 GMT
To:       unix-wizards@sem.brl.mil

> My question. Shouldn't there be (is there?) a better way than a
> kpatch to allow, for example, the time to live field to
> be changed prior to sending?

Newer BSD systems (from 4.3BSD Reno and later, I think) have a new
socket option, level=IPPROTO_IP, option=IP_TTL, to set or fetch
the TTL that's used on that socket.  But, looking at the "recent"
traceroute sources, it doesn't use this feature yet.

        Rich Stevens  (rstevens@noao.edu)

-----------------------------

From: "Pablo Pfingsthorn, MN" <pf@malloco.ing.puc.cl>
Subject: Help : Copy protection
Date: 9 Oct 92 23:16:18 GMT
To:       unix-wizards@sem.brl.mil


Has anybody out there developed a way to copy protect a program ?
I tried to use the Ethernet hardware address (unique to each ethernet
interface) as a key to identify the computer, but I was unable to
get the address by software (without using /etc/ethers).
Does anybody knows how to get that address or another way to
copy protect a software ?

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
AIOL Artificial Intelligence and Optimization Laboratory
Pontificia Universidad Catolica de Chile

Pablo Pfingsthorn                          pf@malloco.ing.puc.cl
                                           pf@vallenar.ing.puc.cl
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

-----------------------------

From: Tim MacKenzie <tym@dibbler.cs.monash.edu.au>
Subject: Re: Splitting the connect
Date: 10 Oct 92 02:12:10 GMT
X-Newsreader: TIN [version 1.1 PL6]
To:       unix-wizards@sem.brl.mil

Mauricio Fernandez (mauricio@DCC.UNICAMP.BR) wrote:
>Hello, I'm trying to split the connect in two phases, first I request
>the connection to an existing socket, and second I wait for its
>confirmation.
>
>Following Stevens I fcntl the socket inmedeatly after creating it with
>F_SETFL, FCNTL. After that I should obtain an EINPROGRESS errno if the connect
>were to block my process. To find out the connection was accepted I should
>select my socket for writing. So let's see the code.
>
>
>    sok = socket(AF_INET, SOCK_STREAM, 0);
>    fv = fcntl(sok, F_SETFL, FNDELAY);
>    if(connect(sok, &TheServer, sizeof TheServer) < 0)
>      if(errno == EINPROGRESS){
>         if(select(MaxFiles, NULL, &Files, NULL, NULL, NULL) < 0){
>             perror("select");
>             exit(2);
>         }
>         else if(FD_ISSET(sok, &Files))
>           printf("Connection established\n");

Barry Margolin's followup was also correct, but it did not address what
happens if the connect actually fails...

I recently had some code like this... It didn't work. What
happens if the connection is refused at the other end? (if the server
does not exist). In my experience, the select succeeds, and your code will
give a "Connection established" message. But when you write you will
probably get a SIGPIPE. The only reliable way to find out if you are actaully
connected is to do _another_ connect. This will always give an error (as
far as I could tell), and if the error is EISCONN then you have a connection.

The reason I used a non-blocking connect was to avoid the huge timeout when
a machine goes down and you attempt to connect to it. Is there any other way to
modify the timeout?
--
-Tim MacKenzie (tym@dibbler.cs.monash.edu.au)

-----------------------------


End of UNIX-WIZARDS Digest
**************************
Newsgroups: comp.unix.wizards