From: friedman@gnu.ai.mit.edu (Noah Friedman)
Newsgroups: comp.unix.shell,comp.unix.programmer
Subject: Re: file locking in shell scripts
Date: 24 Jul 92 09:53:02 GMT
Followup-To: comp.unix.shell
Organization: Free Software Foundation, 675 Mass Ave. Cambridge, MA 02139
In-reply-to: rvloon@cv.ruu.nl's message of 23 Jul 92 12:43:27 GMT

In article <1992Jul23.124327.20843@cv.ruu.nl> rvloon@cv.ruu.nl (Ronald van Loon) writes:
>Somewhere on our system is a log-file. This logfile is used to gather
>statistical information. However, it is possible that people write to the file
>at the same time, thus resulting in either the merge of the input-streams or
>total garbage in the file.
>
>How can I - within a shell script - lock the file exclusive for writing ? If
>that's not possible, what do I need to use to do so from C, for example ?

   If you can design the locking protocol yourself, and make sure that
every interface people will use to write the log file will follow that
protocol, you can specify when a file is locked by creating a separate
lockfile.  Other programs seeing this file will know someone else has a
lock on the log file (many mail programs use this "advisory" locking
method).  You can easily create a lock file in a C program using

            open (lockfile_path, O_CREAT|O_EXCL|..., mode);

(appropriate error checking omitted, since that part is obvious).  This
means that open will only create the file if it does not already exist.
This is an atomic operation, so you shouldn't have to worry about any race
conditions (locally, anyway.. I don't know if anything is guaranteed over
NFS, including other locking methods!)

   So the question is, do any shells provide a way of doing this
sort of exclusive creation?  None of them that I could get the source code
to.  So if you want reliability, don't do it in shell script.  I'm going to
suggest a patch to bash that will make the noclobber feature a little more
reliable (it presently does a stat and then an open, which results in a
race condition), and then it will be possible to do the following safely:

            set -o noclobber
            lockp=$( (> lockfile) 2>&1)

            if [ -z "${lockp}" ]; then
               # acquired the lock
            else
               # Did not acquire the lock---either can't create the file
               # because we don't have permission, or the file already
               # exists and someone else has a lock.  These two conditions
               # are possible to distinguish by doing a series of "test"
               # commands on the file or directory in which you wished to
               # create it, although there may be a race condition in which
               # the lock file has been removed in between tests.
               # You can also try the more shaky method of examining the
               # contents of $lockp.. if it says "Cannot clobber existing
               # file" or whatever, you can probably guess what happened. 
               # In C you could just look at errno (== EEXIST). 
            fi

This relies on bash printing an error message on stderr when it refuses to
clobber a file.  I did it this way becase bash (as of version 1.12) has a
bug: it doesn't return a nonzero exit status when it refuses to clobber
files (yes, I've told the maintainers about this too).  In a
properly-working version (hopefully 1.13) you should be able to just do:

             set -o noclobber
             if 2> /dev/null > lockfile ; then
                # acquired the lock
             else
                # didn't acquire the lock
             fi

This is at least one way to (almost) do locking in pure shell script.  I
suspect other methods are probably more difficult, if not also presently
impossible.  However, you can write a small C program to do the atomic
exclusive file creation, returning an exit status that indicates whether it
succeeded or failed.  Then the rest can probably be done in shell script
(depending on what else you want to do, that is).

   Tom or Larry: can perl do this correctly?


(PS: note that these examples don't work in zsh either, because zsh has
even more bugs than bash does with regards to file clobbering)
Newsgroups: comp.unix.shell