From: henry@zoo.toronto.edu (Henry Spencer)
Subject: Re: inews in C - shell/awk scripts are unreliable!
Date: Fri, 16 Oct 1992 21:59:13 GMT
Organization: U of Toronto Zoology

In article <9978@kralizec.zeta.org.au> nick@kralizec.zeta.org.au (Nick Andrew) writes:
>Shell scripts, especially those which make heavy use of commands like
>awk and sed, are inherently less precise than corresponding C code.
>When coding these scripts, certain assumptions are made which cause
>problems when the input data is not as expected.

In a word, nonsense.  Most problems that crop up are entirely analogous
to problems that crop up in C code; either way, care can avoid them.

>1. The recent fiasco about an "empty newgroup" message. A script just
>ignored the fact that there was no actual group name given, because the
>parsing of the string is more-or-less automatic in a shell script.

Actually, it was because the script's author never considered the possibility
that an argument might not be given.  People do this in C all the time.
Insufficient paranoia about input is the bane of poorly-written C programs,
as witness the infamous gets() botch that the Internet Worm exploited.

>A C programmer would never make such a mistake...

Speaking as someone who has been programming in C, and debugging other
peoples' C programs, since 1975:  boy are you in for a surprise. :-)

>... Furthermore, the programmer would
>probably ensure that the newsgroup name only contained valid characters!

The current C News newgroup script does this, by the way, although its
validity checks should probably be tightened up further (in the same way
that we've steadily tightened up validity checks in our C code).

>2. C-news "rnews" does not support the unbatching feature of B-news.
>This caused me a lot of pain. Anyway the B-news rnews supported the
>detection of #! rnews 12345 at the start of a message, to trigger
>automatic unpacking of the contents into several messages.

What on Earth are you talking about?  Our rnews most assuredly supports
this; it has to, to interoperate with B News systems.

>C-News does not support this. Ok, I can live with it. But what happens
>if I feed such a batch into C-news rnews? It uses an Awk script to
>parse the headers...

Oh, you think that "inews" and "rnews" are the same program.  They aren't.
Please read the documentation.  Inews is for input from humans.  Rnews is
for input (including batches) from machines.  In B News they happened to
be different interfaces to the same code.  Not in C News.

>If the message does not contain a ':' in its first line, the Awk script
>falls through to the reporting (or message body) clause, and then
>cheerfully bombs out with the message "awk: hdrval: not an array"...

This is a bug, pure and simple, like the equally-obscure reporting of
errors that is often found in C programs.  I've actually been harassing
Geoff about fixing this one for some time...

>Furthermore, the character escaping and parsing of shell scripts is
>considerably more complex (and thus error-prone) than C syntax. That
>makes maintenance a nightmare. For example,
>
>	greppat="^`echo $1 | sed 's/\./\\\\./g' ` "
>
>is a typical line from C-news. Modify at your own risk, I say! A slipped
>backslash and you're ash!

Nobody who has dealt with really messy C declarations at length will agree
with you about the above being more complex or more error-prone.  Quick,
what's the type of `xyz' in `void (*xyz(int, void (*)(int)))(int);'?

>Finally, I'd like to point out that C-news shell scripts also contain
>code which is unrelated to the task at hand. For example, let's look
>at the first few lines from the recently-patched newgroup.sh:
>
>: || exec /bin/sh $0 $argv:q
># uncomment the following if you have trouble with older ksh's on USG systems
>#(alias) >/dev/null 2>&1 && exec /bin/sh "$0" ${1+"$@"}

This is really interesting code; where did you find it?  We certainly didn't
ship that.  (We also don't ship anything named "newgroup.sh", but I'm writing
that off to another misunderstanding about names.)

>Shell scripts and awk scripts have a place in the world, but they aren't
>sufficiently robust as programming languages to control a major system
>such as USENET News.

I will content myself with saying that in our experience -- and that of
a lot of other people, several thousand sites -- you're wrong.  The general
weight of opinion supports our own feelings about this:  C News is a good
deal *more* robust than B News.  This is partly because shell scripts
are so much easier to write that we could put a fair bit of effort into
making them clever about management issues like not overflowing disks.

>So in summary I'd like to recommend to the C-news developers to avoid
>scripts wherever possible...

In a word (well, two words):  forget it.  Life is too short to write C
when more powerful tools are available and adequate to the job.  We
unhesitatingly write scripts wherever and whenever we can, and re-code
in C only when necessary.  Anyone who does otherwise is trying to run
a marathon with one foot in a bucket.
-- 
MS-DOS is the OS/360 of the 1980s.      | Henry Spencer @ U of Toronto Zoology
              -Hal W. Hardenbergh (1985)|  henry@zoo.toronto.edu  utzoo!henry