From: edhall@rand.org (Ed Hall)
Newsgroups: comp.unix.sysv386
Subject: Is *your* fread() broken? [Sys5R4]
Keywords: fread, Sys5R4, bugs
Date: 13 Apr 92 19:08:38 GMT
Organization: The RAND Corporation, Santa Monica, CA, USA

I just made a most unhappy discovery with ESIX 4.0.3: the fread() routine
is broken.  I managed to trash a few near-irreplacable files before I
realized this.

Here's the problem:  Unlike the raw read() system call, fread() is
supposed to be able to make several partial read's to satisfy the data
requested by its arguments.  The exceptions are an EOF or an error on
the stream.  This characteristic is quite useful when moving data
through pipes or over network connections, since partial reads are quite
common in these cases.

Well, the version of fread() in ESIX 4.0.3 (and likely other Sys5R4's)
only dones a single physical read, and if it only satifies part of
the requested number of bytes, that's all you get.  This can sting
you even if you carefully check the value returned by fread(), since
the value returned is rounded down to the number of complete "nitems"
read, although your position in the stream can be up to size-1 bytes
beyond that point.  Neither ferror() nor feof() indicate anything is
wrong when this happens.

Now, some of you might be saying, "well, that's the way it's supposed
to work--just like read()."  Wrong.  Not only does the manual page
for fread() indicate only two conditions for returning fewer items than
requested--error or EOF--but the manual page for stdio(3S) reads:

    The macros getchar and putchar, and the higher-level routines fgetc,
    fputs, fprintf, fputc, fputs, fread, fscanf, fwrite, gets, getw,
    printf, puts, putw, and scanf all use or act as if they use getc and
    putc; they can be freely intermixed.

Getc(), as you might know, keeps returning characters until--surprise--
EOF or error.  An fread() call based on it, or one acting as if it were
based on it, would have no way of distinguishing physical read boundries.

Here are a couple of short programs which illustrate the problem; run
the pipeline "wtest | rtest" and see if the fwrites and freads match up:

/*
 * rtest
 */
#include <stdio.h>

int b[65536];

main()
{
	register i, j, r;

	for (i = 0; i < 16; i++)
	{
		r = fread((void *)b, 1, sizeof b, stdin);
		fprintf(stderr, "read %d bytes; buf0 = %d\n", r, b[0]);
	}
	exit(0);
}
/***************************************************************************/

/*
 * wtest
 */
#include <stdio.h>

int b[65536];

main()
{
	register i, j, r;

	for (i = 0; i < 16; i++)
	{
		for (j = 0; j < 65536; j++)
			b[i] = i*65536 + j;

		r = fwrite((void *)b, 1, sizeof b, stdout);
		fprintf(stderr, "wrote %d bytes; buf0 = %d\n", r, b[0]);
	}
	exit(0);
}
/***************************************************************************/

Note that these programs pump a million consecutive ints through a
pipe--you might want to downsize them a bit if your machine is slower;
just make sure that each fwrite/fread is around 16K or more.

If your system is like mine, you'll only get 5120 bytes through for
each fread() instead of 262,144.

What to do?  Well, fortunately, the BSD stdio library is close enough
that, with a few patches, its version of fread() works just fine:

#include <stdio.h>

/*
 * Adapted from BSD; repaired version for fread to replace broken libc
 * version in ESIX 4.0.3
 */
size_t
fread(bptr, size, count, iop)
void *bptr;
size_t size, count;
register FILE *iop;
{
	register size_t s;
	register unsigned char *ptr = (unsigned char *)bptr;
	int c;

	s = size * count;
	while (s > 0) {
		if (iop->_cnt < s) {
			if (iop->_cnt > 0) {
				(void) memmove(ptr, iop->_ptr, iop->_cnt);
				ptr += iop->_cnt;
				s -= iop->_cnt;
			}
			/*
			 * filbuf clobbers _cnt & _ptr,
			 * so don't waste time setting them.
			 */
			if ((c = _filbuf(iop)) == EOF)
				break;
			*ptr++ = c;
			s--;
		}
		if (iop->_cnt >= s) {
			(void) memmove(ptr, iop->_ptr, s);
			iop->_ptr += s;
			iop->_cnt -= s;
			return (count);
		}
	}
	return (size != 0 ? count - ((s + size - 1) / size) : 0);
}
/***************************************************************************/

Alas, I know of no way to replace the bad version in /usr/lib/libc.so.1.
So this version must be explicitly linked into any progarms which are
to use it.

Note that there are PD software packages out there affected by this; for
example, the FBM image-processing routines use fread() in a way that is
affected by this bug.  So even if you aren't using fread() yourself, you
can wind up getting bit by this one.  Statically-linked binaries created
on an unbroken system have no problem; in fact, I discovered this bug
after a simple recompile of a old program (linked under ISC 2.0.1) which
had worked flawlessly before.

		-Ed Hall
		edhall@rand.org
Newsgroups: comp.unix.sysv386