From: john@iastate.edu (John Hascall)
Subject: Re: RFD: Removal of Unused Newsgroups
Organization: Iowa State University, Ames, IA
Date: Sat, 5 Sep 1992 14:47:52 GMT

barmar@think.com (Barry Margolin) writes:
}erodrigu@dcc.uchile.cl (Eduardo Rodriguez) writes:
}>But you must also consider, as someone said, the space in disk wasted
}>due to large .newsrc, many times duplicated, with many groups that
}>actually can be deleted. I think the deletion procedure is OK.

}Disk space is quantized by disk blocks.  Unsubscribed newsgroups take up
}little space in .newsrc files (I occasionally go through my .newsrc getting
}rid of all the article numbers for unsubscribed groups: "1,$s/!.*$/!/" in
}ed/ex/vi.  Getting rid of the lines for nonexistent groups is only likely
}to recover one or two disk blocks per user.  If a site has 1,000 netnews
}readers and they recover an average of one 1K block each, that's only a
}megabyte.

The trn newsreader updates the .newsrc lines for all groups listed in
the Newsgroups: header line (including ones you are not subscribed to).
This tends to produce very big .newsrc file (and we have ~5000 netnews
readers!).  I hacked together a program (below) which mungs lines of the 
form:
	newsgroupname! [#1] [,-#]... [#2]
to:
	newsgroupname! [#1 [- #2]]
so:
	foo.bar.baz! 36-38,87,90,99,101,103,104-107,109
becomes:
	foo.bar.baz! 36-109

Here are the results for my 1566 line .newsrc file:

	% rcmung < .newsrc > .newsrc.short
	% ls -l .newsrc*
	-rw-rw----  1 john       117187 Sep  5 08:23 .newsrc
	-rw-rw----  1 john        42871 Sep  5 09:22 .newsrc.short

	75KB saved! (* 5000 users = 375MB!!)

------------------------- cut here for rcmung.c -----------------------
/*
 * rcmung	- mung .newsrc file
 *		- John Hascall, Project Vincent, ISU Comp Ctr
 *		- 5 Sept 1992
 *		- Provided to the public domain "as is", use at your own risk
 */

#include <stdio.h>

char		buffer[65536];		/* that's a LONG line */
unsigned char	action1[256];
unsigned char	action2[256];

#define	TRUNC	1
#define	BOGUS	2
#define	UNSUB	3
#define	SUB	4

#define	DONE	2
#define	NUMB	3

/*
 * no args currently, add "-s" to mung subscribed groups too?
 */
int main()
{
	char *		p;
	char *		q;
	unsigned char	a;
	int		errs	= 0;
	int		first;			/* first article number */
	int		curr;			/* an article number */
	int		last;			/* last article number */

	/*
	 * I'll take space vs time ...
	 * what's a couple hundred bytes when you've a 65KB line buffer?
	 */
	action1['\0'] = TRUNC;
	action1['\n'] = BOGUS;
	action1[ '!'] = UNSUB;
	action1[ ':'] = SUB;
	action2['\0'] = TRUNC;
	action2['\n'] = DONE;
	action2[ '0'] = NUMB; action2[ '1'] = NUMB;
	action2[ '2'] = NUMB; action2[ '3'] = NUMB;
	action2[ '4'] = NUMB; action2[ '5'] = NUMB;
	action2[ '6'] = NUMB; action2[ '7'] = NUMB;
	action2[ '8'] = NUMB; action2[ '9'] = NUMB;

	/*
	 * read a line of input, look for ':' or '!'
	 * mung '!' lines, leave ':' lines as is
	 * output put the (maybe munged) line
	 * dump bad lines to stderr
	 */
	while (fgets(buffer, sizeof(buffer), stdin) != NULL) {
		for (p = buffer; (a = action1[*p]) == 0; ++p) {}
		switch (a) {
		    case TRUNC:
			++errs;
			(void)fputs(buffer, stderr);
			(void)fputc('\n', stderr);
			break;
		    case BOGUS:
			++errs;
			(void)fputs(buffer, stderr);
			break;
		    case UNSUB:
			first = 0;
			last = 0;
			q = ++p;
			do {
				while ((a = action2[*q]) == 0) ++q;
				switch (a) {
				    case TRUNC:
				    case DONE:
					if (first) {
						if (last) {
							(void)sprintf(p,
							    " %d-%d\n%c",
							    first, last, 0);
						} else {
							(void)sprintf(p,
							    " %d\n%c",
							    first, 0);
						}
					} else {
						(void)sprintf(p, "\n%c", 0);
					}
					break;
				    case NUMB:
					curr = 0;
					do {
						curr *= 10;
						curr += (*q++ - '\0');
					} while (action2[*q] == NUMB);
					if (!first) first = curr;
					else if (curr > last) last = curr;
				}
			} while (a != DONE);
			/*FALLTHROUGH*/
		    case SUB:
			(void)fputs(buffer, stdout);
			break;
		}
	}
	return (errs);
}
---------------------- cut here, rcmung.c above --------------------------------
-- 
John Hascall                                       ``Live with it pink-boy!''
Project Vincent                                                              
Iowa State University Computation Center                     john@iastate.edu
Ames, IA  50011                                      515/294-9551 [fax -1717]