From: hendrik@cca.vu.nl (Hendrik te Winkel)
Newsgroups: news.software.nn
Subject: nntp server patch for LISTGROUP command
Date: 12 Aug 92 12:01:34 GMT
Organization: VU Amsterdam - dienst CCA/Cio


Hello colleagues,

This posting contains a patchfile for the nntp daemon
running on the news server.

- What is it?

The patch adds the command LISTGROUP to the server and is
written by me on the advise of the authors of nn.

It is intended to optimize the expiring of articles on
a nnmaster-database running on the client by nntp.

Since the authors of nn already anticipated in such a
'hack' or patch, there is no need to change any of
the nn sources. The patch works only on the nntp server
and afflicts only 4 files in the nntp/server directory.

- Why this patch?

The answer is obvious: the authors give it themself in
the NNTP doc file in the nn distribution (file ./doc/NNTP)
and in the source for nntp (file ./nntp.c).
The expiring of articles in the database only needs a list
of all articles (==numbers==filenames) in that directory.
This was done by the XHDR command, which involved the
opening of every article by the server.
Since it simply wants a directory listing of the files that
only contains numbers, it can be done much faster.

- How does it work?

Well, I implemented that and tested it out for some weeks now,
and the result is very, very good! The expiring of articles
now goes in some 15-30 minutes whereas it took about twelve
hours before!!
This is of most importance to the newsserver, especially
(as in our case) when a lot of nn clients are obtaining news 
from it by nntp.
It is written and tested for SunOS (4.1.2) and uses
the 'old' direct calls: opendir, readdir and closedir.
Now other files are included then those already included in "common.h"
The whole new command is compiled only when the XHDR command
is defined. (Because of it analogies with that command)

- What else?

There are a few things to know:
1) I talked to the authors of the nntp-package, and they said
   that extensions to Internet protocols (not defined in the RFC's)
   should start with the charcter X. Thus it should be XLISTGROUP.
   I do not follow that, because the authors of nn didn't either.
   With the LISTGROUP addition, no changes need to be made
   in the nn sources.
   It should be noted that is it is better to change this in a
   next version. Please don't flame about that.
2) I talked to the author of the nn-package and he suggested to
   post this in this newsgroup, because it is more important to
   nn users than it is to nntp users. So therfore no cross-posting.

Well, here we go then. This is the patch. You should apply it
to the directory "server" of the nntp disrtibution: (./src/nntp/server
or something like that). It modifies Makefile, help.c and serve.c
and it creates listgroup.c (and listgroup.c.orig with size 0 :-)).

I hope it will be of as much help to you as it was to me.
Good luck,
Hendrik te Winkel
Vrije Universiteit - dep. CCA/Cio
Amsterdam - Netherlands
--------------------
*** Makefile.orig	Wed Aug 12 12:00:44 1992
--- Makefile	Wed Aug 12 12:00:44 1992
***************
*** 5,17 ****
  SRVROBJ = main.o serve.o access.o access_inet.o access_dnet.o active.o \
  	ahbs.o globals.o group.o help.o ihave.o list.o misc.o netaux.o \
  	newgroups.o newnews.o nextlast.o ngmatch.o post.o parsit.o scandir.o \
! 	slave.o spawn.o strcasecmp.o subnet.o time.o xhdr.o fakesyslog.o \
  	batch.o auth.o timer.o ../common/version.o
  
  SRVRSRC = main.c serve.c access.c access_inet.c access_dnet.c active.c \
  	ahbs.c globals.c group.c help.c ihave.c list.c misc.c netaux.c \
  	newgroups.c newnews.c nextlast.c ngmatch.c post.c parsit.c scandir.c \
! 	slave.c spawn.c strcasecmp.c subnet.c time.c xhdr.c fakesyslog.c \
  	batch.c auth.c timer.c ../common/version.c
  
  SRVRINC = common.h ../common/conf.h ../common/nntp.h timer.h
--- 5,17 ----
  SRVROBJ = main.o serve.o access.o access_inet.o access_dnet.o active.o \
  	ahbs.o globals.o group.o help.o ihave.o list.o misc.o netaux.o \
  	newgroups.o newnews.o nextlast.o ngmatch.o post.o parsit.o scandir.o \
! 	slave.o spawn.o strcasecmp.o subnet.o time.o xhdr.o listgroup.o fakesyslog.o \
  	batch.o auth.o timer.o ../common/version.o
  
  SRVRSRC = main.c serve.c access.c access_inet.c access_dnet.c active.c \
  	ahbs.c globals.c group.c help.c ihave.c list.c misc.c netaux.c \
  	newgroups.c newnews.c nextlast.c ngmatch.c post.c parsit.c scandir.c \
! 	slave.c spawn.c strcasecmp.c subnet.c time.c xhdr.c listgroup.o fakesyslog.c \
  	batch.c auth.c timer.c ../common/version.c
  
  SRVRINC = common.h ../common/conf.h ../common/nntp.h timer.h
*** help.c.orig	Wed Aug 12 12:02:51 1992
--- help.c	Wed Aug 12 12:00:46 1992
***************
*** 23,28 ****
--- 23,29 ----
  	printf("IHAVE       NEWNEWS      SLAVE\r\n");
  	printf("\r\nAdditionally, the following extention is supported:\r\n\r\n");
  	printf("XHDR        Retrieve a single header line from a range of articles.\r\n");
+ 	printf("LISTGROUP        List all article numbers in a newsgroup.\r\n");
  	printf("\r\n");
  	printf("Bugs to Stan Barber (Internet: nntp@tmc.edu; UUCP: ...!bcm!nntp)\r\n");
  	printf(".\r\n");
*** serve.c.orig	Wed Aug 12 12:00:48 1992
--- serve.c	Wed Aug 12 12:00:48 1992
***************
*** 26,32 ****
  
  extern	int	ahbs(), group(), help(), ihave();
  extern	int	list(), newgroups(), newnews(), nextlast(), post();
! extern	int	slave(), stat(), xhdr();
  
  extern int errno;
  
--- 26,32 ----
  
  extern	int	ahbs(), group(), help(), ihave();
  extern	int	list(), newgroups(), newnews(), nextlast(), post();
! extern	int	slave(), stat(), xhdr(), listgroup();
  
  extern int errno;
  
***************
*** 60,65 ****
--- 60,66 ----
  	"stat",		0,	ahbs,
  #ifdef XHDR
  	"xhdr",		0,	xhdr,
+ 	"listgroup",	0,	listgroup,
  #endif XHDR
  };
  #define NUMCMDS (sizeof(cmdtbl) / sizeof(struct cmdent))
*** listgroup.c.orig	Wed Aug 12 12:30:26 1992
--- listgroup.c	Wed Aug 12 12:30:02 1992
***************
*** 0 ****
--- 1,133 ----
+ #ifndef lint
+ static char	*sccsid = "@(#)$Header: listgroup.c,v 1.11 90/12/11 23:59:53 sob Exp $";
+ #endif
+ 
+ 
+ #include "common.h"
+ 
+ 
+ /*
+ #ifdef LISTGROUP
+ */
+ #ifdef XHDR
+ 
+ /*
+  * LISTGROUP newsgroup
+  *
+  * newsgroup is a newsgroup name
+  *
+  * LISTGROUP newsgroup		retrieve list of article numbers
+  *
+  * This command is an extension, and not included in RFC 977.
+  * Made through the suggestion from the author of nn, and
+  * written to obtain a better performance when expiring across
+  * the network.
+  * Hendrik te Winkel hendrik@cca.vu.nl
+  * Wed Jul 29 1992
+  * This version uses the old sys/dir.h functions calls, and is
+  * currently only tested with SunOS 4.1.2
+  */
+ 
+ 
+ #define CURRENT "."			/* current dir, skip */
+ #define PARENT  ".."			/* idem for parent dir */
+ #define ALL_NUMBERS	"0123456789"	/* article-files consist of
+ 					 * only numbers */
+ 
+ listgroup(argc, argv)
+ 	int		argc;
+ 	char		*argv[];
+ {
+ 
+ 	char	temp_dir[MAXPATHLEN];
+ 	int	high_msg, low_msg;
+ 	char	*cp;
+ 	char	*reqlist[2];
+ 	DIR     *dir_ptr;
+ 	struct direct   *dp;
+ 
+ 	if (argc != 2) {
+ 		printf("%d Usage: LISTGROUP newsgroup.\r\n", ERR_CMDSYN);
+ 		(void) fflush(stdout);
+ 		return;
+ 	}
+ 
+ 	if (!canread) {
+ 		printf("%d You only have permission to transfer, sorry.\r\n",
+ 			ERR_ACCESS);
+ 		(void) fflush(stdout);
+ 		return;
+ 	}
+ 
+ 	if (index(argv[1], '/') != (char *) NULL) {
+ 		printf("%d Invalid group name (bad format).\r\n", ERR_NOGROUP);
+ 		(void) fflush(stdout);
+ 		return;
+ 	}
+ 
+ 	if (find_group(argv[1], num_groups, &low_msg, &high_msg) < 0) {
+ 		printf("%d Invalid group name (not in active).\r\n",
+ 			ERR_NOGROUP);
+ 		(void) fflush(stdout);
+ 		return;
+ 	}
+ 
+ 	reqlist[0] = argv[1];
+ 	reqlist[1] = NULL;
+ 
+ 	if (ngpermcount) {
+ 		if (ngmatch(s1strneql, ALLBUT,
+ 		    ngpermlist, ngpermcount, reqlist, 1) == 0) {
+ 			printf("%d You're not allowed to read %s, sorry.\r\n",
+ 				ERR_ACCESS, argv[1]);
+ 			(void) fflush(stdout);
+ 			return;
+ 		}
+ 	} else if (ALLBUT == 0) {
+ 		printf("%d You're not allowed to read %s, sorry.\r\n",
+ 			ERR_ACCESS, argv[1]);
+ 		(void) fflush(stdout);
+ 		return;
+ 	}
+ 
+ #ifdef LOG
+ 	syslog(LOG_INFO, "%s group %s", hostname, argv[1]);
+ #endif
+ 
+ 	while ((cp = index(argv[1], '.')) != (char *) NULL)
+ 		*cp = '/';
+ 
+ 	(void) strcpy(temp_dir, spooldir);
+ 	(void) strcat(temp_dir, "/");
+ 	(void) strcat(temp_dir, argv[1]);
+ 
+ #ifdef LOG
+ 	++grps_acsd;
+ #endif
+ 
+ 	if ( ( dir_ptr = opendir( temp_dir ) ) == NULL ) {
+ 		printf( "%s opendir: Can't open %s\r\n",
+ 			ERR_ACCESS, temp_dir );
+ 	} else printf("%d Command implemented\r\n", OK_GROUP );
+ 
+ 	while( ( dp = readdir( dir_ptr ) ) != NULL )
+ 		/* This test gets the filenames that solely exist
+ 		 * from characters from the ALL_NUMBERS define.
+ 		 * This is needed for groups that contain both
+ 		 * articles and subgroups (=directories)
+ 		 */
+ 		if ( strspn( dp->d_name, ALL_NUMBERS ) ==
+ 		  strlen( dp->d_name ) )
+ 			printf("%s\r\n",dp->d_name);
+ 
+ 	putchar('.');
+ 	putchar('\r');
+ 	putchar('\n');
+ 	(void) fflush(stdout);
+ 	closedir( dir_ptr );
+ }
+ 
+ /*
+ #endif LISTGROUP
+ */
+ #endif XHDR
-- 
Hendrik te Winkel hendrik@cca.vu.nl
	Sydney has quadrupled in size since I was young,
	and I suppose I have too.
	- Clive James, visiting the city in which he grew up.
Newsgroups: news.software.nn