HDTenEightyP/NetNews huggingface dataset:
From: smee@bristol.ac.uk (Paul Smee)
Newsgroups: news.software.b
Subject: Re: What to do with files in incoming/bad
Date: 18 Dec 92 09:57:51 GMT
Reply-To: P.Smee@bristol.ac.uk (Paul Smee)
Organization: University of Bristol

In article <BzELqA.Kvq@sunlab1.bath.ac.uk> ccsis@sunlab1.bath.ac.uk (Icarus Sparry) writes:
>In the referenced article, P.Smee@bristol.ac.uk (Paul Smee) writes:
>>First try:  Just move the file from in.coming/bad back up into
>>in.coming.  Usually this works.  
>
>Cnews will process the entire batch on the first attempt. The fact that
>you get no error message the second time is because (as Paul says) it
>has already been processed and is therfore discarded. There is no need
>to ask Cnews to re-process it, you have already got all the good articles
>out of the batch.

I've often suspected that to be the case, but have never had time to
analyse the code to see if it's true.  Nice to have my suspicions
verified.  Seemed reasonable that the process tearing the bundle apart
would run through the whole lot, and that the error would have come
from a child handling only the specific message.  Still, it's
comforting (to me) to let Cnews have a second shot at it, and 'mv
.../*.t ....' is only a little more typing than 'rm .../*.t'.

-- 
Paul Smee, Computing Service, University of Bristol, Bristol BS8 1UD, UK
 P.Smee@bristol.ac.uk - ..!uunet!uknet!bsmail!p.smee - Tel +44 272 303132
Xref: sparky news.software.b:3538 news.admin.misc:788
From: qq11@liverpool.ac.uk (Alan Thew)
Newsgroups: news.software.b,news.admin.misc
Subject: Re: newgroup
Date: 16 Dec 92 11:42:05 GMT
Followup-To: news.software.b,news.admin.misc
Organization: The University of Liverpool
X-Newsreader: TIN [version 1.1 PL8]

Henry Spencer (henry@zoo.toronto.edu) wrote:
: In article <1992Dec14.174342.679@metrosoft.com> gvh@metrosoft.com (Gordon Van Huizen) writes:
: >I have what I would guess is the 'standard' newgroup
: >script that comes with the C News software distribution.
: >
.....
: The newgroup command in the Performance Release, although somewhat
: ill-documented, permits much greater local control over how newgroup
: requests are handled.
Could you post an example of a newgroupperm file which filters out alt.* .
I've tried following the info in ctl/newgroup but without success.

I have:

alt.*	any	v

Thanks
-- 
Alan Thew
alan.thew@liverpool.ac.uk   ...!uknet!liv!alan.thew   +44 51 794 3735
From: rsalz@rodan.UU.NET (Rich Salz)
Newsgroups: news.software.b,news.software.nntp,news.software.readers
Subject: Re: INN newbie has innd, nnrpd, trn interaction questions
Date: 17 Dec 1992 12:42:14 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
Distribution: inet
NNTP-Posting-Host: rodan.uu.net

The INN model is that there are two classes of hosts:
	Those that give you articles via IHAVE;
	Those that do (almost) everything but IHAVE.
Those in the first class are called newsfeeds; those in the latter are
called readers.

This model does not fit everyone.  I never claimed that it did, and
sometimes you have to do a bit of fine-tuning to make your reality fit the
INN model.  I don't want to change the INN model because I believe it's a
good one.  Instead there are some things to help:
	My 'clientlib' emulation can be linked into newsreaders to make
	sure they get to the newsreader.

	You can run the reference implementation nntpd instead of nnrpd,
	and put only true newsfeeds in your hosts.nntp file.  See the
	install manual.
From: urlichs@smurf.sub.org (Matthias Urlichs)
Newsgroups: news.software.b
Subject: Re: This UUCP site runs INN
Date: 19 Dec 1992 18:16:20 +0100
Organization: University of Karlsruhe, FRG
NNTP-Posting-Host: 127.0.0.1

In news.software.b, article <1gv12mINNcpm@grasp1.univ-lyon1.fr>,
  Christophe.Wolfhugel@grasp.insa-lyon.fr (Christophe Wolfhugel) writes:
> 
> A fully runninf "innd" needs a few thousands of pages. On my site
> (not yet a full feed), the size seems to stabilize at around 2500 pages
> which is 10 Megs.  Being on a 16 Megs system, it mostly gets swapped
> out.  On incoming NNTP connections, I have seen around 1000 pages being
> loaded.  I also run INN on a 8 Megs Sun 3. Just fine.  My history is
> relatively small, around 7 Megs.
> 
Sparc, 15 outgoing feeds, history is 28 MB (the .pag file, which is what INN
keeps in core, is about 3 MB):

PID TT STAT  TIME SL RE PAGEIN SIZE  RSS   LIM %CPU %MEM COMMAND
207 ?  S     2:56 19 20   7014 6784  620    xx  0.0  1.0 innd

This is after an uptime on 10 hours and with two incoming feeds (which are
nntplink-run and probably idle right now).

> Maybe could Rich post UUNET's memory pages occupation of innd regarding
> to their history's size which is just in "another dimension" compared to
> my 7 Megs.
> 
I think they now have a 14-day cutoff, which means that keeping more than 
three weeks of history is wasting disk space and CPU time. (The number
of "late" articles which would have to be NNTP-transmitted to be recognized
as too old is too small to affect things.)

> > c)      does fairly important'' mean that INN will crash and
> >        burn if it can't malloc  more than 2.5 MB of VM (e.g.)?
> 
> Did anyone try with so little ?
> 
It will crash and burn but if anybody runs a Usenet site with less than 16 MB
of swap configured, it's their own fault if things don't work.

> INN is doing in one process the job of msgidd + nntpd + relaynews, with
> no temporary files, less input-outputs and faster IO operations.
> 
But note that INN keeps everything in memory (this includes the full text of
every incoming article) and writes out everything in one fell swoop (that's
what writev() is there for, folks ;-). Depending on the file system code in
your kernel, this _may_ cause more system load _if_ other programs can't 
access the file system while INN writes out a new article, or an update of
history.pag.
Fortunately, most systems do this correctly.

-- 
If I were two-faced, would I be wearing this one?
	-- Abraham Lincoln (1809-1865)
-- 
Matthias Urlichs  --  urlichs@smurf.sub.org -- urlichs@smurf.ira.uka.de   /(o\
Humboldtstrasse 7 -- 7500 Karlsruhe 1 -- Germany  --  +49-721-9612521     \o)/
Xref: sparky news.software.b:3562 comp.unix.aix:12649 comp.unix.questions:14767
From: rsalz@rodan.UU.NET (Rich Salz)
Newsgroups: news.software.b,comp.unix.aix,comp.unix.questions
Subject: Re: AIX3.1.5 problem with INN1.2
Date: 17 Dec 1992 12:51:13 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
NNTP-Posting-Host: rodan.uu.net

In <1gq77qINNmm4@rs6000.bham.ac.uk> G.Simpson@bham.ac.uk writes:
>If i run /usr/local/news/bin/expire to expire articles, it runs through
>ok, building a new history.n history file. When it is finished, the new
>history file is copied to the old one, as you'd expect. The problem
>arises when i do a 'df' command on the partition, and i find that
>the space taken up by the two history files has not been reclaimed by
>the system, consequently leaving me with little disk space. By shutting
>down and restarting the server, the space becomes available. 
When it's ready to shuffle files, expire pauses innd.  This causes innd
to close the history database so innd doesn't have the files open.
expire moves the files, and then it tells the server to go, which means
it will re-open the database.  So unless there is an AIX bug, innd is
definitely not keeping the old file open.

Do you have any nnrpd servers that also have the old history file open?
	/r$
From: smrttadm@uniwa.uwa.edu.au (Jeffrey Smartt)
Newsgroups: news.software.b
Subject: What is the advantage of INN over CNews?
Date: 14 Dec 1992 14:41:36 +0800
Organization: The University of Western Australia
NNTP-Posting-Host: uniwa.uwa.edu.au
Keywords: inn cnews

G'day,

can someone please tell me the advantages of INN over Cnews?  Is the
inews program in INN written in 'C' rather than 'sh'?

Thanks,
Jeff.
From: zeke@fasttech.com (Bohdan Tashchuk)
Newsgroups: news.software.b
Subject: Re: What is the advantage of INN over CNews?
Keywords: inn cnews
Date: 17 Dec 92 19:17:31 GMT
Article-I.D.: fasttech.1992Dec17.191731.10567
Organization: Fast Technology

In <1gofc3INNdb@milk.Warren.MENTORG.COM> tal@Warren.MENTORG.COM (Tom Limoncelli) writes:

>Competition is really going to drive netnews software into the 21st
>century.

>I'm glad that there has been >1 netnews software packages for a long
>time and that this will continue.

>(and I can't wait for G&H's nntp code, or even just details, to be
>released!)

I'm speculating that Mentor is feeling some heat from Cadence.

But I think it's whimsical thinking that Cnews, INN, <??????>, will be
working furiously to outdo each other.

I couldn't waste so much of my time reading news were it not for the great
Cnews package. But G&H haven't gotten a dime from me for their efforts.

If G&H never released another line of code I'd be happy. I'm not waiting
for them to help "drive netnews into the 21st century". They probably have
real jobs.
From: Christophe.Wolfhugel@grasp.insa-lyon.fr (Christophe Wolfhugel)
Newsgroups: news.software.b
Subject: Re: This UUCP site runs INN
Date: 19 Dec 1992 12:29:26 +0100
Organization: INSA Informatique (Grasp), Lyon, France
NNTP-Posting-Host: grasp1.univ-lyon1.fr

bruce@sonyd1.Broadcast.Sony.COM (Bruce Lilly) asks:
> INN memory requirements:
> a)      exactly how much is lots''?

A fully runninf "innd" needs a few thousands of pages. On my site
(not yet a full feed), the size seems to stabilize at around 2500 pages
which is 10 Megs.  Being on a 16 Megs system, it mostly gets swapped
out.  On incoming NNTP connections, I have seen around 1000 pages being
loaded.  I also run INN on a 8 Megs Sun 3. Just fine.  My history is
relatively small, around 7 Megs.

Maybe could Rich post UUNET's memory pages occupation of innd regarding
to their history's size which is just in "another dimension" compared to
my 7 Megs.

> c)      does fairly important'' mean that INN will crash and
>        burn if it can't malloc  more than 2.5 MB of VM (e.g.)?

Did anyone try with so little ?

> Does innd have to be running continuously, or can it be run on an
> as-required basis from inetd (suitable for machines with low
> volume feeds and only occasional newsreader connections (nntpd
> can be run from inetd))?

Innd is a permanently running daemon.  But on no NNTP sessions, it's
just swapped out if needed.

INN is doing in one process the job of msgidd + nntpd + relaynews, with
no temporary files, less input-outputs and faster IO operations.

This just improves dramatically the performances on any system and
especially on those with slow IO, such as AIX on rs6k boxes.

The bottlenecks are still the same mainly: forwarding news, news reading
and the way articles are stored on disk.

-- 
Christophe Wolfhugel    |    Email: Christophe.Wolfhugel@grasp.insa-lyon.fr
  "Pour le bien-etre de tous, il est interdit de pisser dans le metro."
Xref: sparky news.software.b:3589 news.software.nntp:1710 news.software.readers:2522
From: sukes@eng.umd.edu (Tasuki Hirata)
Newsgroups: news.software.b,news.software.nntp,news.software.readers
Subject: Re: INN newbie has innd, nnrpd, trn interaction questions
Date: 19 Dec 92 05:47:30 GMT
	<1gqnduINN95s@wintermute.phys.psu.edu>
Distribution: inet
Organization: University of Maryland, College Park
NNTP-Posting-Host: egypt.eng.umd.edu
In-reply-to: kenh@leps5.phys.psu.edu's message of 17 Dec 92 20:20:14 GMT

> Nntp-Posting-Host: leps5.phys.psu.edu

> In article <1gqdsgINNom1@rodan.UU.NET> rsalz@rodan.UU.NET (Rich Salz) writes:
>>INN includes the NNTP1.5 'clientlib' API, built on top of the INN routines
>>documented in libinn.3.  I have built rn with them.  I have never built trn,
>>but as long as it doesn't stray outside the clientlib set, it should be
>>possible to edit trn/Makefile so that it links with libinn.a before 
>>the NNTP clinetlib.

> That's the problem.  If you want to build trn with the ability to read the
> threads via NNTP, it wants a few extra routines in the clientlib (those are
> provided by the patches in the trn/nntp directory).  Maybe if I get time I will
> try to massage the trn patches into INN's clientlib.  (Or has anyone else done
> this? :-) )

It's easy to check.  All I did was use the nntp.h from nntp-1.5.11t 
and link trn against INN's clientlib.  Looked at the undefined symbols and
found them in the trn patch.
The only trn patches that needs to be added into INN's clientlib are
rawcheck_serv and rawget_serv. ( I didn't actually patch it into INN's
clientlib yet. )

The trn binary I built with INN libs seems to run fine.

--
|                                     ** Place Disclaimer Here  **            |
| Tasuki Hirata                "Hell, if you understand everything I say,     |
| sukes@eng.umd.edu                   you'd be me."  --  Miles Davis          |
From: rsalz@rodan.UU.NET (Rich Salz)
Newsgroups: news.software.b
Subject: Re: Dataloss with innd/nntplink and slow servers
Date: 17 Dec 1992 12:49:10 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
NNTP-Posting-Host: rodan.uu.net
Keywords: innd nntplink dataloss

In <1992Dec17.131639.18996@unipalm.co.uk> ian@unipalm.co.uk (Ian Phillipps) writes:
>Dec 17 11:13:57 pipe innd: doc.ic.ac.uk dataloss 5754
This will be fixed in patch3 -- no more dataloss; it will go to the F file.

I do not know enough about nntplink to comment on the advisability of having the
two of them share files.
	/r$
Xref: sparky news.software.b:3569 comp.unix.aix:12681 comp.unix.questions:14797
From: simpsong@dns0.bham.ac.uk (Glyn Simpson)
Newsgroups: news.software.b,comp.unix.aix,comp.unix.questions
Subject: Re: AIX3.1.5 problem with INN1.2
Followup-To: news.software.b,comp.unix.aix,comp.unix.questions
Date: 18 Dec 1992 12:31:04 GMT
Organization: The University of Birmingham, England
Reply-To: G.Simpson@bham.ac.uk
NNTP-Posting-Host: dns0.bham.ac.uk
X-Newsreader: TIN [version 1.1 PL8]

Jerry Heyman (jerry@austin.ibm.com) wrote:

> In article <1gq77qINNmm4@rs6000.bham.ac.uk>, usenet@rs6000.bham.ac.uk (USENET News Service) writes:
> > 
> > If i run /usr/local/news/bin/expire to expire articles, it runs through
> > ok, building a new history.n history file. When it is finished, the new
> > history file is copied to the old one, as you'd expect. The problem
> > arises when i do a 'df' command on the partition, and i find that
> > the space taken up by the two history files has not been reclaimed by
> > the system, consequently leaving me with little disk space. By shutting
> > down and restarting the server, the space becomes available. 

> This isn't necessarily an AIX problem.  As with all Un*x's that I've used,
> if process A has a file open and that file is then deleted by process B,
> the file for all future process has been removed from the directory, but the 
> actual space is not reclaimed for the file system until process A has terminated
> and released that inode.

> What you need to look for is old running processes that may be utilizing that
> old history file and kill them.  The reason the space is being reclaimed when
> you shutdown the server is because any of the processes holding open the file
> are killed when the server is restarted.

ok, i found a reader in a CLOSING state - i know i've seen something somewhere
about how to get around this, but cant for the life of me remember where it
was. 

Just to make matters worse, there was so much news that the disks were getting
very full....

thanks for the help.

--
Glyn Simpson, Systems Officer,  //  Voice: +44 (0) 21 414 6575
University of Birmingham        //  Fax:   +44 (0) 21 414 3952
Email: G.Simpson@bham.ac.uk     //  NIC Handle: GS56
Newsgroups: news.software.b
From: mark@drd.com (Resident.Alien)
Subject: expire problem in cnews: Ver.24-Mar-1991
Organization: DRD Corporation
Date: Mon, 14 Dec 1992 14:25:31 GMT

I'm running cnews with a patchdate of 24-Mar-1991 on an H-P 9000/350
running HPUX Ver.8.02 or so.

Expire has recently been dying with the following message mailed to
root:

---
expire problems:
expire: no slash in article path `comp.un'
---

my explist looks like:
# modified version, no archiving
#
# hold onto history lines 14 days, nobody gets >90 days
/expired/                       x       14      -
/bounds/                        x       0-1-90  -

# override later defaults for some groups of note
sci.space.shuttle,rec.birds     x       7       -

# big non-tech groups held long enough for a long weekend
sci,rec,talk,soc,misc,alt,news,gnu      u       4       -

# real noise gets thrown away fast
news.groups                     x       2       -
junk,tor.news.stats             x       2       -

# throw away some technical stuff not worth archiving
comp.os.vms,comp.mail.maps      x       7       -

# default:  7 days and archive
all                             x       7       -

My active file is too large to include here, but every line with
'comp.un' in it seems to be ok.

What do I need to do so that expire can successfully complete?
-- 
mark.lawrence@drd.com                                     (918)743-3013
DRD Corp., 5506 South Lewis Ave., Tulsa, OK 74105         (918)745-9037 fax
From: tal@Warren.MENTORG.COM (Tom Limoncelli)
Newsgroups: news.software.b
Subject: Re: What is the advantage of INN over CNews?
Date: 16 Dec 1992 18:50:27 -0500
Organization: Mentor Graphics -- IC Group
NNTP-Posting-Host: milk.warren.mentorg.com
Keywords: inn cnews

The best thing is that there IS a choice.

Competition is really going to drive netnews software into the 21st
century.

In the next few years, small problems will become big and optimization
priorities are going to change.  With MIME-based news, a standard (and
searchable!) NOV database, and other issues around the corner,
competition is going to be very important.

I'm glad that there has been >1 netnews software packages for a long
time and that this will continue.

(and I can't wait for G&H's nntp code, or even just details, to be
released!)

Tom

-- 
Tom Limoncelli -- tal@warren.mentorg.com (work) -- tal@plts.uucp (play)
Reality is stranger than fiction #943247:  The IRS granted
ANS approval as a 501c3 "charity" on September 14, 1992.
Newsgroups: news.software.b,news.admin.misc
From: alan@ernest.itg.ti.com (Alan Edmonds)
Subject: Re: newgroup
Organization: Texas Instruments, Inc. - Plano, Tx
Date: Wed, 16 Dec 1992 18:36:59 GMT

In article <BzCoI6.JrD@liverpool.ac.uk> qq11@liverpool.ac.uk (Alan Thew) writes:
>Henry Spencer (henry@zoo.toronto.edu) wrote:
>: In article <1992Dec14.174342.679@metrosoft.com> gvh@metrosoft.com (Gordon Van Huizen) writes:
>: >I have what I would guess is the 'standard' newgroup
>: >script that comes with the C News software distribution.
>: >
>.....
>: The newgroup command in the Performance Release, although somewhat
>: ill-documented, permits much greater local control over how newgroup
>: requests are handled.
>Could you post an example of a newgroupperm file which filters out alt.* .

Here's mine.  It send me mail when an alt group tries to be newgrouped.

alt	any	nv
*	any	yv

>I've tried following the info in ctl/newgroup but without success.
>
>I have:
>
>alt.*	any	v
>

It needs to be in $NEWSCTL (usually /usr/lib/news)

>Thanks

Your welcome.
-- 
Alan Edmonds                                     Texas Instruments, Inc.
I don't speak for TI; TI doesn't speak for me    M/S 8515
Work phone: (214)575-6427                        6620 Chase Oaks Blvd.
Email: edmonds@lobby.ti.com                      Plano, Texas  75023
Newsgroups: news.software.b
From: bruce@blilly.UUCP (Bruce Lilly)
Subject: Re: Automatically flushing C News history.o
Organization: Bruce Lilly
Date: Sun, 20 Dec 92 22:50:15 GMT
Reply-To: lilb@sony.compuserve.com (Bruce Lilly)

In article <MRL.92Dec18123306@sun4.uai.com> mrl@uai.com (Mark R. Ludwig) wrote:
>
>What is the purpose of history.o?  When is it needed?  Can I automate
>the removal of history.o?  I'd like not to have to move things.$$

It's a copy of the old history file.

I've been running with the following for over a year with no problem:

$ grep history.o /usr/lib/newsbin/expire/doexpire

expire $ropt $hopt $* $NEWSCTL/explist 2>/tmp/doex$$ && > $NEWSCTL/history.o


Note that if expire fails, nothing happens to history.o.  If expire works,
history.o is truncated to zero length.

This is based on an old (ca. 1990) doexpire. Modify to suit your local
conditions.
-- 
	Bruce Lilly		blilly!bruce@Broadcast.Sony.COM
					...uunet!sonyusa!sonyd1!blilly!bruce
From: dem@meaddata.com (David Myers)
Newsgroups: news.software.b
Subject: Re: Multi-FS news spool layout question
Date: 15 Dec 1992 22:34:54 GMT
Organization: Mead Data Central, Dayton OH
NNTP-Posting-Host: devil.meaddata.com

glenn> The disk is doing as many I/Os per second as it can, but the incoming
glenn> NNTP feeds are backloged and not catching up noticably.  My solution to
glenn> this; split /usr/spool/news over multiple partions.  But this prompts
glenn> my question: Has anyone done any investigation into layout policies for
glenn> I/O efficient multi-partition news spools?  Any serious speculation or
glenn> real world experience?

   Our news spool is spread over two disks on separate controllers,
and it helps.  We get all the standard hierarchies, plus alt and gnu.
In deciding how to divide the hierachies across disks, I considered
their relative sizes, along with their local popularity according to
arbitron (INN can give you great newsgroup access statistics, too).  I
settled on putting alt and rec on one disk, and everything else on the
other.  The news software, including history, logs, and the mthreads
database, are on a third disk.  Swap space is divided between both the
third disk, and a fourth disk that otherwise isn't used for news.

   What?!  Not everybody has four disks on their news systems?  Don't
tell me: you only have one CPU, too?  :-) :-) :-)

   This has appeared to balance the I/O load reasonably well across
the two news disks.  The third disk is only heavily used when mthreads
runs.  The disk space used is also relatively well balanced, even
though everything expires at the same rate (we don't get a.b.p.e., so
maybe that slants things a bit :-).

glenn> The reason I think this is an issue is that to read an article via a
glenn> sym-link name costs 2 I/Os instead of 1, negating part of the benefit
glenn> of using 2 disks.  This tells me that to maximize performance, I must
glenn> minimize the number of reads by symlink name.  A good heuristic to
glenn> achieve this is to minimize the number of symlinks.

   I worried a bit about symlink overhead, but figured it wouldn't be
as significant a detriment as the double disks would be a benefit, and
that filesystem cacheing might offset it somewhat.  I don't know how
you'd come up with an algorithm to minimize symlinks, as crossposting
seems to me to be largely unpredictable.  Which disk would you put
alt.flame on?  :-)

--
David Myers                                             (513) 865-1343   
Mead Data Central           No news is             Fabrication Systems
P.O. Box 933                 bad news.                dem@meaddata.com
Dayton, Ohio  45401                             ...!uunet!meaddata!dem
From: rsalz@rodan.UU.NET (Rich Salz)
Newsgroups: news.software.b
Subject: Re: What is the advantage of INN over CNews?
Date: 15 Dec 1992 10:59:49 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
NNTP-Posting-Host: rodan.uu.net
Keywords: inn cnews

In <1ghab1INNcko@uniwa.uwa.edu.au> smrttadm@uniwa.uwa.edu.au (Jeffrey Smartt) writes:
>can someone please tell me the advantages of INN over Cnews?
They are targeted for different environments, although both systems seem
to work fairly well outside of their targets.  Several people run INN on
UUCP-only machines.

INN is designed for hosts on the Internet with a fair amount of memory and
multiple fast incoming NNTP feeds.  It's a full Usenet system, but most
traffic is NNTP.  (E.g., rnews unpacks a batch and sends it to the server
via NNTP.)  If you do NNTP, you'll find it easier to maintain the one
single system, rather than two.  INN is also fast, if you care about
that.  Posting is synchronous -- when inews returns, the article has been
written to disk and queued for forwarding.  INN puts a low, constant, load
on your machine.

>Is the inews program in INN written in 'C' rather than 'sh'?
Yes it is written in C, but is a mistake to let that be a concern of yours.
	/r$
Newsgroups: news.software.b