Newsgroups: comp.bbs.waffle
Subject: waf165unix Keeping usenet uptodate
From: rthomas@hakatac.almanac.bc.ca (Robert N Thomas)
Date: Thu, 27 Aug 92 00:19:11 GMT
Organization: Sir HackAlot's Castle BBS, Port Alberni, B.C., Canada
Keywords: usenet, /word/usenet, /system/usenet

    I managed to get tired of trying to keep the /words/usenet and
system/usenet files up-to-date the easy way.  As your news package
creates & deletes usnet news areas automatically, I have created
script which updates the waffle /words/usenet and /system/usenet files
automatically to reflect these changes.  (Is there some 'better way'
one can do this???).  I run this script once a night via cron..

    Basically what it does, is scans the /usr/lib/news/active file, to
get a basic list of conferences available (moderated conferences automatically
get the /mod option in the /system/usenet file).  

    The next thing it does, is appends any "options" I desire to be on some
specific named usenet group.  It also (via the same algorithm) automatically 
creates the words/usenet file which is displayed to the user...

    If you need any explanation, don't hesitate to ask..  The command language
works OK, however, it isn't exactly "perfect" as my version of awk appears
to be pretty braindead..  I am sure someone out there will make imporvements
to this.

    The JOB file follows... (It's about 8K in length)  

#  August 18, 1992   Robert N Thomas         /usr/lib/waffle/system/mkusenet.job
#
# Job #1
#
#    This JOB file is mean to be run periodically, once a month/week to
# update the waffle bbs's /usr/lib/waffle/system/usenet file automatically
# by reading cnew's active file...
#
# Job #2
#  This JOB will also update the /usr/lib/waffle/words/usenet file which 
# is displayed to users trying to JOIN a particular newsgroup.  The second
# job runs in much the same way as the following... 
#
#  Start of Job #1
#
#  Requires:
#
#     /usr/lib/news/active                = cnews active file 
#     /usr/lib/waffle/system/usenet.opts  = options for specific conferences
#        FORMAT:
#         conference.name /options /options /you /want
#
#     /usr/lib/waffle/system/usenet.headr = default usenet options and docs
#        FORMAT:
#          Basically, the first x lines of your system/waffle file.
#          Default options go here as well.
#     /usr/lib/waffle/system/usenet.footr = the footer of waffles usenet file.
#
#  OVERALL FUNCTION:
#     1.  Read the news active file to create a list of all newsgroups.
#         Where a group is "moderated" the output file will have the /mod
#         appended.
#
#     2.  Concatenate the output from "1." to the usenet.opts file and then
#         sort the file.  Where a newsgroup has "special" options desired,
#         there will appear TWO lines.  One, with/without the /moderated
#         option, and one with the "special" options you want enabled for this
#         newsgroup.  Because of the SORT, these two lines will appear one
#         after another.
#
#     3.  Another awk script reads the output from "2." and ensures only
#         one line per unique newsfeed exists.  The options { /mod and
#         whatever user defined ones } are concatenated together to makup
#         the new single line.
#
#
#
#
#  Remove temporary files,  ensure we are in the correct directory
cd /usr/lib/waffle/system
rm jnk    >/dev/null 2>&1
rm jnk.1  >/dev/null 2>&1


# Step 1.
#  awk script reads the cnews active file to get a list of feeds, and
#  places the /mod onto moderated newsgroups.
awk '
$4 != "m" { $4 = "" }
$4 == "m" { $4 = "/mod" }
          { print $1,$4 }
' /usr/lib/news/active >jnk


# Step 2
#      cat jnk usenet.opts | sort   
#   Concatenates the output of Step 1, with the usenet.opts file and sorts
#   the output.
#
# Step 3
#     | awk '.....etc...etc..
#  Taking the output of Step 2 as a pipe, takes this input and merges options
#  together combining dual newsfeed entries into one entry with the options
#  from both entries and mixed them.
#
cat jnk usenet.opts | sort | awk '
$1 != olde {
              if (olde != "") { print olde,oldopts }
              olde = $1
              $1 = ""
              oldopts = $2
              if ( length($3) ) { oldopts = sprintf ("%s %s",oldopts,$3)  }
              if ( length($4) ) { oldopts = sprintf ("%s %s",oldopts,$4)  }
              if ( length($5) ) { oldopts = sprintf ("%s %s",oldopts,$5)  }
              if ( length($6) ) { oldopts = sprintf ("%s %s",oldopts,$6)  }
              if ( length($7) ) { oldopts = sprintf ("%s %s",oldopts,$7)  }
              if ( length($8) ) { oldopts = sprintf ("%s %s",oldopts,$8)  }
              if ( length($9) ) { oldopts = sprintf ("%s %s",oldopts,$9)  }
              if ( length($10) ) { oldopts = sprintf ("%s %s",oldopts,$10)  }
           }

$1 == olde {
              if ( length($2) ) { oldopts = sprintf ("%s %s",oldopts,$2)  }
              if ( length($3) ) { oldopts = sprintf ("%s %s",oldopts,$3)  }
              if ( length($4) ) { oldopts = sprintf ("%s %s",oldopts,$4)  }
              if ( length($5) ) { oldopts = sprintf ("%s %s",oldopts,$5)  }
              if ( length($6) ) { oldopts = sprintf ("%s %s",oldopts,$6)  }
              if ( length($7) ) { oldopts = sprintf ("%s %s",oldopts,$7)  }
              if ( length($8) ) { oldopts = sprintf ("%s %s",oldopts,$8)  }
              if ( length($9) ) { oldopts = sprintf ("%s %s",oldopts,$9)  }
              if ( length($10) ) { oldopts = sprintf ("%s %s",oldopts,$10)  }
           }

END        { if (olde != "") { print olde,oldeopts }   }
'  >jnk.1 


#  Step 4.  Cleanup and concatenate usenet.headr with Step 3's output and
#      usenet.footr to make the new usenet file for waffle to use.
rm jnk
cat usenet.headr jnk.1 usenet.footr >usenet.tmp
mv usenet usenet.TOAST
mv usenet.tmp usenet
rm usenet.TOAST
rm jnk >/dev/null 2>&1
rm jnk.1





# Job #2
#
#  August 18, 1992   Robert N Thomas         /usr/lib/waffle/jobs/mkusenet.job
#
#  This JOB will also update the /usr/lib/waffle/words/usenet file which 
# is displayed to users trying to JOIN a particular newsgroup.  The second
# job runs in much the same way as the following... 
#
#  Requires:
#
#     /usr/lib/news/active                = cnews active file 
#     /usr/lib/waffle/words/usenet.opts  = names for specific conferences
#        FORMAT:
#           conference.name Text Description for this conference for BBS users
#     /usr/lib/waffle/words/usenet.headr = default usenet options and docs
#        The header for the usenet conference listing..
#
#     /usr/lib/waffle/words/usenet.footr = the footer of waffles usenet file.
#
#  OVERALL FUNCTION:
#     1.  Read the news active file to create a list of all newsgroups.
#         Where a group is "moderated" the output file will have the {mod}
#         appended.
#
#     2.  Concatenate the output from "1." to the usenet.opts file and then
#         sort the file.  Where a newsgroup has "special" options desired,
#         there will appear TWO lines.  One, with/without the /moderated
#         option, and one with the "special" options you want enabled for this
#         newsgroup.  Because of the SORT, these two lines will appear one
#         after another.
#
#     3.  Another awk script reads the output from "2." and ensures only
#         one line per unique newsfeed exists.  The options { /mod and
#         whatever user defined ones } are concatenated together to makup
#         the new single line.
#
#
#
#
#  Remove temporary files,  ensure we are in the correct directory
cd /usr/lib/waffle/words
rm jnk    >/dev/null 2>&1
rm jnk.1  >/dev/null 2>&1


# Step 1.
#  awk script reads the cnews active file to get a list of feeds, and
#  places the {mod} onto moderated newsgroups.
awk '
$4 != "m" { $4 = "" }
$4 == "m" { $4 = "{mod}" }
          { print $1,$4 }
' /usr/lib/news/active >jnk


# Step 2
#      cat jnk usenet.opts | sort   
#   Concatenates the output of Step 1, with the usenet.opts file and sorts
#   the output.
#
# Step 3
#     | awk '.....etc...etc..
#  Taking the output of Step 2 as a pipe, takes this input and merges options
#  together combining dual newsfeed entries into one entry with the options
#  from both entries and mixed them.
#
cat jnk usenet.opts | sort | awk '
$1 != olde {
              if (olde != "") { printf ("%-36.36s|%-2.43s\n", olde,oldopts) }
              olde = $1
              $1 = ""
              oldopts = ""
              if ( length($2) ) { oldopts = sprintf ("%s %s",oldopts,$2)  }
              if ( length($3) ) { oldopts = sprintf ("%s %s",oldopts,$3)  }
              if ( length($4) ) { oldopts = sprintf ("%s %s",oldopts,$4)  }
              if ( length($5) ) { oldopts = sprintf ("%s %s",oldopts,$5)  }
              if ( length($6) ) { oldopts = sprintf ("%s %s",oldopts,$6)  }
              if ( length($7) ) { oldopts = sprintf ("%s %s",oldopts,$7)  }
              if ( length($8) ) { oldopts = sprintf ("%s %s",oldopts,$8)  }
              if ( length($9) ) { oldopts = sprintf ("%s %s",oldopts,$9)  }
              if ( length($10) ) { oldopts = sprintf ("%s %s",oldopts,$10)  }
           }


$1 == olde {
              if ( length($2) ) { oldopts = sprintf ("%s %s",oldopts,$2)  }
              if ( length($3) ) { oldopts = sprintf ("%s %s",oldopts,$3)  }
              if ( length($4) ) { oldopts = sprintf ("%s %s",oldopts,$4)  }
              if ( length($5) ) { oldopts = sprintf ("%s %s",oldopts,$5)  }
              if ( length($6) ) { oldopts = sprintf ("%s %s",oldopts,$6)  }
              if ( length($7) ) { oldopts = sprintf ("%s %s",oldopts,$7)  }
              if ( length($8) ) { oldopts = sprintf ("%s %s",oldopts,$8)  }
              if ( length($9) ) { oldopts = sprintf ("%s %s",oldopts,$9)  }
              if ( length($10) ) { oldopts = sprintf ("%s %s",oldopts,$10)  }
           }

END        { if (olde != "") { 
                printf ("%-36.36s|%-2.43s\n", olde,oldopts) 
             }
           }
' >jnk.1 


#  Step 4.  Cleanup and concatenate usenet.headr with Step 3's output and
#      usenet.footr to make the new usenet file for waffle to use.
rm jnk 
cat usenet.headr jnk.1 usenet.footr >usenet.tmp
mv usenet usenet.TOAST
mv usenet.tmp usenet
rm usenet.TOAST
rm jnk >/dev/null 2>&1
rm jnk.1