From: karttu@mits.mdata.fi (Antti Karttunen)
Subject: Re: posting to fj groups (patch to inews)
Organization: MITS, Helsinki, Finland
Date: Mon, 7 Sep 1992 00:07:46 GMT

In article <1992Aug31.193214.900@neopath.wa.com> mike@neopath.wa.com writes:
>
>I want to post an article or two to some of the fj newsgroups, and would
>like some advice from others who may know what they're doing. My main
>question is what code should I use for japanese text?
>
>News I post from here appears to get mangled so that the escape characters
>for JIS codes are stripped, but EUC code *seems* to go out fine - I posted
>some messages with a 'local' distribution.  Are readers in Japan going to
>be able to read a EUC message, or will they get a screen full of garbage?

Well, I think that by now most of the newsgates don't strip the eighth
bits off from articles, as I have seen some articles posted accidentally
in EUC to fj.groups from Japan, coming intact here (Finland). (I think
they come through U.S.A). But it is still not recommended to post
with 8-bit codes (EUC & Shift-JIS) as there might be some machines
still somewhere which strip the eighth bits. So use JIS, which uses
only printable characters from ! (33.) to ~ (126.), and ESC, which
however itself is stripped by default version of inews when _sender_
_posts_ the article. So solution is to patch your local inews shell
script (if you use Unix), as mentioned in articles below:

>--
>mike rosenlof                   (206)455-5932
>mike@neopath.wa.com     NeoPath, Bellevue Washington USA

 
Here are some articles I have saved about patching of inews shell script
so that it doesn't strip JIS-in & out escape sequences off from posted
articles. (Simplest way is to remove ESC (code \33) from those characters
to be removed by tr command in inews script, as explained later below.)
(If you are not system administrator yourself, then you should mail this
to him/her and hope that (s)he has couple of minutes to install this patch ;-)

From: rwk@crl.dec.com (Bob Kerns)
Newsgroups: sci.lang.japan
Subject: Fixing news for nihongo
Date: 7 Mar 92 00:05:16 GMT

OK, folks I found the culprit that's been making my
posts lose their escapes.  (I didn't notice this
before, because another bug was making them come out
in EUC code instead of JIS).

I *REALLY* didn't want to learn about news, sigh.

The culprit is the 'inews' program (a moby shell script
which inserts the news into the system, handling headers
and such).  The following lines are at fault:
---------------- BEGIN BOGUS ----------------
 # strip invisible chars from body, a la B news.  bells and escapes are right out.
 case "$trversion" in
 v7)    tr -d  '\1-\7\13\15-\37' ;;
 v6)    tr -d '[\1-\7]\13[\15-\37]' ;;
 esac <$inbody
---------------- END BOGUS ----------------

Now, I claim it's wrong for the transport agent to be doing this;
it should be done by the display agent, which after all is the
authority on what would be dangerous.  Especially since there's
no guarentee that all transport agents will have performed this
filtering; in other words, it really doesn't provide any protection
at all anyway.

The obvious solution is to replace them with

cat <$inbody

However, if your site administrator is paranoid, below is a simple
C program to do the filtering.  Use the line

ifilter <$inbody

in place of the tr stuff.

---------------- BEGIN ifilter.c ----------------
/* 
 * By Bob Kerns @ Digital Equipment Corpration, Cambridge
 * Research Lab (rwk@crl.dec.com), to more safely censor
 * inews without censoring Japanese.
 *
 * This should replace the following evil lines from inews:
 * 
# strip invisible chars from body, a la B news.  bells and escapes are right out.
 case "$trversion" in
 v7)    tr -d  '\1-\7\13\15-\37' ;;
 v6)    tr -d '[\1-\7]\13[\15-\37]' ;;
 esac <$inbody
 *
 * with 
 *
ifilter <$inbody
 *
 * As a statement of my position, I claim that it should *ALWAYS* be
 * the responsibility of the display agent to not do anything harmful
 * with the data, and not that of the transport agent to perform this
 * kind of censoring.
 */


#include <stdio.h>

#define TAB         9
#define NL          10
#define FF          12
#define ESC         27
#define SPACE       ' '

void copy(FILE *in,FILE *out);
void handleESC(FILE *in,FILE *out);

main(int argc,char **argv)
{
  copy(stdin,stdout);
  exit(0);
}

void copy(FILE *in,FILE *out)
{
  int p1;
  while ((p1 = getc(in)) != EOF) {
    if (p1 >= SPACE) {
      putc(p1,out);
    } else {
      switch (p1) {
      case NL :
      case FF :
      case TAB:
	putc(p1,out);
        break;
      case ESC :
	handleESC(in,out);
        break;
      default:
	/* Otherwise, censor it. */
	break;
      }
    }
  }
}

void handleESC(FILE *in,FILE *out)
{ int temp1,temp2;
  temp1 = getc(in);
  switch (temp1) {
  case EOF:			/*  Premature EOF. */
    return;
  case '$':			/* JIS in */
    temp2=getc(in);
    switch (temp2) {
    case EOF:			/* Premature EOF; fall through, */
      break;			/* as if this were a 1-char sequence. */
    case '@':			/* Old JIS */
    case 'B':			/* New JIS */
      putc(ESC,out);		/* Nihongo; include the ESC */
    default:
      putc(temp1,out); putc(temp2,out);
      return;
    }
  case '(':			/* JIS out (old or new)? */
    temp2=getc(in);
    switch (temp2) {
    case EOF:			/* Premature EOF; fall through, */
      break;			/* as if this were a 1-char sequence. */
    case 'J':			/* Yes, JIS out. */
    case 'B':  /* Bob forgot this. ESC(B is also used sometimes. Added by AK */
      putc(ESC,out);
    default:
      putc(temp1,out); putc(temp2,out);
      return;
    }
  case 'K':			/* NEC in */
  case 'H':			/* NEC out */
    putc(ESC,out);		/* Nihongo; include the ESC. */
  default:
    putc(temp1,out);
  }
}
---------------- END ifilter.c ----------------

From: moore@srl.mew.mei.co.jp (W. Phillip Moore)
Newsgroups: sci.lang.japan
Subject: Re: Fixing news for nihongo
Date: 9 Mar 92 01:16:46 GMT
Organization: Matsushita Electric Works Ltd., Osaka, Japan.

Bob> The culprit is the 'inews' program (a moby shell script
Bob> which inserts the news into the system, handling headers
Bob> and such).  The following lines are at fault:

You have discovered the one and only place in all of Cnews which needs to
be hacked for Japanese.  

Bob> However, if your site administrator is paranoid, below is a simple C
Bob> program to do the filtering.  

Far simpler is the following patch to the Cnews inews script, which is
exactly what we use in Japan, and what anyone else using Cnews and getting
fj.* out to be doing.  

W. Phillip Moore	   				 Phone: 81-6-908-1431
UNIX/Network System Administrator			   FAX: 81-6-908-9273
Semiconductor Research Laboratory		  E-mail: moore@mew.mei.co.jp
       Matsushita Electric Works, Ltd.	1048 Kadoma, Osaka 571, Japan

============Cut Here===========
*** relay/sh/inews.dst	Wed Mar 27 07:23:31 1991
--- relay/sh/inews	Wed Mar 27 08:06:43 1991
***************
*** 177,184 ****
  
   # strip invisible chars from body, a la B news.  bells and escapes are right out.
   case "$trversion" in
!  v7)	tr -d  '\1-\7\13\15-\37' ;;
!  v6)	tr -d '[\1-\7]\13[\15-\37]' ;;
   esac <$inbody
  
   if test -r $HOME/.signature; then
--- 177,184 ----
  
   # strip invisible chars from body, a la B news.  bells and escapes are right out.
   case "$trversion" in
!  v7)	tr -d  '\1-\7\13\15-\32\34-\37' ;;
!  v6)	tr -d '[\1-\7]\13[\15-\32][\34-\37]' ;;
   esac <$inbody
  
   if test -r $HOME/.signature; then



--------------------- CUT HERE ----------------------------

In our system (UNIX System V/386 Release 3.2 by Interactive
Systems Corporation) that inews shell-script is linked at least to
the following four places:

/usr/lbin/inews
/usr/lib/news/inews
/usr/lib/newsbin/inject/inews
/usr/local/bin/inews

-- 
Antti Karttunen       --  $B%"%s%C%F%#!!%+%k%C%H%%%M%s(J  (et{ty|t|n)
karttu@mits.mdata.fi  --  Apatheist, although not particularly proud of it.
Newsgroups: sci.lang.japan