HDTenEightyP/NetNews huggingface dataset:
From: Kari.Hurtta@Helsinki.FI (Kari E. Hurtta)
Subject: Re: 8-bit news
Organization: Department of Computer Science, University of Helsinki, Finland
	<davecb.724685064@yorku.ca> <1gtep3INNbm8@rodan.UU.NET>
	<berger.724949297@gram>
Date: Mon, 21 Dec 1992 18:53:42 GMT

In article <berger.724949297@gram> berger@imag.fr (Gilles BERGER SABBATEL) wrote:
> Which restriction?  On SunOs 4.1.2, sendmail transmits 8 bits chars
> without problem, and it do use SMTP.  Obviously, if you send messages to
> 7 bits hosts, the 8th bit will be lost (which give funny chars...), but
> there should not be other problem, unless the software is bugged.

IF you don't put 8 bits chars to address headers !

--
- K E H                                      /  Elm on monimutkaista
  Kari.Hurtta@Helsinki.FI
From: avg@rodan.UU.NET (Vadim Antonov)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 1 Jan 1993 16:56:34 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
NNTP-Posting-Host: rodan.uu.net
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages

In article <1993Jan1.114158.17149@prl.dec.com> boyd@prl.dec.com (Boyd Roberts) writes:
>In article <1i0s05INNnfn@rodan.UU.NET>, avg@rodan.UU.NET (Vadim Antonov) writes:
>> 
>> A good encoding should support easy (i'd say natural) localization.
>> It should provide simple algorithms for simple functions
>> like getting string length, searching a character, case-insensitive
>> comparison, lexicographical comparison.
>> 
>
>Well that's where you're wrong.  The characters and how they are used
>are distinct problems.

Don't you realize that having trivial programs to ask which language
they're doing operation in effectively defeats the entire purpose of
Unicode? Should my shell ask me about language of every [a-z] in my
commands? If it shouldn't then it has to get the information somewhere,
right? If the information is kept outside the text (file names in this case)
then why do i need all those extra bits -- my program *already* knows the exact
(small) alphabet.

"Unicode -- a code for texts which will never be sorted!" Great.

>UNICODE is
>a good example of this:  not only does it specify the code -> glyph
>mapping (ie the encoding) it has support for left -> right, right -> left
>writing styles and a bunch of other stuff, and this part of UNICODE is a mess.

Yuck. Right->left is nothing more than a character with negative width.

>Problem 2 (localisation) is damn hard.

Tell me. I've spent ten years doing *real* localization and i know
the price of ill-thought solutions on the ground level (aka character
set ordering).

>Should Problem 1 cater for the fact I type `localisation' whereas
>you type `localization'?  We're both using Engligh, typed on American
>keyboards (I guess, oops mine's made in West Germany) so where are you
>going to draw the line.  Is this Problem 1?  I say it's Problem 2.

The example is artificial and has nothing to do with the character sets.
As you well aware it is different words in the same alphabet.

--vadim
Xref: sparky comp.std.internat:930 news.admin.misc:915
From: emv@msen.com (Edward Vielmetti)
Newsgroups: comp.std.internat,news.admin.misc
Subject: Re: 8-bit news
Followup-To: comp.std.internat,news.admin.misc
Date: 25 Dec 1992 22:34:41 GMT
Organization: Msen, Inc. -- Ann Arbor, Michigan
NNTP-Posting-Host: garnet.msen.com
X-Newsreader: TIN [version 1.1 PL8]

Damon (dhd@exnet.co.uk) wrote:

: Yes but *LISTEN*!  8-bit only makes it right for (say) European
: languages.  It hardly lets Chinese or Japanese speakers avoid the
: `spelling mistakes' as you put it.

Chinese and Japanese already have several encodings into character
sets that can be sent around through netnews - the fj.* news hierarchy
carries quite a bit of stuff in it right now.  As it happens the fj
groups chose a 7 bit encoding, but that doesn't remove the need for
extra tools (from a Latin alphabet point of view) to read them.

The ideal news reader would be flexible enough to treat the various
character set encodings sensibly and not put undue burdens on the
news posting software to get all sorts of headers right.  So if I'm
reading an article in sfnet.* the } is a Finnish character, but in
news.admin it's a right brace, and in relcom.* it's something Cyrillic.
If people do post with MIME headers that explicitly set the language
and character encoding that's better, but for most distributions there's
a sensible default that can be under the control of the news reader.

  Edward Vielmetti, vice president for research, Msen Inc. emv@Msen.com
        Msen Inc., 628 Brooks, Ann Arbor MI  48103 +1 313 998 GLOB
From: tt@tarzan.jyu.fi (Tapani Tarvainen)
Newsgroups: comp.std.internat
Subject: Re: What are the ISO paper sizes? (Was: An alternative I18N paradigm)
Date: 29 Dec 92 07:39:24 GMT
Organization: University of Jyvaskyla
Originator: tt@tarzan.math.jyu.fi

In article <cline.725593960@ebony.cs.scarolina.edu> cline@cs.scarolina.edu (Ernest Cline) writes:

>However, what I would like to know is what size is A4 or C5 or the rest
>of these map references that ISO uses for paper.
>Please reply by e-mail or posting with either the sizes or suggestion on
>where to look for a listing of _all_ the sizes.


All A sizes can be determined as follows:

A0 is exactly one square meter in area and the ratio of its sides (or
the sizes of any A size paper) is sqrt(2), and each A(n+1) is exactly
half of A(n) in area (and thus cutting an A(n) in the middle results
in two A(n+1) papers, exactly).

Consequently, the longer side of A0 is the 4th root of 2 meters
or about 1189 mm, the shorter side that divided by sqrt(2) or 841 mm,
and A4 those divided by sqrt(2) to the 4th (=4) or 297*210 mm.

I.e., the sides of An paper are 2^(0.25-n/2) and 2^(0.25-(n+1)/2) meters.
I don't know if any rounding has been specified.
Actually I don't even know if the above rule has been written in the
ISO standard or whether they've just listed the sizes, but that's the
way the sizes were derived.  Anybody know better?

I don't know about the C sizes, but they appear to follow a similar
rule.  Roughly, Cn is 1.075 times An on each side.
--
Tapani Tarvainen  (tt@math.jyu.fi, tarvainen@finjyu.bitnet)
From: erik@poel.juice.or.jp (Erik M. van der Poel)
Newsgroups: comp.std.internat
Subject: Re: What are the ISO paper sizes? (Was: An alternative I18N paradigm)
Date: 3 Jan 93 15:28:27 GMT
Organization: sometimes

> As far as I know, these paper sizes are not only used by ISO, but by nearly
> all countries on this world apart from North America and Japan.

Japan also uses A4, etc.  Perhaps you were thinking of NTSC?
-
-- 
Erik M. van der Poel                               erik@poel.juice.or.jp
Newsgroups: comp.std.internat
From: avg@rodan.UU.NET (Vadim Antonov)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 1 Jan 1993 19:03:51 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
NNTP-Posting-Host: rodan.uu.net
Keywords: ISO10646 Unicode

In article <id.68CW.A16@ferranti.com> peter@ferranti.com (peter da silva) writes:
>In article <1i13rrINNars@rodan.UU.NET> avg@rodan.UU.NET (Vadim Antonov) writes:
>> We were talking about lexicographical sorting, not abouth phonetics.
>
>But lexicographic sorting (actually, lexicograhic ordering) is a minor part of
>this. Most sorting computers do is algorithmic ordering, to optimise some
>combination of operations on data structures (searching, for example). The
>character set is irrelevant there.

Wrong-o. Nobody does numerical sorts since invention of secondary
indices.

The problem is not in searching -- the problem is in presenting
the information and in regular expressions ([a-z] - does it include "o?)

>I want a character set that differentiates between parts of speech.

Let's solve simplier problems first. I merely want a character set
which allows me to use my screen editor without fussing around
every search pattern i use.

><para><sentence><phrase>The semantic in <acronym>ASCII</> is <jargon>
>hard-coded</></><dash><phrase>it is the <phrase>order of letters</>
>and <phrase>the trivial <jargon>upper-case</> to <jargon>lower-case</>
>conversion</></></sentence></para>

You got me.

--vadim
From: avg@rodan.UU.NET (Vadim Antonov)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 31 Dec 1992 18:03:05 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
NNTP-Posting-Host: rodan.uu.net
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages

In article <8490@charon.cwi.nl> dik@cwi.nl (Dik T. Winter) writes:
>Wrong Vadim.  You cannot even do it in the European languages.  You cannot
>even do it in German.  How would you assign codes...

Dik, i never insisted that all European languages belong to
the single group -- how many are the ISO Latin-X sets?
My point was that there obviously are identifyable meta-alpahbets
covering several languages.

>A-umlaut sorts as if it is the letter combination AE, and at the same
>time the umpteenth letter of Swedish (after Z).  How would you encode
>spanish where the letter combinations CH and LL are regarded as single
>letters? 
>Or Maltese where the GH-crossbar combination is a single letter
>that does not sort in the neighbourhood of G or H-crossbar but between
>P and Q?  Or dutch, where the letter combination ij is sorted either
>amongst i as a double letter, or amongst y as a single letter, or
>between y and z as a single letter, depending on who does the sorting?

If a combination of letters is treated as a letter IT IS A LETTER.
Then add it to the alphabet and let the keyboard driver (which surely
knows the language -- simply because there are different keyboard
layouts) to handle the matter. FYI, English has some compound letters
too (though they're used only in typesetting) -- ff, fff, fi, ffi, fl, ffl..

The idea of visual encoding (and one letter-onr glyph is nothing more
than a compressed image of the text) is simply wrong because it
drops valuable information readily available at the point of the CREATION
of the text but not later. Sure, the information can (*must* if you're
going to do trivial things like sorting or case-insensitive comparisons)
be preserved off-text (in mail headers or in file attributes, for
example) but it effectively defeats the very purpose of ISO10646 --
why on the Earth do i need to spare bits for encoding glyphs if
i already know the language and 8 (or 16 for oriental languages) bits
is quite enough to map the alphabet. Don't you see this gap in
the logic nullifying all benefits of 10646?

10646 was meant as an encoding eliminating the necessity to carry off-text
information (which is not a piece of cake, especially in multi-lingual
texts). However, the "single glyph" approach ruined the very intent
because you need the off-text information to do trivial tasks anyway!
What's the gain? More wasted bits, yeah?

Take a life, guys. We in Russia did that mistake (DKOI and "GOST" encodings)
many years ago and came to realize that this solution is too simple to
be correct.

--vadim
From: boyd@prl.dec.com (Boyd Roberts)
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages
Organization: Digital Equipment Corporation - Paris Research Laboratory
Date: Thu, 31 Dec 1992 20:31:01 GMT

There are two problems:

    1. Getting an encoding of the characters.

    2. Getting local conventions right.

Problem 2 is hard.  Problem 1 should not address problem 2.

Divide and conquer.

Plan 9 utf solves Problem 1.


Boyd Roberts			boyd@prl.dec.com

``When the going gets wierd, the weird turn pro...''
From: ishikawa@personal-media.co.jp (Chiaki Ishikawa)
Newsgroups: comp.std.internat
Subject: Re: Data tagging (was: 8-bit representation, plus an X problem)
Date: 22 Dec 92 09:08:03 GMT
	<1gtrpdINN6c4@corax.udac.uu.se> <24455@alice.att.com>
Reply-To: ishikawa@personal-media.co.jp
Organization: Personal Media Corp., Tokyo Japan
In-reply-to: andrew@alice.att.com's message of 20 Dec 92 06:37:41 GMT
X-Md4-Signature: d014e4083e841a53c4f8ad47ee0edd19


Hello. I am a Japanese working at a Japanese software company in
Tokyo, Japan.

In article <24455@alice.att.com> andrew@alice.att.com (Andrew Hume) writes:

	[long text deleted]

	   for mostly these reasons, Plan 9 chose a byte-stream encoding
   (initially UTF-1 and then UTF-2) and applied it uniformly according
   to a single rule: all byte streams interpreted as characters shall
   be interpreted as a sequence of 10646 characters encoded as UTF-2.
   this applies everywhere: it applies to the kernel and file server,
   it applies to the window system and the user's display, it applies
   to names in archives and tar files. and best of all, the existing
   system and its text is, because we were an ascii site, already
   correctly encoded. (actually, we were a Latin-1 system, but we were
   willing to make user's convert latin-1 text to the new format.)

	   normally, such a solution
   requires everything entering/leaving the plan 9 universe be converted.
   however as the encoding we use is backward compatible with ASCII,
   no conversion needs be done for the only important case (text files on
   networked filesystems). it also has the advantage that all programs
   can display text uniformly; users don't have to write S-JIS editors
   because the regular editor (sam or ed) edits kana/kanji just fine.
   all the conversion effort can be, and is, confined to one place
   (a program called tcs [translate character sets]). the hope is
   that is most cases, this conversion can happen automatically
   (which is how this stream arose originally; the case of mail
   and news should be easy to make happen).

The work done for plan 9 seems to be very well done in terms of I18N
character support. I think I read an article about Plan 9 itself in a
Usenix publication, but is there a technical paper specicifically
written about I18N aspect of plan 9 available? (BTW, is plan 9 named
after "Plan 9 from outer space"? Now there is a computer game based on
this movie, I have found out.)

	   i believe these system (design and migration) issues have been
   essentially ignored in all the work and fuss on unicode/10646.
   i know that deep within unicode and in places like X/Open, there are
   efforts to develop support libraries for wide characters but this simply
   ignores the system issues.

			   andrew hume

I agree. Characters alone don't make a system I18N. With all the
hoopla in POSIX standardization, meaning of locale still leaves so
many loose ends. However, I can say we are defining the problems
clearly now. The solutions are not in sight, though.

ishikawa@personal-media.co.jp
From: dik@cwi.nl (Dik T. Winter)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages
Date: 1 Jan 93 11:09:47 GMT
Organization: CWI, Amsterdam

In article <1i0vnmINN352@rodan.UU.NET> avg@rodan.UU.NET (Vadim Antonov) writes:
 > In article <8492@charon.cwi.nl> dik@cwi.nl (Dik T. Winter) writes:
 > > > My point was that there obviously are identifyable meta-alpahbets
 > > > covering several languages.
 > >I do think that the number in several is very small.
 > 
 > With a trivial trick of including several codes for identical glyphs
 > for letters from different languages you can put all of them in ONE
 > meta-alphabet.
 > 
As somebody else mentioned already: should a spelling-checker allow a
German A-umlaut in a Swedish word?  If so, what are the consequences,
if not, why not?

Moreover, one question: how would you encode the German A-umlaut such that
it sorts properly (i.e. as if it is the letter combination AE)?  Even in
a single language (German) you can not come up with a coding that gives
proper sorting.  And also how to code the German eszet such that when
uppercased it becomes the *two* letters 'SS' (and how do you lowercase
that again?  Things are not as simple as you appear to think.
-- 
dik t. winter, cwi, kruislaan 413, 1098 sj  amsterdam, nederland
home: bovenover 215, 1025 jn  amsterdam, nederland; e-mail: dik@cwi.nl
From: dik@cwi.nl (Dik T. Winter)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 3 Jan 93 01:46:37 GMT
Organization: CWI, Amsterdam

In article <1993Jan2.230101.20871@enea.se> sommar@enea.se (Erland Sommarskog) writes:
 > The problem with your idea is that you believe that everything is
 > known at input time. It isn't. If you have a list of names which 
 > is to be used in Sweden, Norway, Denmark and Finland, the list will
 > sort differently depending on the reader, not on who is entering the 
 > text. The Swedish and Finnish alphabets ends with A-ring, A-dots,
 > O-dots. The Danish and Norwegian ends with AE-ligature, O-slash,
 > A-ring. Looks trivial for a simple bit-order sort? Nope. Because
 > the dotted A is equivlent to the AE ligature and so is dotted O and
 > O-slash. Thus Danish and Norwegian names with slashed O should
 > appear together with Swedish and Finnish names with dotted O. So
 > the sort algorithm must make no distinction between the two, except
 > when everything else in the same. And the sort algorithm must know
 > in which order the user wants the text to be presented.
Not to mention that the Danish/Norwegian 'aa' is sometimes equivalent
to 'a-ring'.
 > 
 > This is a simple end-user requirement which your proposal is not
 > incapabale to handle.
I think you intended to say that Vadims proposal is not capable to
handle it (I agree with that).
 > 
 > >which is not easy and sometimes ruins the whole logic of the
 > >program (see shell globbing example in my previous posting os
 > >tr example before).
 > 
 > You've talked a lot about regular expressions etc. Frankly I
 > don't give a damn about those. The main bulk of computer users
 > are not programmers and don't know what a regular expression
 > is, so why focus such specific issues?

I agree completely here.  Shell globbing, regular expressions and
sorting by 'ls' are not relevant.  (There are still systems around
that do not sort the list of files at all.  Can you say IBM VM/CS?)
Those things are indeed used by programmers only, end-users search
on fixed strings and would be surprised if their dotted-a shows no
match because of language differences.
-- 
dik t. winter, cwi, kruislaan 413, 1098 sj  amsterdam, nederland
home: bovenover 215, 1025 jn  amsterdam, nederland; e-mail: dik@cwi.nl
Newsgroups: comp.std.internat
From: tt@tarzan.jyu.fi (Tapani Tarvainen)
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Originator: tt@tarzan.math.jyu.fi
Organization: University of Jyvaskyla
	<1i13rrINNars@rodan.UU.NET>
Date: Fri, 1 Jan 1993 12:34:49 GMT

In article <1i13rrINNars@rodan.UU.NET> avg@rodan.UU.NET (Vadim Antonov) writes:

>We were talking about lexicographical sorting

When sorting a list of names, the sorting rules are determined by the
language of the context, not by the language of the names -- indeed,
I'd say names don't _have_ any inherent, context-independent language.

Think about it:  How would you sort a list containing Finnish and
German names -- all Finnish ones first, then all German ones??  Yuck.

Distinguishing between Cyrillic T and Latin T is perhaps sensible.
Distinguishing between German a-umlaut and Finnish a-umlaut isn't.
Wanting enough information in the mere characters to allow sorting
isn't possible without forcing all languages that use (partially)
common character set to use same sorting rules, which I deem
politically impossible in the foreseeable future.
--
Tapani Tarvainen  (tt@math.jyu.fi, tarvainen@finjyu.bitnet)
From: andrew@alice.att.com (Andrew Hume)
Newsgroups: comp.std.internat
Subject: Re: Data tagging (was: 8-bit representation, plus an X problem)
Summary: que?
Date: 28 Dec 92 06:39:52 GMT
Article-I.D.: alice.24494
Organization: AT&T Bell Laboratories, Murray Hill NJ

In article <2563@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
> In article <24479@alice.att.com>
> 	andrew@alice.att.com (Andrew Hume) writes:
> >japanese better get rendered as japanese no matter what your locale is.
> 
> I heard that, microsoft's NT will have a locale mechanism so that
> it can print Japanes Han as Japanese and Chinese Han as Chinese,
> which is impossible with bare 10646/Unicode.

	this is a fundamental and persistent misapprehension of
10646. it was neither designed for, nor can it allowd a system
to display identical Han characters (same 10646 codepoint)
in different fonts WITHOUT some other (non-10646) convention.
10646 simply defines codepoints, allowing the unambiguous specification
of the characters. if you want to set (or switch) fonts, then you
have to invent some scheme to do it, as say ISO 2022 does
(in a slightly clumsy way).
	i claim NT has some convention in addition to the locale
(perhaps, it is part of the lcoale's definition) so it can do this.
but i don't know.

> >another example is the issue of system error messages; what language
> >(for example) should they be in?
> 
> Shouldn't we use error numbers internally and interprete it by local
> programs?

	we couldn't figure out a way to do that in plan 9.
the problem is that a file server, say, may have an arbitrary
number of error conditions, many of which may not be known locally.
originally, plan 9's file system protocol returned numbers for
errors but part of the protocol was getting the text messgae for an
error number (so you could cache these lcoally in a program).
	when the protocol was updated, this was taken out
and errors are now just text strings (64 bytes i think).

	the bottom line is, when you connect to remote services,
there is simply no way to know what they might report as errors.

	andrew
Newsgroups: comp.std.internat,news.admin.misc
From: dhd@exnet.co.uk (Damon)
Subject: Re: 8-bit news
Organization: ExNet Systems Ltd Public Access News, London, UK
Date: Tue, 22 Dec 1992 05:01:50 GMT

In article <berger.724949297@gram> berger@imag.fr (Gilles BERGER SABBATEL) writes:
>Anyway, the problem is no longer that it is difficult to use 8 bits
>chars.  We need them, and we will use them.  It is no longer acceptable
>in current computer technology to be unable to write one's own language
>without spelling mistakes!  This is simply anachronistic.

Yes but *LISTEN*!  8-bit only makes it right for (say) European
languages.  It hardly lets Chinese or Japanese speakers avoid the
`spelling mistakes' as you put it.

So use something like MIME which can cope with a much bigger character
set than 8-bit and begin to solve the problem for all (human, natural)
languages.

IE, don't be parochial and assume that because it solves *your* problem
it's more then moving from a 1% solution to a 1.1% solution...

(An Englishman telling a Frensh person not to be parochial.  B^>)

(Probably, in fact, moving from a 40% to a 60% solution going from US
ASCII (English) to Latin-1 (most common European languages) when you
add on those who would not be happy to converse in English, but would
converse in Spanish, French, German, Italian, etc.)

And funnily enough, MIME doesn't need 8 bits and the changing of all
the existing software, does it?  It may be ``simply anachronistic'' to
be forced to converse in accentless Latin script to communicate
electronically, but is it *clearly* *simply unreasonable* to make
*everyone* convert *all* their mailer/news software to solve your
unique problem when there is a solution which is more general and less
intrusive.

Or am I missing something again, not being a real linguist or
ISO-character-set expert?

Damon
-- 
Damon Hart-Davis                          Internet: dhd@exnet.co.uk, d@hd.org

Public-access UNIX (Suns), news and mail for UK#5 per month.  FIRST MONTH FREE.
[1.35] Cheap Sun eqpt.  UUCP news/mail feeds.  Tel/Fax: +44 81 755 0077.