HDTenEightyP/NetNews huggingface dataset:
From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 9 Jan 1993 17:35:04 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: wheat-chex.ai.mit.edu
Keywords: Unicode ISO10646 CharacterEncoding

In article <1993Jan9.024546.26934@fcom.cc.utah.edu> you write:
>[ First a clarification of something which is my fault because of my
>  background in comm software:  I have been informed that the currently
>  "blessed" correct terminlogy for what I have been calling "Runic
>  encoding" is "Process code", "File code", or "Interchange code".  I'll
>  try to call it "Interchange code" from now on (I feel the other terms
>  imply applications, some of which I disagree with). ]

I should have been more clear.  A "process code" is a fixed-width encoding
suitable for internal processing, e.g., ASCII, Unicode, 10646 UCS2, and
10646 UCS4, EUC wide char; a "file code" or "interchange code" is a
potentially variable length encoding suitable for file storage (non memory
mapped environments) or interchange, e.g., UTF1 and UTF2 (FSS-UTF),
Shift JIS, EUC Multibyte.

[My objection to your use of the word "rune" was (1) you weren't clear
about which of these encodings you were referring to, and (2) I hate
cute terminology which is opaque when perfectly transparent terminology
already exists.]

One should not in general use an interchange code (UTF1 or UTF2) for
processing.  While one may use a process code for interchange, some
communication channels may have difficulties with data transparency
(e.g., Unicode and 10646 UCS[24] allow NULL bytes and ISO2022 C0/C1 control
code bytes in any byte position of their "process codes").

I can't imagine why anyone in their right mind would want to use UTF[12]
or any other ostensibe interchange code for processing, given the problems
of variable length encodings.  However, that doesn't mean that unaware
applications can't effectively use an interchange code internally, e.g.,
8-bit clean applications which interpret only the ASCII (ISO646) characters
could use UTF2 (FSS-UTF) without difficulty.  But if one is to create
an aware application which uses more than the ASCII subset, or if it is
to memory map files, then use of a fixed-with process code (even for backing
store) becomes much more sensible.

Glenn Adams
From: sandstro@cac.washington.edu (Bob Sandstrom)
Newsgroups: comp.std.internat
Subject: Unicode and EACC
Date: 7 Jan 1993 01:45:22 GMT
Organization: UW Networks and Distributed Computing
Distribution: world
NNTP-Posting-Host: shiva1.cac.washington.edu

I'm trying to pin down some facts regarding character encodings.

My copy of The Unicode Standard (Version 1.0, Volume 1, page 113)
has the number 13481 for Z39.64, and 13053 for Big Five.
My copy of The Unicode Standard (Version 1.0, Volume 2, page 20)
has the number 13053 for Z39.64, and 13481 for Big Five.
The change does not appear in Volume 2, Appendix D.
Please explain.

Also, I took some notes from Z39.64 itself, ISSN 1041-5653,
as published by Transaction Publishers.  I wrote down
that Z39.64 has a total of
	9718 on layer 1,
	3603 on layers 2-9
	0    on layers 10-12
	2365 on layers 13-14
This is a total of 13321 on the Chinese layers, and 15686 on all layers.
Where does the 13481 quoted by Unicode come from? 

Other comments about the adaptation of REACC or EACC to post-1985 additions
to CCCII welcomed.

Many thanks.
Bob Sandstrom
University of Washington
sandstro@cac.washington.edu
Newsgroups: comp.std.internat
From: melby@dove.yk.fujitsu.co.jp (John B. Melby)
Newsgroups: comp.std.internat
Subject: Re: Language tagging
Date: 8 Jan 93 10:27:29 GMT
Organization: Open Systems Group, Fujitsu Limited, Yokohama
In-reply-to: rap@news's message of 6 Jan 93 13:44:15 GMT

>About the only radical where the simplified form doesn't
>get a different codepoint (unless something else in the character changes)
>is the "grass" radical -- "+ +" (4 strokes) vs "++" (3 strokes) -- but
>that shouldn't interfere too much with legibility.

Thanks for the clarification.  It has also been pointed out that the
traditional forms of the "eating" radical with the strokes "/\" and "="
at the bottom are not distinguished, except in the case of "inryou no in"/
"yin3liao4 de yin3" (for the reason that it has two codepoints in JISX0208).

-----
John B. Melby
Fujitsu Limited, Yokohama
melby@yk.fujitsu.co.jp
From: jxl@ontario.opl.com (John X. Laporta)
Newsgroups: comp.std.internat
Subject: Multilingual IO library
Date: 7 Jan 93 21:34:42 GMT
Distribution: comp
Organization: opl.com


Some months ago, Yutaka Kataoka of Omron posted information
about the net location of a complete software product that
performed multilingual input/output under XWindows. This included
an extensive set of POSIX locale files.

I cannot remember the name of the package or where it
located. I seem to have lost the email that announced the
software. Since I don't remember the name the usual search
techniques are fruitless.

Please send me email if you know anything about this,
or post your answer to this group.
--
    John X. Laporta
    Technical Specialist
    jxl@opl.com
    TEL: (617) 965-0220 x 2579
    FAX: (617) 965-7599
From: maf@dtek.chalmers.se (Martin Forssen)
Newsgroups: comp.std.internat
Subject: Re: Let's develop ISO sorting rules
Date: 6 Jan 93 18:45:46 GMT
Organization: Chalmers University of Technology, Gothenburg Sweden
X-Newsreader: NN version 6.4.19

unrza3@cd4680fs.rrze.uni-erlangen.de (Markus Kuhn) writes:
>flaps@dgp.toronto.edu (Alan J Rosenthal) writes:
>>unrza3@cd4680fs.rrze.uni-erlangen.de (Markus Kuhn) writes:
>>>Believe me, German users who know what they are talking about won't
>>>complan if "a, "o and "u are not sorted as ae, oe and ue.
>>I believe you.  I'm sure that the analogous statement is true for French.  But
>>I'm almost as sure that Swedish readers won't want "a-circle" to be anywhere
>>near "a".  I think your flexibility on this is a language-relative phenomenon.
>This would be really bad news, could someone from Sweden coment on this?

 I'm afraid a lot of Swedes will refuse to accept any sorting order which
 places aring and adiaeresis adjacent to a. In swedish these are completely
 different characters and aring and adiaeresis comes after z. All of this
 IMHO of course.

 I think it would be great if one could design a single sorting order based
 on character positions, but IMHO this is impossible. There is a too great
 mass of people out there who won't change... I think we will have to define
 some sort of list which defines the local sorting order.

	/MaF

--
Martin Forssen: maf@dtek.chalmers.se or maf@math.chalmers.se
System administrator at math and dtek at Chalmers univ. of technology
From: unrza3@cd4680fs.rrze.uni-erlangen.de (Markus Kuhn)
Newsgroups: comp.std.internat
Subject: Re: ISO paper sizes
Date: 5 Jan 1993 22:04:01 +0100
Organization: Regionales Rechenzentrum Erlangen
Reply-To: mskuhn@immd4.informatik.uni-erlangen.de
NNTP-Posting-Host: cd4680fs.rrze.uni-erlangen.de

walters@HQ.Ileaf.COM (Tim Walters) writes:

>In article <1i9j4gEINN8nd@uni-erlangen.de> mskuhn@immd4.informatik.uni-erlangen.de writes:
>}Here for the aprox. 5% of the world population that don't use ISO paper sizes
>}the dimensions of 3 of the most popular sizes:
>}
>}    A4    210 x 297 mm
>}    C5    162 x 229 mm (envelope for A4 folded to A6 format)
>}    DL    110 x 220 mm (envelope for A4 folded twice parallel)

>Printers in Germany and Japan use different sizes for the B series.
>The German value is specified by DIN; I believe the Japanese value is
>specified by some JIS standard:

>        DIN             JIS    
>B4      250x353         257x364
>B5      176x250         182x257
>B6      125x176         128x182

That's strange news!

However, the DIN standard says that it is compatible with the ISO standard,
only the tolerances allowed by DIN are smaller than those allowed by ISO,
which isn't an incompatibility.

How have the JIS people defined the B series? In ISO and DIN, it is
just the geometric mean between two A series sizes (geometric mean of
a and b is sqrt(a*b)).

Still confused ...

Perhaps you mean the following: DIN/ISO define paper end formats. You write
'printers ... use'. Do you mean the size of the paper delivered to the printer
before it is sent to the offset printing machine and cut to e.g. B4? Ok,
that is not the 'paper end format' that is of interest for the common user.

Markus

-- 
Markus Kuhn, Computer Science student -=-=- University of Erlangen, Germany
Internet: mskuhn@immd4.informatik.uni-erlangen.de  |  X.500 entry available
----- Anyone participating in the use of MS-DOS, Heroin or Cocaine is -----
---- simply not getting the most out of life possible. (Brian Downing) ----
From: avg@rodan.UU.NET (Vadim Antonov)
Newsgroups: comp.std.internat
Subject: Re: Language tagging
Date: 6 Jan 1993 17:07:30 -0500
Organization: UUNET Technologies Inc, Falls Church, VA
Distribution: comp
NNTP-Posting-Host: rodan.uu.net

In article <PREECE.93Jan6092809@predator.urbana.mcd.mot.com> preece@urbana.mcd.mot.com (Scott E. Preece) writes:
>There is an important additional approach:
>
>	Text is represented as a sequence of text objects, each of which
>	has its own locale and cultural tagging.

Agreed, though it is a variation of "register switching" approach
using some control characters. I'd like to see a standard way to
create balanced-parenthesis textual structures; thouhg it will
surely be incompatible with most of the modern text processing
tools.

--vadim
From: mckenney@gauss.cims.nyu.edu (Alan M. McKenney)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Summary: German has more than one "alphabetical order"
Keywords: ISO10646 Unicode German Sorting
Date: 5 Jan 93 16:15:02 GMT
Organization: Courant Institute, NYU, NY, NY, USA

In article <8496@charon.cwi.nl> dik@cwi.nl (Dik T. Winter) writes:
>In article <1i2durINN2pj@rodan.UU.NET> avg@rodan.UU.NET (Vadim Antonov) writes:
  ....

> > >Moreover, one question: how would you encode the German A-umlaut such that
> > >it sorts properly (i.e. as if it is the letter combination AE)?
> > 
> > The sorting order should be strict -- if you have two identical words
> > with a-umlaut and ae in the middle is it the same word? If it is then
> > ae IS a variation of a-umlaut and should always be treated as a signle
> > letter.
> > 
>I do not think you understand.  From the AVON (Amtliches Verzeichnis der
>Ortnetzkennzahlen) edition 1985, which gives area codes for the places in
>Germany.  The next is a selection of places mentioned ("o is o-umlaut):
>	Modautal
>	M"ockm"uhl
>	...
>	M"ornsheim
>	Moers
>	M"ossingen
>	...
>	M"otzingen
>	Mogendorf
>now come up with a coding that allows this (standard German) sorting.
 [and further mentions that "o is equiv. to oe, but oe is not equiv. to
  "o.]

     I'm afraid that Dik Winter, as well-informed as he is, has
     oversimplified things a little. :-)  There is not one sorting
     scheme used in Germany, but (at least?) two.

     The scheme described by Dik *is* the scheme that is used in
     telephone books, (and generally for proper names?) for very good
     reasons.

     It is *not* the sorting that is used in any German dictionary I
     have seen.  In dictionaries, a-umlaut is equivalent to a (not ae)
     for sorting purposes, o-umlaut to o, u-umlaut to u, and "sharp s"
     to ss.

     This further complicates Vadim Antonov's scheme for having character
     codes encode sorting order: we would have to somehow specify that
     under certain circumstances  "a  sorts one way, and under others
     it sorts another.

     I even seem to recall a third scheme, but I don't remember what
     it was, and it could have been something only used by users
     of English-"speaking" computers.


     (However, I am inclined to think that totally different alphabets,
     e.g., the Cyrillics and the Latins, should be represented separately.
     I am not familiar with 10646, but some posts here have suggested
     that 10646 uses the same code for all letters that look like, e.g.,
     T -- I can understand someone not being happy with that, if it is
     true.  However, I don't have much experience in producing a unified
     character set.)


-- 
Alan McKenney        E-mail:  mckenney@cims.nyu.edu         (INTERNET)
Courant Institute,NYU,USA     ...!cmcl2!cims.nyu.edu!mckenney   (UUCP)
From: melby@dove.yk.fujitsu.co.jp (John B. Melby)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 12 Jan 93 10:46:57 GMT
	<1ipo2kINN6g2@life.ai.mit.edu>
	<1993Jan11.193710.29580@fcom.cc.utah.edu>
Organization: Open Systems Group, Fujitsu Limited, Yokohama
In-reply-to: terry@cs.weber.edu's message of 11 Jan 93 19:37:10 GMT

>Consider the worst case scenerio: a Japanese text telling how to write
>Chinese characters.

I don't think this can be considered a worst case scenario.  The style
differences we are looking at between Japanese and Chinese can probably
be summarized as follows:

Chinese Ming:  slender strokes, pronounced hooks
Japanese Ming:  strokes slightly thicker, hooks not as pronounced
Chinese Gothic:  stroke width depends on the spacing of the strokes
Japanese Gothic:  stroke width is generally constant

There may very well be some characters that share a single Unicode point
between Chinese and Japanese but have different forms that are not
completely interchangeable with forms that differ only in a stylistic
manner.  (I assume that Mr. Ohta knows of many such examples, but I
have yet to see a concrete example posted here.)

Besides, in the example above, each character would be graphically
represented to show the stroke order.  (The use of a printed character
would be quite misleading, since the printed version of a character often
seems to have more strokes than the handwritten version.)

-----
John B. Melby
Fujitsu Limited, Yokohama
melby@yk.fujitsu.co.jp
Newsgroups: comp.std.internat
From: erik@poel.juice.or.jp (Erik M. van der Poel)
Newsgroups: comp.std.internat
Subject: islands
Date: 10 Jan 93 06:23:18 GMT
Organization: sometimes

> plan 9 is inherently very distributed and so we see
> the difficulties of teh island approach earlier or more clearly.

The way I see it, Plan 9 is itself an island.  The island may become
very large if enough people see that UTF is a good solution for
predominantly ASCII environments.  The island may even subsume the
Latin-1 communities if they haven't invested too much already in their
code.  I wonder how successful UTF will be in Japan.


> on the other hand, i'd be the last person to advocate massive system
> upheavals just for the sake of using 10646.

Yes, in some cases it's too close to an "all pain, no gain"
proposition, especially since the need for truly multilingual text
isn't as great as that for one's own language (plus English).


> if you are willing to be
> an island, and for many systems it may be the practical choice, then
> all power to you.

Bell Labs was willing to be an island, and UTF was a practical choice,
since most of their text files are ASCII.  That is why Plan 9 could
take the UTF approach.  (Congratulations, NOT flames.)

It will be interesting to see how organizations with truly distributed
systems (as in "selling hardware all over the world") try to cope with
this Unicode thing.  For example, Sun may not be able to take the
all-the-world-is-UTF approach, since they have a fairly large number
of customers using non-ASCII text.  Two major examples of this are
Japanese EUC and European EUC (aka Latin-1).

They currently use environment variables (e.g. "LANG") for "file
typing".  This is how they can use the same "vi" binary in both Japan
and Germany.  One conceivable approach is to extend this rudimentary
file typing to actual file attributes ("groan, not this guy again" I
hear all of you saying), extending NFS as well with a negotiated file
type option.

Then again, that may be too much pain too.  Like I said, it will be
interesting to see what they do about Unicode.

I read that Microsoft is planning to support both Unicode and
Shift-JIS for Windows NT in Japan.  I wonder if they have some sort of
file typing.  Byte-order mark, perhaps?  Anybody know?

Maybe Sun should go for such a "magic number" method?
-
-- 
Erik M. van der Poel                               erik@poel.juice.or.jp
From: boyd@prl.dec.com (Boyd Roberts)
Newsgroups: comp.std.internat
Subject: Re: islands
Date: 10 Jan 93 14:03:23 GMT
Organization: Digital Equipment Corporation - Paris Research Laboratory

In article <C0MKEu.5or@poel.juice.or.jp>, erik@poel.juice.or.jp (Erik M. van der Poel) writes:
> 
> The way I see it, Plan 9 is itself an island.

I'm not sure at all that Plan 9 _is_ an island;  utf and 9P have been
published, you can actually read all of the documentation and understand
it (in contract to things like X and the almost unbounded number of RFC's),
it's more `open' than anything else that goes by the same moniker.

Reminds me of Geoff Collyer's comment [in ;login:] that the [PDP] 11/70 was
`open' because you could get the documentation, read it and build something.
It seems that the wheel has turned full circle in that it's again difficult
to build things because of the intentional (or otherwise) complexity,
obscurity and the broken set of tools to hand.

And while I'm at it, I think that GNU are as much a part of the problem rather
than the solution.  Yes you can get the source, but it's of no use because
it has been crippled by comlexity.  Look at gcc: a zillion command line
options when it should probably have _less_ than 5; the compiler should
just compile a program that adheres to the language spec, no more, no less.

So, what's my plan?  Talk utf and 9P and get some connectivity happening.
TCP/IP access to a public 9P file-server would be a start.  I'd almost be
tempted to `teach' NFS about 9P.  Yes, I'm probably that crazy.


Boyd Roberts			boyd@prl.dec.com

``When the going gets wierd, the weird turn pro...''
Newsgroups: comp.std.internat
From: peter@ferranti.com (peter da silva)
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages
Organization: Xenix Support, FICC
Date: Mon, 4 Jan 1993 17:19:39 GMT

In article <1i2emiINN2td@rodan.UU.NET> avg@rodan.UU.NET (Vadim Antonov) writes:
> Don't you realize that having trivial programs to ask which language
> they're doing operation in effectively defeats the entire purpose of
> Unicode?

Luckily, they don't have to.

> Should my shell ask me about language of every [a-z] in my
> commands?

No, what happens if you have a file name in French in the middle of a
directory full of English names? I've got just that situation on my Amiga,
in the preferences files for some programs: Espa~na.lang, Fran,cais.lang.
With UTF I'm sure I'll have Cyrillic, Hebrew, Han, and Czech filenames.

In most cases it doesn't matter. Where it does, it's not a trivial matter
(for example, in Holland you probably want files to sort using the Dutch
 dictionary order... not the Scandinavian one... no matter what language
 the word is actually in).
-- 
Peter da Silva                                            `-_-'
Ferranti International Controls Corporation                'U` 
Sugar Land, TX  77487-5012 USA
+1 713 274 5180                            "Zure otsoa besarkatu al duzu gaur?"
From: boyd@prl.dec.com (Boyd Roberts)
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: ISO10646 Unicode
Organization: Digital Equipment Corporation - Paris Research Laboratory
Date: Mon, 4 Jan 1993 21:41:09 GMT

In article <2605@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
> 
> In Japanese, the only natural sorting is on pronouciation, whose
> information is missing from character code. Otherwise, any sorting
> is almost as good as other sorting.
> 

Applying your argument to languages that use the glyphs 0, 1, 2, 3, 4, 5,
6, 7, 8 and 9 to represent integers then it is only `natural' to sort
them based on their pronounciation.

So in Japanese I will find myself sorting them in ascending order:

    1 ichi
    9 kyu
    5 go
    3 san
    4 shi
    7 shichi
    0 zero
    2 ni
    8 hachi

But that would be wrong, because I can pronounce:

    0 as rei
    4 as yon
    7 as nana
    9 as ku

So just how will you sort these glyphs based on `natural' pronounciation?

Is it a case of ``gaijin dakara wakaranai''?


Boyd Roberts			boyd@prl.dec.com

``When the going gets wierd, the weird turn pro...''
Newsgroups: comp.std.internat
From: andrew@alice.att.com (Andrew Hume)
Newsgroups: comp.std.internat
Subject: Re: Data tagging (was: 8-bit representation, plus an X problem)
Date: 7 Jan 93 06:49:50 GMT
Article-I.D.: alice.24564
Organization: AT&T Bell Laboratories, Murray Hill NJ

this is probably pointless but i'll have one more go at extracting some
facts out of this interchange.

In article <2630@titccy.cc.titech.ac.jp>, mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
> In article <24551@alice.att.com> andrew@alice.att.com (Andrew Hume) writes:
> >~ Then, how can I create a text file which contains both Chinese
> >~ and Japanese?
> >
> >	i said i didn't know; presumably, there are magic commands (a la RTF)
> >that change fonts.
> 
> Very good.
> Then, isn't the magic commands a special escape sequence or something
> like that?
> Mustn't we know how that particular font specification works?
> Doesn't the font specification make everuthing stateful, so we can't
> seek or anything?

	the layer handling character transport need not know.
one would presume the display layer needs to know. as for seeking,
we can find word boundaries and all sorts of useful things WITHOUT
knowing higher-level information such as font changes or size changes.

> According to your criteria, 10646 is just horrible and utterly hopeless
> to us Japanes.
> Thus, 10646 can't be a internationally usable standard.

	we've come a long way; from a guess of mine as to how
NT does internationalisation, you've deduced 10646 is utterly hopeless.
this is called sophistry.

> >~ Making everything English text is another fatal misdesign of Plan 9
> >~ as an internationalized OS.
> >
> >	damned if i know where i said the text was english.
> 
> How damned Plan9 is. It is much better to be said "Not a tty" than
> "iwmczal aslkdjf lskjancm aslkio" on inappropriate ioctl.

	what? you said plan 9 is crap because it uses english. i replied
that i never said plan 9 had to use english. you reply plan 9 is crap because
it doesn't use english. this must be the logic equivalent of bistro math.

> Internationalization of "errno" interpretation is a simple and
> already-solved problem.

	really? i would think so for the cases like posix where there is
a predefined list of alternatives and implementations such
as a list of language-dependent strings indexed by error numbers work
just fine. when the errors are not so constrained, i am unaware
of any general solution. does anyone know of one? if so, could you
describe it here or email me please?


			andrew hume
Newsgroups: comp.std.internat
From: peter@ferranti.com (peter da silva)
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages
Organization: Xenix Support, FICC
Date: Mon, 11 Jan 1993 23:38:23 GMT

So store the fixed field data in 32-bit form, and store the variable field
data in runic form. You should only use fixed fields where you absolutely
have to anyway.
-- 
Peter da Silva                                            `-_-'
Ferranti International Controls Corporation                'U` 
Sugar Land, TX  77487-5012 USA
+1 713 274 5180                            "Zure otsoa besarkatu al duzu gaur?"
Newsgroups: comp.std.internat