HDTenEightyP/NetNews huggingface dataset:
From: melby@dove.yk.fujitsu.co.jp (John B. Melby)
Newsgroups: comp.std.internat
Subject: Re: Radicals Instead of Characters
Date: 21 Jan 93 14:47:39 GMT
	<1jhvstINNbcq@flop.ENGR.ORST.EDU>
	<1993Jan20.233154.19733@fcom.cc.utah.edu>
Organization: Open Systems Group, Fujitsu Limited, Yokohama
In-reply-to: terry@cs.weber.edu's message of 20 Jan 93 23:31:54 GMT

>What is the min, max, and median number of radicals per glyph?  How about
>per word?  As was pointed out by someone else, Japanese may take multiple
>glyphs to encode a word.

Looking at Han characters in a probabilistic sense probably is not going
to help much, since the positioning of radicals varies widely between
characters.

The majority of rare characters needed for normal purposes can be composed
by putting a "productive" radical on the left, top, bottom, or right side
of a character which already has an existing codepoint.  There are two
major catches:  (1) some rare characters cannot be expressed in this manner,
and (2) allowing the display of arbitrary characters using this sort of
composition does not mean that their components will be aesthetically
spaced.

>In terms of benefit to the Japanese, any answer over and above "2" is more
>expensive than raw encoding for a 16 bit font, or "4" if a 32 bit font is
>used.

A 16 bit font is insufficient for encoding rare characters, whichever way
you look at it, although having 16-bit CJK unification and a user-defined
character facility may be sufficient for an average user.

-----
John B. Melby
Fujitsu Limited, Yokohama
melby@yk.fujitsu.co.jp
From: mohta@necom830.cc.titech.ac.jp (Masataka Ohta)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages
Date: 24 Jan 93 12:03:10 GMT
Organization: Tokyo Institute of Technology

In article <1jnlg0INN9n4@life.ai.mit.edu>
	glenn@wheat-chex.ai.mit.edu (Glenn A. Adams) writes:


Glenn, you stop saying anything on Han. You know nothing.

>>Its shape in Japanese or mainland China is

>>whose stroke count is 3, while its shape in Taiwan and Korea is

>>whose stroke count is 4. But, these different characters with differently
>>shaped radicals are unified in Unicode.
>
>Yes, this is a good example of where there is a difference in how
>people count strokes.

Wrong.

>No, this is not the way it is counted in these countries.  That is, you
>cannot say that all of Taiwan and Korea always count 3 strokes; similarly
>you cannot say that all of China and Japan always count 4 strokes.
>Different people count (or write) these radicals differently independently
>of their country of origin.

Wrong.

All Chines, Japanese, Korean ALWAYS cont the stroke of

        *        *
     ****************
        *        *

3 and the stroke of

        *        *
     *******  *******
        *        *

4.

There is no exception.

>However, in all cases there is absolutely no difference in meaning.

Meaning is meaningless here. You can write Greek script with Latin alphabet
without changing meaning.

>In these cases, the CJK-JRG chose a normative count to assign to
>these characters;

You don't understand how stroke count is used.

Normative count is completely useless.

							Masataka Ohta
From: mohta@necom830.cc.titech.ac.jp (Masataka Ohta)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Unicode ISO10646 CharacterEncoding
Date: 21 Jan 93 17:24:36 GMT
Organization: Tokyo Institute of Technology

In article <1ippo5INN7b3@life.ai.mit.edu>
	glenn@wheat-chex.ai.mit.edu (Glenn A. Adams) writes:

>>On the other hand, one can't use process code for interchange, unless
>>you are living in the closed world, because other applications won't
>>accept it.

>Oh, how about ASCII?  It makes a pretty decent process *and* interchange
>code for English text.

ASCII = American Standard Code for Information Interchange.

As I stated, there is no reason that we can't use interchange code for
processing.

Likewise, we can use UTF for processing.

>>You should be crazy. If you are mapping a file under possibly networked
>>environment (these days, all environments are so), you can't use multiple
>>octet fixed width code because of endeaness. Don't say signature, because
>>it makes everything complex and, thus, slow, and your reasoning to map
>>files should be for efficiency and for simplicity.

>Signatures are a far cry simpler than variable length encodings.

Dependence on global state is much worse than variable length encoding.

You can't even concatinate files.

							Masataka Ohta
From: klaus@diku.dk (Klaus Ole Kristiansen)
Newsgroups: comp.std.internat
Subject: Re: European characters (was 8-bit news)
Date: 22 Jan 93 11:37:02 GMT
Organization: Department of Computer Science, U of Copenhagen

melby@dove.yk.fujitsu.co.jp (John B. Melby) writes:

>The Northern European subsets of ISO 646 are 7-bit codes, so there
>is no high bit to be stripped.  Although they are relatively
>obsolete, they are commonly used on the net, partially because
>the net is not 8-bit transparent, and partially because they are
>easily recognizable even by users with ASCII fonts.

>[ {	= AE or A:, ae or a:
>] }	= AA aa (more precisely, aring, a.k.a. Angstrom symbol)
>\ |	= O/ or O:, o/ or o:

The assignment of the Danish/Norwegian letters are very logical.

ae, o/ and aa follows z in that order in the alphabet, so they
get the next characters after z {|}. The Swedish alphabet,
however, ends "o, "a, aa. So in "Swedish ASCII", the
letters are not even in alphabetical order. Someone must
have decided that it was more important to have Swedish
letters print as the letters with the same pronounciation
on Danish and Norwegian printers.

Klaus O K
From: mohta@necom830.cc.titech.ac.jp (Masataka Ohta)
Newsgroups: comp.std.internat
Subject: Re: Cleanicode
Date: 22 Jan 93 09:35:37 GMT
Organization: Tokyo Institute of Technology

In article <2179@blue.cis.pitt.edu>
	djbpitt+@pitt.edu (David J Birnbaum) writes:

>LGC (Latin/Greek/Cyrillic) are clearly distinct culturally, although
>opponents of Han unification have argued the same for CJK characters. I 
>find the argument more persuasive for LGC than for CJK, but my
>information about CJK is entirely second-hand and this may be an area
>that is subjective enough to allow reasonable people to disagree.

LGC are used mostly by Indo-European languaes. They share the single
culture, the European culture. They even share the major religion,
Chiritianity.

On the other hand, the grammer of Chinese language is totally different
from JK languages, while JK languages are similar just as English and
German are similar.

But, as JK languages are so special, the origin of JK languages is yet
unknown.

In Japan, Shintoism and Buddhism are the two major religions. In Korea,
Confucianism is dominant. In China, you can find co-existence of
Confucianism, Buddhism, Taoism, Communism and several other religions.

That is, while LGC are all alike, there are clear cultural difference
between CJK.

>Finally, LGC are graphically similar in some places and graphically
>divergent in others. <A> is everyone's favorite example, where all three
>scripts have similar form and function (although G lc <a> differs from
>LC lc <a> graphically). Things get messier one character over, however,

With Han characters, you can find thouthands of messy cases.

>If we were to unify, how would we do it?

Don't do that.

						Masataka Ohta
From: ishikawa@personal-media.co.jp (Chiaki Ishikawa)
Newsgroups: comp.std.internat
Subject: Re: Script Unification [was: Re: Cleanicode]
Date: 22 Jan 93 11:36:04 GMT
	<2179@blue.cis.pitt.edu> <1jlojhINNqv3@life.ai.mit.edu>
Reply-To: ishikawa@personal-media.co.jp
Organization: Personal Media Corp., Tokyo Japan
In-reply-to: glenn@muesli.ai.mit.edu's message of 21 Jan 93 08:58:25 GMT
X-Md4-Signature: 3907e450aebcdeaf2f7b6802e3c547a4


I understand (I hope!) Glenn's proposition although I don't agree
fully what are in there. But, again, I have to thank him for his easy
to understand exposition of important ideas.

[I believe now that opposition to CJK unification is maily due to the
difference people have about the degree with which to judge if two
glyphs are similar enough for unification or not. The question is not
black and white. One of my colleage I talked thinks some kind of
unification is a good idea. But he also says he can find inconsistent
unification in CJK characters.]

Now, I thought I knew European scripts as well as an average Japanese
college graduate in science [and studied in Canada for one year to top
it] ought to know. I took German for two years, and tried to learn
Russian and French as extra curricular activity.  However, after
reading a paragraph in Glenn's posting, I need some inputs to make
sure if I understand European scripts.

>Of course it is true that CJK unification does have certain costs,
>e.g., different implicit sort orders cannot be maintained without
>language tags, minor distinctions in the glyphic representation of
>CJK character data cannot be made without language tags, and so
>forth.  However, and this is important to consider, such distinctions
>are not maintained by character set standards practices for other
>scripts either:  the English, German, French, and Spanish alphabets,
>all distinct in their ordering rules, all potentially requiring slightly
>different glyphic displays, 

The question is here. I can tell that French c cedille is different
from ordinary English c. Or German eszet(?) is different from English
s or B. Now, I have no objection to assign different code to c cedille
and c or to eszet and the rest of the L/C/G of characters.

But, here is the dumb question. Are 'a', 'b', 'c' in English and, say,
the similar looking characters in French given slightly different
glyphic display under similar circumstances?! I have never thought
they would be.  I don't know why, but I had the impression from
reading the above paragraph and the text which followed that there is
a circumstance where 'a', 'b', 'c' in different languages use slightly
different glyphic display in different countries. (I mean is there
Spanish-style 'a', English-style 'a', and so forth???)

Please let me know. I might have missed important assumption held by
the users of these languages.

 Chiaki Ishikawa,  Personal Media Corp.,  MY Bldg, 1-7-7 Hiratsuka,
 Shinagawa, Tokyo 142, JAPAN. FAX:+81-3-5702-0359, Phone:+81-3-5702-0351
 UUNET: ishikawa@personal-media.co.jp
From: unrza3@cd4680fs.rrze.uni-erlangen.de (Markus Kuhn)
Newsgroups: comp.std.internat
Subject: Re: Script Unification [was: Re: Cleanicode]
Date: 25 Jan 1993 14:57:56 +0100
Organization: Regionales Rechenzentrum Erlangen
Reply-To: mskuhn@immd4.informatik.uni-erlangen.de
NNTP-Posting-Host: cd4680fs.rrze.uni-erlangen.de

glenn@wheat-chex.ai.mit.edu (Glenn A. Adams) writes:

>My point in this paragraph is that existing character sets like ISO8859-1
>(IsoLatin1), or the Windows ANSI set, ...

Can anyone tell me, what't the difference between ISO 8859-1 and the
ANSI/Windows set? They both seem to be exactly identical in my eyes!

Markus

-- 
Markus Kuhn, Computer Science student -=-=- University of Erlangen, Germany
Internet: mskuhn@immd4.informatik.uni-erlangen.de  |  X.500 entry available
--- Wer, wie, was? Wieso, weshalb, warum? Wer nichts fragt bleibt dumm. ---
From: arnet@hpcupt1.cup.hp.com (Arne Thormodsen)
Date: Thu, 21 Jan 1993 16:17:15 GMT
Subject: Re: No-break space & soft hyphen?
Organization: Hewlett Packard, Cupertino
Newsgroups: comp.std.internat

I don't think that the No-break space is in the 8859 sets as an aid
to word processing.  It is there for field-oriented data I/O to indicate
that a particular "space" is NOT a valid place to break up input data.

--arne

Arne Thormodsen
CSO Internationalization
Hewlett-Packard
From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 22 Jan 1993 01:53:02 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: wheat-chex.ai.mit.edu
Keywords: Unicode ISO10646 CharacterEncoding

In article <2771@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
>Dependence on global state is much worse than variable length encoding.
>You can't even concatinate files.

For once, we strongly agree:  locales are brain damaged as far as
determining character encoding or language of text is concerned.

Glenn Adams
From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Date: 22 Jan 1993 02:17:36 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: wheat-chex.ai.mit.edu
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages

In article <2770@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
>Consider, for example, a very common radical used for plant-related
>characters, such as grass and flowers (character code from 827C to 864F
>of DIS 10646-1.2).
>
>Its shape in Japanese or mainland China is
>
>        *        *
>     ****************
>        *        *
>
>whose stroke count is 3, while its shape in Taiwan and Korea is
>
>        *        *
>     *******  *******
>        *        *
>
>whose stroke count is 4. But, these different characters with differently
>shaped radicals are unified in Unicode.

Yes, this is a good example of where there is a difference in how
people count strokes.

No, this is not the way it is counted in these countries.  That is, you
cannot say that all of Taiwan and Korea always count 3 strokes; similarly
you cannot say that all of China and Japan always count 4 strokes.
Different people count (or write) these radicals differently independently
of their country of origin.  Different fonts also make different choices.
However, in all cases there is absolutely no difference in meaning.  One
can freely substitute in writing either form of this radical; few people
will even notice unless they look closely.

Take another example, such as the component form of radical radical 163
and 170 (identical except for position), which is counted as either 2 or
3 strokes.  The different standards and dictionaries have:

  GB 2312 = 2 strokes
  JIS = 2 strokes
  CNS = 3 strokes
  No way of telling with KSC, but Dae Jaweon = 3 strokes
  Han-Viet Tu-dien = 3 strokes

But that these are not consistent nationally:

  China: Kangxi Zidian = 3 strokes
  Japan: Dai Kan-Wa Ziten = 3 strokes

In China (PRC), even though the normative dictionary has 3 strokes,
GB2312 has 2; in Japan, even though a normative dictionary has 3 strokes,
JIS has 2.  A count of two is clearly an innovation; the more traditional
sources count 3.

The point here is that there is not a consistent treatment of stroke
counting in certain cases in the communities which use these character
forms.  Furthermore, in these cases, stroke count does not change
the meaning of the text whatsoever.  It is like having a dollar sign
glyph with one stroke or two strokes through an S, or having one with
a single vertical stroke that intersects the whole S, or extends vertically
from the top and bottom arches without fully intersecting the S.

In these cases, the CJK-JRG chose a normative count to assign to
these characters; doing this does not cause a loss of reliable information,
since,  in these casese, stroke counting isn't reliable in the
first place.

Glenn Adams
From: ishikawa@personal-media.co.jp (Chiaki Ishikawa)
Newsgroups: comp.std.internat
Subject: Re: Radicals Instead of Characters
Date: 22 Jan 93 11:03:33 GMT
	<1993Jan20.233154.19733@fcom.cc.utah.edu>
	<MELBY.93Jan21144739@dove.yk.fujitsu.co.jp>
	<1jlngtINNqnk@life.ai.mit.edu>
Reply-To: ishikawa@personal-media.co.jp
Organization: Personal Media Corp., Tokyo Japan
In-reply-to: glenn@muesli.ai.mit.edu's message of 21 Jan 93 08:39:57 GMT
X-Md4-Signature: 0acf8c72732eb25f081037f0e68cc893

I just need some clarification or opinions from netnews readers.
In article <1jlngtINNqnk@life.ai.mit.edu> glenn@muesli.ai.mit.edu (Glenn A. Adams) writes:

   >A 16 bit font is insufficient for encoding rare characters, whichever way
   >you look at it, although having 16-bit CJK unification and a user-defined
   >character facility may be sufficient for an average user.

   Keep in mind that there is no necessary relation between a 16-bit character
   encoding and a 16-bit font.  One can have a 16-bit character encoding like
   Unicode (with 20,902 precomposed Han characters, and possibly a collection
   of combining radical characters) and display with a 16-bit font that contains
   2^16 Han glyphs, or even with a 24-bit font, a 32-bit font, etc.  The
   relation of Unicode character code to font code is not defined by the
   Unicode display model.

Is the last statement understood by many netnews readers, and is it
shared by UNICODE proponents? I had a vague understanding about this,
and yet it was a revelation to read such a clear statement.

 Chiaki Ishikawa,  Personal Media Corp.,  MY Bldg, 1-7-7 Hiratsuka,
 Shinagawa, Tokyo 142, JAPAN. FAX:+81-3-5702-0359, Phone:+81-3-5702-0351
 UUNET: ishikawa@personal-media.co.jp
From: daniel@borland.com (Dan Veditz)
Subject: Re: No-break space & soft hyphen?
Originator: daniel@genghis.borland.com
Organization: Borland International--Scotts Valley, California
Date: Thu, 21 Jan 1993 19:25:34 GMT

ct@dde.dk (Claus Tondering) writes:
>ag129@cus.cam.ac.uk (Alasdair Grant) writes:
>
>>ISO 8859-1 includes "no-break space" and "soft hyphen" which are intended
>>to print as space and hyphen without being considered suitable targets
>>for automatic line breaks and hyphenation.  Do these characters have a
>>future?  Are they widely used?
>
>I doubt it very much. It seems to me that the people behind 8859 have
>confused two issues here: Character sets and word processing.

I doubt they are used as such, but the NBSP exists in all the European
codepages for DOS PC's I've seen, as character 255 (decimal).  The
soft-hyphen exists in DOS codepage 850 as character 240.

-Daniel Veditz
Newsgroups: comp.std.internat
From: ishikawa@personal-media.co.jp (Chiaki Ishikawa)
Newsgroups: comp.std.internat
Subject: Re: Cleanicode
Date: 22 Jan 93 11:09:40 GMT
	<2179@blue.cis.pitt.edu>
Reply-To: ishikawa@personal-media.co.jp
Organization: Personal Media Corp., Tokyo Japan
In-reply-to: djbpitt+@pitt.edu's message of 20 Jan 93 12:32:19 GMT
X-Md4-Signature: b44eed65a063fdaf031892b90fedceff


In article <2179@blue.cis.pitt.edu> djbpitt+@pitt.edu (David J Birnbaum) writes:
   The preceding rhetorical question (no need to answer) raises a general
   problem that seems not to have received adequate attention and that I
   confront as I attempt to assemble a Slavic Cyrillic inventory.
   Specifically, characterhood depends on form and function, but the
   relationship between form and function is not constant over time. More
   on this on another occasion.

I would appreciate if you could elaborate this in some future time.  I
couldn't fathom what "form and function" you meant.  No, not
necessarily now. I think this group is already too crowded.

 Chiaki Ishikawa,  Personal Media Corp.,  MY Bldg, 1-7-7 Hiratsuka,
 Shinagawa, Tokyo 142, JAPAN. FAX:+81-3-5702-0359, Phone:+81-3-5702-0351
 UUNET: ishikawa@personal-media.co.jp
From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages
Date: 24 Jan 93 18:05:27 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: wheat-chex.ai.mit.edu

In article <2807@titccy.cc.titech.ac.jp> mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
>Glenn, you stop saying anything on Han. You know nothing.

My, how polite you are today!

Yes, you are right that nobody would count a 4 stroke grass radical
as 3 strokes, or a 3 stroke grass radial as 4 strokes.  As usual, you
don't seem to grasp what I'm saying.  What I'm saying is that the
choice of whether to write a grass radical with 3 or 4 strokes is
not cast in concrete; nor does it even matter, except for aesthetic
reasons.  I will grant that certain governments have tried to standardize
on certain printed forms, but that doesn't extend to the way people
write by hand.  Then again, these same governments routinely attempt
to limit the number of Han characters used in printing, to little avail.

You still don't seem to grasp the distinction between characters and
glyphs.  Perhaps you slept through part of your formal education?  You
must have missed the classes which explained the notion of abstraction.

Unicode is not a font standard, it does not specify the glyph to be
used to depict a character.  Any font designer (or display software)
is free to use a glyph with a 3 or 4 stroke grass radical.  Even if
a display system uses a glyph with the different radical form, a
user will not have any trouble understanding the text.  Yes, they
may notice that an unusual glyph shape was used (i.e., different than
expected), but that won't affect their ability to read the text.
This is the definition of "plain text":  legible display.

If you want your software to display different glyphs according to
whether the character represents Chinese, Japanese, or Korean text,
then support either language attribution or font (style) attribution.
In the mean time, stop whining to this newsgroup about your pet
peeves.  The rest of us (i.e., myself) are too busy building systems
that *will* support language tags and font tags to produce typographically
correct results to spend all of our time answering your childish
innuendoes.

Glenn Adams
Newsgroups: comp.std.internat
From: sommar@enea.se (Erland Sommarskog)
Subject: Re: Cleanicode
Organization: Enea Data AB
Date: Sun, 24 Jan 1993 17:02:52 GMT

Chiaki Ishikawa (ishikawa@personal-media.co.jp) writes:
>Why not unify Latin/Cyrillic/Greek 'A'?  This simple question also is
>the cause of uncomfortable feeling many Japanese programmers seem to
>have (including myself).

Were we to design a 16-bit character for this machine invented yesterday
and which we tentively named "computer" I would be inclined to agree
with you. Somehow I have this feeling that having the same character
in two code points will at some time cost someone a lot of time and
money. If he is lucky he will just spend a week trying to find out
why he his program won't compile. If he is unlucky, he will draw the
wrong conclusion of a silently failed search.

But the situation at hand is different. We have compatibility issues
which are very compelling, so the issue is moot.

This also serves as a reply to Masataka Ohta (mohta@necom830.cc.titech.ac.jp):
>>It must equalize all the umpteen instances, yes. And it better do if
>>the user is not going to be violent when he is told "you got the wrong
>>sort of 'W' here, please replace".
>
>What? How do you think about a system which distigushes 'A's in
>Latin/Greek/Cyrillic and tells "you got the wrong sort of 'A' here,
>please replace"?
>
>The system is just poorly desinged.

Yes, however:
1) it's slightly easier to accept that the machine keep scripts apart
   than languages.
2) from my Western European horizon it is a much smaller problem, since
   most users won't be hit by the problem.
-- 
Erland Sommarskog - ENEA Data, Stockholm - sommar@enea.se
From: mohta@necom830.cc.titech.ac.jp (Masataka Ohta)
Newsgroups: comp.std.internat
Subject: Re: islands
Date: 22 Jan 93 08:50:51 GMT
Organization: Tokyo Institute of Technology

In article <keld.726930296@login.dkuug.dk>
	keld@login.dkuug.dk (Keld J|rn Simonsen) writes:

>> The big problem, as Ohta pointed out, is the "cat" problem.
>
>The cat problem is not a big problem, for text files.
>You can just automatically convert all of the input files,
>which can be in different coded character sets, into the
>coded character set of the output file.

If you can't 'cat' two text files just as two binary files,
the OS is not UNIX.

						Masataka Ohta
Newsgroups: comp.std.internat