HDTenEightyP/NetNews huggingface dataset:
From: sommar@enea.se (Erland Sommarskog)
Subject: Alphabets
Organization: Enea Data AB
Date: Sun, 24 Jan 1993 17:23:23 GMT

Masataka Ohta (mohta@necom830.cc.titech.ac.jp) writes:
>Kosta Kostis (kosta@blues.kk.sub.org) writes:
>>Greek, Latin and Cyrillic are three different *alphabets*.
>>
>>When I say "alphabets" I'm referring to a small number of characters with
>>an order that allow the "user" of such an alphabet to write text in their
>>language. At least some characters in the above mentioned alphabets look
>>different and have different meanings or can't be found in all alphabets.
>
>German alphabet has a character "A with umlaut" which can't be found in
>French alphabet.

Masataka is right on the comment, but wrong on the example since the
French and German alphabets are the same.

But saying that Latin script is one alphabet as Kosta did, is not very
appropriate, since with "alphabet" we usually mean an ordered set of
letters from which we in their original or modified shape form words
of our language. What is included in the alphabet is very much a con-
vention; recently we had a discussion in a group about the Swedish
language on whether W was part of the Swedish alphabet or not. (It is 
not.)

There are about as many Latin alphabets as there are languages written
with the Latin script, however many of them coincide. But if even if
they coincide, the languages may use different modifiers.
-- 
Erland Sommarskog - ENEA Data, Stockholm - sommar@enea.se
From: crowl@jade.CS.ORST.EDU (Lawrence Crowl)
Newsgroups: comp.std.internat
Subject: Re: Radicals Instead of Characters
Date: 22 Jan 1993 19:52:28 GMT
Organization: Computer Science Department, Oregon State University
NNTP-Posting-Host: jade.cs.orst.edu

In article <2791@titccy.cc.titech.ac.jp>
mohta@necom830.cc.titech.ac.jp (Masataka Ohta) writes:
>In article <1jfgq1INNqmn@flop.ENGR.ORST.EDU>
>	crowl@jade.CS.ORST.EDU (Lawrence Crowl) writes:
>
>>The question I was asking was "can you _identify_ a han/kanji character
>>based on a sequence of radicals"
>
>No, you can't. Radicals are for indexing only. The rest of the character
>has its own complex shape.

If you can use radicals for indexing, then you can use them to identify
characters.  The process of identifying a character need not be able
to, by itself, generate an acceptable image.

>>and "would it be reasonable to encode
>>han/kanji on that basis".
>
>Such encoding is too lengthy.

An encoding every variant of every character ever written is not?  At
214 radicals, we can represent a radical in eight bits, and a character
in 8*(average number of radicals per character).  Your non-unified
approach would require roughly eighteen bits per character.

>>Agreed.  However, there is no natural size for tables.  Table sized of
>>4000 are much cheaper than table sizes of 64000.
>
>If you use radical based encoding, it makes everything complex.

Could you please elaborate?  Your argument leaves me unconvinced.

>Moreover, you will have to have sixteen 4000 entry tables which is as
>large as a single 64000 entry table.

No, I don't have to have sixteen 4000 entry tables.  I only need one.

>>But, can sixteen bits represent _all_ historical Han characters _and_
>>the historical texts of all other languages?  My guess is 16 bits can
>>_if_ Han characters are coded as radicals,
>
>Maybe nor may not be. Many complex Han characters are just unique.

Unique in what sense?  Examples?

>BTW, from the view point of programmers, combining characters are
>just unusable.

I am a programmer.  It is from a programmer's view that I made my
proposal.  What I am proposing is not unusable.  In many ways it is
more usable that straight character coding.

-- 
  Lawrence Crowl		503-737-2554	Computer Science Department
			   crowl@cs.orst.edu	Oregon State University
	      ...!hplabs!hp-pcd!orstcs!crowl	Corvallis, Oregon,  97331-3202
Newsgroups: comp.std.internat
From: ketil@edb.tih.no (Ketil Albertsen,TIH)
Subject: Re: European characters (was 8-bit news)
Organization: T I H / T I S I P 
Posting-Front-End: Winix Conference v 92.05.15  1.20 (running under MS-Windows)
Date: Fri, 22 Jan 1993 12:25:46 GMT

In article <2261@blue.cis.pitt.edu>, wbdst+@pitt.edu (William B Dwinnell) 
writes:

>Whatever happened to Unicode?

Well and alive, isn't it? I don't have the funds to order any ISO standards
at the time (the last one cost me approx $10/page!), but as far as I have
read in various magazines, IS 10646 is accepted now. Well, that might have
been as a DIS, not an IS, but from DIS to IS there should be no technical
changes. 10646/1 is supposedly a 32-bit identification scheme intended to
cover just about any printable symbol in the world, while 10646/2, or Unicode,
is a subset using 16 bit codes for text symbols (characters or Asian-
style symbols). They haven't allocated all 65536 codes by far yet; large
fractions of the table are white spots, but I am quite happy with that...
From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Characters vs. Glyphs [was: Re: Radicals Instead of Characters]
Date: 23 Jan 1993 17:15:16 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: wheat-chex.ai.mit.edu

In article <ISHIKAWA.93Jan22200347@ds5200.personal-media.co.jp> ishikawa@personal-media.co.jp writes:
>In article <1jlngtINNqnk@life.ai.mit.edu> glenn@muesli.ai.mit.edu (Glenn A. Adams) writes:
>   Keep in mind that there is no necessary relation between a 16-bit character
>   encoding and a 16-bit font.  One can have a 16-bit character encoding like
>   Unicode (with 20,902 precomposed Han characters, and possibly a collection
>   of combining radical characters) and display with a 16-bit font that
>   contains 2^16 Han glyphs, or even with a 24-bit font, a 32-bit font, etc.
>   The relation of Unicode character code to font code is not defined by the
>   Unicode display model.
>
>Is the last statement understood by many netnews readers, and is it
>shared by UNICODE proponents? I had a vague understanding about this,
>and yet it was a revelation to read such a clear statement.

Just to make thinks more clear still, I will quote from The Unicode Standard,
Volume 1, pp. 12-13, under the section entitled "Glyphs":

  "The Unicode standard was designed to encode characters.  There are
   various relationships between character and glyph: a single glyph may
   correspond to a single character, or to a number of characters, or
   multiple glyphs may result from a single character."

  "Characters reside only in the machine, as srings in memory or on disk, in
   the backing store.  The Unicode standard deals only with character codes.
   In contrast to characters, glyphs appear on the screen or paper as
   particular representations of one or more backing store characters.  A
   repertoire of glyphs comprises a font."

  "Glyph shape and glyph identifier assignments are the responsibility of
   individual font vendors and of the glyph identifier standards."

  "The process of mapping from characters in the backing store to glyphs
   is one aspect of text rendering.  The final appearance of rendered
   text is dependent on context (neighboring characters in the backing
   store), variations in typographic design of the fonts used, and
   formatting information (point size, superscript, subscript, and so on).
   The results on screen or paper can differ considerably from the
   expected or prototypical shape of a letter or character.  The glyph 'A'
   displayed on the screen must not be confused with the character 'A' in
   the backing store."

Glenn Adams

Newsgroups: comp.std.internat,soc.culture.nordic,soc.culture.german,soc.culture.french,alt.folklore.computers,news.future,news.admin.misc
From: ingvar@ufs.uf.se (Ingvar Mattsson)
Subject: Re: European characters (was 8-bit news)
Organization: Unga Forskare Stockholmsdistriktet
Date: Fri, 22 Jan 1993 18:05:48 GMT

ddawson@rodan.acs.syr.EDU (Dick Dawson) writes:

(stuff deleted)
>Another interesting bit:  Some news postings have various punctuations
>in the places of Scandinavian characters that do not exist in the
>American alphabet.  I don't remember the equivalences.  They always
>show as the puncutations, curly braces, vertical bar, etc, on my
>terminals and on the Xerox 4075 laser printer I usually use.  No
>surprise, as the data links at Syracuse Univ. are all 7bit.  Except,
>apparently to a HP Laserjet/Postscript printer... where the actual
>Scandinavian characters appear, about half of the time.  I'm guessing:
>some writers are substituting US punctuations and probably are on
>machines that can't generate Scandinavian characters but others are
>apparently using machines that actually do create Scandinavian
>characters and are sending them in postings; they appear as US
>punctuations when the high bit is stripped.

Well, some of us in Sweden use the old (at least 8 years old) standard
that replaces "a-diaresis" with "{", "a-ring" with "}" and
"o-diaresis" with "|". Normal rules apply to convert from/to UPPER.
Usually when I encounter "swedish characters with their high bit
stripped" they appear as "d", "e", "v" and that makes for fascinating
problems in reading them.

-- 
+ Ingvar Mattsson    + I would like to be a fish, for they have no problems. +
+ Vasav{gen 83       + You are not a fish, how do you know about their mind? +
+ 175 32  J[RF[LLA   + You are not me, how do you know what I know?          +
Xref: sparky comp.std.internat:1271 soc.culture.nordic:8383 soc.culture.german:10152 soc.culture.french:9883 alt.folklore.computers:19171
From: melby@dove.yk.fujitsu.co.jp (John B. Melby)
Newsgroups: comp.std.internat,soc.culture.nordic,soc.culture.german,soc.culture.french,alt.folklore.computers
Subject: Re: European characters (was 8-bit news)
Date: 22 Jan 93 12:11:43 GMT
	<1993Jan3.013554.521@newstand.syr.edu>
	<1993Jan21.005656.25514@newstand.syr.edu>
Followup-To: comp.std.internat
Organization: Open Systems Group, Fujitsu Limited, Yokohama
In-reply-to: ddawson@rodan.acs.syr.EDU's message of 21 Jan 93 05:56:56 GMT

>I'm guessing:
>some writers are substituting US punctuations and probably are on
>machines that can't generate Scandinavian characters but others are
>apparently using machines that actually do create Scandinavian
>characters and are sending them in postings; they appear as US
>punctuations when the high bit is stripped.

The Northern European subsets of ISO 646 are 7-bit codes, so there
is no high bit to be stripped.  Although they are relatively
obsolete, they are commonly used on the net, partially because
the net is not 8-bit transparent, and partially because they are
easily recognizable even by users with ASCII fonts.

[ {	= AE or A:, ae or a:
] }	= AA aa (more precisely, aring, a.k.a. Angstrom symbol)
\ |	= O/ or O:, o/ or o:

If these characters were written in Latin-1, stripping the 8th bit would
often result in different letters (such as "x" for "o:").

-----
John B. Melby
Fujitsu Limited, Yokohama
melby@yk.fujitsu.co.jp
Newsgroups: comp.std.internat
From: hpa@merle.acns.nwu.edu (H. Peter Anvin N9ITP)
Subject: Re: Radicals Instead of Characters
Reply-To: hpa@nwu.edu (H. Peter Anvin)
Organization: You must be kidding!
Date: Fri, 22 Jan 1993 15:19:17 GMT

In article <ISHIKAWA.93Jan22200347@ds5200.personal-media.co.jp> of comp.std.internat,
  ishikawa@personal-media.co.jp writes:
> I just need some clarification or opinions from netnews readers.
> In article <1jlngtINNqnk@life.ai.mit.edu> glenn@muesli.ai.mit.edu (Glenn A. Adams) writes:
> 
>    Keep in mind that there is no necessary relation between a 16-bit character
>    encoding and a 16-bit font...
> 
> Is the last statement understood by many netnews readers, and is it
> shared by UNICODE proponents? I had a vague understanding about this,
> and yet it was a revelation to read such a clear statement.
> 

This is indeed shared by UNICODE proponents, at least it better be!
There are scripts within UNICODE, such as Arabic, that *cannot* be
represented 1:1 between UNICODE points and glyphs (although there *is*
a set of glyphs in the Compatibility Zone, it only covers the basic
Arabic language, not the complete coverage of the script.)   Arabic
characters can have up to four forms: standalone, initial, medial and
final, plus that some ligatures are apparently mandatory.  In
addition, there are diacritical marks to be accounted for, just as in
Latin script.

Newsgroups: comp.std.internat
From: mazur@bostech.com
Subject: Looking for xpg(4?)
Organization: Boston Technology, Inc.
Date: Thu, 21 Jan 93 22:38:57 GMT

Netters:

1) Does anyone know where I can get the most recent xpg libs?  Is xpg4 the
   most recent?

2) Is there an FAQ for this newsgroup?  If so, where can I grab a copy?

Thanks in advance,
dave
From: glenn@muesli.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Re: Radicals Instead of Characters
Date: 21 Jan 1993 08:39:57 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: muesli.ai.mit.edu

In article <MELBY.93Jan21144739@dove.yk.fujitsu.co.jp> melby@dove.yk.fujitsu.co.jp (John B. Melby) writes:
>Looking at Han characters in a probabilistic sense probably is not going
>to help much, since the positioning of radicals varies widely between
>characters.

The idea being discussed for Han decomposition would have different
combining radicals for each of the possible positions the radical
could take; e.g. MAN-LEFT, MAN-TOP, MAN-BOTTOM, etc.

>(1) some rare characters cannot be expressed in this manner,

Characters which could not be decomposed in this manner would be
represented in their entirety (i.e., as non-decomposed symbols).

>(2) allowing the display of arbitrary characters using this sort of
>composition does not mean that their components will be aesthetically
>spaced.

A system that displayed such decomposed symbols would most likely
employ a font which either (1) contained glyphs that represented the
entire symbol; or (2) contained internal instructions that would allow
it to position the radical properly.  In both cases, the correct
display geometry would be used.  The display engine would have to
map multiple coded character elements to single glyph references
or mutliple glyph references as appropriate.

>A 16 bit font is insufficient for encoding rare characters, whichever way
>you look at it, although having 16-bit CJK unification and a user-defined
>character facility may be sufficient for an average user.

Keep in mind that there is no necessary relation between a 16-bit character
encoding and a 16-bit font.  One can have a 16-bit character encoding like
Unicode (with 20,902 precomposed Han characters, and possibly a collection
of combining radical characters) and display with a 16-bit font that contains
2^16 Han glyphs, or even with a 24-bit font, a 32-bit font, etc.  The
relation of Unicode character code to font code is not defined by the
Unicode display model.

Glenn Adams
From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.std.internat
Subject: Re: Alphabets
Date: 24 Jan 1993 20:25:00 GMT
Organization: MIT Artificial Intelligence Laboratory
NNTP-Posting-Host: wheat-chex.ai.mit.edu

In article <1993Jan24.172323.2706@enea.se> sommar@enea.se (Erland Sommarskog) writes:
>Masataka is right on the comment, but wrong on the example since the
>French and German alphabets are the same.

I disagree.  See below.

>There are about as many Latin alphabets as there are languages written
>with the Latin script, however many of them coincide. But if even if
>they coincide, the languages may use different modifiers.

I believe your terminology is problematic here.  There is only one Latin
alphabet, namely, the one used by the Romans.  What I think you want to
say is that there are many Latin script-based alphabets.  We can distinguish
between the Latin alphabet (as used by the romans), and a somewhat
artificial construct, the Latin script (as used by various alphabets).
It might be better to call the Latin alphabet the Roman alphabet just
to be more clear (and then not use the phrase "Latin alphabet").

The Latin script can be though of as the union of symbols contained in
all alphabets which are predominantly derived from the symbols used in
the Roman alphabet.

The notion of alphabet can be specified further still.  If we define
a "writing system" as a set of conventions and rules (an orthography)
applied to a set of symbols (from one or more scripts) in order to
represent some language (sound, meaning, syntax, style, etc.), then
we can say that for each writing system there is one and only one
alphabet, and that such an alphabet is comprised of the collection
of symbols employed by the writing system; furthermore, certain
conventions and rules of the writing system's use of these symbols
are also captured in the alphabet, in particular, the default order
which convention dictates for these symbols.

From the above definition, it is clear that the French and German
alphabets are distinct because (1) they are different writing systems
(i.e., they write different langauges) and (2) they employ different
sets of symbols, e.g., ESS-ZET in German, and (3) they employ different
conventions about these symbols.

The encoding of character sets for computer usage tends to represent
abstract forms which correspond to the elements of a script, irrespective
of their membership (or properties) in specific alphabets (i.e., in the
context of particular writing systems).

The decision on whether to unify the symbols drawn from a collection of
alphabets (i.e., to form a unified script which can represent the
different alphabets) should be based on the utility of performing
unification and not on linguistic or cultural relations among the
different alphabets [see my previous message on Script unification].

Glenn Adams
From: dik@cwi.nl (Dik T. Winter)
Newsgroups: comp.std.internat
Subject: Re: Radicals Instead of Characters
Date: 23 Jan 93 03:01:05 GMT
Organization: CWI, Amsterdam

In article <1jpj9sINNlie@flop.ENGR.ORST.EDU> crowl@jade.CS.ORST.EDU (Lawrence Crowl) writes:
(removed all other attributions).
 > >>The question I was asking was "can you _identify_ a han/kanji character
 > >>based on a sequence of radicals"
 > >
 > >No, you can't. Radicals are for indexing only. The rest of the character
 > >has its own complex shape.
 > 
 > If you can use radicals for indexing, then you can use them to identify
 > characters.  The process of identifying a character need not be able
 > to, by itself, generate an acceptable image.
How come?  I can use the base letter A for indexing, it can not be used to
identify the actual letter (which may have one or more diacritics floating
around).  (Sorry you nordic people that I make a diacritic from the ring!)
 > 
 > An encoding every variant of every character ever written is not?  At
 > 214 radicals, we can represent a radical in eight bits, and a character
 > in 8*(average number of radicals per character).  Your non-unified
 > approach would require roughly eighteen bits per character.
As I understand it each character has only one radical.  (I learned that
a character consists of a radical plus possibly additional strokes.)  You
have the misunderstanding that a character is composed of a number of
radicals.  (BTW, this is true of the Korean Hangul, but that is actually
an alphabetic script, each base symbol represents a sound.)
 > 
...
 > Unique in what sense?  Examples?
Unique in the sense that they do not contain radicals.

(And excuses to you Japanese, Chinese and Korean people for the inaccuracies
I wrote.)
-- 
dik t. winter, cwi, kruislaan 413, 1098 sj  amsterdam, nederland
home: bovenover 215, 1025 jn  amsterdam, nederland; e-mail: dik@cwi.nl
Newsgroups: comp.std.internat
From: dik@cwi.nl (Dik T. Winter)
Newsgroups: comp.std.internat
Subject: Re: Cleanicode
Date: 21 Jan 93 02:34:48 GMT
Organization: CWI, Amsterdam

In article <ISHIKAWA.93Jan20182546@ds5200.personal-media.co.jp> ishikawa@personal-media.co.jp writes:
 > Why not unify Latin/Cyrillic/Greek 'A'?  This simple question also is
 > the cause of uncomfortable feeling many Japanese programmers seem to
 > have (including myself).
 > 
I understand that.  I think some unification might be done with the LCG
glyphs.  But the LCG scripts have the feature that each glyph comes in
two forms: majuscule and minuscule.  The distinction between the two
forms is very small, in many cases it does not matter whether the
majuscule form or the minuscule form is used (e.g. sorting).  But that
breaks down unification of Latin/Cyrillic and Greek 'A' because the
minuscule form is different.  Still worse are examples like 'T' and
'B' where the minuscule form is different for all three.  On the other
hand, I do not think Unicode is consistent (I do not know for sure, when
I tried to buy the book it was sold out).  I think that Turkish dotless
and dot-having 'I' both share half a code point with the Latin 'I'.  My
preference would be three (times two) code points: Latin 'I', Turkish
'I' with dot and Turkish 'I' without dot.  But I (as a westerner)
understand why it is not done.  It is impossible to distinguish the
majuscule Latin 'I' from the Turkish majuscule dotless 'I'.  Which
would make it more difficult for the user.  On the other hand, as a
programmer, I see the difficulty in doing a case insensitive search.
With the two Turkish 'I's there will be more false matches unless
language is coded also, but again, that makes it more difficult for
the user.  But I think that unification of those majuscule/minuscule
glyphs that are (upto font differences) identical would make sense.
This includes Latin/Cyrillic 'A/a', 'J/j' (is the latter included
in Cyrillic?) and Latin/Cyrillic/Greek 'O/o'.

As I understand it, part of the problems with CJK unification are of
a similar nature.  While the base character is the same there may be
different simplifications.  But that is only as far as I understand it.
-- 
dik t. winter, cwi, kruislaan 413, 1098 sj  amsterdam, nederland
home: bovenover 215, 1025 jn  amsterdam, nederland; e-mail: dik@cwi.nl
From: unrza3@cd4680fs.rrze.uni-erlangen.de (Markus Kuhn)
Newsgroups: comp.std.internat
Subject: Re: No-break space & soft hyphen?
Date: 26 Jan 1993 11:29:19 +0100
Organization: Regionales Rechenzentrum Erlangen
Reply-To: mskuhn@immd4.informatik.uni-erlangen.de
NNTP-Posting-Host: cd4680fs.rrze.uni-erlangen.de

>ag129@cus.cam.ac.uk (Alasdair Grant) writes:

>ISO 8859-1 includes "no-break space" and "soft hyphen" which are intended
>to print as space and hyphen without being considered suitable targets
>for automatic line breaks and hyphenation.  Do these characters have a
>future?  Are they widely used?

All ISO 8859 sets have these two special characters at the same positions.
The purpose of SHY is to represent a hyphen, that has only been inserted
for a line break within a word and that will be removed again, if the
word moves near the middle of the line again during another reformat.

I hope that these two characters will gain more support in future text
editors  la emacs (I'm NOT speaking about real wordprocessing). Many text
editors have some limited paragraph formating capabilities, which have
proofed to be very useful (e.g. for writing USENET articles, README files,
etc.). With plain ASCII it is impossible to add a simple hyphenation 
feature (e.g.  la good old WordStar), because ASCII files will forget,
whether the "-" has been inserted by the user as a real character or only
for a line break. An editor supporting soft hyphen (SHY) will remove
unnecessary SHYs before reformating.

Especially in languages, where long words are quite common (e.g. German),
a line formating function supporting hyphenation is even VERY desireable
in text editors (not only in DTP systems!), because paragraph formatting
without hyphenation often causes ugly paragraphs.

I think, that both NBSP and SHY are a very welcome extension!

Markus

-- 
Markus Kuhn, Computer Science student -=-=- University of Erlangen, Germany
Internet: mskuhn@immd4.informatik.uni-erlangen.de  |  X.500 entry available
--- Wer, wie, was? Wieso, weshalb, warum? Wer nichts fragt bleibt dumm. ---