From: keld@login.dkuug.dk (Keld J|rn Simonsen)
Newsgroups: comp.fonts,comp.protocols.iso,comp.os.os2.programmer,comp.os.ms-windows.programmer.misc,comp.os.misc,sci.lang
Subject: Re: Character Sets (AGAIN)
Date: 30 Sep 92 21:22:45 GMT
Organization: DKnet

Erik Naggum <enag@ifi.uio.no> writes some remarks on my RFC 1345, that
I would like to comment on:

>The tables are generally unreadable, they're full of errors, and they can't
>be debugged by inspection, so you can't even find the errors without
>doing very time-consuming comparisons with the original material.

It is true that there are some errors in the tables, and this was 
expected. The tables have been checked extensively by various
sources, and the most important tables have been exposed to the
public for more than a year and tested by programs against
other machine readable specifications. Erik Naggum has pointed out
to me some errors in the supplementary Japanese 16-bit charset
and a Arabic EBCDIC, and general errors about EBCDIC control charactes,
which I appreciate and will correct in a future release. If you are
in nessecity of such corrections, please contact me.

Wrt readability, I think my tables are easier to check with regards
to alignment, eg upper-case/lowercase alignment, and they are
in a quite compact format, which makes the whole specification
of character sets more handy. The RFC 1345 is already 102 pages,
and a more elaborate format, ala Eriks would make the volume
go into a magnitude of more paper. The tables in RFC 1345
are available also as POSIX charmaps, where the full ISO 10646
name is included for every character, with the code and the 
short identifier; this can be handy for some checking.
The POSIX charmaps are available in dkuug.dk:i18n/charmaps.all

>Compare the following two definitions of ISO 8859-1 (ISO Latin 1):

>From RFC 1345:

>(Note: this is really four different character sets, including two
>control character sets, and two graphic character sets: ASCII, and the
>right half of ISO 8859-1, which is known as ISO registration number 100,
>and it's only this right half which has ISO 2022 escape code (g1esc)
>"ESC 2/13 4/1" (or x2d41).  The complete ISO 2022 escape code sequence
>is "ESC 2/0 4/3 ESC 2/1 4/0 ESC 2/8 4/2 ESC 2/2 4/3 ESC 2/13 4/1".  We
>see that this RFC does a major disservice to the community by pretending
>that an escape sequence is more than it is, and lists four character
>sets where only one is actually identified (ISO #100).  It should also
>be said that ISO #100 is intended to be used with ISO #6 (ASCII, or the
>new IRV), but it is still identified as only 96 graphic characters, not
>256 graphic and control characters.  It should also be noted that not
>all control characters in C1 (PA, HO, BH...) had been standardized at
>the time of publication of the RFC, and many are in fact withdrawn by
>now.  In general, C1 control codes are used as escape sequences, too.
>E.g., CSI (CI in the above) is uniformly used as "ESC 5/11", not as
>"9/11".  ISO 10646 requires that escape sequences shall be used.)

ISO 8859-1 is an 8-bit standard, read the standard Erik!
It defines 190 graphic characters. ISO 2375 registry 100 is a 96
character registration, and as such does not define the full
ISO 8859-1. If you read the text of the RFC you see that 
the esc specification only designates the RH part of the
specification. The issues that you are bringing up here
are also adressed in the RFC, so I do not understand your complaints.

The RFC is for use in Internet communications, and ordinary email
users do not consider their 8-bit character encoding as 4 separate
character sets, they call it LATIN1 and expect it to both contain
ASCII characters and accented characters (and rightly so, according
to 8859-1) and also to have some control characters like TAB, CR, NL
etc (and this is actually not part of 8859-1). 

>Compare this with my encoding of the ISO 2375 Register of Character Sets
>to Be Used with Escape Sequences (according to ISO 2022), of all four
>character sets.  (The descriptions are according to ISO 8879 (SGML)
>character set declarations, and weremainly intended for use with SGML,
>but have greater utility than that.  Comments are surrounded by "--".)

Erik then has an example of about 270 lines, compared to my
table of about 20 lines. It is in an ISO defined format (SGML),
but it is actually not far from my specifications in POSIX format.

Actually I welcome that we will have some firm specifiations 
of character sets, this has been needed for many years, and is
a constant source of grief (and keeps us employed!) in the
various products that I am engaged in maintaing at my job.

Keld Simonsen