From: jon@hook.corp.mot.com (Jon Whalen)
Subject: Character Sets: Summary (Part II)
Organization: MOTOROLA 
Date:  3 Oct 92 21:47:10 GMT
Keywords: character, font, glyph, code sets, ascii


The following is a summary of the replies to my posting Re: Character Sets

[This is second of a two part posting.]

--jon

*Jon S. Whalen                                        Phone: (708) 576-0166*
*Staff Engineer, Motorola, Inc.                         Fax: (708) 576-0892*
*Corporate, Computer & Communications R&D                                  *
*Internet:  jon@hook.corp.mot.com / Compuserve: 76665,3043 / AOL:JonSWhalen*

----------------------------------------------------------------------------

III. Encodings

   I asked:

>III. I'm looking for pointers to documentation on standard and ad-hoc mappings
>between 7-bit ascii and 8-bit charsets. For example, on the mailing list
>GAELIC-L, the accented vowels: a_acute, e_acute, i_acute, o_acute, u_acute,
>a_grave, e_grave, i_grave, o_grave, u_grave; are represented as vowel + /
>and vowel + \, respectively. What other encodings are in use? Where might I
>get documentation (formal or informal) on them? Are there ISO standards
>in existence or under development?

I received the follwoing replies, 

A. Hungarian

Andra1s Kornai writes:

>on the Hungarian mailing lists you often find x1 for TeX \'{x}, x2 for
>\"{x}, and x3 for \H{x}, where x is any vowel a u i o e (the
>combinations a2 a3 i2 i3 e2 e3 don't actually occur in Hungarian).
>This code, originated by Ga1bor Pro1sze1ky at around 1980, is very
>popular among people working with Hungarian corpora, less popular (but
>still frequently seen) in political, cultural, etc. Hungarian e-mail
>forums (fora?) of which there are at least five different ones. A
>Hungarian Electronic Resources FAQ is often posted on
>soc.culture.magyar, you'll find more info there if you are interested.

and Vera Horvath write:

>In response to your query on the Corpora list I would like to share with you
>what I know about character-mappings in Hungarian.
>At the Linguistic Institute of the Academy (in Budapest), the compilers of
>a Hungarian
>electronic corpus (which is still under preparation) decided on the use of
>numbers after vowels to mark accent. Consequently, accented ("long") a is a1,
>long "i" is i1, long e is e1. The number for the umlaut is 2 : o2, u2
>the number for the long umlaut is 3: o3, u3.
>
>On the Hungarian mailing lists there is no consesus at all. Some people use the
>
>numbers, others omit accents altogether, still others use single quote (') for
>one accent, tilde (~) for umlaut and double quote (") for two accents.
>(Graphically, these are the closest approximations of the "real" diacritics.)

B. Vietnamese

Cuong T. Nguyen writes:

>For Vietnamese, please refer to the VIQR (VIetnamese Quoted-Readable)
>standard from Viet-Std, which models the prevalent convention used
>on electronic media worldwide.  A PostScript version of the standards
>is available for anonymous FTP from Sonygate.Sony.COM (192.65.137.2),
>under /tin/viet-std/viet-std.eng.ps.Z.


C. The Text Encoding Initiative

Lou Burnard responded with a copy of the introduction to:

                       Guidelines for Electronic

                     Text Encoding and Interchange



                               Edited by


                 C. M. Sperberg-McQueen and Lou Burnard

                           TEI P2, Chapter 21
                        Text Encoding Initiative
                            Chicago, Oxford


                     (c) 1990, 1992 ACH, ACL, ALLC

                             July 15, 1992

                     Draft Version 2, July 15, 1992
                     
In the body of that document it states:

>   Updates on the status of the draft as a whole will be distributed
>automatically to all subscribers of the Listserv list TEI-L.  To sub-
>scribe to TEI-L, send electronic mail to the address LISTSERV@UICVM (or
>Listserv@uicvm.uic.edu) containing the single line
>
>        subscribe tei-l J. Smith
>
>(substituting your name for "J. Smith").  TEI-L is also the appropriate
>place to pose questions or offer public comments on the TEI guidelines
>and other relevant issues.  The TEI-L file server contains all the sec-
>tions of TEI P2 thus far released, as well as other TEI materials.  Its
>contents are shadowed at various other sites around the world; for more
>information consult one of the following documents:
>
>*   TEI ED J8, "Obtaining the Second Version of the TEI Guidelines (TEI
>    P2)" (describes how to retrieve electronic copies of TEI P2 and the
>    various formats they are available in)
>*   TEI ED J9, "Obtaining Paper Copies of the Second Version of the TEI
>    Guidelines (TEI P2)" (describes how to request paper copies of TEI
>    P2, for those without electronic mail access)
>
>These documents are available from the TEI-L file server or through the
>editors at the addresses given on the User Response and Comment Form.

In the introductory note for the TEI-L list it says:

>  Contributions sent to this list are  automatically archived. You can obtain a
>list of  the available  archive files  by sending an  "INDEX TEI-L"  command to
>LISTSERV@UICVM. These  files can  then be  retrieved by means  of a  "GET TEI-L
>filetype" command, or using the database search facilities of LISTSERV. Send an
>"INFO DATABASE" command for more information on the latter.

D. Hebrew, Aramaic, Greek, Arabic, Coptic

Bob Kraft replies:

>The following material is easily forwardable so I append
>it for your interest. It probably is not what you had in
>mind! Bob Kraft, UPenn/CCAT
>=====

[Editor's note: due to the length of the selection, I haven't inserted '>'
 at the beginning of the following, rather I have surrounded it with a 
 separator.]

-------------------------------- BEGIN INCLUDED ----------------------------

Appendix 1: CODING FOR TRANSLITERATION OF HEBREW

The Hebrew, Syriac, and Aramaic texts are coded according to the Michigan-
Claremont scheme:

      Hebrew          Coding                  Hebrew          Coding

       alef             )                      patah            A
       bet              B                      qametz           F
       gimel            G                      hireq            I
       dalet            D                      segol            E
       he               H                      tsere            "
       waw              W                      holam            O
       zayin            Z                      qibbuts          U
       het              X                      shureq           W.
       tet              +                      schwa            :
       yod              Y                      holem waw        OW
       kaf              K                      hateph-pathah    :A
       lamed            L                      hateph-qametz    :F
       mem              M                      hateph-segol     :E
       nun              N                      maqqeph          -
       samek            S                      dagesh           .
       ayin             (                      rape             ,
       pe               P                      ketiv            *
       zade             C                      qere             **
       qof              Q
       resh             R
       sin/shin         #
       sin              &
       shin             $
       taw              T


HEBREW ACCENTS/CANTILLATION CODING

The accent and cantillation markings are named and cross referenced as in the
TABULA ACCENTUM insert card in BHS.  The Michigan-Claremont coding is in the
first column.  The alternate coding used by the French and Belgian projects is
also noted.
                                                      French   Belgian
    Michigan-Claremont                      BHS       CATAB     CIB

at end (to left) of word, above
    00  ;  ---  sop pasuq [end of verse]                 -        -
    _  ,         rape [see above]                                80
    _  -  //---  abbreviation                            -       83
    _  -   /---  abbreviation                            -       84
    01  .:---  segolta                     I.3          14       7
    02   )---  zarqa, sinnor             I.9,II.7       20        -
    03   ---  pashta, azla legarmeh    I.10,II.12      21        -
    04   &---  telisha parvum              I.25         47       30
    05   |---  paseq [separator]          "Nota"        11        -
     -  |-,--  legarmeh (74 + 05)          I.18       40 + 11     -

at start (to right) of word, below
    10    ---<  yetib (yetiv)              I.11         23      (42)
    13    ---  dehi or tipha             II.9         (49)       -

at start (to right) of word, above
    11    ---/  (81 + ) mugrash           II.5           -        -
    14    ---%  telisha magnum             I.17         27       31

above word
    24   -&--  telisha qetannah (med)       -            -        -
    44   -%--  telisha magnum (med)         -            -        -
    60   --<-  ole or mahpakatum         (~I.2)          -       43
    61   -/--  geresh or teres             I.13         25       81
    62   -"--  garshajim                   I.14         26       82
    63   ---  azla, azla or qadma      I.24,II.19     45=46      8
    64   -,--  illuj                      II.15          -       44
    65   -#--  shalshelet (magn,parv)    I.4,II.6+20    15       33
    80   -:--  zaqep parvum                 I.5         16        6
    81   -.--  rebia (magnum=parvum)      I.7,II.4=8    19        5 (cf 80)
    82   --)-  sinnorit                    II.21       (20)       9
    83   -+--  pazer                    I.15,II.10      28       90
    84  -&%--  pazer mag. or qarne para    I.16         29       32
    85  -|:--  zaqep magnum                I.6          17       60

below word
    35  -F|:--  meteg (med)                  -         (12)      (,)
    70   -<--  mahpak or mehuppak       I.20,II.11+18   43       42
    71   -/--  mereka                    I.21,II.14     41        1
    72  -//--  mereka kepulah (duplex)      I.22        42       11
    73   ---  tipha, tarha               I.8,II.16     18        2 (munah)
     -  ----  majela [= 73]                I.27        49        -
    74   -,--  munah                  I.18-19,II.13     40        4 (dehi/tarha)
    75   -|--  silluq [meteg (left)]      I.1,II.1      12        ,
    91  -./--  tebir                        I.12        22       10
    92   -^--  atnah                      I.2,II.3      13        3
    93   -v--  galgal or jerah           I.26,II.17     48       41
    94   -s--  darga                        I.23        44       40
    95   -|--  meteg (right) [cf 35,75]      -         (12)       -

rak 4/02/86

Appendix 2: CODING FOR TRANSLITERATION OF GREEK AND COPTIC

              Letter         Greek (TLG)          Coptic (RAK)

              alfa             A                   A
              beta             B                   B
              gamma            G                   G
              delta            D                   D
              epsilon          E                   E
              digamma/vau (=6) V                   V
              zeta             Z                   Z
              eta              H                   H
              theta            Q                   Q
              iota             I                   I
              kappa            K                   K
              lamda            L                   L
              mu               M                   M
              nu               N                   N
              ksi              C                   C
              omicron          O                   O
              pi               P                   P
              koppa (=90)      #3
              rho              R                   R
              sigma            S                   S
             [sigma final      j ]
              tau              T                   T
              upsilon          U                   U
              phi              F                   F
              chi              X                   X
              psi              Y                   Y
              omega            W                   W
              sampi(=900)      #5

              smooth breathing )         shai      s
              rough breathing  (         fai       f
              iota subscript   |         chai(Bo)

              acute accent     /         hori      h
              grave accent              janjia    j
              circumflex acc.  =         gima      g
                                         ti        t
                                         chi-rho   x
              diaeresis        +         overline   (backslash)
              midpoint punct.  :         dash      - (hyphen)

              capital letter   * (precedes)

RAK 5/23/88

Appendix 3: CODING FOR THE TRANSLITERATION OF ARMENIAN (Leiden, AIAS)

      Armenian        Code            Code
                      Uncial          Minuscule
        ayb             A               a
        ben             B               b
        gim             G               g
        da              D               d
        ec              E               e
        za              Z               z
        e               E/              e/
        et'             E^ or E:        e^ or e:
        t'o             T^              t^
        ze              Z^              z^
        ini             I               i
        liwn            L               l
        xe              X               x
        ca              C               c
        ken             K               k
        ho              H               h
        ja              J               j
        lat             L^              l^
        ce              C/              c/
        men             M               m
        yi              Y               y
        nu              N               n
        sa              S^              s^
        o               O               o
        c'a             C^              c^
        pe              P               p
        je              J^              j^
        ra              R^              r^
        se              S               s
        vew             V               v
        tiwn            T               t
        re              R               r
        c'o             C|              c|
        hiwn            W or U          w or u
        p'iwr           P^              p^
        k'e             Q               q

        O               O/              o/
        fe              F               f

        full stop                       :
        semicolon                       ;
        half-comma                      
        question                        ?
        acute accent                    ]

Other codes in the text correspond to various editorial markings which are
difficult to translate into ASCII.  A little experimentation will render them
useful.  A few characters need no transliteration.

The following characters and editorial markings do not appear to be in the
documents in the collection, either by coincidence or because they were
filtered out during transmission and/or reformatting, but are included
in the AIAS coding scheme:

        dash -                          -
        tilde ~                         =
        crux                            %
        left curly bracket {            <
        right curly bracket }           >
        upper left half bracket Z       [
        upper right half bracket ?      $
        lower left half bracket @       +
        lower right half bracket        *

 [rvh 2/88]
Appendix 4: CHART OF CODINGS FOR TRANSLITERATION OF HEBREW-ARABIC

There is as yet no substantial agreement as to what codes to use for the
transliteration of Hebrew and Arabic.  This collection uses the Michigan-
Claremont scheme for Hebrew texts, and the Oxford schemes for Arabic.
The following chart records various codings, along with some suggestions
towards standardization of Semitic language coding:

              current     proposed      ANSI-Heb        Arabic
Hebrew        Mich-Clar   by Kraft      1975            Oxford     Arabic

 alef            )           A           @               A         elif
 bet             B           B           B               B         ba
 gimel           G           G           G               J, G      g'im, g'ain
 dalet           D           D           D               D, d      dal, d_al
 he              H           H           H               O, X      ha, h_a
 waw             W           W           W               W         waw
 zayin           Z           Z           Z               Z, &      zay, z.a
 het             X           X           X               H         h.a
 tet             +           t           J               V         t.a
 yod             Y           Y           Y              ?
 kaf             K           K, k        K, K/           K         kaf
 lamed           L           L           L               L         lam
 mem             M           M, m        M, M/           M         mim
 nun             N           N, n        N, N/           N         nun
 samek           S           S           S              ?
 ayin            (           J           &               E         (ain
 pe              P           P, p        P, P/           F         fa
 zade            C           C, c        C, C/           C, c      s.ad, d.ad
 qof             Q           Q           Q               Q         k.af
 resh            R           R           R               R         ra
 sin/shin        #           s           F
 sin             &           f           $               S, s      sin, s'in
 shin            $           F          ?               ?
 taw             T           T           T               T, t      ta, t_a
                                                         ?         ta marbut.a
 ---------------------------------- END INCLUDED ----------------------------
                                                        
 patah           A           h           A/              P         hamsa
 qametz          F           a           A
 hireq           I           i           I
 segol           E           e           E
 tsere           "           y           E/
 holam           O           o           O
 qibbuts         U           u           U
 shureq          W.          W*          W*
 holem waw       OW          oW          OW
 schwa           :           %           %
 hateph-pathah   :A          %h          %O
 hateph-qametz   :F          %a          %A
 hateph-segol    :E          %e          %E
 maqqeph         -           -
 dagesh          .           *                           :         shadda
 rape            ,           ^
 ketiv           *           |
 qere           **          ||

[rak-rvh 2/88]
Appendix 5: CODINGS FOR SANSKRIT

Each Sanskrit document in this collection is coded in a slightly
different way.  In order to reduce the possibilities of confusion, we
shall first list the fifty symbols needed, then chart them by
document. The assistance of J. Hubbard and D. Wujastyk in this
endeavor is gratefully acknowledged:

Description of phoneme / standard scholarly representation

 1  Short a / a
 2  Long a / a with macron above
 3  Short i / i
 4  Long i / i with macron above
 5  Short u / u
 6  Long u / u with macron above
 7  Short vocalic r / r with dot beneath
 8  Long vocalic r / r with dot beneath and macron above
 9  Vocalic l / l with dot beneath
10  e / e
11  Diphthong ai / ai
12  o / o
13  Diphthong au / au
14  Unaspirated unvoiced velar / k
15  Aspirated unvoiced velar / kh
16  Unaspirated voiced velar / g
17  Aspirated voiced velar / gh
18  Velar nasal / n with a dot above
19  Unaspirated unvoiced palatal / c
20  Aspirated unvoiced palatal / ch
21  Unaspirated voiced palatal / j
22  Aspirated voiced palatal / jh
23  Palatal nasal / n with tilde above
24  Unaspirated unvoiced retroflex / t with dot beneath
25  Aspirated unvoiced retroflex / t with dot beneath followed by h
26  Unaspirated voiced retroflex / d with dot beneath
27  Aspirated voiced retroflex / d with dot beneath followed by h
28  Retroflex nasal / n with dot beneath
29  Unaspirated unvoiced dental / t
30  Aspirated unvoiced dental / th
31  Unaspirated voiced dental / d
32  Aspirated voiced dental / dh
33  Dental nasal / n
34  Unaspirated unvoiced labial / p
35  Aspirated unvoiced labial / ph
36  Unaspirated voiced labial / b
37  Aspirated voiced labial / bh
38  Labial nasal / m
39  Palatal semivowel / y
40  Retroflex semivowel / r
41  Dental semivowel / l
42  Labial semivowel / v
43  Palatal sibilant / s with acute accent above
44  Retroflex sibilant / s with dot beneath
45  Dental sibilant / s
46  Sonant aspirate h / h
47  Pure nasal / m with dot beneath
48  Hard breathing / h with dot beneath
49  Elided short vowel / apostrophe
50  Punctuation mark / diagonal slash


LETTER CODES BY DOCUMENT
                Heart                           Bhagavad-       Kali-
  Phoneme       Sutra           Rigveda         gita            dasa
  by number
     1          a               A               a               a
     2          A               A:              aa              aa
     3          i               I               i               i
     4          I               I:              ii              ii
     5          u               U               u               u
     6          U               U:              uu              uu
     7          r               /R              .r              *r
     8          RI              R:              r*              &r
     9          L               /L              .l              /l
    10          e               E               e               e
    11          ai              AI              ai              ai
    12          o               O               o               o
    13          au              AU              au              au
    14          k               K               k               k
    15          kh              KH              kh              kh
    16          g               G               g               g
    17          gh              GH              gh              gh
    18          ng              /G             [?]              n*
    19          c               C               c               c
    20          ch              CH              ch              ch
    21          j               J               j               j
    22          jh              /H              jh              /h
    23          nj              /J              n#              n#
    24          T               /T              .t              /t
    25          TH              /TH             .th            [?]
    26          D               /D              .d              .d
    27          DH              /DH             .dh  ?          /dh
    28          N               /N              .n              .n
    29          t               T               t               t
    30          th              TH              th              th
    31          d               D               d               d
    32          dh              DH              dh              dh
    33          n               N               n               n
    34          p               P               p               p
    35          ph              PH              ph              ph
    36          b               B               b               b
    37          bh              BH              bh              bh
    38          m               M               m               m
    39          y               Y               y               y
    40          R               R               r               r
    41          l               L               l               l
    42          v               V               v               v
    43          S              [?]              z               z
    44          SH              /S              .s              .s
    45          s               S               s               s
    46          h               H               h               h
    47          M               /M              .m              .m
    48          :               /H              .h              .h
    49          '              [?]              '               '
    50          /               /

[rvh 2/88]

----------------------------------------------------------------------------

The End :-)
--
*Jon S. Whalen                                        Phone: (708) 576-0166*
*Staff Engineer, Motorola, Inc.                         Fax: (708) 576-0892*
*Corporate, Computer & Communications R&D                                  *
*Internet:  jon@hook.corp.mot.com / Compuserve: 76665,3043 / AOL:JonSWhalen*
Newsgroups: sci.lang