From: glenn@wheat-chex.ai.mit.edu (Glenn A. Adams)
Newsgroups: comp.fonts
Subject: Unicode and Fonts
Date: 26 Aug 92 01:11:08 GMT
Article-I.D.: life.27197
Organization: MIT Artificial Intelligence Laboratory

After reading Berthold's good submission and the ensuing responses,
I feel obliged to point out some pertinent facts:

1. Unicode is explicitly and vehemently **not** a font encoding.

2. In principle, no (typographically optional) ligatures or (predictable)
   contextual variants need to be encoded for representing the basic content
   of text; they are retained in Unicode **only** for 1-1 round-trip
   compatibility with certain important character sets which served as
   the source for Unicode's collection of characters.

3. The fact that many programmers (and font designers) have confused characters
   and glyphs is a historical accident of the writing systems employed to
   write English, Chinese, Japanese, and languages with similar orthographies.
   The largely 1-1 correspondence between character and glyph in these
   writing systems fails to hold for Arabic, Indic, and other scripts, which,
   in general, employ M-N mappings between orthographic units and the glyphs
   used to display these units.

4. Textual information is meant for more than simply display.  Optimizing the
   representation of text for display alone produces suboptimal behavior for
   many other operations.  Say I encode the Latin ligature <ff> as a single
   unit and then wish to do an incremental search for words starting with
   <of> as in "office"; I now have the problem of decomposing the ostensibly
   lexical unit <ff> into its subparts for searching.

5. Unicode is meant to be a foundational encoding for representing the
   primary content of textual information; it is not the end all in this
   process.  Just like I can't do much good composition and layout with VI
   (or EMACS) as long as I use ASCII alone, I won't be able to do any better
   if these applications used Unicode.  And I don't expect to be able to
   do better; nor should you expect better.

6. To build high quality systems around Unicode, implementors (that is, you
   out there - me included), are going to have to build rich text information
   standards that both build on and go beyond the simple Plain Text standard
   provided by Unicode.  I can easily see such systems providing a way to
   allow the use to directly access or reference a glyph in a font; perhaps
   by displaying the font and allowing the user to select the glyph desired.
   This glyph reference could then be incorporated into the rich text format
   which represents a more sophisticated combination of FORM and CONTENT.

   [AFII - Association for Font Information Interchange provides a standard
   way to refer to glyphs; AFII, by the way, *does not encode character sets* -
   at most it may include glyphs which can be used to display character data.]

7. The issue of whether a character encodes FORM, CONTENT, or both is a
   complex one.  A good distinction between a glyph and a character is that
   a glyph represents *only* FORM and nothing else: it carries no lexical
   information whatsoever (its image may be given some interpretation by
   a user, but that is not its express purpose).  In contrast, a character
   is primarily a bearer of lexical information; however, it does so by means
   of symbolic reference, and, as we all know, a symbol possesses a dual
   nature.  A symbol is a form, which, in a particular linguistic or symbolic
   context, possesses a meaning, i.e., a functional interpretation.  Now,
   if we built a character set in which each character were assigned a unique
   form/function combination, i.e., a unique symbolic interpretation, then
   we would have to encode perhaps the cross product of all forms and all
   functions - a clearly impossible task.  Alternatively, we might determine
   what the collection of atomic forms were across all symbolic instances and
   represent these alone - allowing their function to be determined by context.
   We could even push this approach further by analyzing certain forms into
   constituent forms, which, in combination, represent a larger form.

   By and large, character sets have traditionally represented symbols
   irrespective of their symbolic interpretation; e.g., we don't have two
   characters for <i> in "pine" and "pin" even though have a different
   symbolic meanings in terms of their phonemic interpretation.  Since
   Unicode inherited all of these characters from existing character sets,
   it is clear that Unicode, like all other character sets, tends to
   represent formal abstractions which may map to multiple functional
   interpretations.  At the same time, some characters are developed
   functional encodings, e.g., A WITH RING ABOVE vs. ANGSTROM SYMBOL.
   Again, Unicode inherits these characters too.  On the other hand,
   with the Arabic script, we have a case where we need to be a bit more
   abstract still in regard to form, encoding only the graphemic elements
   which map to the appropriate contextual variant glyphs at display time.
   Then again, Unicode inherits these contextual variant glyphs as encodings
   from other character sets.  The only conclusion that can be reached is
   that characters should be associated with (at least) a 2-dimensional
   basis that specifies FORM and FUNCTION.  However, even when form is
   predominant in a character, it is the lexical (symbolic) function which
   that character possesses that makes it a character (and not a glyph).

   The standard way of dealing with characters that are present for
   compatibility reasons is to map them to their canonical counterpart
   at import time.  Only when the 1-1 correspondence is absolutely needed
   would they not be so normalized.

8. Dingbats and other relatively glyphic encodings in Unicode are there
   because it was important to retain data compatibility with a large
   installed base of existing data in which these encodings are present.
   These are *not* present in Unicode because Unicode is trying to pretend
   to be a font encoding.

9. Unlike some statements I've seen here, I do not believe Unicode will die
   if it does not encode ligatures.  It's intent is not to represent a final
   form document; rather, it is a minimum level of content representation
   that a revisable form document can be built from.  Really useful revisable
   form documents will add more sophistication to this minimum level that
   will support such features as explicit glyph referencing, swash variants,
   small caps variants, alternant Burmese Letter YEIHCA forms, optional
   Latin ligatures, etc.  As has been pointed out previously, Unicode (and
   any real character set encoding) has to draw a line somewhere; viz.,
   between the basic content, and that content plus some unbounded amount
   of extra "stuff," the "stuff" from which rich text is made.

Glenn Adams