From: bkph@rice-chex.ai.mit.edu (Berthold K.P. Horn)
Newsgroups: comp.fonts
Subject: Re: Unicode and Fonts
Date: 26 Aug 92 15:41:11 GMT
In-reply-to: glenn@wheat-chex.ai.mit.edu's message of 26 Aug 92 01:11:08 GMT


In article <27197@life.ai.mit.edu> glenn@wheat-chex.ai.mit.edu (Glenn A. Adams) writes:

   After reading Berthold's good submission and the ensuing responses,
   I feel obliged to point out some pertinent facts:

   1. Unicode is explicitly and vehemently **not** a font encoding.

   2. In principle, no (typographically optional) ligatures or (predictable)
      contextual variants need to be encoded for representing the basic content
      of text; they are retained in Unicode **only** for 1-1 round-trip
      compatibility with certain important character sets which served as
      the source for Unicode's collection of characters.

	... and much other good stuff deleted ...

      will support such features as explicit glyph referencing, swash variants,
      small caps variants, alternant Burmese Letter YEIHCA forms, optional
      Latin ligatures, etc.  As has been pointed out previously, Unicode (and
      any real character set encoding) has to draw a line somewhere; viz.,
      between the basic content, and that content plus some unbounded amount
      of extra "stuff," the "stuff" from which rich text is made.

   Glenn Adams

Much has been said in regard to UNICODE and ligatures.  
Yes, (I think)  I understand the philosophy of characters versus glyphs.  
NO, I do not want text search for the word 'file' to fail because the
ligature 'fi' appears in the file.  

The problem is the discrepancy between the nice theory and the practial
world of real operating systems and graphical user interfaces.  
The problem I have is with imposing a UNICODE solution without making sure
that operating systems and graphical user interfaces can cope with it.

The crux of the matter is exactly that UNICODE encodes characters,
not glyphs.  It is the 'right thing' for textual input and interchange.
The problem is that at the moment there is no equally accepted
encoding for glyphs.  Worse yet, while there are plans for supporting
UNICODE, there are no plans for supporting a uniform glyph encoding
scheme.  In particular, if systems only have ONE way of representing
both characters and glyphs (which is pretty much true across the
board today) then they should not use UNICODE, since UNICODE does not
represent glyphs.

My gripe is exactly that in most operating systems this distinction
between character encoding and glyph encoding is not made.  Hence the
nice philosophy behind UNICODE does the implementor no good.  There
are just streams of bytes or words, both were the application reads
the text from the file and where it communicates to the display
driver or the printer driver what it wants rendered. 

The religious converts would have us believe that somehow magically
the operating system or the graphical user interface takes care of
this.  First of all, it doesn't.  There is nothing in Windows as it
stands now, or as it is currently envisioned that deals with
ligatures.  You can't have them!  This is true even pre-UNICODE,
since the Windows ANSI character set does not contain ligatures.
It is certainly also true for TrueType fonts.

And for Type 1 fonts, you have to give up on using Windows ANSI.
This requires special tools to create new PFM (metric files) and to
reencode the PFB (outline font) file itself.  It can be done (and is
done all the time for previewing TeX DVI files using Type 1 fonts),
but you then face the horrible problem that ATM for Windows has
hard-wired positions for accents.  

If an operating system does not have a different representation for
characters than for glyphs, then imposing UNICODE as it stands is
disastrous.  The application MUST be able to deal with ligatures and
other glyphs directly.  How else can it compute proper line breaking
in a paragraph for example?  The problem then is that the application
must be able to ask the graphical user interface for metric
information about arbitrary glyphs and must be able to ask it to
render arbitrary glyphs in the font --- NOT just those that happen to
correspond to characters in UNICODE.  

This should not require some special mechanism for ligatures.  Glyphs
for ligatures are not different from glyphs that happen to correspond
directly to characters in UNICODE.

(*) If UNICODE cannot be used for specfying ligatures, then we must
have another widely accepted standard coding system for specifying glyphs.

(*) If UNICODE does not contain codes for specifying ligatures, then
operating systems should not use UNICODE in the graphical user
interface.  The system is asked to render glyphs not characters. 

The problem with MS Windows, is that it will use UNICODE throughout.
When I pass a string to ExtTextOut, it will be taken to contain codes
that are interpreted in the UNICODE space.  So I cannot show
ligatures and other glyphs that do not happen to directly correspond
to characters in UNICODE.

The problem on the Macintosh is different and somewhat more nebulous
(at least to me).  There appears to be this mysterious process called
'line layout' which gets in between the application and what gets
rendered.  Maybe somebody could explain how this is supposed to work?

In any case, its not going to fly if it requires use of different
system calls.  The application has to be able to deal with kerning
and ligatures directly.  And in a way that is backward compatible.

A simple compromise would be to add the commonly used ligatures to
UNICODE. This would permit UNICODE to be used for specifying glyphs
in the interim, while new operating systems evolve that have a
politically correct view of characters and glyphs.  But I know this
is heresy...

Berthold K.P. Horn
Cambridge, Massachusetts, USA