From: rebecca@cco.caltech.edu (Mark L. Fussell)
Newsgroups: comp.text.sgml
Subject: Parting PSGML
Date: 15 Apr 92 12:14:57 GMT
Organization: California Institute of Technology, Pasadena


Howdy all,

This will be my last comment to this news group on this issue, since I am  
losing an argument by nobody seriously considering what I have to say or the  
examples I give -- at least for the posted responses.  Eric's and James's  
comments were welcome and helpful (although the overall tone wasn't so good).   
I have read, tested, and pondered over everything posted or mailed to me,  
contrary to others' implications or statements.


I hoped for a bit better reception since I have nothing against SGML, and PSGML  
isn't in anyway designed to replace or insult it.  It is meant to be very  
similar to the `C' preprocessor: handling needs people may have that are only  
mildly related to the target language.  If you don't have the needs, it doesn't  
hurt anything.  I wanted comments from "SGML people" to see how much they  
really think I violated or interfered with SGML and to help keep PSGML in line,  
but the responses didn't go in that direction.
    Certainly my approach did not seem to help any: the examples were meant to  
make people think about the problems or ask more about PSGML.  I also chose  
that approach because the specification wasn't complete and polished at the  
time, and I thought it would waste your time to read an incomplete  
specification, unless requested.  But it did not work.  My parting (long)  
remarks will try to remedy this a little.
    
    
To summarize why I even started the PSGML project:

   1)   I have numerous documents I want to convert to SGML while still
        being extremely readable in the source form. I also want 
        document conversion to be mostly automatic, so the documents 
        should be fairly "sticky" both aesthetically and structurally.
        
   2)   I want to use popular public DTDs (QWERTZ, TEI, etc.) without
        modifying them significantly, so the DTDs should be very
        sticky.

   3)   When modifying the shortrefs for SGML to handle these documents, 
        it was impossible to handle many of the structures as nicely 
        as I wanted.  Also, modifying the shortrefs for a DTD requires
        rewritting that section of the DTD, not just adding to it.
        
   4)   Since I was concerned with a problem for hundreds of documents,
        I attempted to generalize simple solutions for specific problems 
        I was encountering.
   
    5)  I did not want to lose any of the features of SGML.


And here is a long list of the problems encountered (pretty much in the order  
discovered):

    1)  Spaces between a tag and the next non-space character on the
        same line are significant (no matter what).  This prevents 
        visual space separating tags from "important" directly readable
        information.  It also prevents aligning multiple rows of data
        that are more readable aligned (when tags are different 
        lengths).
    
    2)  Spaces at the beginnings of lines are significant (unless a 
        shortref removes them for that particular structure).

            2a) The shortref is structure dependent, and not document
                flexible, so not formatting flexible.  If you actually
                try to deal with this problem, you will see this to be
                significant.
            
            2b) If the shortref form is used, putting a space at the
                beginning of a line requires inserting a null entity 
                or tag ("&null", "<!>") which can be too intrusive.
                
    3)  Spaces at the ends of line are significant... (see above). 
    
    4)  Removing blank lines dependent on the following tag is 
        impossible -- although most of these occurrences can be partially
        resolved by smarter DTDs (an example do to Erik Naggum), this
        is not resolved in many of the cases I have to deal with (counter
        example to Erik's: what if paragraphs are left flush?).
    
    5)  Alternate tag names and multiple tags can not be used.

            5a) A possibly document specific GI (like <fable>) can
                not be converted to another GI (like <section>), nor
                can it be converted to multiple tags (<section><heading>).
                Nasty DTD GIs can not be renamed.
            
            5a) An omitted tag can not have empty content without
                SGML complaining.
    
    6)  Entities can not be separated from text by spaces (without
        them being significant). 
    
            6a) Entities behave differently from TeX macros, causing 
                unintended spaces in the document for converted
                TeX users.
    
    7)  Entities can not have alternate names, and can not have most 
        special characters in them (without a major change to the 
        DTD):
            &C++;   &:-);   
        are not allowed.
    
    8)  The closed entity form:         stuff stuff &stuff; stuff.
        can be less readable than:      stuff stuff \stuff/ stuff.
    
    
    9)  Using entities for accents and fractions is visually unpleasing.
    
    10) Shortrefs can be in effect for text symbols (&,%,@,#,etc.), and
        converting these to entity form is visually unpleasing.
    
    
    11) Newlines in content may not really be desired (this is only 
        conceptually important to me since I always format in TeX or 
        Nroff which usually don't care about \n-s).

    12) Printed SGML code (usually for examples) has to have some symbols 
        converted to be parsed.
    
    13) Shortrefs consider spaces before or after significant unless you
        add 3 more maps for all the permutations.
            "*"     <item>
            "B*B"   <item>
            "*B"    <item>
            "B*"    <item>
            
    14) Tag/entity macros can not be designed in SGML.  This isn't to say 
        we should have really complicated macros, but the statement:
            <=identitymatrix 4 4>
        is both easier to type and read than expanding it out (at least
        if it is not typeset).

    15) Some documents have hyphens at the ends of lines which should be
        removed (and the hyphenated syllables concatenated).
    

The generalized solutions I worked out:
    1)      Alternate form of tag open and close that indicates spaces before
            or after a tag are insignificant.  This looks almost identical
            to SGML, and so is not very distracting.
        
    2,3)    Spaces at the end and beginning of lines can be removed by
            PSGML.  This is the default mode since these spaces are rarely
            intended.  It can be turned off if desired, or spaces can be
            protected in several simple ways.
            
    4)      PSGML can be told to ignores blank lines before and after 
            certain tags.
            
    5)      PSGML can remap tags to a new GI, a GI plus attributes, and
            multiple tags.  The SGML document will then look "standard",
            even if you like (possibly for very good reasons) a different 
            name set.  Multiple SGML DTDs could also be mapped to with
            a single source document.  
            
            The omitted tag problem is solved by included multiple tags
            in the mapping:
                PSGML:  <ing>       <what>water 
                SGML:   <ingredient><amount><what>water
             
    6)      All PSGML "tokens" have a full form with a starting '<' and 
            ending '>', so you can use (1) to remove spaces.  Also, the
            alternate entity form removes spaces after an unclosed 
            entity.  Also, PSGML has space removal forms.
            
    7,8)    Entities can be mapped (&aa; --> &aring;), and two alternate 
            entity forms are introduced: \C++/, and <':-)>.  NOTE: all
            examples of mappings are just examples, so can be adjusted
            for each DTD, but the ISO standard for entities would be the
            common map.
    
    9)      PSGML has a form for accents similar to TeX allowing: 
                ^'o     -->     &oacute;
                ^^o     -->     &ocirc;
                ^co     -->     &ocedil;
               <^" o>   -->     &ouml;       (example of full form)
            Fractions can be enterred by:
                <#1/2>  -->     &onehalf;
        
    10)     PSGML has a simple form (common to many computer languages) 
            for the printable entity of a special character:
                \@      -->     &commat;
                \^      -->     &circum;
        
    11)     Newlines can be removed by PSGML with a trivial notation.   
                
    12)     PSGML can convert a block of code: 
                * to be converted to printable entities for SGML,
                * to be ignored by PSGML (so sent straight through to SGML),
    
    13)     Shortrefs are extended allowing spaces before and after to 
            be removed, blank lines before and after to be removed, and
            more characters than are normally allowed in shortrefs to be
            used.
        
    14)     PSGML has macros, but they are implementation dependent at the
            moment (and probably forever since PSGML isn't supposed to get
            complicated, but it may allow alternate macros for the possible 
            implementations).   
    
    15)     PSGML has a mode to convert these hyphens.
    

So quite a number of problems and a short statement of the solutions chosen for  
PSGML.  All of the problems are real: even when SGML can partially handle them   
it does so in an unpleasant manner or with much greater difficulty than simply  
letting the pre-processor handle it.  That is the point of PSGML: it solves  
some real problems that I have encountered.  And after getting constructive  
feedback, I desired to make PSGML available to those who might have similar  
problems.  That is all.
    
                                    
                                      -- Mark

Mark L. Fussell
rebecca@cco.caltech.edu     


PS: For those that actually expressed an interest in PSGML: I will continue to  
finish it (I am removing the Perl specifics at the moment) as time becomes  
available, but the interest was limited.