ISO/IEC JTC1/SC18/WG8 **N1729**

# ISO/IEC JTC1/SC18/WG8 Document Processing and Related Communication--- Document Description and Processing Languages {#isoiec-jtc1sc18wg8-document-processing-and-related-communication-document-description-and-processing-languages .Heading-0-(nobreak)}

  -------------------- --------------------------------------------------
  Title:               Towards a Simple Model of an SGML Document

  Source:              Sam Wilmott

  Project:             1.18.15.1 (SGML)

  Project editor:      C. F. Goldfarb

  Status of document:  Individual contribution

  Requested action:    For information

  Date                 25 August 1994

  Distribution:        WG8 and liaisons

  References:          

  Supersedes:          
  -------------------- --------------------------------------------------

# Towards a Simple Model of an SGML Document {#towards-a-simple-model-of-an-sgml-document .Heading-0}

Sam Wilmott

This note presents some of the issues that need discussing in building a
model of the Òinformation contentÓ of an SGML Document, and outlines a
starting point for those discussions. In particular, it suggests what
should and what should not be part of the model used in the SGML
standard.

## Information in an SGML Document

A parsed SGML Document can be viewed in a number of different ways:

¥ Parsed token information: The delimiters, data characters and
name-like tokens of a document form an a simply ordered sequence as seen
by an SGML parser.

¥ Markup information: Markup information is a linear sequence of the
different kinds of markup and data characters in a document instance as
recognized by an SGML parser after the replacement of text entities, the
recognition of delimiters, data characters and other lexical structures
(e.g. names), after the conversion and ignoring of white space and after
the recognition of declarations and tags, but prior to the
interpretation of that markup and data as element structure, entity
structure and marked section structure. Markup information includes
comments, marked section declarations, USEMAP and USELINK declarations
and short reference delimiters.

Markup information can be a structure, like those described below, but
it differs from them in that it is linear, not hierarchical: in the
simplest view of an SGML document, markup and data, together with the
boundaries of entity replacement text, form a single sequence.

From the markup information can be derived (at least) five different
kinds of structure. They are:

¥ Element structure: Element structure is a hierarchical tree structure
with elements as its non-terminal nodes and with attribute values,
declared and contingently empty elements, data characters, data entity
references and processing instructions as its terminal nodes.

¥ Entity structure: Entity structure is a hierarchical tree structure
with text entity references of all sorts as its non-terminal nodes and
with parsed and unparsed character data, and data entity references as
its terminal nodes.

¥ Marked section structure: Marked section structure is a hierarchical
tree structure with marked section declarations as its non-terminal
nodes and all data and other markup as its terminal nodes.

¥ Short reference mapping structure: There is a subset of markup
information, represented by USEMAP declarations in both the DTD and
instance, that is not part of the element structure, that can
nonetheless be viewed as a companion to the element structure. The way
in which short reference maps and link sets are ÒpushedÓ and ÒpoppedÓ by
element boundaries makes the subparts of content influenced by them
properly nested with respect to element structure.

Note that short reference mapping structure is an artifact of SGML
parsing and may very well be of no interest to any application of SGML.

¥ Link set mapping structure: There is a subset of markup information,
represented by USELINK declarations, that is analogous to short
reference mapping structure.

A number of observations can be made about the different types of
structure:

¥ There is a mapping between element structure and markup information in
the sense that for each node in the element structure tree, markup or
data characters that correspond to that node can be identified in the
markup information. This mapping includes identification of the
locations of omitted tags. Note that this mapping is Òone-wayÓ in that
although it maps all components of the element structure into markup
information, it does not map all components of markup information into
element structure. Unmapped components include marked section and
comment declarations in the instance, white space matched by the ÒsÓ
production in ISO 8879, and, most importantly, the non-terminal nodes of
the entity structure.

¥ The components which are unique to markup information (e.g.
declarations and ÒsÓ white-space) cannot in general, based on ISO 8879,
be assigned unique locations within element structure. The element to
which a comment applies, for example, may be the one containing it, or
it may, especially if intervening end tags are omitted, be the following
element. The fact that, in a linear parse, these markup components are
ÒdiscoveredÓ within a particular node of the element structure is an
artifact of the parsing process, so care must be taken in attaching too
much significance to the exact point in the parsing process at which
they are discovered.

¥ As with element structure, there is a mapping between entity structure
and markup information. Markup and data can be mapped to the entities of
which they are the replacement text and vice-versa. This mapping, unlike
the element structure mapping, is fully Òbi-directionalÓ Ñ every
component of markup information can be identified as resulting from
parsing the replacement text of a particular entity.

¥ There is also a mapping between markup information and marked section
structure. This mapping is Òbi-directionalÓ, in the same sense as is
entity structure: all components of the markup information have a
well-defined location in the marked section structure.

¥ Element structure, entity structure and marked section structure
overlap: Apart from entities referenced within attribute values and
markup declarations, text entities can start and end anywhere in element
or mixed content. Even short references declared as STARTTAG, for
example cannot, in general, be mapped to individual tags Ñ whether such
an entity is recognized as a start tag depends on its replacement text
and from where it is referenced. Similarly, INCLUDE marked sections can
overlap with the starts and ends of elements.

## Addressing Parsed Document Structures

Any node of a tree structure can be addressed by the simple expedient of
first selecting a node, say the ÒrootÓ node of the tree or a node within
the tree identified by a ÒlabelÓ, and then counting across the subnodes
of the selected node to a specific subnode and recursively counting
across its subnodes until the addressed node is reached. This process
allows any node of a tree structure to be identified. In SGML terms the
initially selected node of an element structure can be the document
element of the current document instance, the document element of
another document that is identified by an external identifier, or an
element identified by a specified ID attribute value (its label). HyTime
provides a set of SGML-based standard notations for representing such
addressing.

There is no unique addressing scheme. For example, HyTime clumps
together contiguous sequences of data characters into tree nodes called
ÒpseudoelementsÓ. Character addressing is done by first addressing the
containing pseudoelement and then the character within it. Each data
character could have as easily been considered a direct subnode of its
containing element and addressed as such.

Addressing is not addressed by ISO 8879. Many applications of SGML need
to support some addressing scheme as part of processing the document.
HyTime was developed in order to help standardize addressing schemes
between applications, and in particular, to define a standard notation
for representing such addresses within SGML documents. As a consequence,
the contributors to the HyTime standard agreed on a set of addressing
schemes that were deemed most appropriate for the widest range of
applications.

The choice of a particular addressing scheme is based on the
requirements of a particular application or set of applications. For
example, text-based addressing often needs to address ÒwordsÓ, as
defined by a human language or a computer-based notation, and may wish
to address the tenth word of a paragraph, even though the word is the
third word within an \<emph\> element that follows the seventh word of
the paragraph. Similarly, an internal SDATA entity may represent what is
to the application, a letter, a word break or a sequence of characters,
and the preferred method of counting both words and characters may be
affected by the application interpretation of the entityÕs text. As
these examples show, although there are many benefits to using a
standard addressing scheme, particular application requirements may lead
to greater benefits in not adhering to a particular standard in some
contexts.

## The Simplest Model of an SGML Document

The simplest and most complete model of an SGML document is the linear
markup information, fleshed out with rules that specify, given a
particular markup information, how to determine the represented element
structure, entity structure and marked section structure. Design of the
markup information model will require determining which things are
ÒsignificantÓ and which are not, although the requirements for fully
supporting markup-sensitive applications of SGML will tend to favor the
view that ÒeverythingÓ is significant at the markup level, including
such things as ignored white space. The real questions will arise when
determining what is properly part of the derivative element structure
and entity structure models, but this is detail work.

The simplest model on which to base the derivative element structure,
entity structure and marked section structure is a simple hierarchical
tree, each node having associated with it an ordered sequence of zero or
more subnodes.

The purpose of these models is descriptive. As a consequence, no
application-specific structures are represented by these models, but
rather the models form the basis on which application-specific
structures are described and defined. In this context, ÒapplicationÓ
includes other standards which are based on ISO 8879. Examples of
application-specific structures include ÒclumpingÓ of data characters.
In the simplest, and, in the context of describing SGML (though not
necessarily preferred in the context of any particular application of
SGML), preferred model, each data character is a direct subnode of the
element, attribute or other structure that contains it.

## Modularity of Standards and Their ÒEnablingÓ Role

The purpose of technological standards is to ÒenableÓ new technologies
by minimizing the effort that is spent on common details and allowing
application designers and implementors to spend their time on
Òhigher-levelÓ and application-specific issues. Care has to be taken
that the strictures imposed by standards are, in fact, enabling and do
not prevent the development of new technology.

There are now many ISO standards, many of which overlap or are even
mutually exclusive in their provisions. It is important, for standards
to be maximally useful, for standards to be modular: that is, each
standard should deal with the problems it addresses, and as little as
possible directly deal with issues addressed by other standards.

In addition, the models in terms of which standards are defined should
be as simple as possible, consistent with the descriptive requirements
of the standards.

In the context of SGML, this means that the normative text of SGML
standard itself should not step into the realms of addressing or
object-oriented models, but should provide as simple a model as possible
that enables these models. The use and form of addressing and
object-oriented models can, and should be noted in ISO 8879, through
references to other standards and to technical reports, and through
non-normative annexes. Note that doing all this still allows the use of
terminology consistent with non-normative models, in particular,
object-oriented terminology.