Abstract:
A method, system, and computer program product for flexibly including descriptive information items, and the various media resources used for those items, in a user interface (UI). One markup language document is used for the UI data content, and a separate markup language document is used for specifying the descriptive items and media resources to be used along with that content. This technique enables dynamically altering the descriptive items and media resources to be used for a UI document, without requiring change to the UI document itself This technique also enables developing the descriptive items and media resources separately from developing the UI document, without a requirement to share common files between these development efforts. The specifications of descriptive information items and media resources can be efficiently reused among multiple UI documents.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates to the field of computer programming, and deals more particularly with a method, system, and computer program product for including information items having various media formats (such as text, image, audio, etc.) in a user interface that is specified in, and will be rendered from, a markup language document (such as an Extensible Markup Language, or “XML”, document). 
     2. Reservation of Copyright 
     A portion of the disclosure of this patent document contains material to which a claim of copyright protection is made. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but reserves all other rights whatsoever. 
     3. Description of the Related Art 
     The advanced user interfaces used by state-of-the-art applications often provide information using a wide variety of media, both for the user interface (UI) and for descriptive information items associated with the interface. The types of descriptive information items in common use today include tool tips; audio or visual explanations for an element presented via the UI, where these explanations may be selectable by a user; help text; hints, including “hover help” information that is provided when a user allows the cursor of a pointing device to hover over an element of a displayed UI; etc. The media types in which these descriptive items are provided may include text, video clips, graphic images, audio or speech, etc. Because these kinds of media are typically human-language specific, they are by convention stored in files which are separate from the files containing the software code of the application and/or the code from which the UI is created. This separate storage facilitates translating the stored information into different natural languages, sometimes referred to as “internationalization” of the information, without requiring changes or recompilation of the executable software. “Resource files” is a term commonly used to refer to these separately-stored media files, referring to the fact that the stored media information is used as a resource by an application program. An application typically includes a particular resource file by specifying a reference to the file system location where the resource file is stored (e.g. by providing the file system name to a compiler and/or linker utility). If multiple resource files are needed by the application, then multiple of these references are used. Resource files may be shared among multiple applications. These concepts are well known in the art. 
     Techniques for referencing media resource files from markup language documents are also known in the art. In the Hypertext Markup Language (“HTML”) notation, for example, the HTML syntax which specifies a Web page may include a reference to a separately-stored image, or video clip, or other type of media resource. Typically, the Web developer specifies an explicit reference to the separately-stored media resource using its Uniform Resource Locator (URL) or Uniform Resource Identifier (URI). The developer of an Extensible Markup Language (“XML”) document may also specify references to separately-stored media resources, which is also typically done by specifying a URL or URI reference. HTML and XML are both tag languages, where a tag language uses specially-designated constructs referred to as “tags” to delimit (or “mark up”) information. In the general case, a tag is a keyword that identifies what the data is which is associated with the tag, and is typically composed of a character string enclosed in special characters. “Special characters” means characters other than letters and numbers, which are defined and reserved for use with tags. Special characters are used so that a parser processing the data stream will recognize that this a tag. A tag is normally inserted preceding its associated data: a corresponding tag may also be inserted following the data, to clearly identify where that data ends. As an example of using tags, the syntax “&lt;email&gt;” could be used as a tag to indicate that the character string appearing in the data stream after this tag is to treated as an e-mail address; the syntax “&lt;/email&gt;” would then be inserted after the character string, to delimit where the e-mail character string ends. 
     While HTML uses a predefined set of tags, the syntax of XML is extensible and allows document developers to define their own tags which may have application-specific semantics. Because of this extensibility, XML documents may be used to specify many different types of information, for use in a virtually unlimited number of contexts. (A number of XML derivative notations have been defined, and continue to be defined, for particular purposes. “VoiceXML” is an example of one such derivative. References herein to “XML” are intended to include XML derivatives and semantically similar notations such as derivatives of the Standard Generalized Markup Language, or “SGML”, from which XML was derived. Refer to ISO 8879, “Standard Generalized Markup Language (SGML)”, (1986) for more information on SGML. Refer to “Extensible Markup Language (XML), W3C Recommendation 10-Feb.-1998” which is available on the World Wide Web from the World Wide Web Consortium, or “W3C”, for more information on XML.) 
     The present invention is concerned with XML documents which are used to specify a user interface. In this UI context, XML tags may be defined which specify media resources that may be used to provide the descriptive items described above, such as help text, hints, etc. For example, if a UI has an element such as a text entry field requesting a user to enter his or her last name, this element may have an associated hint which reminds the user that this is the surname. The element may also have associated help information which provides more detailed information, such as guidance on any syntactic data entry requirements for the surname, references to other places where the surname value may have been previously entered, etc. This hint and/or help information may be rendered using text, audio, etc. A UI designer may in some cases choose to provide multiple media types for one or more of the descriptive information items associated with a particular UI element. In the surname example, the user may be given a choice of receiving the hint in audio or text format, in which case the UI XML document must specify or reference both types of media resource files. FIG. 1 provides an example of prior-art XML syntax  100  that may be used for specifying UI information for a single UI field. In this example, a &lt;STRING&gt; tag pair  105 ,  160  represents information for a last name, as indicated at  115  for the NAME attribute  110 . The &lt;HINT&gt; tag pair  120 ,  130  brackets a textual hint  125 . The &lt;HELP&gt; tag pair  140 ,  150  brackets textual help  145  which is enclosed within &lt;META-TEXT&gt;  142  and &lt;/META-TEXT&gt;  147  tags. (In this example, a shortcut notation for the text within the &lt;HINT&gt; tag has been used, where this hint text has not been enclosed within &lt;META-TEXT&gt; tags.) 
     The techniques which exist in the prior art have a number of drawbacks, however. As will be obvious, as a UI XML document provides resource tags for more types of descriptive items, and/or more different media types for each descriptive item, the tagging in the source document becomes more verbose and cluttered, as well as more complex. The data content within the UI document may become obscured due to these multiple resource tags. In addition, a great deal of time can be expended when the developer of the UI XML document must explicitly and manually specify each type of descriptive information item to be provided for the UI content, and each media type to be provided for those items. In complex projects where one person or team develops the UI content and a different person or team develops the descriptive items, there may be a significant amount of overhead effort expended to coordinate sharing of files between these persons or teams during the development and testing phases, and to perform an integration of the final results. Furthermore, once such information has been statically specified for a particular UI XML document, it can be a time-consuming and error-prone process to revise that document such that it uses different descriptive items and/or different media types. The resource capabilities which are currently available in the art (such as in the Windows operating system and Java programming language) are rather limited. There is no capability for cross-referencing resources within a file to allow multiple use of a resource, for example, and very little structured support for resources. (For example, Windows allows definition of a dialog box, but this dialog box definition cannot reference other resources such as strings or other types of media in the resource file.) 
     Accordingly, what is needed is an improved technique for including descriptive information items, and the various media resources used for those items, in the markup language document from which a user interface is to be rendered. 
     SUMMARY OF THE INVENTION 
     An object of the present invention is to provide an improved technique for including descriptive information items, and the various media resources used for those items, in a user interface. 
     Another object of the present invention is to provide this technique by using one markup language document for the UI data content, and a separate markup language document for specifying the descriptive items and media resources to be used along with the content of that UI document. 
     Still another object of the present invention is to provide a technique which enables dynamically altering the descriptive items and media resources to be used for a UI document, without requiring change to the UI document itself. 
     Yet another object of the present invention is to provide a technique whereby the descriptive information items and media resources to be used for a UI document are specified in an MRI document, where the UI document and the MRI document may be developed independently. 
     Another object of the present invention is to provide a technique for efficiently reusing specifications of descriptive information items and media resources. 
     A further object of the present invention is to provide this reuse of specifications among multiple UI documents. 
     Other objects and advantages of the present invention will be set forth in part in the description and in the drawings which follow and, in part, will be obvious from the description or may be learned by practice of the invention. 
     To achieve the foregoing objects, and in accordance with the purpose of the invention as broadly described herein, the present invention provides a computer program product, a system, and a method for flexibly including descriptive items and media types in a user interface. This technique comprises: providing a UI document, the UI document comprising a specification of one or more UI elements to be rendered to a user; providing a machine readable information (MRI) document, the MRI document comprising one or more aggregations of MRI elements, each of the aggregations specifying one or more descriptive items in one or more media formats, and wherein selected ones of the UI elements reference selected ones of the aggregations of MRI elements; applying the selected ones of the aggregations to expand the references by the selected ones of the UI elements, thereby creating a media-enhanced UI document; and rendering the media-enhanced UI document. 
     The MRI elements may comprise one or more of: media specifications, references to include one or more of the media specifications, descriptive media containers, and references to include one of more of the descriptive media containers. The media specifications may comprise one or more of: a text format, a video format, an audio format, and an image format. The technique may further comprise one or more exclusion statements to exclude particular ones of the descriptive media containers, and the applying may further comprise omitting the particular ones when expanding the references to create the media-enhanced UI document. 
     The descriptive media containers may comprise one or more of: a caption container, a hint container, a tip container, and a help container. In this case, the UI document may further comprise one or more exclusion statements to exclude particular media types within particular ones of the descriptive media containers, and the applying may further comprise omitting the particular media types when expanding the references to create the media-enhanced UI document. 
     The applying may further comprise creating a hash table for quick and efficient retrieval of the MRI elements for use by the rendering. 
     The UI document and the MRI document may be specified in an Extensible Markup Language (XML) derivative., which may be an abstract UI markup language (AUIML). 
     An alternate version of the MRI document may be used by the applying, thereby altering results of the rendering without requiring change to the UI document. The technique may further comprise: providing second UI document, this second UI document comprising second UI elements which reference particular ones of the aggregations of MRI elements; applying the particular ones of the aggregations to expand the second UI elements, thereby creating a media-enhanced second UI document by reusing the particular ones of the aggregations from the MRI document; and rendering the media-enhanced second UI document. 
     The present invention will now be described with reference to the following drawings, in which like reference numbers denote the same element throughout. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is an example specifying descriptive items for a field of a user interface according to the prior art; 
     FIG. 2 is a block diagram of a computer workstation environment in which the present invention may be practiced; 
     FIG. 3 provides a table of data types that may be used in the markup notation processed by a preferred embodiment of the present invention; 
     FIG. 4 illustrates a more complex example of the prior art technique with which descriptive items, and various media forms thereof, may be specified in a UI document; 
     FIG. 5A provides an example of a prior art UI document specified in an extended XML notation, and FIGS. 5B and 5C then show how a preferred embodiment of the present invention uses two separate documents for separately specifying the user interface content, and the descriptive items and media types to be used along with that content, from this prior art document; 
     FIG. 6 provides tags that may be used in marking up UI documents and media resource documents, according to a preferred embodiment of the present invention; 
     FIG. 7 provides information that may be used with an optional exclusion feature of the present invention when reusing descriptive items and media resources; 
     FIG. 8 provides an example of this optional exclusion feature of the present invention whereby a specification of descriptive information items and media resources may be efficiently reused among UI documents having different needs; 
     FIGS. 9 through 28 illustrate flow charts which set forth the logic which may be used to implement a preferred embodiment of the present invention; and 
     FIGS. 29 through 35 illustrate flow charts which set forth the logic which may be used in an alternative embodiment of the present invention. 
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT 
     FIG. 2 illustrates a representative workstation hardware environment in which the present invention may be practiced. The environment of FIG. 2 comprises a representative single user computer workstation  210 , such as a personal computer, including related peripheral devices. The workstation  210  includes a microprocessor  212  and a bus  214  employed to connect and enable communication between the microprocessor  212  and the components of the workstation  210  in accordance with known techniques. The workstation  210  typically includes a user interface adapter  216 , which connects the microprocessor  212  via the bus  214  to one or more interface devices, such as a keyboard  218 , mouse  220 , and/or other interface devices  222 , which can be any user interface device, such as a touch sensitive screen, digitized entry pad, etc. The bus  214  also connects a display device  224 , such as an LCD screen or monitor, to the microprocessor  212  via a display adapter  226 . The bus  214  also connects the microprocessor  212  to memory  228  and long-term storage  230  which can include a hard drive, diskette drive, tape drive, etc. 
     The workstation  210  may communicate with other computers or networks of computers, for example via a communications channel or modem  232 . Alternatively, the workstation  210  may communicate using a wireless interface at  232 , such as a CDPD (cellular digital packet data) card. The workstation  210  may be associated with such other computers in a local area network (LAN) or a wide area network (WAN), or the workstation  210  can be a client in a client/server arrangement with another computer, etc. All of these configurations, as well as the appropriate communications hardware and software, are known in the art. 
     Software programming code which embodies the present invention is typically accessed by the microprocessor  212  of the workstation  210  from long-term storage media  230  of some type, such as a CD-ROM drive or hard drive. The software programming code may be embodied on any of a variety of known media for use with a data processing system, such as a diskette, hard drive, or CD-ROM. The code may be distributed on such media, or may be distributed to users from the memory or storage of one computer system over a network of some type to other computer systems for use by users of such other systems. Alternatively, the programming code may be embodied in the memory  228 , and accessed by the microprocessor  212  using the bus  214 . The techniques and methods for embodying software programming code in memory, on physical media, and/or distributing software code via networks are well known and will not be further discussed herein. 
     In the preferred embodiment, the present invention is implemented as one or more modules (also referred to as code subroutines, or “objects” in object-oriented programming) of one or more computer software programs. This software may operate on a user&#39;s workstation. Alternatively, the software may operate on a server in a network, or on any other suitable device, or partly on a workstation and partly on a server or other device in a distributed fashion. These environments and configurations are well known in the art. 
     The present invention provides a technique for flexibly including descriptive information items and various media types in a user interface, where that UI is specified in (and is to be rendered from) a markup language document. 
     The present invention will be described with reference to the Extended Markup Language, and an intent-based extension thereof which is designed for user interface markup documents and which is referred to herein as the Abstract User Interface Markup Language, or “AUIML”. In the preferred embodiment, the present invention is used with UI documents specified in XML or in the AUIML. Note, however, that other markup notations may exist or may be developed in the future which have similar properties to XML and the AUIML. Accordingly, while the present invention will be described with reference to examples expressed in the AUIML notation, it may also be used advantageously with information content expressed in other markup languages without deviating from the inventive concepts disclosed herein. 
     The AUIML is an XML extension developed by IBM. A description of the AUIML notation may be found in “AUIML: A Language for Marking-up Intent-based User Interfaces”, a copy is which is found herein in Table I. The AUIML provides a particular XML vocabulary which is designed for specifying a user interface definition, where that specification conveys not only the data content to be used but also formatting information that is to be used when rendering the content. In AUIML, data content and information regarding interactions with a user (e.g. which items have a mandatory response, what to display if the user requests a data entry “hint”, relationships among multiple data items and their values, etc.) are specified within &lt;DATA-GROUP&gt; and &lt;/DATA-GROUP&gt; markup tags. An arbitrary number of &lt;GROUP&gt;, &lt;/GROUP&gt; tags may be specified to define logical groups within a &lt;DATA-GROUP&gt;, where each group has an arbitrary number of data items. Data items may have a simple elementary data type, a user-defined type, or a complex data type. Examples of elementary types include: a string of characters representing a person&#39;s name, a Boolean value used with a checkbox, a number, etc. The types currently supported in the AUIML, as well as types which may be supported in extensions thereto, are shown in the table in FIG.  3 . In the AUIML as in XML, the organization of the data items within the groups, and the organization of groups within the data groups, indicates the structure of the information content. 
     For more information on how AUIML documents are written and processed in the prior art, refer to the AUIML specification in Table I and/or to U.S. Pat. No. 6,476,828, entitled “Systems, Methods and Computer Program Products for Building and Displaying Dynamic Graphical User Interfaces” (Ser. No. 09/321,700, filed May 28, 1999), which is incorporated herein by reference and is referred to hereinafter as “the referenced invention”. 
     According to the AUIML, descriptive information may be provided for each individual item of a UI or for an aggregation (i.e. a group) of UI items. The tags which are used in the preferred embodiment of the present invention to specify descriptive information include, but are not limited to, the following: &lt;CAPTION&gt;, &lt;HINT&gt;, &lt;TIP&gt;, and &lt;HELP&gt;. A &lt;CAPTION&gt; tag is used for specifying information that is preferably to be rendered as the title of the data item or group with which it is associated. A &lt;HINT&gt; tag is used for specifying information that is preferably to be rendered as a tooltip for its associated data item or group. A &lt;TIP&gt; is preferably rendered in a tip window for its associated data item or group, and &lt;HELP&gt; is preferably rendered as a separate page of information (for example, in a pop-up window). As will be obvious, an alternative tag syntax may be used, other types of descriptive items may be added, fewer of the descriptive items may be supported, and the manner of rendering the descriptive items may be altered, all without deviating from the concepts of the present invention. 
     FIG. 4 illustrates an example of the prior art technique with which these descriptive items, and various media forms thereof, may be specified in a UI document. This example expands on the previously-described example in FIG. 1, adding a &lt;CAPTION&gt;  410  which is comprised of a textual caption  420  and an audio caption  440 . The textual caption  420  is similar to the textual help  140 . However, note that the content for audio caption  440  is not defined directly in the UI document (i.e. as the value of the &lt;CAPTION&gt; tag), as is the textual caption  420 . Instead, a reference to a separately-stored audio resource file is specified using a SRC=attribute format. The value of the SRC attribute  442  is a URI  444 , indicating that a separately-stored audio resource file having this URI  444  is to be located and used to provide some type of audio caption information. A similar URI reference syntax may be used, if desired, to specify &lt;META-AUDIO&gt;, &lt;META-VIDEO&gt;, and/or &lt;META-IMAGE&gt; resource files for each of the types of descriptive information. 
     The preferred embodiment of the present invention will now be discussed in more detail with reference to FIGS. 5 through 28. 
     An example document of the prior art, where multiple resource tags are specified for the last name component  510  and first name component  530 , is shown in FIG.  5 A. This document  500  has been encoded using the AUIML, wherein the data content of a UI is enclosed within a &lt;DATA-GROUP&gt; and formatting (layout) information for that content is enclosed within layout tags such as &lt;DIALOG&gt; or &lt;PANEL&gt;, &lt;PROPERTY-SHEET&gt;, or &lt;WIZARD&gt; tags. (Refer to the AUIML specification and/or the referenced invention for more information on the prior art AUIML syntax.) As seen in this example, a significant number of tags are used to specify the descriptive information and media resources (see elements  515  and  535 ). 
     Rather than specifying the descriptive items and media resources directly within the UI document, in the manner which was depicted in the prior art examples of FIGS. 1,  4 , and  5 A, the present invention specifies this information in a separate “MRI document” or “MRI file”, so called because it provides the Machine Readable Information (“MRI”) for use with the UI. In the preferred embodiment, the MRI document is correlated to the UI document through the value of an MRI-SRC attribute on the root tag of the UI document. In one aspect of the present invention, the multiple resource tags which were used in the prior art for a component of the UI are now replaced by a single tag. This single tag provides a reference to an element in the MRI document where this element (which may itself comprise one or more tags) then provides the MRI information to be used for that component. (In a second aspect of the present invention, to be described below, multiple tags may be used in the UI document when specifying the media resources for a component.) The descriptive items and media resources to be used are then specified in this separate MRI document. In the preferred embodiment, this single tag has the syntax &lt;INSERT-MRI MRI-NAME=“MRI-file-name”/&gt;, and the separate MRI document is enclosed by an opening &lt;MRI&gt; tag and a closing &lt;/MRI&gt; tag. Within this &lt;MRI&gt; definition in the MRI document, an &lt;MRI-AGGREGATION NAME=“MRI-file-name”&gt; is defined. The value of the MRI-NAME attribute on the &lt;INSERT-MRI&gt; tag and the NAME attribute on the &lt;MRI-AGGREGATION&gt; tag are then used to correlate a reference for a component of the UI document with its associated MRI definition in the MRI document. FIGS. 5B and 5C illustrate how the single UI document of FIG. 5A is changed to use this approach of the present invention. FIG. 5B shows the new form of UI document (element  510 ), wherein an MRI-SRC attribute  511  indicates that the MRI information is stored in a file  512 . (In this example, the file directory “ENGLISH” indicates that this MRI file presumably contains media in the English language. Translation of MRI may be facilitated in this manner, where the same MRI may be translated into other languages in the same or another directory, and the appropriate language version is then used for a particular rendering of the UI document by altering the MRI-SRC attribute value.) FIG. 5B also shows that the multiple resource tags  515  of FIG. 5A (for the last name field) are now replaced by the single &lt;INSERT-MRI&gt; tag  520 , and similarly the multiple resource tags  535  (for the first name field) are replaced by the single tag  540 . FIG. 5C depicts an example MRI document  550  that is used, along with the UI document  510  of FIG. 5B, to flexibly include descriptive information items of various media types in the UI which will be rendered. The values  521 ,  561  of the MRI-NAME attributes in the UI document  510  (in FIG. 5B) and MRI document  550  (in FIG. 5C) provide the correlation which ties the MRI for the last name component between the two documents together at run-time. (In this example, the MRI in FIG. 5C provides only the media resources for the last name field; the media resources for the first name field may be provided in a similar manner within this MRI document  550 , or may use different features of the present invention that will be described.) 
     FIG. 6 provides a table showing the different types of MRI tags, and examples thereof, that may be used in the preferred embodiment. (Note that these examples are provided by way of illustration, and not of limitation: other tag types may be used instead of, or in addition to, these examples to support different media. Furthermore, a particular embodiment of the present invention is not required to support all of the tags shown in FIG.  6 . As will be obvious, changes to the syntax of the tags may be made without deviating from the scope of the present invention.) These tags are divided into five different types: media, media inclusion, media aggregation, media aggregation inclusion, and MRI aggregation. The file structure of an MRI document has a uniform reference mechanism with which these references are resolved, where each of these types of MRI tags follows particular inclusion rules, as will now be described. 
     A Document Type Definition (DTD) expressing the grammar of the MRI markup language (MRIML) used by the preferred embodiment of the present invention is provided herein in Table II. 
     The media tags are used when specifying content in-line, or specifying a URI reference to a stored media file. According to the present invention, media types are specified in the MRI document. See elements  591 ,  593 ,  595 , and  597  of FIG.  5 C. These media types can then be referenced from media inclusion tags. (Note that in the prior art, a descriptive item such as a caption could not specify a media item in-line except through use of a &lt;META- . . . &gt; tag, nor was resource file indirection—as provided by the present invention—supported.) 
     Media inclusion tags include their content using name correspondence and media type matching. An example of this is shown in FIG. 5C, where the &lt;META-TEXT&gt; tag  581  has the MRI-NAME value of “LastNameText”: by finding a corresponding name for a corresponding media type, it can be seen that this tag  581  will be associated at run-time with the &lt;TEXT&gt; tag  591  (the matching media type) and the NAME “LastNameText”  592  (the matching name value). Tags  583  and  587  are also media inclusion tags. 
     Media aggregation tags include their content by allowing a plurality of children tags to be included, where the children tags specify the media content for the aggregation. A media aggregation tag provides a definition, and has the attribute “NAME=”. An example is shown in FIG. 5C, where the &lt;CAPTION&gt; media aggregation tag  575  includes two child tags  581 ,  583 , where these child tags specify textual and image content for the last name caption, respectively. Tag  585  is another example of a media aggregation tag, and has a single child  587 . Media aggregation tags are referenced from a media aggregation inclusion tag. (Media aggregation tags  575  and  585 , for example, are referenced from media aggregation inclusion tags  571  and  573 , respectively.) 
     Media aggregation inclusion tags do not define media content, but rather include media content by reference to a definition in a media aggregation which has the same container type and name correspondence. Media aggregation inclusion tags do not have child tags, and use an “MRI-NAME=” attribute to provide the reference to the associated definition of the same container type. The &lt;META-CAPTION&gt; tag  571  in FIG. 5C, as an example, refers to the content defined in &lt;CAPTION&gt; tag  575 , because these tags are of the same container type of CAPTION and have the same attribute value “LastNameCaption” (see elements  572  and  576 ). 
     MRI aggregation tags bracket the MRI for a UI component, and include their contents by specifying media aggregation inclusion tag(s) which use an MRI-NAME attribute. In the example of FIG. 5C, the MRI aggregation tag  560  specifies two media aggregation inclusion tags  571 ,  573 . The first tag  571  is associated by container matching and name correspondence with tag  575  (including its children  581 ,  583 ). The second tag  573  is associated, again using container matching and name correspondence, with tag  585 . 
     For a given UI component, either an INSERT-MRI tag may be included (as shown in FIG. 5B at  520  and  540 ), as in the first aspect described above, or in a second aspect of the present invention, selected ones of the plurality of media aggregation inclusion tags may be used. Whereas the style used in FIG. 5B specifies a single tag  520  to include all the MRI defined in MRI document  550  at &lt;MRI-AGGREGATION&gt; tag  560  for use in rendering the last name field, the alternative style of using multiple media aggregation inclusion tags (up to one each of the media aggregation inclusion tags depicted in FIG. 6) may be used in the UI document to explicitly reference MRI in the MRI document according to this second aspect. As an example, a caption and help for the Last Name field may be referenced as follows: 
     &lt;STRING NAME=“LastName”&gt; 
     &lt;META-CAPTION MRI-NAME=“LastNameCaption”/&gt; 
     &lt;META-HELP MRI-NAME=“EmployeeInfoHelpPanel”/&gt; 
     &lt;/STRING&gt; 
     According to an optional feature of the present invention, MRI inclusion tags (i.e. the media inclusion tags and media aggregation inclusion tags) may be specified in the layout of the UI document, in addition to just allowing them in the data group. When this option is supported and used within an UI document, the definitions used in the layout preferably include media resources into the UI document by overriding any definitions in the data group for a particular UI component. (The flowcharts which are described below automatically implement this optional override feature through the manner in which an AUIML document and its associated MRI document are processed.) 
     According to another optional feature of the present invention, “EXCLUDE” tags may be defined. An EXCLUDE tag enables re-using previously specified MRI, but omitting selected parts thereof when the MRI is being rendered. Use of this exclusion feature enables efficiently reusing descriptive information items and media resources among UI documents having different needs. For example, all textual MRI from a previously specified MRI may be excluded, or all captions may be excluded therefrom. In the preferred embodiment of this optional feature, exclusion is supported only when the &lt;INSERT-MRI&gt; style of providing MRI for a UI component is used, although alternatively an exclusion feature may be provided with media aggregation inclusion tags as well. (For example, a &lt;META-CAPTION&gt; tag might have a child tag such as &lt;EXCLUDE-AUDIO /&gt;.) Furthermore, while the preferred embodiment allows the exclusion syntax as one or more child elements of the &lt;INSERT-MRI&gt; tag, this is by way of illustration and not of limitation: an implementation may be provided where the exclusion tags are allowed at other places within the UI document, without deviating from the scope of the present invention. FIG. 7 provides a table showing the types and classes that may be used with a preferred embodiment of this exclusion feature. (The presence of an asterisk in the leftmost column of the table in FIG. 7 indicates a media class, which comprises one or more media types. When a media class is specified on an exclude tag, such as &lt;EXCLUDE-VISUAL&gt;, this is an indication that all of the associated media are to be excluded.) This exclusion feature will be particularly useful when supporting user interfaces having different modalities, such as audio-only or IVR (Interactive Voice Response), and for supporting pervasive devices where memory for dealing with a large amount of meta information may be limited. (Note that these media types and classes are merely examples of those that may be supported, and are provided by way of illustration, and not of limitation. Furthermore, a particular embodiment of the present invention is not required to support all of the media types and media classes shown in FIG. 7, and different or additional types and classes may be supported without deviating from the scope of the present invention.) 
     An example of using MRI exclusion is shown in UI document  800  of FIG.  8 . As can be seen by comparing FIG. 8 with FIG. 5B, both UI document  510  and UI document  800  reference MRI for a last name field (elements  520  and  820 , respectively). However, in UI document  800  the exclusion feature has been used to selectively exclude portions of the &lt;MRI-AGGREGATION&gt;  560  in MRI document  550  of FIG. 5C which is referenced by MRI-NAME attribute  821 . In particular, all forms of HINT information are excluded by the &lt;EXCLUDE-HINT/&gt; tag  830 , and caption information in image form is excluded by the tags at  840 . 
     The logic with which the preferred embodiment of the present invention may be implemented will now be described with reference to the flowcharts in FIGS. 9 through 28. 
     The process for resolving the MRI for a particular UI document begins at Block  900  of FIG. 9, where a UI document (in the preferred embodiment, a UI document expressed using the AUIML) is parsed into a “DOM tree”, and a pointer “ADP” (for “AUIML Document Pointer”) is set to point to the root node of this tree. As is well known in the art, when a parser for XML or an XML derivative processes an input file, it reads the file and constructs a DOM tree based on the syntax of the tags embedded in the file and the interrelationships between those tags. The tag syntax is stored in the nodes of the DOM tree, and the shape of the tree is determined from the tag relationships. “DOM” is an acronym for “Document Object Model”, which is a language-independent application programming interface (“API”) for use with documents specified in markup languages including XML. DOM is published as a Recommendation of the World Wide Web Consortium, titled “Document Object Model (DOM) Level 1 Specification, Version 1.0” (1998) and available on the World Wide Web from the W3C. 
     “DOM tree” refers to the logical structure with which a document is modeled using the DOM. A DOM tree is a hierarchical representation of the document structure and contents. Each DOM tree has a root node and one or more leaf nodes, with zero or more intermediate nodes, using the terminology for tree structures that is commonly known in the computer programming art. A node&#39;s predecessor node in the tree is called a “parent” and nodes below a given node in the tree are called “child” nodes. A DOM tree can be streamed to a markup language document and back again, using techniques which are well known in the art. The DOM API enables application programs to access this tree-oriented abstraction of a document, and to manipulate document structure and contents (that is, by changing, deleting, and/or adding elements). Further, the DOM enables efficiently navigating the structure of the document. 
     Block  905  asks whether a parsing error was encountered while parsing the AUIML document. If so, then the document cannot be rendered, and processing stops. Otherwise, the MRI document is parsed (Block  910 ) into a DOM, and a pointer “MDP” (“MRI Document Pointer”) is set to point to the root node of this MRI DOM tree. Block  915  tests whether a parsing error occurred while parsing the MRI document. If so, the MRI provided according to the present invention cannot be resolved, and processing is halted. Otherwise, Block  920  invokes the logic of FIG. 10 to perform a DOM walk process. Control returns to the processing of FIG. 9 when the DOM walk has completed and the MRI has been resolved, after which an expanded UI document resulting from this resolution process can be rendered for the document user. (The actual rendering of the expanded document uses prior art techniques which are well known, and accordingly is not shown.) 
     The DOM walk process depicted in FIG. 10 is a recursive process which is first invoked from Block  920  of FIG. 9, and may be subsequently invoked from Block  1025 . Upon entering the logic of this process, the ADP pointer is pushed onto a stack (Block  1005 ). (Note that all pointers referenced herein are preferably implemented as global variables.) Block  1010  then moves the ADP pointer to point to its first child for the first invocation of Block  1010  for a traversal through FIG. 10 for a parent node, or to the next child of this parent node for subsequent invocations. Block  1015  then asks whether there was such a child node. If there was, control transfers to Block  1030 ; otherwise, processing continues at Block  1020 , which pops the ADP pointer from the stack and returns to the invoking logic. 
     Block  1030  invokes the logic of FIG. 11 to process the AUIML child nodes in this branch of the AUIML DOM tree, searching for MRI tags. As previously stated, the tags in the UI document may be of the form &lt;INSERT-MRI&gt;or &lt;META- . . . &gt;. Upon returning from this invocation in Block  1030 , a recursive invocation of the logic of FIG. 10 is made from Block  1025 , after which control transfers to Block  1010  to begin processing another child node. 
     FIG. 11 depicts the logic used to process MRI inclusion nodes of the AUIML DOM tree. Block  1105  asks whether the AUIML node pointed to by ADP has a child node representing an &lt;INSERT-MRI&gt; tag. If so, control transfers to Block  1115  which sets a pointer “IMN” (for “Insert MRI Node”) to point to this child node, and Block  1120  invokes the logic of FIG. 12 to process this INSERT-MRI node. When control reaches Block  1110 , the alternate style used for providing MRI for a UI component was used in the UI document, where one or more &lt;META- . . .&gt; tags will be present. Block  1110  invokes the logic of FIG. 13 to process each of the nodes representing these tags. (While Block  1110  is shown in FIG. 11 as a single invocation, it will be obvious to one of ordinary skill in the art how to repeat this invocation for each META node which is a child of the node pointed to by ADP.) Upon returning from the invocation of FIG. 12 or FIG. 13, control returns to the logic from which FIG. 11 was invoked. 
     An INSERT-MRI node is processed using the logic of FIG.  12 . The processing begins at Block  1205 , which invokes the logic of FIG. 14 to resolve the MRI-NAME reference for this node. Block  1210  then checks a “MAP” pointer (“MRI Aggregation Pointer”) that is set during the processing of FIG. 14, to see if the pointer has a null value. If it does, then the matching MRI-AGGREGATION node (i.e. the node of the DOM tree representing the &lt;MRI-AGGREGATION&gt; tag having the same attribute value as specified as an attribute value for the &lt;INSERT-MRI&gt; tag) could not be found, and the processing of FIG. 12 ends. Otherwise, when the pointer is not null, Block  1215  invokes the logic of FIG. 15 to process the located MRI-AGGREGATION node. 
     FIG. 13 represents the logic which is preferably used to process the &lt;META- . . .&gt; inclusion nodes of the AUIML DOM tree when this style of specifying MRI for a UI component was used. Block  1305  checks to see whether the AUIML node pointed to by ADP has a child node representing a &lt;META-TEXT&gt; tag. If so, Block  1310  invokes the processing of FIG. 20 to process this META-TEXT node, after which control transfers to Block  1315 . Block  1315  checks to see if there is a child node representing a &lt;META-IMAGE&gt; node. If so, Block  1320  invokes the processing of FIG. 21 to process this META-IMAGE node. In a similar manner, Blocks  1325  and  1330  are used to invoke the processing of FIG. 22 for a META-AUDIO node, and Blocks  1335  and  1340  invoke FIG. 23 to process a META-VIDEO node. (It will be obvious to one of ordinary skill in the art how to modify FIG. 13 if a particular implementation of the present invention supports fewer of these media inclusion tags, or additional inclusion tags for alternative media types.) 
     FIG. 14 resolves the MRI-NAME reference for an INSERT-MRI node, and is invoked from Block  1205  of FIG.  12 . Block  1405  initializes the MAP pointer, which will be used to point to the MRI-AGGREGATION node in the MRI DOM tree, to a null value. Block  1410  sets a pointer “P” to point to the first child of the node being pointed to by MDP. (With reference to the example MRI document  550  in FIG. 5C, the MDP pointer points to the node representing the &lt;MRI&gt; tag, and the P pointer will thus now point to its child node which represents &lt;MRI-AGGREGATION&gt; tag  560 .) Block  1415  asks whether P points at an MRI-AGGREGATION node. If so, control transfers to Block  1420 . Otherwise, control transfers to Block  1430 . 
     Block  1420  checks whether the value of the NAME attribute for the node to which P is pointing is the same as the value of the MRI-NAME attribute of the INSERT-MRI node (i.e. the node pointed to by IMN). If so, then the matching MRI-AGGREGATION node has been found. Block  1425  therefore sets MAP to point to this node, and control exits from FIG. 14 to the invoking logic. Otherwise, control transfers to Block  1430 . 
     Block  1430  is reached when the MRI-AGGREGATION node has not yet been located. A breadth-wise traversal of the MRI DOM tree is then performed by moving the pointer P to the next child of the node pointed to by MDP. Block  1435  asks whether there was such a node; if so, control returns to Block  1415  to evaluate this child node. Otherwise, when there are no more child nodes to evaluate, then the INSERT-MRI cannot be successfully resolved and the processing of FIG. 14 ends by returning control to the invoking logic. 
     FIG. 15 begins the processing of an MRI-AGGREGATION node which was located by the processing of FIG. 14, and is invoked from Block  1215  of FIG. 12 with the MAP pointer pointing to the MRI-AGGREGATION node. FIGS. 16-18 continue the processing of the MRI-AGGREGATION node, as will be described. Block  1515  asks whether the node pointed to by MAP has a child which is a META-CAPTION node, If not, control transfers to Block  1600  of FIG.  16 . Otherwise, Block  1520  sets an “MRIN” pointer (“MRI Node”) to point to the child node containing the META-CAPTION, and Block  1525  sets a variable “STYPE” to the value CAPTION. (This STYPE variable is subsequently used to find the associated container node in the process of container matching and name correspondence, as previously described with reference to finding the CAPTION container  575  for &lt;META-CAPTION&gt; tag  571  of FIG. 5C.) Block  1530  then invokes the processing of FIG. 19 to resolve this META-CAPTION node. Upon returning from FIG. 19, Block  1535  asks whether the matching CAPTION container node (which will be pointed to by pointer NP) was found. If so, control transfers to Block  1505 , and if not, control transfers to Block  1600  of FIG.  16 . 
     Block  1505  sets a parameter or variable “ETYPE” (for “Exclusion Type”) to CAPTION. (The ETYPE parameter is subsequently used during the processing of FIG. 24.) Block  1510  then invokes the processing of FIG. 24 to clone (i.e. make a copy of) this CAPTION node (referred to in FIG. 15 as the “NP” node because a node pointer NP points to it during subsequent processing) for inclusion in the AUIML DOM and prune it from the MRI DOM, after which control transfers to Block  1600  of FIG. 16 to continue processing the MRI-AGGREGATION node. (Note that references herein to making a clone of a node are to be interpreted as copying the node and any descendant nodes which it may have.) 
     FIG. 16 continues the processing of the MRI-AGGREGATION node, and performs similar processing for hints to that performed for captions by FIG.  15 . Block  1615  asks whether the node pointed to by MAP has a child which is a META-HINT node. If not, control transfers to Block  1700  of FIG.  17 . Otherwise, Block  1620  sets MRIN to point to the child node containing the META-HINT, and Block  1625  sets the STYPE variable to the value HINT. Block  1630  then invokes the processing of FIG. 19 to resolve this META-HINT node. Upon returning from FIG. 19, Block  1635  asks whether the matching HINT container node was found. If so, control transfers to Block  1605 , and if not, control transfers to Block  1700  of FIG.  17 . 
     Block  1605  sets ETYPE to HINT, and Block  1610  invokes the processing of FIG. 24 to clone this HINT node for inclusion in the AUIML DOM and prune it from the MRI DOM, after which control transfers to Block  1700  of FIG. 17 to continue processing the MRI-AGGREGATION node. 
     FIG. 17 continues the processing of the MRI-AGGREGATION node, and performs similar processing for tips to that of FIGS. 15 and 16 for captions and hints. Block  1715  asks whether the node pointed to by MAP has a child which is a META-TIP node. If not, control transfers to Block  1800  of FIG.  18 . Otherwise, Block  1720  sets MRIN to point to the child node containing the META-TIP, and Block  1725  sets the STYPE variable to the value TIP. Block  1730  then invokes the processing of FIG. 19 to resolve this META-TIP node. Upon returning from FIG. 19, Block  1735  asks whether the matching TIP container node was found. If so, control transfers to Block  1705 , and if not, control transfers to Block  1800  of FIG.  18 . 
     Block  1705  sets ETYPE to TIP, and Block  1710  invokes the processing of FIG. 24 to clone this TIP node for inclusion in the AUIML DOM and prune it from the MRI DOM, after which control transfers to Block  1800  of FIG. 18 to continue processing the MRI-AGGREGATION node. 
     FIG. 18 completes the processing of the MRI-AGGREGATION node, and performs similar processing for help to that performed for captions, hints, and tips by FIGS. 15-17. Block  1815  asks whether the node pointed to by MAP has a child which is a META-HELP node. If not, control transfers to Block  1835 , where the INSERT-MRI node (which is pointed to in the AUIML DOM by the IMN pointer, and which has now been completely resolved and expanded) is deleted as a child of the node pointed to by ADP. Control then returns to the invoking logic (Block  1215  of FIG.  12 ). Otherwise, Block  1820  sets MRIN to point to the child node containing the META-HELP, and Block  1825  sets the STYPE variable to the value HELP. Block  1830  then invokes the processing of FIG. 19 to resolve this META-HELP node. Upon returning from FIG. 19, Block  1840  asks whether the matching HELP container node was found. If so, control transfers to Block  1805 , and if not, control returns to the invoking logic in FIG.  12 . 
     Block  1805  sets ETYPE to HELP, and Block  1810  invokes the processing of FIG. 24 to clone this HELP node for inclusion in the AUIML DOM and prune it from the MRI DOM, after which control transfers to Block  1835 , described above. 
     FIG. 19 resolves a META aggregation node, and is invoked from Blocks  1530 ,  1630 ,  1730 , and  1830  for resolving META-CAPTION, META-HINT, META-TIP, and META-HELP media aggregation inclusion nodes, respectively. FIG. 19 uses container matching and name correspondence to search the MRI DOM to find the media aggregation node which corresponds to the media aggregation inclusion node for which FIG. 19 was invoked, where this media aggregation node provides the media content definition. Block  1905  initializes the node pointer NP to a null value. Block  1910  then sets the pointer P to point to the first child node of the node currently pointed to by MDP. Block  1915  asks whether P now points to a node having a container type which matches the value of variable STYPE. If so, Block  1920  checks the name correspondence between the name of the node pointed to by P and the node pointed to by MRIN. If the value of the NAME attribute in P&#39;s node is the same as the value of the MRI-NAME attribute in MRIN&#39;s node, then the names correspond and Block  1920  has a positive result. Block  1935  then sets the NP pointer to also point to the node pointed to by P, and control returns to the invoking logic. Otherwise, when the names do not correspond, control transfers from Block  1920  to Block  1925 . Block  1925  is also reached when P did not point to a node of the appropriate STYPE. Therefore, Block  1925  moves the pointer P to point to the next child of MDP, to continue searching for the node having the appropriate STYPE and attribute value. Block  1930  asks if there was another child. If not, then the processing of FIG. 19 exits. Otherwise, control returns to Block  1915  to evaluate this child node for a match. Checking of the STYPE and attribute value continues until there are no more children to evaluate (a negative result in Block  1930 ) or a match is found (a positive result in Block  1920 ). 
     FIG. 20 comprises the logic for processing a META-TEXT node, and is invoked from Block  1310  of FIG.  13 . Block  2005  asks whether the node pointed to by MDP has a child node which is a TEXT node, and where the value of the NAME attribute for that child corresponds to the value of the MRI-NAME attribute of the META-TEXT node in the AUIML DOM which is currently pointed to by ADP. If there is no such child, then the META-TEXT node cannot be resolved, and the processing of FIG. 20 exits. Otherwise, Block  2010  clones this matching TEXT node from the MRI DOM. Block  2015  then replaces the META-TEXT node in the AUIML DOM tree with this cloned TEXT node, and processing of FIG. 20 ends. 
     META-IMAGE nodes are processed by the logic of FIG. 21, which is invoked from Block  1320  of FIG.  13 . Block  2105  asks whether the node pointed to by MDP has a child node which is an IMAGE node, and where the value of the NAME attribute for that child corresponds to the value of the MRI-NAME attribute of the META-IMAGE node in the AUIML DOM which is currently pointed to by ADP. If there is no such child, then the META-IMAGE node cannot be resolved, and the processing of FIG. 21 exits. Otherwise, Block  2110  clones this matching IMAGE node from the MRI DOM. Block  2115  then replaces the META-IMAGE node in the AUIML DOM tree (at the location where ADP points) with this cloned IMAGE node, and processing of FIG. 21 ends. 
     META-AUDIO nodes are processed by the logic of FIG. 22, which is invoked from Block  1330  of FIG.  13 . Block  2205  asks whether the node pointed to by MDP has a child node which is an AUDIO node, and where the value of the NAME attribute for that child corresponds to the value of the MRI-NAME attribute of the META-AUDIO node in the AUIML DOM which is currently pointed to by ADP. If there is no such child, then the META-AUDIO node cannot be resolved, and the processing of FIG. 22 exits. Otherwise, Block  2210  clones this matching AUDIO node from the MRI DOM. Block  2215  then replaces the META-AUDIO node in the AUIML DOM tree with this cloned AUDIO node, and processing of FIG. 22 ends. 
     META-VIDEO nodes are processed by the logic of FIG. 23, which is invoked from Block  1340  of FIG.  13 . Block  2305  asks whether the node pointed to by MDP has a child node which is a VIDEO node, and where the value of the NAME attribute for that child corresponds to the value of the MRI-NAME attribute of the META-VIDEO node in the AUIML DOM which is currently pointed to by ADP. If there is no such child, then the META-VIDEO node cannot be resolved, and the processing of FIG. 23 exits. Otherwise, Block  2310  clones this matching VIDEO node from the MRI DOM. Block  2315  then replaces the META-VIDEO node in the AUIML DOM tree with this cloned VIDEO node, and processing of FIG. 23 ends. 
     FIGS. 24-28 depict a preferred embodiment of the logic used to clone a node from the MRI DOM for inclusion in the AUIML DOM, and to then prune the node from the MRI DOM. Note that these FIGS. 24-28 are primarily concerned with the logic which supports the optional exclusion feature which was previously described. If the exclusion feature is not implemented, then only the functionality of Blocks  2410  and  2425  is to be implemented, as appropriate for each supported type of descriptive information. 
     On entry to this logic, the variable or parameter ETYPE indicates which type of node (CAPTION, HINT, TIP, or HELP) is being processed. The processing of FIG. 24 begins at Block  2402 , which concatenates this ETYPE value to the syntax “EXCLUDE-”, and stores the result as the value of “ETAGN” (“Exclude Tag Name”). At Block  2405 , a test is made to determine whether the node of the AUIML DOM pointed to by IMN contains a node representing the appropriate ETAGN value (e.g. an EXCLUDE-CAPTION tag when FIG. 24 is being used to exclude a caption). If not, then there is no exclusion processing to be performed for the tag name with which FIG. 24 was invoked, and processing continues at Block  2407 . Otherwise, Block  2430  tests to see if the ETYPE parameter is some value other than CAPTION (i.e. HINT, TIP, or HELP) and if the node in the AUIML DOM pointed to by IMN has a child node which specifies that all user assistance (i.e. a node representing an &lt;EXCLUDE-USER-ASSISTANCE&gt; tag) is to be excluded. If both of these conditions are true, then the node pointed to by NP is to be excluded, so the processing of FIG. 24 exits. If one or both of the conditions are false, then the node pointed to by NP is not necessarily to be excluded, and control transfers to Block  2440  where a pointer “ECN” (“Exclude Child Node”) is set to point to the child of the node pointed to by IMN which has the value of ETAGN. With reference to the example of exclusion tags in FIG. 8, during a traversal through the logic of FIG. 24, IMN would point to a node representing the &lt;INSERT-MRI&gt; tag  820 , ETAGN would have the value EXCLUDE-CAPTION, and ECN would thus point to a node representing the &lt;EXCLUDE-CAPTION&gt; node in the AUIML DOM. Block  2450  then checks to see if the ECN node has any child exclusion nodes (i.e. nodes which would exclude only certain media types from the caption, hint, tip, or help which is being processed). If this test has a positive result, control transfers to Block  2410 ; when it has a negative result, then the entire media aggregation (i.e. CAPTION, HINT, etc.) is to be excluded, and the processing of FIG. 24 exits without copying any information for this media aggregation into the AUIML DOM. With reference to the example in FIG. 8, the &lt;EXCLUDE-CAPTION&gt; tag has an &lt;EXCLUDE-IMAGE&gt; tag as a child, and thus the test in Block  2450  has a positive result. 
     At Block  2407 , the ETYPE (i.e. CAPTION, HINT, TIP, or HELP) node from the MRI DOM (which is a child of the node to which MAP currently points and which is pointed to by NP) is cloned, and a pointer “CN” (cloned node) is set to point to this clone. Control then transfers to Block  2425 . 
     Block  2410  performs the same cloning process and setting of the CN pointer as Block  2407 , after which Block  2420  then invokes the processing of FIG. 25 to prune any media-specific exclusions (i.e. exclusions using tags such as &lt;EXCLUDE-AUDIO&gt;) from this clone. Upon returning from FIG. 25, Block  2425  attaches the possibly-modified clone to the AUIML DOM as a child of the node currently pointed to by ADP. This attachment (whether the cloned node has been modified or not) has the effect of expanding the AUIML syntax in the UI document so that it dynamically reflects the current contents of its associated MRI document. Control then returns to the invoking logic (in one of FIGS.  15 - 18 ). 
     The processing of FIG. 25 is invoked from Block  2420  of FIG. 24, and begins the exclusion processing of the node (and any descendants it may have) which has just been cloned by Block  2410 . Block  2505  asks whether the node pointed to by ECN (which is a child of the AUIML DOM node pointed to by IMN) contains (i.e. has a child which is) an EXCLUDE-TEXT node. If it does, control transfers to Block  2515 . Otherwise, another test is performed at Block  2510  to determine whether the node pointed to by ECN contains an EXCLUDE-VISUAL node. With reference to the table in FIG. 7, it can be seen that an EXCLUDE-VISUAL tag is an indication that all types of visual media, comprising text, image, and video, are to be excluded. When the test at Block  2510  has a positive result, control transfers to Block  2515 . When the test has a negative result, then neither text nor visual media is to be excluded from the node pointed to by ECN, so control transfers to Block  2600  of FIG. 26 to continue evaluating the node for other exclusions. (ECN points to an &lt;EXCLUDE-CAPTION&gt; node in the AUIML DOM, for example, when traversing through the logic of FIG. 25 for processing nodes corresponding to element  840  of FIG. 8; in this example, the tests in Blocks  2505  and  2510  both have a negative result.) 
     Control reaches Block  2515  when either text or all visual media is to be excluded from the media aggregation (i.e. CAPTION, HINT, etc.) being processed. Block  2515  then asks whether the cloned CN node includes a TEXT node. If so, processing continues to Block  2520  where this TEXT node is deleted as a child of the cloned CN node. When the test in Block  2515  has a negative result, and following processing of Block  2520 , control transfers to Block  2600  of FIG.  26 . 
     FIG. 26 continues the processing of the “EXCLUDE- . . .” node pointed to by ECN. At Block  2605 , a test is made to determine whether the node pointed to by ECN has a child that represents an EXCLUDE-IMAGE node. If it does, control transfers to Block  2615 . Otherwise, another test is performed at Block  2610  to determine whether the node pointed to by ECN contains an EXCLUDE-VISUAL node. When the test at Block  2610  has a positive result, control transfers to Block  2615 . When the test has a negative result, then neither image nor visual media is to be excluded from the media aggregation being processed, so control transfers to Block  2700  of FIG. 27 to continue evaluating the node pointed to by ECN for other exclusions. 
     Control reaches Block  2615  when either image or all visual media is to be excluded from the media aggregation being processed. Block  2615  then asks whether the cloned CN node includes an IMAGE node. If so, processing continues to Block  2620  where this IMAGE node is deleted as a child of the cloned CN node. When the test in Block  2615  has a negative result, and following processing of Block  2620 , control transfers to Block  2700  of FIG.  27 . 
     FIG. 27 performs the exclusion processing for AUDIO and AURAL media. At Block  2705 , a test is made to determine whether the node pointed to by ECN contains an EXCLUDE-AUDIO node. If it does, control transfers to Block  2720 . Otherwise, another test is performed at Block  2710  to determine whether the node pointed to by ECN contains an EXCLUDE-AURAL node. When this test has a positive result, control transfers to Block  2720 . Otherwise, Block  2715  then checks whether the node pointed to by ECN contains an EXCLUDE-RT-META-DATA node (to exclude all real-time meta-data media resources). If it does, control transfers to Block  2720 . When the test has a negative result, then none of the media types which have been checked are to be excluded from the media aggregation being processed, so control transfers to Block  2800  of FIG. 28 to continue evaluating the node pointed to by ECN for other exclusions. 
     Control reaches Block  2720  when either audio or all aural or real-time media is to be excluded from the media aggregation being processed. Block  2720  then asks whether the cloned CN node includes an AUDIO node. If so, processing continues to Block  2725  where this AUDIO node is deleted as a child of the cloned CN node. When the test in Block  2720  has a negative result, and following processing of Block  2725 , control transfers to Block  2800  of FIG.  28 . 
     FIG. 28 finishes the processing of the node pointed to by ECN. At Block  2805 , a test is made to determine whether the node pointed to by ECN has a child that represents an EXCLUDE-VIDEO node. If it does, control transfers to Block  2820 . Otherwise, another test is performed at Block  2810  to determine whether the node pointed to by ECN contains an EXCLUDE-VISUAL node. When the test at Block  2810  has a positive result, control transfers to Block  2820 . Otherwise, Block  2815  then checks whether the node pointed to by ECN contains an EXCLUDE-RT-META-DATA node. If so, control transfers to Block  2820 . When the test has a negative result, then none of the media types which have been checked are to be excluded from the media aggregation being processed, so the processing of the node pointed to by ECN is complete and the traversal through the logic of FIGS. 25-28 ends, returning control to the invoking logic in FIG.  24 . 
     Control reaches Block  2820  when either video or all visual or real-time media is to be excluded from the media aggregation being processed. Block  2820  then asks whether the cloned CN node includes a VIDEO node. If so, processing continues to Block  2825  where this VIDEO node is deleted as a child of the cloned CN node. When the test in Block  2820  has a negative result, and following processing of Block  2825 , control returns to the invoking logic of FIG.  24 . 
     In an alternative embodiment of the present invention, a hash table is used to optimize the process of looking up MRI to resolve the references from a UI document, rather than retrieving the information from the DOM. This optimization takes advantage of the fact that in the MRIML grammar (see Table II), items that are referenced by name are never META- . . . nodes, and all referenced nodes (i.e. those that contain content, and are not META- . . . nodes) are direct descendants of the root &lt;MRI&gt; tag. Since compound identifiers are not used, nor are nested names within groups, the hash table can be used to store and quickly look up (by name and type) every node that needs to be resolved. Preferably, an entry is created in the hash table for each node that is not a META- . . . node, where the hash table key is comprised of a concatenation of the node&#39;s type and name (where a separator such as a “.” may be used between the type and name values). For example, the caption for the last name in FIG. 5C (element  575 ) would be stored using the key “CAPTION.LastNameCaption”. 
     This alternative embodiment will now be discussed in more detail with reference to the logic in FIGS. 29 through 35. 
     FIG. 29 replaces the logic of FIG. 9, and has identical function except in Blocks  2910  and  2915 . Rather than parsing the MRI document into a DOM, as in Block  910  of FIG. 9, Block  2910  sets a collection TS (for “TypeSet”) to contain the types which are valid (which in this embodiment are MRI-AGGREGATION, CAPTION, HINT, TIP, HELP, TEXT, AUDIO, VIDEO, and IMAGE). Block  2915  then invokes a new hashing and parsing process which is described in FIG.  30 . Upon returning from this hashing and parsing process, the processing of FIG. 29 is identical to returning from Block  910  of FIG.  9 . 
     The hashing and parsing process of FIG. 30 begins at Block  3005 , where the hash table “HT” is created. Block  3010  then parses the MRI document, and sets MDP to point to the root of the DOM tree created from this parsing. Block  3015  pushes MDP onto a stack, and Block  3020  moves MDP so that it now points to one of its children. (On the first pass through this logic for a parent node, the pointer will point to the first child; on subsequent passes, it will point in succession to each sibling of that first child.) Block  3025  asks whether there was a child node. If not, Block  3030  pops MDP from the stack, and control returns to the invoking logic. Otherwise, processing continues at Block  3035  where a test is made to determine whether the node type of the node to which MDP is pointing is contained within the set of valid types in TS. If this test has a negative result, then no hash table entry will be created for this node, and control transfers to Block  3020  to process the next sibling node. When the test in Block  3035  has a positive result, Block  3040  creates the hash key that will be used for the current node by concatenating its node type and the value of its MRI-NAME attribute. Block  3045  then computes a hash for this hash key, and inserts the node&#39;s information into the hash table at that location. Control then returns to Block  3020  to process the next sibling. 
     FIGS. 10-13 of the preferred embodiment are used in this alternative embodiment for processing the AUIML DOM, except for the following changes: (1) Block  1205  of FIG. 12 now invokes the processing of FIG. 31, rather than FIG. 14; (2) Blocks  1310 ,  1320 ,  1330 , and  1340  of FIG. 13 now invoke the processing of FIGS. 32,  33 ,  34 , and  35 , respectively, rather than FIGS. 20,  21 ,  22 , and  23 ; and (3) FIGS. 32,  33 ,  34 , and  35 , replace FIGS. 20,  21 ,  22 , and  23 , respectively. 
     FIG. 31 is used to resolve the MRI-NAME reference for an INSERT-MRI node, and is invoked from Block  1205  of FIG.  12 . Block  3105  initializes the MAP pointer, which will be used to point to the MRI-AGGREGATION node in the MRI DOM tree, to a null value. Block  3110  sets the hash key to the concatenation of variables RTYPE (which is set in FIGS. 32-35 to one of TEXT, IMAGE, AUDIO, or VIDEO) and RNAME (which is set in those same figures to the value of the MRI-NAME attribute for the node being processed). Block  3115  then uses this hash key to retrieve information from the hash table HT, and sets the pointer “N” to point to this retrieved information. Block  3120  asks whether N in fact points to a node. If not, then no information was located in the hash table, and control returns to the invoking logic. Otherwise, the test in Block  3120  has a positive result, and Block  3125  then sets the MAP pointer to point where N is pointing. The processing of FIG. 31 then exits. 
     The logic of FIG. 32 is invoked from Block  1310  of FIG.  13 . Beginning at Block  3205 , the RTYPE variable is set to the value TEXT because a META-TEXT node is being processed. Block  3210  then sets the RNAME variable to the value of this META-TEXT node&#39;s MRI-NAME attribute. Block  3215  then invokes the logic of FIG. 31 to retrieve the information for this RTYPE, RNAME hash key from the hash table. On exit from FIG. 31, the pointer MAP points either to a null value (when no entry was found in the hash table) or to a located entry for a node. Block  3220  thus tests the value of MAP to see if it points to a node. If not, Block  3235  signals an error, and the processing of FIG. 32 ends. Otherwise, Block  3225  clones the TEXT node currently pointed to by MAP, and Block  3230  replaces the META-TEXT node in the AUIML DOM which is pointed to by ADP with this cloned TEXT node. The processing of FIG. 32 then ends. 
     FIG. 33 is invoked from Block  1320  of FIG. 13, and processes a META-IMAGE node. At Block  3305 , the RTYPE variable is set to the value IMAGE, and Block  3310  then sets the RNAME variable to the value of this META-IMAGE node&#39;s MRI-NAME attribute. Block  3315  then invokes the logic of FIG. 31 to retrieve the information for this RTYPE, RNAME hash key from the hash table. On returning from FIG. 31, Block  3320  tests the value of MAP to see if it points to a node. If not, Block  3335  signals an error, and the processing of FIG. 33 ends. Otherwise, Block  3325  clones the IMAGE node currently pointed to by MAP, and Block  3330  replaces the META-IMAGE node in the AUIML DOM which is pointed to by ADP with this cloned IMAGE node. The processing of FIG. 33 then ends. 
     The logic of FIG. 34 is invoked from Block  1330  of FIG. 13, and processes a META-AUDIO node. Beginning at Block  3405 , the RTYPE variable is set to the value AUDIO, and Block  3410  then sets the RNAME variable to the value of this META-AUDIO node&#39;s MRI-NAME attribute. Block  3415  then invokes the logic of FIG. 31 to retrieve the information for this RTYPE, RNAME hash key from the hash table. On returning from FIG. 31, Block  3420  tests the value of MAP to see if it points to a node. If not, Block  3435  signals an error, and the processing of FIG. 34 ends. Otherwise, Block  3425  clones the AUDIO node currently pointed to by MAP, and Block  3430  replaces the META-AUDIO node in the AUIML DOM which is pointed to by ADP with this cloned AUDIO node. The processing of FIG. 34 then ends. 
     FIG. 35 is invoked from Block  1340  of FIG. 13, and processes a META-VIDEO node. At Block  3505 , the RTYPE variable is set to the value VIDEO, and Block  3510  then sets the RNAME variable to the value of this META-VIDEO node&#39;s MRI-NAME attribute. Block  3515  then invokes the logic of FIG. 31 to retrieve the information for this RTYPE, RNAME hash key from the hash table. On returning from FIG. 31, Block  3520  tests the value of MAP to see if it points to a node. If not, Block  3535  signals an error, and the processing of FIG. 35 ends. Otherwise, Block  3525  clones the VIDEO node currently pointed to by MAP, and Block  3530  replaces, the META-VIDEO node in the AUIML DOM which is pointed to by ADP with this cloned VIDEO node. The processing of FIG. 35 then ends. 
     As has been demonstrated, the present invention provides an efficient, flexible technique for including descriptive information of various media types in a user interface. The type of descriptive information, and the media formats of that information, can be easily changed by altering the MRI document; the UI document does not need to be changed. This technique is particularly beneficial for translation-sensitive information, where MRI in alternative languages can easily be substituted as needed. Furthermore, using the optional exclusion feature, a particular UI document specification can reference an MRI document but make changes to that UI document as suited to the needs of a particular situation. This feature is particularly beneficial for when a UI document is to be rendered on devices which support different modalities. 
     While the preferred embodiment of the present invention has been described, additional variations and modifications in that embodiment may occur to those skilled in the art once they learn of the basic inventive concepts. The techniques disclosed herein are based partially upon certain predefined characteristics of the XML and AUIML notations. It will be obvious to one of ordinary skill in the art that the inventive concepts disclosed herein may be adapted to changes in the notations, should they occur. The inventive concepts may also be used with notations other than XML and AUIML that provide the same characteristics described herein for XML and AUIML documents, and with modeling techniques other than DOM trees that exhibit the required properties discussed for DOM trees (i.e. the modeling technique creates a tree that represents the structure and content described in the particular notation.) Therefore, it is intended that the appended claims shall be construed to include both the preferred embodiment and all such variations and modifications as fall within the spirit and scope of the invention.