Effective transmission of documents in hypertext markup language (HTML)

A data processor controlled display system for displaying documents including natural language text representative of data transmitted in a markup language such as HTML to a display station from a location remote from said station. The HTML tags are tokenized prior to transmission, comments are removed from the HTML file prior to transmission and the file is then subjected to a conventional general text compression. After the HTML file is received at the display station, it is decompressed, detokenized and the resulting HTML file is used to create a document or page on the display.

TECHNICAL FIELD
 The present invention relates to computer managed communication networks
 and particularly to the transmission of documents in markup language
 format from remote distribution points to computer controlled user
 interactive display terminals which would be available to interactive
 users in a networking environment such as the internet or equivalent
 proprietary or public networks.
 BACKGROUND OF THE INVENTION
 The 1990's decade has been marked by a societal technological revolution
 driven by the convergence of the data processing industry with the
 consumer electronics industry. Like all such revolutions, it unleashed a
 great ripple effect of technological waves. The effect has in turn driven
 technologies which have been known and available but relatively quiescent
 over the years. Two of these technologies are the internet-related
 distribution and object oriented programming systems. The convergence of
 the electronic entertainment and consumer industries with data processing
 exponentially accelerated the demand for wide ranging communications
 distribution channels, and the World Wide Web or internet which had
 quietly existed for over a generation as a loose academic and government
 data distribution facility reached "critical mass" and commenced a period
 of phenomenal expansion. With the expanded accessibility of tens of
 thousands of programmers to each other, not to mention to potential users
 of such programs via the expanded internet client base, an obvious need
 became apparent: cooperative programming systems wherein program
 developers could coact to continuously expand and enhance existing
 programs in a distributed programming environment. Object oriented
 programming offered the solution. With its potentially interchangeable
 objects or units, object oriented programming systems have found
 acceptance as the programming system for the internet. In all areas of
 data processing and communications, as well as the electronic
 entertainment and consumer industries having anything to do with the
 internet, there has been a substantial movement to object oriented
 programming systems and particularly to the Java programming system. A
 major significance of Java is that it is an internet or World Wide Web
 distributed programming system where literally thousands of program
 developers and users are continually upgrading and changing the programs.
 There arose a need for programmers and users to readily being able to
 display program documentation in a clear and comprehensive manner in
 natural language. Hypertext Markup Language (HTML), which had been the
 documentation language of the internet world wide web for years, offered
 an answer and more. It offered direct links between pages and other
 documentation on the web and a variety of related data sources which were
 at first text and then evolved into media, i.e. "hypermedia".
 Now the combination of Java object oriented programming and HTML has taken
 an even greater advance in Web pages: Java code and programs are being
 created which will exist on web pages where they offer the user the option
 of downloading and executing such code or programs when the page is being
 browsed. These programs are embedded in the web page as applets.
 With all of these rapidly expanding functions of web pages and like
 documentation, it should be readily understandable that the demand for web
 documents has been expanding exponentially in recent years. In addition to
 the proliferating standard uses of HTML for text and media related world
 wide web pages for commercial, academic and entertainment purposes, the
 Java documentation program, JavaDoc, will produce standard HTML files for
 outputs to computer controlled displays to provide standard natural
 language displays of the program documentation. Thus, HTML has become the
 display language of choice for the internet or World Wide Web. It is used
 there for all forms of display documentation including the markup of
 hypertext and hypermedia documents, usually stored with their respective
 documents on an internet or web server in addition to the above described
 programming distribution. HTML is an application of SGML (Standard
 Generalized Markup Language), an ISO standard for defining the structure
 and contents of any digital document. It should be recognized that any of
 the aspects of the present invention illustrated with respect to HTML
 would be equally applicable to SGML. For further details on Java, JavaDoc
 or HTML, reference may be made to the texts "Just Java", 2nd Edition,
 Peter van der Linden, Sun Microsystems, Inc., 1997, or "Java in a
 Nutshell", 2nd Edition, by David Flanagan, O'Reilly publisher, 1997.
 It must be recognized that HTML, as well as other markup languages for web
 pages, originated at a time when the demand for web pages and related
 documentation was relatively modest. It could be said that the present day
 demand on web page resources and markup languages was probably
 inconceivable when these were developed. Accordingly, it is the objective
 of the present invention to provide implementations which go well beyond
 conventional general data compression whereby the markup languages used in
 web page development, transmission and use may be downloaded and used more
 effectively with less wasted time.
 SUMMARY OF THE INVENTION
 The present invention relates to a computer controlled display systems for
 displaying documents e.g., web pages including natural language text
 representative of data transmitted to display stations from a location
 remote from said stations e.g., web servers where the data is stored in a
 markup language format including tags identifying the contents of said
 data. One key aspect of the present invention is the provision of means
 for tokenizing the tags prior to the transmission of data from the server
 to the display station and then detokenizing said tokenized tags received
 at the display station whereby said received data is restored to said
 markup language format. Then, there are means associated with the display
 station for translating said received data from said markup language
 format into said displayed natural language text documents. The preferable
 markup language is of course HTML. Another aspect of the invention is
 directed to portions of the data to be transmitted, identified by tags, as
 comments. Means are provided for removing the data portions identified as
 comments prior to transmitting said data.
 All of the above expedients may be combined with standard general data
 compression means for compressing, prior to said transmitting, the data
 portions remaining after tag tokenizing comments removal, and means for
 decompressing said compressed data portions received at the display
 station so that the received data is restored to said markup language
 format.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
 Before going into the details of specific embodiments, it will be helpful
 to understand from a more general perspective the various elements and
 methods which may be related to the present invention. Some of the
 documents on which the present invention may be used are in the Java
 Programming system which is an object oriented system utilizing the Java
 programming language. The Java system and language are extensively
 familiar to those skilled in the art. The text, "Just Java", Peter van der
 Linden, described above comprehensively details the system and language.
 The HTML language is also described in detail in the above "Just Java" text
 and particularly at Chapter 7, pp. 249-268, dealing with the handling of
 web pages with embedded Java programs and also in the text, "Mastering the
 Internet", Cady and McGregor, published by Sybex, San Francisco, 1996 and
 particularly pp. 637-642 on HTML in the formation of web pages.
 Referring to FIG. 1, a typical data processing system is shown which may be
 used in conjunction with object oriented software such as Java (.TM. of
 Sun Microsystems, Inc.) and HTML in implementing the present invention on
 the receiving interactive workstation. A central processing unit (CPU),
 such as one of the PowerPC microprocessors available from International
 Business Machines Corporation (PowerPC is a trademark of International
 Business Machines Corporation) is provided and interconnected to various
 other components by system bus 12. An operating system 41 runs on CPU 10
 and provides control and is used to coordinate the function of the various
 components of FIG. 1. Operating system 41 may be one of the commercially
 available operating systems such as DOS or the OS/2 operating system
 available from International Business Machines Corporation (OS/2 is a
 trademark of International Business Machines Corporation). Any
 conventional network browser system involving HTML language with embedded
 Java applets forms part of application 40, runs in conjunction with
 operating system 41 and provides output calls to the operating system 41
 which implements the various functions to be performed by the HTML
 application 40. The browser program, in combination with the operating
 system, provides the basic receiving workstation on which the web pages,
 according to the present invention, may be implemented.
 A read only memory (ROM) 16 is connected to CPU 10, via bus 12 and includes
 the basic input/output system (BIOS) that controls the basic computer
 functions. Random access memory (RAM) 14, I/O adapter 18 and
 communications adapter 34 are also interconnected to system bus 12. It
 should be noted that software components including the operating system 41
 and the browser and HTML applications 40 are loaded into RAM 14 which is
 the computer system's main memory. I/O adapter 18 may be a small computer
 system interface (SCSI) adapter that communicates with the disk storage
 device 20, i.e. a hard drive. Communications adapter 34 interconnects bus
 12 with an outside network enabling the workstation to communicate with
 web servers over a local area network (LAN), wide area network (WAN) which
 includes, of course, the internet or World Wide Web. I/O devices are also
 connected to system bus 12 via user interface adapter 22 and display
 adapter 36. Keyboard 24, trackball 32 and mouse 26 are all interconnected
 to bus 12 through user interface adapter 22. Display adapter 36 includes a
 frame buffer 39 which is a storage device that holds a representation of
 each pixel on the display screen 38. Images may be stored in frame buffer
 39 for display on monitor 38 through various components such as a digital
 to analog converter (not shown) and the like. By using the aforementioned
 I/O devices, a user is capable of inputting event data and other
 information to the system through the keyboard 24, trackball 32 or mouse
 26 and receiving output information from the system via display 38.
 An embodiment of the present invention will be described which uses HTML to
 document Java APIs. JavaDoc, which is described in additional detail on
 pp. 253 to 256 of the above referenced "Java in a Nutshell" text, is the
 Java documentation generator. It generates API documentation in HTML
 format for a specified Java package. It parses the source files, reads
 them and generates HTML documentation files that describe each public
 class defined in the source files. An HTML file at its most basic level
 is, in effect, a blueprint of how the web page is to be constructed. As
 set forth above, because of the increased function for which web pages are
 used, their structure and syntax have grown substantially. With the use of
 comprehensive web technologies such as Forms, Java and JavaScript for
 example, an increasing number of HTML tags are required in an HTML file as
 well as additional text. HTML files have increased to the point that it is
 not uncommon for an HTML file to be larger than the resources to which it
 refers such as a Java applet or .GIF file. Also adding to the size of the
 HTML file are comments which are typically left in the file by the
 programmer or HTML file generator. These comments add substantially to the
 size of the file without enhancing web page function. Surprisingly, we
 have found that the download time for a transmitted HTML file may be
 substantially reduced by tokenizing the HTML tags before transmission of
 the files to display stations. Because of the increased complexity and
 size of HTML files, the tags within the files have become so numerous that
 their reduction in size by tokenizing significantly reduces download time.
 The present invention is network related. We will not go into great detail
 in describing the networks to which the present invention is applicable.
 U.S. Pat. Nos. 5,295,244, Dev et al. and 5,353,399, Kuwamoto et al.
 adequately detail conventional networks to which the present invention
 would be applicable including appropriate network management and display
 terminal access to such networks. Reference has also been made to the
 applicability of the present invention to a global network such as the
 internet. For details on internet nodes, objects and links, reference is
 made to the above referenced text, Mastering the Internet, G. H. Cady et
 al., published by Sybex, Inc., Alameda, Calif., 1996.
 Any data communication system which interconnects or links computer
 controlled systems or network objects at various sites or network nodes
 defines a communications network. A network may be as simple as two linked
 computers or it may be any combination of LANS (Local Area Networks) or
 WANS (Wide Area Networks). Of course, the internet is a global network of
 a heterogeneous mix of computer technologies and operating systems.
 A generalized diagram of a portion of an internet which the computer 56
 controlled display terminal 57 used for web page or other document display
 of the present invention is connected as shown in FIG. 2. Computer 56 and
 display terminal 57 are the computer system shown in FIG. 1 and connection
 58 (FIG. 2) is the network connection shown in FIG. 1.
 Reference may be made to the above-mentioned text, Mastering the Internet,
 Cady et. al., particularly at pp. 136-147 for typical connections between
 local display workstations to the internet via network servers any of
 which may be used to implement the system on which this invention is used.
 The system embodiment of FIG. 2 is one of these known as a host-dial
 connection. Such host-dial connections have been in use for over 30 years
 through network access servers 53 which are linked 51 to the net 50. The
 servers 53 are maintained by a service provider to the client's display
 terminal 57. The host's server 53 is accessed by the client terminal 57
 through a normal dial-up telephone linkage 58 via modem 54, telephone line
 55 and modem 52. The HTML files are downloaded to display terminal 57
 through controlling computer 56 via the telephone line linkages from
 server 53 which may have accessed them from the internet 50 via linkage
 51.
 The present invention will now be described with respect to the flowcharts
 of FIGS. 3 and 4. In the embodiment, HTML, Java language and code are
 used. These are fully described in the three previously referenced texts.
 FIG. 3 is a chart showing the development of the system of the present
 invention. Step 60: means are provided on network access server 53 in FIG.
 2 for tokenizing each HTML tag in sequence using an appropriate lookup
 table, step 61 which will be described in greater detail subsequently with
 respect to FIG. 4. Since the HTML tags, which are tokenized for
 transmission to the receiving workstation, 56 and 57, FIG. 2, will have to
 be detokenized before use on display 57, means will have to be set up on
 display control computer 56 to detokenize the received data, step 62, FIG.
 3. Also, means must be provided on the network access server 53 for
 parsing any comments out of the HTML files prior to transmission, step 63.
 Then, means are provided on the network access server for doing a general
 text compression on the tokenized HTML files prior to transmission, step
 64. Then, after transmission, means are provided on the receiving
 workstation computer 56, FIG. 2, for doing a general text decompression,
 step 65, which complements the general text compression of step 64.
 Finally, step 66, means are provided for receiving computer 56 for
 converting the detokenized, decompressed HTML into a natural language web
 page on display 57, FIG. 2.
 Now with respect to FIG. 4, we will proceed through more specific steps in
 implementation of the procedures and programs of FIG. 3. In the running of
 the program, the HTML documentation for a particular web page selected by
 an interactive user at receiving workstation 56, 57, FIG. 2, is accessed
 from the internet 50 or World Wide Web by network access server 53. The
 HTML file which represents the web page is stored in its entirety, step
 70, and a copy is made for transmission, step 71. Prior to transmission of
 this copy, the program runs through the sequence of HTML tags, gets the
 next tag, step 72, and tokenizes it, step 73, using a lookup table to
 substantially reduce the quantity of data representing the original HTML
 files which would otherwise have to be transmitted. The following table is
 a representative portion of the lookup table:
 LOOKUP TABLE
 TAG Start End TAG Start End
 &lt;HTML&gt; &lt;HT&gt; &lt;ht&gt; HREF HA n/a
 &lt;BODY&gt; &lt;BD&gt; &lt;bd&gt; BACKGROUND BG n/a
 &lt;HEAD&gt; &lt;HD&gt; &lt;hd&gt; BGCOLOR BC n/a
 &lt;TITLE&gt; &lt;TI&gt; &lt;ti&gt; COLOR CR n/a
 &lt;TABLE&gt; &lt;TB&gt; &lt;tb&gt; SIZE SZ n/a
 &lt;CENTER&gt; &lt;CN&gt; &lt;cn&gt; INPUT IN n/a
 &lt;FORM&gt; &lt;FM&gt; &lt;fm&gt; TYPE TP n/a
 &lt;FONT&gt; &lt;FT&gt; &lt;ft&gt; VALUE VL n/a
 &lt;CAPTION&gt; &lt;CP&gt; &lt;cp&gt; BORDER BD n/a
 &lt;SELECT&gt; &lt;SL&gt; &lt;sl&gt; WIDTH WD n/a
 &lt;TD&gt; &lt;TD&gt; &lt;td&gt; TEXT TX n/a
 &lt;BLOCKQUOTE&gt; &lt;BQ&gt; &lt;bq&gt; NAME NM n/a
 &lt;P&gt; &lt;P&gt; &lt;p&gt; CELLPADDING CL n/a
 &lt;B&gt; &lt;B&gt; &lt;b&gt; LINK LK n/a
 &lt;TT&gt; &lt;TT&gt; &lt;tt&gt; VLINK VL n/a
 &lt;TR&gt; &lt;TR&gt; &lt;tr&gt; ALINK AL n/a
 &lt;APPLET&gt; &lt;AP&gt; &lt;ap&gt; MAXLENGTH ML n/a
 &lt;OPTION&gt; &lt;OP&gt; n/a AREA AR n/a
 The HTML tags are changed in sequence and at each tag, decision step 74, a
 determination is made as to whether the last tag in the document has been
 reached. If No, then the process is returned to step 72 and the next tag
 is obtained. When the last tag is reached, the program moves to step 75
 wherein all comments are removed from the HTML file. The comments in the
 HTML file are of value primarily to programmers or those modifying or
 updating programs. These comments have no value to those browsing the web
 pages and do not justify the strain their transmission puts on downloading
 HTML pages.
 At this point, the tokenized HTML file is compressed, step 76, by using any
 conventional method known in the art for doing general text compression.
 One example of a general text compression method which may be used is that
 described in U.S. Pat. No. 5,051,745, String Searcher and Compressor,
 Using the Same, Katz. After the general compression, the remaining HTML
 file is transmitted to the receiving display 57, via computer 56, FIG. 2,
 via branch A in FIG. 4, step 77. It should be noted that this transmission
 need not be made immediately. This optimized version of the original HTML
 file may be stored for later transmission.
 Upon receiving the transmitted HTML file, a decompression is done, step 78,
 on workstation computer 56 by using the complement of the general text
 compression technique. Then, detokenization is carried out by sequentially
 getting the next tokenized tag, step 80 and detokenizing it via the above
 Lookup Table techniques, step 81. A determination is made, decision step
 82, as to whether the last tag has been reached. If not, the process is
 returned to step 80 and detokenization is continued. When the last tag has
 been detokenized, then, step 83, the HTML file is converted into a natural
 language display on workstation display 57, FIG. 2, by using conventional
 HTML conversion techniques.
 Although certain preferred embodiments have been shown and described, it
 will be understood that many changes and modifications may be made therein
 without departing from the scope and intent of the appended claims.