Abstract:
A data processor controlled display system for displaying documents including natural language text representative of data transmitted in a markup language such as HTML to a display station from a location remote from said station. The HTML tags are tokenized prior to transmission, comments are removed from the HTML file prior to transmission and the file is then subjected to a conventional general text compression. After the HTML file is received at the display station, it is decompressed, detokenized and the resulting HTML file is used to create a document or page on the display.

Description:
TECHNICAL FIELD 
     The present invention relates to computer managed communication networks and particularly to the transmission of documents in markup language format from remote distribution points to computer controlled user interactive display terminals which would be available to interactive users in a networking environment such as the internet or equivalent proprietary or public networks. 
     BACKGROUND OF THE INVENTION 
     The 1990&#39;s decade has been marked by a societal technological revolution driven by the convergence of the data processing industry with the consumer electronics industry. Like all such revolutions, it unleashed a great ripple effect of technological waves. The effect has in turn driven technologies which have been known and available but relatively quiescent over the years. Two of these technologies are the internet-related distribution and object oriented programming systems. The convergence of the electronic entertainment and consumer industries with data processing exponentially accelerated the demand for wide ranging communications distribution channels, and the World Wide Web or internet which had quietly existed for over a generation as a loose academic and government data distribution facility reached “critical mass” and commenced a period of phenomenal expansion. With the expanded accessibility of tens of thousands of programmers to each other, not to mention to potential users of such programs via the expanded internet client base, an obvious need became apparent: cooperative programming systems wherein program developers could coact to continuously expand and enhance existing programs in a distributed programming environment. Object oriented programming offered the solution. With its potentially interchangeable objects or units, object oriented programming systems have found acceptance as the programming system for the internet. In all areas of data processing and communications, as well as the electronic entertainment and consumer industries having anything to do with the internet, there has been a substantial movement to object oriented programming systems and particularly to the Java programming system. A major significance of Java is that it is an internet or World Wide Web distributed programming system where literally thousands of program developers and users are continually upgrading and changing the programs. There arose a need for programmers and users to readily being able to display program documentation in a clear and comprehensive manner in natural language. Hypertext Markup Language (HTML), which had been the documentation language of the internet world wide web for years, offered an answer and more. It offered direct links between pages and other documentation on the web and a variety of related data sources which were at first text and then evolved into media, i.e. “hypermedia”. 
     Now the combination of Java object oriented programming and HTML has taken an even greater advance in Web pages: Java code and programs are being created which will exist on web pages where they offer the user the option of downloading and executing such code or programs when the page is being browsed. These programs are embedded in the web page as applets. 
     With all of these rapidly expanding functions of web pages and like documentation, it should be readily understandable that the demand for web documents has been expanding exponentially in recent years. In addition to the proliferating standard uses of HTML for text and media related world wide web pages for commercial, academic and entertainment purposes, the Java documentation program, JavaDoc, will produce standard HTML files for outputs to computer controlled displays to provide standard natural language displays of the program documentation. Thus, HTML has become the display language of choice for the internet or World Wide Web. It is used there for all forms of display documentation including the markup of hypertext and hypermedia documents, usually stored with their respective documents on an internet or web server in addition to the above described programming distribution. HTML is an application of SGML (Standard Generalized Markup Language), an ISO standard for defining the structure and contents of any digital document. It should be recognized that any of the aspects of the present invention illustrated with respect to HTML would be equally applicable to SGML. For further details on Java, JavaDoc or HTML, reference may be made to the texts “Just Java”, 2nd Edition, Peter van der Linden, Sun Microsystems, Inc., 1997, or “Java in a Nutshell”, 2nd Edition, by David Flanagan, O&#39;Reilly publisher, 1997. 
     It must be recognized that HTML, as well as other markup languages for web pages, originated at a time when the demand for web pages and related documentation was relatively modest. It could be said that the present day demand on web page resources and markup languages was probably inconceivable when these were developed. Accordingly, it is the objective of the present invention to provide implementations which go well beyond conventional general data compression whereby the markup languages used in web page development, transmission and use may be downloaded and used more effectively with less wasted time. 
     SUMMARY OF THE INVENTION 
     The present invention relates to a computer controlled display systems for displaying documents e.g., web pages including natural language text representative of data transmitted to display stations from a location remote from said stations e.g., web servers where the data is stored in a markup language format including tags identifying the contents of said data. One key aspect of the present invention is the provision of means for tokenizing the tags prior to the transmission of data from the server to the display station and then detokenizing said tokenized tags received at the display station whereby said received data is restored to said markup language format. Then, there are means associated with the display station for translating said received data from said markup language format into said displayed natural language text documents. The preferable markup language is of course HTML. Another aspect of the invention is directed to portions of the data to be transmitted, identified by tags, as comments. Means are provided for removing the data portions identified as comments prior to transmitting said data. 
     All of the above expedients may be combined with standard general data compression means for compressing, prior to said transmitting, the data portions remaining after tag tokenizing comments removal, and means for decompressing said compressed data portions received at the display station so that the received data is restored to said markup language format. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a block diagram of a data processing system including a central processing unit and network connections via a communications adapter which is capable of implementing a user interactive workstation on which the received data may be converted into a web page in accordance with the present invention; 
     FIG. 2 is a generalized diagrammatic view of an internet portion upon which the present invention may implemented; 
     FIG. 3 is a flowchart describing the operations involved in substantially reducing HTML file transmission time in accordance with the present invention; and 
     FIG. 4 is a flowchart showing how the operations of FIG. 3 may be implemented in specific programming steps. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
     Before going into the details of specific embodiments, it will be helpful to understand from a more general perspective the various elements and methods which may be related to the present invention. Some of the documents on which the present invention may be used are in the Java Programming system which is an object oriented system utilizing the Java programming language. The Java system and language are extensively familiar to those skilled in the art. The text, “Just Java”, Peter van der Linden, described above comprehensively details the system and language. 
     The HTML language is also described in detail in the above “Just Java” text and particularly at Chapter 7, pp. 249-268, dealing with the handling of web pages with embedded Java programs and also in the text, “Mastering the Internet”, Cady and McGregor, published by Sybex, San Francisco, 1996 and particularly pp. 637-642 on HTML in the formation of web pages. 
     Referring to FIG. 1, a typical data processing system is shown which may be used in conjunction with object oriented software such as Java (™ of Sun Microsystems, Inc.) and HTML in implementing the present invention on the receiving interactive workstation. A central processing unit (CPU), such as one of the PowerPC microprocessors available from International Business Machines Corporation (PowerPC is a trademark of International Business Machines Corporation) is provided and interconnected to various other components by system bus  12 . An operating system  41  runs on CPU  10  and provides control and is used to coordinate the function of the various components of FIG.  1 . Operating system  41  may be one of the commercially available operating systems such as DOS or the OS/2 operating system available from International Business Machines Corporation (OS/2 is a trademark of International Business Machines Corporation). Any conventional network browser system involving HTML language with embedded Java applets forms part of application  40 , runs in conjunction with operating system  41  and provides output calls to the operating system  41  which implements the various functions to be performed by the HTML application  40 . The browser program, in combination with the operating system, provides the basic receiving workstation on which the web pages, according to the present invention, may be implemented. 
     A read only memory (ROM)  16  is connected to CPU  10 , via bus  12  and includes the basic input/output system (BIOS) that controls the basic computer functions. Random access memory (RAM)  14 , I/O adapter  18  and communications adapter  34  are also interconnected to system bus  12 . It should be noted that software components including the operating system  41  and the browser and HTML applications  40  are loaded into RAM  14  which is the computer system&#39;s main memory. I/O adapter  18  may be a small computer system interface (SCSI) adapter that communicates with the disk storage device  20 , i.e. a hard drive. Communications adapter  34  interconnects bus  12  with an outside network enabling the workstation to communicate with web servers over a local area network (LAN), wide area network (WAN) which includes, of course, the internet or World Wide Web. I/O devices are also connected to system bus  12  via user interface adapter  22  and display adapter  36 . Keyboard  24 , trackball  32  and mouse  26  are all interconnected to bus  12  through user interface adapter  22 . Display adapter  36  includes a frame buffer  39  which is a storage device that holds a representation of each pixel on the display screen  38 . Images may be stored in frame buffer  39  for display on monitor  38  through various components such as a digital to analog converter (not shown) and the like. By using the aforementioned I/O devices, a user is capable of inputting event data and other information to the system through the keyboard  24 , trackball  32  or mouse  26  and receiving output information from the system via display  38 . 
     An embodiment of the present invention will be described which uses HTML to document Java APIs. JavaDoc, which is described in additional detail on pp. 253 to 256 of the above referenced “Java in a Nutshell” text, is the Java documentation generator. It generates API documentation in HTML format for a specified Java package. It parses the source files, reads them and generates HTML documentation files that describe each public class defined in the source files. An HTML file at its most basic level is, in effect, a blueprint of how the web page is to be constructed. As set forth above, because of the increased function for which web pages are used, their structure and syntax have grown substantially. With the use of comprehensive web technologies such as Forms, Java and JavaScript for example, an increasing number of HTML tags are required in an HTML file as well as additional text. HTML files have increased to the point that it is not uncommon for an HTML file to be larger than the resources to which it refers such as a Java applet or .GIF file. Also adding to the size of the HTML file are comments which are typically left in the file by the programmer or HTML file generator. These comments add substantially to the size of the file without enhancing web page function. Surprisingly, we have found that the download time for a transmitted HTML file may be substantially reduced by tokenizing the HTML tags before transmission of the files to display stations. Because of the increased complexity and size of HTML files, the tags within the files have become so numerous that their reduction in size by tokenizing significantly reduces download time. 
     The present invention is network related. We will not go into great detail in describing the networks to which the present invention is applicable. U.S. Pat. Nos. 5,295,244, Dev et al. and 5,353,399, Kuwamoto et al. adequately detail conventional networks to which the present invention would be applicable including appropriate network management and display terminal access to such networks. Reference has also been made to the applicability of the present invention to a global network such as the internet. For details on internet nodes, objects and links, reference is made to the above referenced text,  Mastering the Internet,  G. H. Cady et al., published by Sybex, Inc., Alameda, Calif., 1996. 
     Any data communication system which interconnects or links computer controlled systems or network objects at various sites or network nodes defines a communications network. A network may be as simple as two linked computers or it may be any combination of LANS (Local Area Networks) or WANS (Wide Area Networks). Of course, the internet is a global network of a heterogeneous mix of computer technologies and operating systems. 
     A generalized diagram of a portion of an internet which the computer  56  controlled display terminal  57  used for web page or other document display of the present invention is connected as shown in FIG.  2 . Computer  56  and display terminal  57  are the computer system shown in FIG.  1  and connection  58  (FIG. 2) is the network connection shown in FIG.  1 . 
     Reference may be made to the above-mentioned text,  Mastering the Internet,  Cady et. al., particularly at pp. 136-147 for typical connections between local display workstations to the internet via network servers any of which may be used to implement the system on which this invention is used. The system embodiment of FIG. 2 is one of these known as a host-dial connection. Such host-dial connections have been in use for over 30 years through network access servers  53  which are linked  51  to the net  50 . The servers  53  are maintained by a service provider to the client&#39;s display terminal  57 . The host&#39;s server  53  is accessed by the client terminal  57  through a normal dial-up telephone linkage  58  via modem  54 , telephone line  55  and modem  52 . The HTML files are downloaded to display terminal  57  through controlling computer  56  via the telephone line linkages from server  53  which may have accessed them from the internet  50  via linkage  51 . 
     The present invention will now be described with respect to the flowcharts of FIGS. 3 and 4. In the embodiment, HTML, Java language and code are used. These are fully described in the three previously referenced texts. FIG. 3 is a chart showing the development of the system of the present invention. Step  60 : means are provided on network access server  53  in FIG. 2 for tokenizing each HTML tag in sequence using an appropriate lookup table, step  61  which will be described in greater detail subsequently with respect to FIG.  4 . Since the HTML tags, which are tokenized for transmission to the receiving workstation,  56  and  57 , FIG. 2, will have to be detokenized before use on display  57 , means will have to be set up on display control computer  56  to detokenize the received data, step  62 , FIG.  3 . Also, means must be provided on the network access server  53  for parsing any comments out of the HTML files prior to transmission, step  63 . Then, means are provided on the network access server for doing a general text compression on the tokenized HTML files prior to transmission, step  64 . Then, after transmission, means are provided on the receiving workstation computer  56 , FIG. 2, for doing a general text decompression, step  65 , which complements the general text compression of step  64 . Finally, step  66 , means are provided for receiving computer  56  for converting the detokenized, decompressed HTML into a natural language web page on display  57 , FIG.  2 . 
     Now with respect to FIG. 4, we will proceed through more specific steps in implementation of the procedures and programs of FIG.  3 . In the running of the program, the HTML documentation for a particular web page selected by an interactive user at receiving workstation  56 ,  57 , FIG. 2, is accessed from the internet  50  or World Wide Web by network access server  53 . The HTML file which represents the web page is stored in its entirety, step  70 , and a copy is made for transmission, step  71 . Prior to transmission of this copy, the program runs through the sequence of HTML tags, gets the next tag, step  72 , and tokenizes it, step  73 , using a lookup table to substantially reduce the quantity of data representing the original HTML files which would otherwise have to be transmitted. The following table is a representative portion of the lookup table: 
     
       
         
               
               
               
               
               
               
             
           
               
                 LOOKUP TABLE 
               
               
                   
               
               
                 TAG 
                 Start 
                 End 
                 TAG 
                 Start 
                 End 
               
               
                   
               
             
             
               
                 &lt;HTML&gt; 
                 &lt;HT&gt; 
                 &lt;ht&gt; 
                 HREF 
                 HA 
                 n/a 
               
               
                 &lt;BODY&gt; 
                 &lt;BD&gt; 
                 &lt;bd&gt; 
                 BACKGROUND 
                 BG 
                 n/a 
               
               
                 &lt;HEAD&gt; 
                 &lt;HD&gt; 
                 &lt;hd&gt; 
                 BGCOLOR 
                 BC 
                 n/a 
               
               
                 &lt;TITLE&gt; 
                 &lt;TI&gt; 
                 &lt;ti&gt; 
                 COLOR 
                 CR 
                 n/a 
               
               
                 &lt;TABLE&gt; 
                 &lt;TB&gt; 
                 &lt;tb&gt; 
                 SIZE 
                 SZ 
                 n/a 
               
               
                 &lt;CENTER&gt; 
                 &lt;CN&gt; 
                 &lt;cn&gt; 
                 INPUT 
                 IN 
                 n/a 
               
               
                 &lt;FORM&gt; 
                 &lt;FM&gt; 
                 &lt;fm&gt; 
                 TYPE 
                 TP 
                 n/a 
               
               
                 &lt;FONT&gt; 
                 &lt;FT&gt; 
                 &lt;ft&gt; 
                 VALUE 
                 VL 
                 n/a 
               
               
                 &lt;CAPTION&gt; 
                 &lt;CP&gt; 
                 &lt;cp&gt; 
                 BORDER 
                 BD 
                 n/a 
               
               
                 &lt;SELECT&gt; 
                 &lt;SL&gt; 
                 &lt;sl&gt; 
                 WIDTH 
                 WD 
                 n/a 
               
               
                 &lt;TD&gt; 
                 &lt;TD&gt; 
                 &lt;td&gt; 
                 TEXT 
                 TX 
                 n/a 
               
               
                 &lt;BLOCKQUOTE&gt; 
                 &lt;BQ&gt; 
                 &lt;bq&gt; 
                 NAME 
                 NM 
                 n/a 
               
               
                 &lt;P&gt; 
                 &lt;P&gt; 
                 &lt;p&gt; 
                 CELLPADDING 
                 CL 
                 n/a 
               
               
                 &lt;B&gt; 
                 &lt;B&gt; 
                 &lt;b&gt; 
                 LINK 
                 LK 
                 n/a 
               
               
                 &lt;TT&gt; 
                 &lt;TT&gt; 
                 &lt;tt&gt; 
                 VLINK 
                 VL 
                 n/a 
               
               
                 &lt;TR&gt; 
                 &lt;TR&gt; 
                 &lt;tr&gt; 
                 ALINK 
                 AL 
                 n/a 
               
               
                 &lt;APPLET&gt; 
                 &lt;AP&gt; 
                 &lt;ap&gt; 
                 MAXLENGTH 
                 ML 
                 n/a 
               
               
                 &lt;OPTION&gt; 
                 &lt;OP&gt; 
                 n/a 
                 AREA 
                 AR 
                 n/a 
               
               
                   
               
             
          
         
       
     
     The HTML tags are changed in sequence and at each tag, decision step  74 , a determination is made as to whether the last tag in the document has been reached. If No, then the process is returned to step  72  and the next tag is obtained. When the last tag is reached, the program moves to step  75  wherein all comments are removed from the HTML file. The comments in the HTML file are of value primarily to programmers or those modifying or updating programs. These comments have no value to those browsing the web pages and do not justify the strain their transmission puts on downloading HTML pages. 
     At this point, the tokenized HTML file is compressed, step  76 , by using any conventional method known in the art for doing general text compression. One example of a general text compression method which may be used is that described in U.S. Pat. No. 5,051,745,  String Searcher and Compressor, Using the Same,  Katz. After the general compression, the remaining HTML file is transmitted to the receiving display  57 , via computer  56 , FIG. 2, via branch A in FIG. 4, step  77 . It should be noted that this transmission need not be made immediately. This optimized version of the original HTML file may be stored for later transmission. 
     Upon receiving the transmitted HTML file, a decompression is done, step  78 , on workstation computer  56  by using the complement of the general text compression technique. Then, detokenization is carried out by sequentially getting the next tokenized tag, step  80  and detokenizing it via the above Lookup Table techniques, step  81 . A determination is made, decision step  82 , as to whether the last tag has been reached. If not, the process is returned to step  80  and detokenization is continued. When the last tag has been detokenized, then, step  83 , the HTML file is converted into a natural language display on workstation display  57 , FIG. 2, by using conventional HTML conversion techniques. 
     Although certain preferred embodiments have been shown and described, it will be understood that many changes and modifications may be made therein without departing from the scope and intent of the appended claims.