Abstract:
A markup language parser processes markup language data and provides processed data to an application. A reader is configured to tokenize the markup language data and store resulting tokens in a buffer associated with the reader. A scanner is configured to create string objects based on the tokens and to provide the created string objects to an application. The scanner is further configured to determine a need by the application for the string objects. For each at least some of the string objects created by the scanner, the scanner is configured to selectively create those string objects based on a determination of the need by the application for the string objects.

Description:
TECHNICAL FIELD 
     The present invention is in the field of markup language parsers that provide processed markup language data to applications and, in particular, relates to a markup language parser that expends processing resources based at least in part on application request. 
     BACKGROUND 
     A markup language parser is, in a general sense at least, a program to determine the syntactic structure of a string of symbols in markup language. A markup language (or, perhaps more precisely, markup specification language) describes information (text or data), usually for storage, transmission, or processing by a program. The markup language typically does not specify what should be done with the data. 
       FIG. 1  illustrates a conventional markup language parser  100  from one simplistic point of view. In broad terms, the parser  100  processes markup language source from a file  106  and provides processed data for use by one or more applications  101 . From the simplistic point of view illustrated in  FIG. 1 , the parser  100  can be considered to include two primary components—a reader  102  and a scanner  104 . 
     The reader  102  reads the contents of the file  106  (including markup language statements which, in the example, are XML) to be processed and stores the contents into a buffer  108 , typically of fixed predetermined size. If the size of the file  106  is more than the size of the buffer  108 , then the buffer  108  is refreshed with the unread markup language data once the scanner  104  has processed the data that is currently in the buffer  108 . 
     The reader  102  is configured to check for valid markup language characters, tokenize the markup language content (e.g., for XML in one example, to tokenize the markup language content to XMLNames, values and content), and provide the tokens to the scanner  104 . 
     The scanner  104  is configured to process the tokens generated by the reader  102  and to provide string objects and/or values (generically, data  103 ) to the application  101  based on the tokens. For example, the scanner  104  may operate as a state machine. The string objects and/or values provided to the application  101  by the scanner  104  may be, for example, an XMLName (element name, attribute name), attribute value, element content, etc. 
     We now briefly discuss circumstances surrounding the conventional passing of data between the reader  102  and the scanner  104 . The scanner  104  passes pointer objects to the reader  102 . The pointer objects passed by the scanner  104  to the reader  102  are essentially just shells, to be populated by the reader  102 . After processing by the reader  102 , a pointer object points to a token in the buffer  108 , and control is returned to the scanner  104 . More particularly, the pointer object indicates an offset into the buffer  108  as well as the length of the token. Then, depending on the type of token being processed, the scanner  104  processes a populated pointer object to either create string objects or to copy data into a buffer  110  in the scanner  104 . 
     It is desired to streamline the operation of the parser. 
     SUMMARY 
     A markup language parser processes markup language data and provides processed data to an application. A reader is configured to tokenize the markup language data and store resulting tokens in a buffer associated with the reader. A scanner is configured to create string objects based on the tokens and to provide the created string objects to an application. 
     The scanner is further configured to determine a need by the application for the string objects. For each at least some of the string objects created by the scanner, the scanner is configured to selectively create those string objects based on a determination of the need by the application for the string objects. 
    
    
     
       BRIEF DESCRIPTION OF FIGURES 
         FIG. 1  illustrates a conventional configuration of a markup language parser. 
         FIG. 2  illustrates a configuration of a markup language parser in which particular operations of the parser are carried out based on an indication from the application of need for the results of the operations. 
         FIG. 3  illustrates an alternate configuration of the  FIG. 2  markup language parser. 
     
    
    
     DETAILED DESCRIPTION 
     It has been realized that, conventionally, the scanner of a markup language parser executes processing to create string objects or copy data into its buffer (depending upon the type of token being processed) even in situations where the application does not ask for or otherwise require a string object and/or data. In accordance with aspects of the invention, at least a portion of processing by the parser, which formerly was carried out “open loop,” is now carried out by the parser based on a request or other indication of need by the application, for the result of processing by the parser. 
     For example, in accordance with one aspect, the string objects are created by the scanner, to provide to the application(s), when requested by the application. In accordance with another aspect, the values are copied from the buffer in the reader, to the scanner for processing and providing to the application(s), based on an internal state of the scanner or the type of token being read. 
     We now discuss these aspects with reference to embodiments illustrated in  FIG. 2  and  FIG. 3 . 
     Referring first to  FIG. 2 , like the conventional parser  100  illustrated in  FIG. 1 , the  FIG. 2  parser  200  includes a reader  202  and a scanner  204 . Like the buffer  108  in the  FIG. 1  conventional parser  100 , the reader  202  includes a buffer  208  into which contents of an XML file  206  are stored. Also like the  FIG. 1  example, the reader  202  is configured to check for valid markup language characters in the markup language source  206  and to tokenize the markup language content. 
     Also like the scanner  104 , the scanner  204  maintains a register  212  containing a pointer object that references back to the buffer  208  in the reader  202 . For example, the reader  202  may have informed the scanner  204  of offset and length information pertaining to data relevant to a particular token, and the scanner  204  maintains the register  212  containing a pointer object that references back to this data. 
     Unlike the scanner  104 , the scanner  204  typically does not (open loop) copy the pointed-to data from the reader buffer  208  into a buffer associated with the scanner  204  upon completion of processing of the pointer object (i.e., population) by the reader  202 . Rather, when string objects are created by the scanner, the data is accessed directly from the reader buffer  208 . 
     To avoid data being overwritten in the reader buffer  208  before it can be accessed by the scanner  204 , the scanner  204  is notified when the referenced data in the buffer  208  may be no longer available (e.g., will be overwritten in the buffer  208 ). At this point, if it is determined by the scanner  204  that the referenced data will not be needed, then the referenced data need never be copied out of the buffer  208 . 
     In one example, a “listener” mechanism is used to notify the scanner  204  when the referenced data in the buffer  208  may be no longer available. That is, the arrow  214  in  FIG. 2  emanating from listener logic  213  indicates specification by the scanner  204  of a listener object to the reader  202 . When the referenced data in the buffer  208  may be no longer available (e.g., when the reader  202  is about to load new data from the markup language document  206  into the buffer  208 ), the reader  202  gives a callback (arrow  216 ) to the listener logic  213  of the scanner  204 . 
     At this point, the scanner  204  determines whether the referenced data may be needed, based on the internal state of the scanner  204  or the type of token that is being/was read. For example, if the state of the scanner  204  is such that the referenced data has already been processed or is past the state in which the referenced data would have been processed, then the referenced data is no longer needed. If the referenced data in the buffer  208  may be needed, then the referenced data is copied from the buffer  208  of the reader  202  into a buffer  210  of the scanner  204 . 
     Furthermore, whereas in the  FIG. 1  example, objects  103  are created without regard for need by the application(s)  101 , in accordance with another aspect, the objects  203  ( FIG. 2 ) are created when the application(s)  201  asks for data from the parser. Thus, for example, when an application  201  asks for data from the parser, the objects  203  are created using data from the buffer  208  (of the reader  202 ) or from the buffer  210  (of the scanner  204 ) as appropriate. 
       FIG. 3  illustrates an example where the scanner  304  maintains a pool  312  of pointer objects. This is particularly useful where, for example, a token may have multiple attributes in the buffer  308  of the reader  302 . Each pointer object references, for example, a different one of the attributes in the buffer  308 . The processing with respect to the pointer object references, and the data pointed to by the pointer objects, may be similar to that discussed above with reference to  FIG. 2 . As with the  FIG. 2  example, at least some of the string objects  303  are created only when a need is indicated by the application. It is noted that, in the  FIG. 3  example, a buffer like the buffer  210  of the scanner  204 , in the  FIG. 2  example, is not shown. 
     Put another way, in general, there may be two types of buffers, and there are two pools, one for each type of buffer. Both types of buffers are used to store attribute values. For example, in the following string: 
     attrNameOne=“Value1” 
     the buffers may be used to store “Value1”. 
     With regard to the two types of buffers, a first type of buffer is used when the attribute is simple and all that is needed to get the value is the starting and ending offsets to the buffer. This is the type of buffer  212  illustrated in  FIG. 2 . That is, each pointer object in the buffer  212  is just a pointer to a set of characters in the reader buffer  208 . 
     As discussed above, the first type of pointers are converted to string values and read into the scanner  204  in two scenarios. The first scenario is that an attribute is read by the application and the corresponding attribute value is requested by the application. The second scenario is that a callback  216  is received, indicating the buffer  208  is about to be written over. In the second scenario, the attribute value is copied into the buffer  210 . 
     A second type of buffer (illustrated in the simplified view in  FIG. 3 ) is used when attribute values are not simple, and more processing is applied before giving an attribute value to the application. The processing of the attribute value is applied when the scanner  304  is processing the buffer  308  in the reader (again, either due to the attribute being requested by the application  301  or due to a callback  316  being received by listener logic  313 , corresponding to a listener object specified by the scanner  304  to the reader  302  indicated by arrow  314 , indicating that the buffer  308  in the reader is about to be written over. The processed attribute values are stored in a byte array  311  in the scanner. 
     In this case, there is a one-to-one correspondence between the first type of object (in array  312 ) and the second type of object (in byte array  311 ). That is, each of the first type of object (in array  312 ) points to the buffer  308  in the reader  302 . In this case, when a copy is made, the type one object points to a type  2  object (in byte array  311 ) in the scanner  304  that holds the actual values. 
     As described, a scanner of a markup language parser can omit execution of processing to create string objects or copy data into its buffer (depending upon the type of token being processed) in situations where the application does not ask for or otherwise require a string object and/or data.