Annotation data filtering of computer files

A method and apparatus for annotation filtering of computer files. An embodiment of a method for annotation filtering of computer files includes receiving a serial data stream input, where the serial data stream input representing one or more computer files. A data type description of a computer file is detected in the data stream, and an annotation is detected in the computer file. A determination is made whether to continue processing of the annotation.

TECHNICAL FIELD

Embodiments of the invention generally relate to the field of computer systems and, more particularly, to a method and apparatus for annotation data filtering of computer files.

BACKGROUND

Computer files, such as Java class files, may have specific standard formats. The standard formats of computer may limit the data that can be provided in relation to the files. For this reason, annotations may be provided to add additional information regarding computer files. Annotations may potentially be found anywhere within a set of computer files.

In a particular example, Java allows annotations to Java class files, with the practice now being specifically described in annotations under Java release 5.0 (Java under the JDK (Java Development Kit) 5.0) as provided in JSR-175 recommendation regarding code annotations. The annotations may add guidance regarding certain class files. Thus, a Java class file may include one or more annotations associated with program elements.

It may become necessary or useful to filter program files in order to obtain annotation values and annotated program elements that may be present in the files. However, the filtering of the program files may require a significant amount of processing time because the annotations may be scattered throughout the computer files in an unpredictable manner. Further, the filtering process may be not easily adaptable to dynamic changes in filtering requirements that may occur as annotations are examined in the process.

SUMMARY OF THE INVENTION

A method and apparatus are provided for annotation data filtering of computer files.

In a first aspect of the invention, an embodiment of a method for annotation filtering of computer files includes receiving a serial data stream input, where the serial data stream input representing one or more computer files. A data type description of a computer file is detected in the data stream, and an announcement of an annotation is detected in the computer file. A determination is made whether to continue processing of the annotation.

In a second aspect of the invention, an embodiment of an annotation filtering system includes a data scanning module, where the data scanning module is to receive a serial data stream input containing one or more computer files and to filter out annotations of interest in the one or more computer files. The data scanning module is to detect a data type description of a computer file in the data stream, detect an announcement of an annotation in the computer file, and determine whether to continue processing of the annotation. The system further includes a data handling module, where the data listening module is to receive detected annotation data from the data scanning module and to generate a data stream output containing the detected annotation data.

DETAILED DESCRIPTION

Embodiments of the invention are generally directed to type dependency processing of computer files.

As used herein:

“Annotation” means additional information or metadata that is associated with or attached to a particular point in a computer program. The term annotation includes formal and informal annotation methods, including, but not limited to, annotations under Java release 5.0 (Java under the JDK (Java Development Kit) 5.0) as provided in JSR (Java Specification Request)175recommendation regarding code annotations (“A Metadata Facility for the Java Programming Language”).

“Data type” means a classification of a computer file. Data type includes, but is not limited to, a type of a Java class file.

“Computer file” means any file structure used in a computer system. Computer files include files with specific required structures, including Java class files.

“Class file” means a Java class file. A Java class file is a defined format for compiled Java code, which may then be loaded and executed by any Java virtual machine. The format and structure for a Java class file is provided in JSR 000202, Java Class File Specification Update (Oct. 2, 2006) and subsequent specifications.

“Traversal” means a process for progressing through the elements of a computer system, including a process for progressing through the elements of a computer archive.

“Archive” means a single file that may contain one or more separate files. An archive may also be empty. The files within an archive are extracted, or separated, from the archive for used by a computer program. The files contained within an archive are commonly compressed, and the compressed files are decompressed prior to use. An archive may further include data required to extract the files from the archives “Archive” may also refer to act of transferring one or more files into an archive.

In an embodiment of the invention, computer files are filtered to obtain needed annotations found within the computer file. In an embodiment, the computer files are in the form of a serial data stream, with the data stream being filtered to separate annotations that are of interest in the computer files. In one embodiment, a set of computer files includes a set of Java class files. In an embodiment of the invention, computer files are extracted into a neutral format to allow efficient filtering of annotations.

In an embodiment of the invention, a dynamic annotation filtering process includes receiving a serial data stream input, where the serial data stream input is representing one or more computer files. In particular, the computer files may be Java class files that have been converted into a serial data stream. In an embodiment, the serial data stream may include a data type description prior to any annotations relating to the data type, and may include an annotation announcement prior to an annotation value. In this process, a data type description for a computer file may be detected in the data stream, and an announcement of an annotation is detected in the computer file. A determination is then made whether to continue processing of the annotation. In an embodiment, the annotation is skipped if the annotation is not of interest, with the process then continuing with detecting additional annotations in the computer file, if any. In an embodiment, the program elements in a computer file are skipped if the program elements are not of interest, with the process then continuing with detecting data type descriptions of additional computer files in the data stream, if any.

If the processing of an annotation is continued, then the processing of the annotation includes obtaining an annotation value in the computer file, and determining whether to continue processing of the annotation value. If processing of the annotation value is continued, then an annotated program element of the computer file associated with the annotation value is obtained, and an output is generated that includes the annotated program element and the annotation value.

In an embodiment of the invention, a processing system includes a scanning module to scan the data stream input and output a data stream that includes selected annotated elements and annotation values. In an embodiment, the processing system further includes a handler module to handle the data stream output. In an embodiment, the handler module further provides feedback to the scanning module to direct the selection of annotations by the scanning module. In an embodiment, the file processor may include additional handlers to provide other functions. In one embodiment of the invention, a dedicated, independent processing module is provided for annotation processing, but embodiments of the invention are not limited to a dedicated module implementation.

In an embodiment of the invention, a scanner operates by sending data to the handler as it is identified in the data stream. In an embodiment, a type definition is thus received before receiving any of the program elements within the type, and thus also prior to any annotation value for the program elements within the type. In this manner, the handler may receive a type definition and may provide a callback to the scanner if the handler is not interested in annotations for type definition. If the handler indicates that it is not interested in any annotations for the type definition, the scanner may then skip any annotation values for the type definition.

In an embodiment of the invention, a set of computer files are scanned in a single pass as a serial data stream without requiring multiple readings of the file data. In an embodiment, the same serial data stream format is maintained both on input and output, thereby allowing further processing of computer files without further file conversion.

In an embodiment, the conversion of the data into a data stream allows processing without any dependency on random access files, and broadens the applicable scope of the process for the input. In an embodiment, the processing of class files as a data stream allows processing without requiring use of, for example, Java library utilities that may normally be required to conduct the file processing.

In an embodiment of the invention, the conversion of computer files to a data stream allows for the use of a protocol for both the data producer (the computer file processor) and the data consumer without creating a complete file representation, thereby simplifying the data structure. In an implementation for Java class files, the processing system operates with a class file data model, without requiring the addition of any major abstraction for data processing.

In an embodiment, the conversion of computer files to a serial data format may include, but is not limited to, the operation of a traversal of a hierarchical data structure or of a data archive as provided respectively in patent application Ser. No. 11/648,065, entitled “Computer File System Traversal”, filed Dec. 30, 2006. Other processes for conversion of a set of files to a serial data stream may also be utilized in embodiments of the invention.

In an embodiment of the invention, processing is designed to provide sufficient performance for overall computer file processing. For example, in an embodiment a system includes stream buffering to buffer data as it is obtained and processed. In addition, an embodiment of the invention provides a compact internal file state in the data stream, thereby minimizing the amount of data that will be required in the process of transferring and processing the computer files.

In an embodiment of the invention, a data scanner may be provided in multiple implementations, depending on the system requirements. A data scanner may be a portion of a file processor. In one example, native processing implementations may be provided for a computer file scanner, with the native implementations being based upon relevant Java standards. In another example, a non-native implementation may be provided, as required. A particular non-native implementation may include a BCEL (Byte Code Engineering Library) implementation, with the BCEL API being a toolkit for the static analysis and dynamic creation or transformation of Java class files.

In an embodiment of the invention, a data consumer that receives output data is a main framework extension point for which neutral utility implementations might be required. In an embodiment of the invention, a file processor (the data producer) operates using the same data protocol as the data consumer protocol. In an embodiment of the invention, the data consumer may have control over the data to be provided to the data consumer. In an embodiment, the data producer and the data consumer may cooperate to agree on the data to be provided from the serial data stream. In an embodiment of the invention, a system may include complexity control, including configuring the file processor to deliver the data of interest. In an embodiment, the data of interest includes data meeting a certain degree of detail, or certain types of data. In an embodiment of the invention, the structure of the data processing may allow for a system to be utilized with loose semantics and implementation constraints. For example, the technical framework and protocol data types may be defined. However, there may be leeway for implementation characteristics, such as the result order sequence and analysis capabilities.

In an embodiment of the invention, file processing may be included within a set of tools that are provided to search files. The tools may, for example, provide for the conversion of files into serial form by a traversal process, the scanning of data for desired elements, and other related processes.

FIG. 1is an illustration of annotation filtering of computer files. In this illustration, a data stream125is received, the data stream representing a serial version of a set of computer files. In particular, the computer files may be Java class files, which have been converted into a serial data stream. In a particular embodiment, the Java class files are arranged in a serial format such that type definition is encountered prior to any program elements in a class file, and such that an annotation announcement is made prior to reading any annotation value. In this process, a data type, such as a type definition for a Java class file is encountered105. The process continues130by determining if an annotation is present115. If not, then the process may continue to the next data type definition, such as the next class file.

If an annotation is encountered, it is announced115, and the process may continue135with obtaining the annotation value120and the annotated program element110. Upon encountering the annotation, the process may alternatively skip the annotation150if the annotation is not of interest, with the process remaining with the program element110to determine whether another annotation is encountered and announced115, or skip the program elements if the elements in the data type are not of interest, with the process continuing to determine whether another data type105is encountered. Upon encountering the annotation value120, the process may alternatively skip the annotation150if the annotation is not of interest or skip the program elements if the elements in the data type are not of interest. The process may stop155if there are no more data elements encountered in the data stream125or if another condition occurs that requires the stopping of the filtering process.

FIG. 2is an illustration of an embodiment of processing of computer file data. In this illustration, a computer file conversion module205is provided to convert computer file data200into a serial data stream210. The computer file data200may be, but is not limited to, Java class file program elements. The conversion of the computer file data may include, but is not limited to, the traversal of a hierarchical file or archive. The output of the processing of computer file data is a serial data stream210representing the computer file data.

In an embodiment, the serial data stream includes one or more annotations. For example, the data stream210is illustrated as a series of data elements arriving as a data stream230. In this data stream, there is a type definition prior to any program elements within the type, and an annotation announcement is made prior to any annotations. For example, Type1235is a first type definition, which is followed by program element Element1240within Type1. Element1240is associated with an annotation, with ANNO1245being a first annotation descriptor for annotation value AnnoValue1250. The data stream further includes a second type definition Type2255, which includes program elements Element2260and Element3265. Element3265is associated with a second annotation, as shown by second annotation descriptor ANNO2270and annotation value AnnoValue2275.

In an embodiment of the invention, the serial data stream210then is provided to a data scanner215, which processes the data, including scanning the data stream for data elements of interest, including annotations to the data elements within the data stream. The scanner215may contain multiple modules or sub-modules, depending on the particular embodiment. The scanner215outputs an extracted data stream220, which represents elements of the data stream that have been selected by the scanner215. In this implementation, the extracted data stream would contain the annotated data elements and associated annotation values. The extracted data stream220then is eventually provided to a data consumer225. The consumer225may receive additional reports or data processing as required for the needs of the consumer225.

FIG. 3is an illustration of a computer file processing system300. While this illustration shows the processes occurring within a single system for simplicity in description, the processes may occur in multiple systems, including multiple systems within a network. In this illustration, a computer file data stream input305is provided to a file processor310, which may include a scanner to scan the data for desired program elements. The data stream305may, for example, represent Java class file data that has been converted into a serial data stream. The file processor310may include multiple components, depending on the particular embodiment of the invention. The file processor310generates an extracted computer file data stream315, which may be presented to a data consumer320.

In an embodiment of the invention, the operation of the computer file processing system300is directed by certain inputs and settings. The operation of the file processor310may be directed by a scanner configuration325. In addition, a data mode configuration330affects both the file processor310and the data consumer320. The file processor310also may include one of multiple implementations. In particular embodiments, the implementation may be a native implementation335or a BCEL (Byte Code Engineering Library) implementation340. The BCEL implementation340may include the Apache BCEL process345, as developed by the Apache Software Foundation. In addition, the consumer320may utilize a framework utility350and a framework extension355in the operation of the computer file processing.

FIG. 4is an illustration of an embodiment of a system to process annotated data elements. The system400may include a data scanner410and a data handler420. The data scanner410may, for example, represent the file processor310illustrated inFIG. 3or a subpart of the file processor310. The data handler may represent the data consumer320illustrated inFIG. 3or a subpart of the data consumer310. In this illustration, the data scanner410is to scan a received data stream input405for annotations, and to produce a data stream containing selected annotated data elements and annotation values. The data handler420is to receive and handle the output of the scanner410. The operation of the data handler includes the provision of feedback to the data scanner415. As illustrated, in addition to any other functions, the data handler420provides a callback control flow425to inform the scanner whether particular data elements are desired. For example, the data scanner410may encounter a particular data type, and the data handler420may inform the data scanner410via the callback control flow425that annotations for the particular class type are not of interest. Upon being informed via the callback control flow425that annotations for the particular class type are not of interest, the data scanner410may then skip the elements in class type.

The data scanner410may include a native implementation440and a BCEL implementation445, illustrated with Apache BCEL450. The implementations may be associated with a parsing module to recognize type descriptors and identify the appropriate types. Also illustrated are the scanner configuration430and the data mode configuration435.

FIG. 5is a flowchart to illustrate an embodiment of an annotation data filtering process. In an embodiment of the invention, a set of class files are received as a serial data stream505. The class file type description is read510and there is a determination whether an annotation announcement is encountered in the class file515. If not, then there are no annotations in the class file, and there is a determination whether there are more class files550. If there are more class files, the process continues with reading the type description of the next class file510, and if not then the process stops555.

If an annotation is announced515, then there is determination whether the program elements are of interest. If not, then the process continues with a determination whether there are more class files to be encountered in the data stream550. If the program elements are of interest, then there is a determination whether the annotation is of interest525. If not, then the process continues with a determination whether there are more class files to be encountered in the data stream550. If the annotation is of interest, the annotation value is read530, and there is a determination whether there are program elements of interest535. If not, then the process continues with a determination whether there are more class files to be encountered in the data stream550. If there are program elements of interest, then there is a determination whether the annotation is of interest540. If not, then there is a return to determining whether is an annotation announcement515. If the annotation is of interest, then the annotated program element is read545, and there is a return to determining whether is an annotation announcement515.

FIG. 6is an illustration of an embodiment of a class file program element as a data stream. In this illustration, a class file program element606is shown within a code walk module602(used in the traversal of class files). The class file program element606is represented by an element type608(including an element kind, type name, and type flags), an element field610(also including an element name and element flags), and one or more element methods612(also including a method signature), and an element parameter614(paramldx). The element further includes a class file element record616in the code walk implementation604, including one or more operations (defining field accessors, shallow or deep equality, ordering relation, binary serialization, and XML serialization).

FIG. 7is an illustration of an embodiment of an annotation to a class file program element presented as a data stream. In this illustration, an annotation is represented in the code walk702as a class file annotation value706, including whether the value is visible at runtime, a type name, and the annotation elements. The class file annotation value706is related to a particular named program element708, which includes the element name. The annotated program element710includes the element tag and element value, as well as tag-specific accessors. The program element710is shown in relation to the annotation714, as well as either Boolean716, char (character)718, double, float720, byte, short, int (integer), or long722. The program element further may include a string724or class726, an enum (enumeration constant)728, and an array730. The enumeration constant728is illustrated712as including an enumeration type and enumeration literal.

The annotation is further illustrated as a class file annotation record732in a code walk implementation704. The class file annotation record732includes operations, including shallow or deep equality, the ordering relation, binary serialization, and XML serialization732. The class file annotation record732is shown in relation with the named element734. Also illustrated are the annotated element736and the enumeration constant738.

FIG. 8illustrates an embodiment of library utilities.FIG. 8may illustrate software modules, hardware modules, or modules including a combination of software and hardware. In this illustration, the utilities relate to an interface layer comprising code walk interfaces (code.walk880); for class file processing and file walk interfaces (file.walk810) for locating files; and further to an implementation toolbox comprising code processing850and a code walk implementation (code.walk.impl860) for class file processing, and file processing855and a file walk implementation (file.walk.impl830) for locating files.

In the interface layer, the code walk interfaces880may include a class file annotation value interface module882, a class file program element interface module884, a class file annotation handler interface module886, a class file annotation scanner interface module888, a class file dependency scanner interface module890, and a class file dependency listener interface module892. The file walk interfaces then may include a file condition interface module812, a file name classifier interface module814, a directory walker handler interface module816, a directory walker interface module818, a zip walker handler interface module (“zip” indicating use for archives)820, a zip walker interface module822, and a file notification interface module824.

In an embodiment of the invention, the code processing850may provide for parsing types from class file descriptors. Code processing850may include a class file format helper module852and a class file descriptor parser module. The code walk implementation860for class file processing may include a class file annotation record module862, a class file element record module864, a class file annotation filter866, a class file annotation for native elements868, a class file dependencies module for native elements870, a class file dependencies module for BCEL (Byte Code Engineering Library) elements872, a class file dependency concentrator module874, and a class file dependency filter876.

In an embodiment of the invention, the file processing855may include a comma separated value (CSV) formatter and a CSV scanner. The file walk implementation830for locating files may include a simple file condition module832, a basic file name classifier module834, a directory finder module836, a directory walker implementation module838, a walk recorder module840, a zip (archive) condenser module842, and a zip walker implementation module844.

FIG. 9is an illustration of a computer system in an embodiment of the invention. The computer system may be utilized as a system for processing of computer files in the form of a data stream, or may represent one of multiple systems used in such processing. The computing system illustrated inFIG. 9is only one of various possible computing system architectures, and is a simplified illustration that does include many well-known elements. As illustrated, a computing system900can execute program code stored by an article of manufacture. Computer system900may be a J2EE system, ABAP (Advanced Business Application Program) system of SAP AG, or administration system. A computer system900includes one or more processors905and memory910coupled to a bus system920. The bus system920is an abstraction that represents any one or more separate physical buses, point-to-point connections, or both connected by appropriate bridges, adapters, or controllers. The bus system920may include, for example, a system bus, a Peripheral Component Interconnect (PCI) bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus, sometimes referred to as “Firewire”. (“Standard for a High Performance Serial Bus” 1394-1995, IEEE, published Aug. 30, 1996, and supplements thereto)

As illustrated inFIG. 9, the processors905are central processing units (CPUs) of the computer system900and control the overall operation of the computer system900. The processors905execute software stored in memory910. A processor905may be, or may include, one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), or the like, or a combination of such devices.

Memory910is or includes the main memory of the computer system900. Memory910represents any form of random access memory (RAM), read-only memory (ROM), flash memory, or the like, or a combination of such devices. Memory910stores, among other things, the operating system915of the computer system900.

Also connected to the processors905through the bus system920are one or more mass storage devices925and a network adapter935. Mass storage devices925may be or may include any conventional medium for storing large volumes of instructions and data930in a non-volatile manner, such as one or more magnetic or optical based disks. In an embodiment of the invention, the mass storage devices may include storage of file or an archive932that requires processing. In an embodiment of the invention, the processors905may operate to traverse the files or archive932, the traversal of the files or archive932resulting in output of a serial data stream representing selected elements of the archive. The processor905may scan the serial stream for desired program elements within the computer files. In another embodiment the computer system900may provide for the conversion of the computer files into a serial data stream, while another system or systems is responsible for scanning the data stream for desired program elements.

The network adapter935provides the computer system900with the ability to communicate with remote devices, over a network940and may be, for example, an Ethernet adapter. In one embodiment, the network adapter may be utilized to output data including, for example, an extracted serial data stream representing selected elements of the files or archive932.

FIG. 10illustrates an embodiment of a client-server network system. As illustrated, a network1025links a server1030with client systems1005,1010, and1015. Client1015may include certain data storage1020, including computer files in the form of, for example, a computer file hierarchy or computer archive1022. Server1030includes programming data processing system suitable for implementing apparatus, programs, and/or methods in accordance with one or more embodiments of the present invention. Server1030includes processor1035and memory1040. Server1030provides a core operating environment for one or more runtime systems, including, for example, virtual machine1045, at memory1040to process user requests. Memory1040may include a shared memory area that is accessible by multiple operating system processes executing in server1030. For example, virtual machine1045may include an enterprise server (e.g., a J2EE-compatible server or node, Web Application Server developed by SAP AG, WebSphere Application Server developed by IBM Corp. of Armonk, N.Y., and the like). Memory1040can be used to store an operating system, a Transmission Control Protocol/Internet Protocol (TCP/IP) stack for communicating over network1025, and machine executable instructions executed by processor1035. The memory1045may also include data1050for processing, including the processing of data that includes data of one or more computer file hierarchies or computer archives1052. In an embodiment, the data has been converted into a serial data stream for processing. In some embodiments, server1035may include multiple processors, each of which can be used to execute machine executable instructions.

Client systems1005-1015may execute multiple application or application interfaces. Each instance or application or application interface may constitute a user session. Each user session may generate one or more requests to be processed by server1030. The requests may include instructions or code to be executed on a runtime system, such as virtual machine1045on server1030.

The present invention may include various processes. The processes of the present invention may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor or logic circuits programmed with the instructions to perform the processes. Alternatively, the processes may be performed by a combination of hardware and software.

It should also be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature may be included in the practice of the invention. Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims are hereby expressly incorporated into this description, with each claim standing on its own as a separate embodiment of this invention.