Detection of encryption and compression applications

Detection of an encryption or compression application program may be based on similarity between read files read by a process of the application program and write files written by the process. Read fingerprints of the read files and write fingerprints of the write files are generated. A listing of the read fingerprints is searched for presence of matching write fingerprints to find matched fingerprints. The similarity is calculated based on the read fingerprints and matched fingerprints.

COPYRIGHT NOTICE

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to computer security, and more particularly but not exclusively to methods and apparatus for detecting applications.

2. Description of the Background Art

Classification of application programs, which is also referred to herein as “applications”, has various uses in the field of computer security. For example, an application may be classified to determine if the application is a computer virus. One way of classifying an application is by behavior monitoring. As a particular example, an application may be monitored to determine if it behaves like a computer virus.

Unfortunately, there is no known general solution for detecting every class of applications. While behavior monitoring is suitable for classifying applications to detect computer viruses, behavior monitoring is not effective in detecting other applications, such as encryption and compression applications.

SUMMARY

In one embodiment, detection of an encryption or compression application program may be based on similarity between read files read by a process of the application program and write files written by the process. Read fingerprints of the read files and write fingerprints of the write files are generated. A listing of the read fingerprints is searched for presence of matching write fingerprints to find matched fingerprints. The similarity is calculated based on the read fingerprints and the matched fingerprints.

DETAILED DESCRIPTION

Referring now toFIG. 1, there is shown a schematic diagram of a computer100in accordance with an embodiment of the present invention. The computer100may have fewer or more components to meet the needs of a particular application. The computer100may include a processor101. The computer100may have one or more buses103coupling its various components. The computer100may include one or more user input devices102(e.g., keyboard, mouse), one or more data storage devices106(e.g., hard drive, optical disk, Universal Serial Bus memory), a display monitor104(e.g., liquid crystal display, flat panel monitor, cathode ray tube), a computer network interface105(e.g., network adapter, modem), and a main memory108(e.g., random access memory). The computer network interface105may be coupled to a computer network109, which in this example includes the Internet.

The computer100is a particular machine as programmed with software modules, which comprise computer-readable program code stored non-transitory in the main memory108for execution by the processor101. In the example ofFIG. 1, the software modules include an encryption/compression application detector201, processes203(i.e.,203-1,203-2, . . . ,203-n), and other software modules not shown (e.g., operating system). The processes203are processes of various application programs that are currently running.

The detector201may comprise computer-readable program code that determines whether any of the processes203is a process of an encryption/compression application, i.e., an application for performing file encryption, file compression, or both. In other embodiments, the detector201may be implemented as programmed logic (e.g., as an application specific integrated circuit), in firmware, or other implementation.

Generally speaking, file encryption/compression involves reformatting a file in a reversible way. More specifically, a file encryption application may encrypt an input file into an encrypted file. The encrypted file may be decrypted back into the input file using a key and the appropriate decryption algorithm. Compression is similar to encryption but also involves reducing the size of the input file. Typical compression applications reformat the input file into a smaller compressed file. Compression may also require a key, a particular decompression algorithm, or both, to decompress the compressed file back into the input file.

Encryption/compression applications may be employed to perform data leakage or other cybercrime. To get around data leakage prevention (DLP) technology, cybercriminals may pre-process stolen confidential information, such as personal information, company trade secrets, and the like, by using an encryption or compression application to encrypt or compress a file containing the confidential information. The resulting encrypted or compressed file can't be properly scanned by DLP technology without the requisite key or algorithm to restore the file back to its original format.

It is also not easy to detect encryption/compression applications. Detection by application name is not feasible because application names can be readily changed. Furthermore, some encryption/compression applications are not well known, so their names and other identities are not readily recognizable.

In one embodiment, the detector201monitors a process203running in the computer100, collects files that are read by the process203(also referred to herein as “read files”), collects files that are written by the process203(also referred to herein as “write files”), generates fingerprints of the read files (also referred to herein as “read fingerprints”), generates fingerprints of the write files (also referred to herein as “write fingerprints”), and determines similarity between the read files and the write files from the read fingerprints and the write fingerprints. In one embodiment, the detector201deems a process203to be that of an encryption/compression application if the similarity is less than a similarity threshold. The detector201may perform various responsive actions upon detection of an encryption/compression application including informing another program, such as a DLP program, or an administrator, for example.

FIG. 2shows a flow diagram of a method200of generating a listing of read files R and a listing of write files W in accordance with an embodiment of the present invention. In one embodiment, the method200is performed by the detector201. Other components may also be employed without detracting from the merits of the present invention.

In the example ofFIG. 2, the detector201performs process monitoring (see arrow210) to monitor the activities of processes203of various applications running in the computer100. The detector201may monitor file operations and process operations of the processes203. For performance reasons, the detector201may employ a whitelist of particular known processes (see arrow211) that are excluded from monitoring, such as processes of web browsers and compilers. The whitelist may contain process identifiers of known applications that do not perform encryption or compression.

Upon detection of creation of a process203(see arrow212), the detector201allocates storage areas (e.g., buffer, array) to hold a listing of read files R and a listing of write files W (see arrow213) for the process203. In one embodiment, each process203has its own listings of read files R and read files W. Upon detection of exiting of a process203(see arrow214), the detector201cleans up (e.g., by erasing) the storage areas allocated for that process's203listings of read files R and write files W (see arrow215). In one embodiment, under the Microsoft Windows™ operating system, process creation and exiting may be detected by registering a callback in the kernel to get process create and exit events. A similar callback may be set to get process creation and exit events in other platforms.

Upon detection of a read or write operation of a process203, the detector201may apply filter conditions to determine if the file being read or written by the process203is to be included in the corresponding listing of read files R or listing of write files W (see arrow216). In one embodiment, the filtering improves the accuracy and performance of the detector201by selectively including in the listings only those files that are normally accessed by the process203for read and write operations for editing. In one embodiment, the detector201only collects edit files for inclusion in the listing of read files R and listing of write files W, and ignores image files. Examples of edit files include Microsoft Word™ doc files, Microsoft Excel™ xls files, Adobe Acrobat™ pdf files, Microsoft Notepad™ cpp files, and the like.

In one embodiment, the detector201ignores image files, such as executable files (e.g., exe files and dynamic link library (DLL) files) that are loaded by a process203. Image files may be identified by location and type. For example, the filtering conditions may indicate ignoring files located in the same folder as the process's203image file, files in the same folder as public libraries, and files of particular types (e.g., exe, DLL, so, and lib file types). Files ignored by the detector201as indicated by the filtering conditions are not collected for inclusion in the listing of read files R and the listing of write files W.

File create, read, write, and close operations performed by a process203are monitored by the detector201. Under the Microsoft Windows™ operating system, the detector201may include or communicate with a driver to monitor file operations. A file operation may include a process ID (identifier) that identifies the process203performing the file operation.

The detector201may collect files that have been read and written by a process203. In the example ofFIG. 2, a file that has been read by a process203and has not been filtered out (see arrow217) is included in a listing of read files R of that process203(see arrow218). A listing of read files R may be represented as R={r1, r2, . . . , rn}, where rnis a read file, i.e., a file that has been read by the corresponding process203. Similarly, a file that has been written by the process203and has not been filtered out (see arrow219) is included in a listing of write files W of the process203(see arrow220). A listing of write files W may be represented as W={w1, w2, . . . , wm}, where wnis a write file, i.e., a file that has been written by the corresponding process203.

FIG. 3shows a flow diagram of a method300of detecting an encryption/compression application in accordance with an embodiment of the present invention. In one embodiment, the method300is performed by the detector201. Other components may also be employed without detracting from the merits of the present invention. In the example ofFIG. 3, the method300is illustrated as classifying a single process203in the interest of clarity. It is to be noted that in practice, the method300may be employed to classify a plurality of processes203.

In the example ofFIG. 3, the detector201generates the listing of read files R by collecting files that have been read by the process203(see arrow303). The detector201also generates the listing of write files W by collecting files that have been written by the process203(see arrow305). In one embodiment, only files that have not been filtered out are collected for inclusion in the listing of read files R and listing of write files W.

In the example ofFIG. 3, the detector201includes a fingerprint generation engine350that generates a set of read fingerprints frnfor each read file rn(see arrow304) and a set of write fingerprints fwmfor each write file wm(see arrow306). In one embodiment, a set of read fingerprints frnis a vector and may be represented as
frn={frn1,frn2, . . . ,frnk(n)}  (EQ. 1)
where k(n) is the total number of read fingerprints for the corresponding read file rn. Similarly, a set of write fingerprints fwmmay be a vector and may be represented as
fwm={fwm1,fwm2, . . . ,fwmk(m)}  (EQ. 2)
where k(m) is the total number of write fingerprints for the corresponding write file wm. That is, for the listing of read files R={r1, r2, . . . , rn}, the fingerprint generation engine350may generate a listing of read fingerprints FR={fr1, fr2, . . . , frn}, with fr1being a set of read fingerprints of the read file r1, fr2being a set of read fingerprints of the read file r2, and so on. Likewise, for the listing of write files W={w1, w2, . . . , wm}, the fingerprint generation engine350may generate a listing of write fingerprints FW={fw1, fw2, . . . , fwm}, with fw1being a set of write fingerprints of the write file w1, fw2being a set of write fingerprints of the write file w2, and so on. The listing of read fingerprints FR may thus be represented as
FR={fr1,fr2, . . . ,frn}  (EQ. 3)
FR={fr11,fr12, . . . ,fr1k(1),fr21,fr22, . . . ,fr2k(2), . . . ,frn1,frn2, . . . ,frnk(n)}  (EQ. 4)
Similarly, the listing of write fingerprints FW may be represented as
FW={fw1,fw2, . . . ,fwm}  (EQ. 5)
FW={fw11,fw12, . . . ,fw1k(1),fw21,fw22, . . . ,fw2k(2), . . . ,fwm1,fwm2, . . . ,fwmk(m)}  (EQ. 6)
In one embodiment, the listing of read fingerprints FR={fr1, fr2, . . . , frn} and listing of write fingerprints FW={fw1, fw2, . . . , fwm} are in strictly ascending order.

Generally speaking, a set of file fingerprints is a representation of the input file and is unique to the input file; a different input file will result in a different set of file fingerprints. In one embodiment, the fingerprint generation engine350creates a unique set of fingerprints for each read file and unique set of fingerprints for each write file, with each set of fingerprints being stable with changes to the corresponding read file and write file. That is, the set of fingerprints is not only unique to the input file, which is a read or write file in this case, but also does not change even with some changes to the input file. The stability of the set of fingerprints with respect to changes to the input file depends on the algorithm employed to generate the fingerprint.

In one embodiment, the fingerprint generation engine350employs the fingerprinting algorithm disclosed in commonly-assigned U.S. Pat. No. 8,359,472, which is incorporated herein by reference in its entirety. The fingerprinting algorithm disclosed in U.S. Pat. No. 8,359,472 includes normalizing a text string, applying a first hash function with sliding hash window to the normalized text string to generate an array of hash values, applying a first filter to the array of hash values to select candidate anchoring points, applying a second hash function to the candidate anchoring points to select anchoring points, and applying a second hash function to substrings located at the selected anchoring points to generate hash values for use as fingerprints. Other suitable fingerprinting algorithms may also be employed.

In the example ofFIG. 3, the detector201further includes a search engine351. In one embodiment, the search engine351searches the listing of read fingerprints for presence of each of the write fingerprints that are included in the listing of write fingerprints. That is, the search engine351searches FW={fw1, fw2, . . . , fwm} in FR={fr1, fr2, . . . , frn} to find matched fingerprints. Searching for the write fingerprints in the listing of read fingerprints yields a listing of matched fingerprints F={f1, f2, . . . , ft}, with a matched fingerprint ftbeing a fingerprint that is found in both the listing of read fingerprints FR and listing of write fingerprints FW. The search engine351may employ a dual binary search algorithm to find the matched fingerprints. More particularly, in one embodiment, the search engine351employs the search algorithm disclosed in commonly-assigned U.S. Pat. No. 8,266,150, which is incorporated herein by reference in its entirety. An example pseudo-code for a binary search algorithm that may be employed by the search engine351is disclosed in Appendix A. Other suitable search algorithms may also be employed.

Continuing withFIG. 3, the detector201performs a similarity calculation to determine the similarity between the files read by the process203and the files written by the process203(see block352). In one embodiment, the detector201performs the similarity calculation based on the matching read fingerprints and write fingerprints, which in the example ofFIG. 3are included in the listing of matched fingerprints F={f1, f2, . . . , ft}. The listing of matched fingerprints F provide an indication of how similar the read files are to the write files. By calculating the ratio of the number of matched fingerprints (see arrow310) to the number of read fingerprints (see arrow311), the detector201is able to determine the percentage of read files that are similar to the write files. In one embodiment, the detector201determines the similarity S as follows

S=FFR(EQ.⁢7)
where |*| is the size of the vector/listing. For example, assuming there are 1,000 matched fingerprints and there are 2,000 read fingerprints, the similarity S between read files and write files is equal to 0.5, or 50%, meaning half of the files read by the process203are similar to the files written by the process203.

The detector201may compare a calculated similarity value to a similarity threshold to determine if a process203is that of an encryption/compression application (see arrow312). In one embodiment, the detector201deems that a process203with a similarity that is less than a similarity threshold is a process of an encryption/compression application. For example, assuming a similarity threshold X % is set to 20%, a process203with a similarity value of less than 20% is deemed to be a process of an encryption/compression application. Otherwise, when the similarity value of the process203is greater than 20% in that example, the process203is deemed to be of some other application that is not an encryption/compression application.

The similarity threshold may be adjusted depending on the particulars of the processes203, the computer platform, and the fingerprinting algorithm employed. For example, experiments performed by the inventors indicate that in personal computers running the Microsoft Windows™ operating system, a similarity threshold X % may be set to 20%. In that case, a process203with a similarity less than 20% is deemed to be a process of an encryption/compression application.

FIG. 4shows a flow diagram of a method400of detecting an encryption/compression application in accordance with an embodiment of the present invention. The method400may be performed by the detector201. Other components may also be employed without detracting from the merits of the present invention.

In the example ofFIG. 4, the detector201monitors a plurality of processes203running on the computer100(step401). For a process203in the plurality of processes203, the detector201performs file filtering to determine if a file being read by the process203, i.e., a read file, is an edit file or an image file (step402). The detector201also performs the same filtering for files written by the process203, i.e., write files. In one embodiment, the detector201only collects edit files and ignores image files. The detector201adds edit files read and written by the process203into a listing of read files and a listing of write files, respectively (step403). The detector201generates fingerprints of the read files and the write files (step404). In one embodiment, the detector201generates a set of read fingerprints for each read file and a set of write fingerprints for each write file. The read fingerprints are included in a listing of read fingerprints and the write fingerprints are included in a listing of write fingerprints. The detector201searches the listing of read fingerprints for matching write fingerprints (step405). A write fingerprint that matches a read fingerprint, i.e., a matched fingerprint, is included in a listing of matched fingerprints. The detector201thereafter calculates a similarity value that indicates similarity between the read and write files of the process203. In the example ofFIG. 4, the detector201determines the similarity between the read and write files from the read fingerprints and the matched fingerprints (step406). For example, the detector201may calculate the similarity value based on the size of the listing of matched fingerprints in relation to the size of the listing of read fingerprints. The detector201may compare the calculated similarity value to a similarity threshold to determine if the process203is a process of an application that performs encryption, compression, or both (step407). In one embodiment, the detector201deems a process203to be that of an encryption or compression application when the similarity value is less than a similarity threshold.

Tables 1 and 2 show test results of implementing the detector201in a personal computer running the Microsoft Windows™ operating system. Table 1 shows the results of similarity calculations for processes of three different encryption/compression applications, namely, TrueCrypt™, Winrar™, and 7Zip™ applications. Note that these applications result in a similarity of 0%, allowing for detection by the detector201.

Table 2 shows the results of similarity calculations for processes of four other applications that are not encryption/compression applications, namely, Microsoft Word™ Microsoft Notepad™, Microsoft Excel™, and Adobe Acrobat Pro™ applications. These other applications were employed to perform some editing operations (copy, cut, delete, change) on read files. The resulting write files yield similarity values greater than 20% and as much as 99.5% in some cases. The similarity threshold may be adjusted as needed to obtain satisfactory detection rates while reducing false positives.

Methods and apparatus for detecting encryption and compression applications have been disclosed. While specific embodiments of the present invention have been provided, it is to be understood that these embodiments are for illustration purposes and not limiting. Many additional embodiments will be apparent to persons of ordinary skill in the art reading this disclosure.