Document Classification Part 4: Variations of This Approach (Malware Detection Using Document Classification)
By: Chan Woo Kim
Tags: Malware Detection, NLP, Classification, Malware, Machine Learning

Static Analysis vs. Dynamic Analysis

The motivation behind most machine learning projects to detect malware is to have a dynamic approach to analyzing malware. A dynamic approach analyses the behavior of a program. In contrast, static malware analysis, the prevailing approach until just this decade, looks at static characteristics of programs to determine if they’re malicious. For example, the simplest static approach would constitute of taking hashes (long strings that summarize a given program) of programs and compare them to hashes of malware that we already know of. If there is a match, we know that this is a program we’ve identified to be malicious before. That is static analysis.

The glaring problem with that is that as malware become more advanced, they are able to adapt to these mechanisms. For instance, with the hash example, due to the fact that hashes are so sensitive to change (what else can you expect from just a long string that summaries hundreds of thousands of lines of code?), some programs are able to change their own code automatically and recompile to have malware that is technically never seen before. Also consider that if you are comparing a given malware to what is seen before, a new variant would be virtually undetectable in that regard.

What we want to pursue instead is looking at the behavior of programs and see if patterns emerge from behavior. This is dynamic analysis and it is often done with machine learning approaches. This is what we tried to do for our project.

We needed to first choose what aspects of behaviors of programs we wanted to specifically look at. What we chose were system calls.

System Calls

First some background information. Every program that runs on an operating system like Windows has a series of system calls. These system calls are requests that the program makes of the computer.

Example System Call Trace (truncated to five calls, from the paper mentioned above)

An easy analogy of this is the little things that go on in our muscle to just move our arm. A program just booting up would be executing a ton of system calls already. Every line of a trace of system calls asks the operating system to do certain really specific tasks, the accumulation of which does what the program intends to do.

So our approach with our research was to analyze these system calls. This was a direct way to analyze what a given program wanted to do in the operating system. But how do we use these system calls in detecting malware?

Document Classification In Malware Detection Using System Call Traces

This approach became a document classification task. We collected traces of system calls and the project soon become one where we were developing a machine learning program that can differentiate between logs of benign programs and malicious programs.

We collected around 250 traces of benign and malicious programs. We did this by collecting malware samples online and running them in a closed virtual machine. It is said that malware has become advanced enough to escape the virtual machine to infect the host but I had no other choice (I do not know if it was coincidence that my computer’s keyboard stopped working due to a software problem).

Here is a sample system call trace, taken directly from my project folder:

This can’t be worked with!

There’s too much stuff going on. If I created a vector that contains every token that appears here, I would have an unmanageably immense vector that doesn’t help anything.

This is the case with most ML tasks. This is why you would almost always want to go through a pre-processing phase. Here’s one line taken from the example above:

NtTraceControl (CtrlCode=0xf, InputBuffer=0x19f748, InputBufferLength=0xa0, OutputBuffer=0x19f748, OutputBufferLength=0xa0, ReturnLength=0x19f73c [0xa0]) => 0

You can think of every system calls as functions. For those that are familiar with programming languages, the content inside the parentheses are parameters that specify its needs in executing the system calls. Although it would’ve been great to analyze the parameters as well, this was beyond the scope of our research. So our pre-processing step included cleaning that up:

Before:

NtTraceControl (CtrlCode=0xf, InputBuffer=0x19f748, InputBufferLength=0xa0, OutputBuffer=0x19f748, OutputBufferLength=0xa0, ReturnLength=0x19f73c [0xa0]) => 0 After:

NtTraceControl

So the trace that the machine learning programs worked with rather looked like:

The following is just a portion of my folder of logs of malware:

Each of these text files contains a log of system calls like what was just shown above. Now the objective of whatever machine learning program I create was to differentiate if certain logs belonged to a malicious program or a benign one.

N-Grams & TF-IDF

These concepts are explained in part 2 of this series.

Microsoft isn’t going to create a system call specifically for malicious purposes. It is the particular order of the use of these system calls that could constitute a malicious process. This is why we integrated the use of N-Grams:

Taken From My Paper

What we want to analyze is sequences of system calls, not just individual system calls. This is why we used n-gram representations of system calls. We actually used 8-grams.

Furthermore, every one of these system call logs starts with the same system calls. There are system calls that are executed for all program executions in general in an operating system. Therefore, some sequences are more important than others. This is the rationale behind integrating TF-IDF weighting.

So our vector representations of system calls were comprised of TF-IDF values of 8-grams of system calls. This data was fed into a Support Vector Machine algorithm optimized through Stochastic Gradient Descent.

Testing & Results

We split the dataset into a 1:4 ratio. 80% of the data was used to train the classifiers and 20% was used to test their quality. The following is the result of the best-performing classifier:

Results

You can learn more about what these scores mean here, but it should be intuitive for most people that 98% precision score is a valid testimony to the performance of this classifier.

Cool! So this project was me and my friend’s entry to Intel International Science and Engineering Fair 2018. Our project was named to be one of the top 5 projects in the Intel Cup regional round in Chengdu, China, and we’re going go off to participate in the finals at Pittsburgh, Pennsylvania in two months! At the local fair we also won the Yale Science and Engineering Award.

This article was just an intuitive and simplified explanation of the research project.

Once again, for those that really want to get into the details about the project, you can find the paper here.