## 4 Feature Extraction and Selection
### Feature Extraction
With min-max normalization, smaller file offsets and lengths would acquire very small normalized values that would be almost indistinguishable from each other. For instance, 80% of the files have a size of less than 8 GB, but they would be normalized to less than 0.25 with min-max normalization, while the remaining 20% would acquire values between 0.25 and 1. The fifth root normalization spreads the normalized values in a more representative fashion. In the above example, those 80% of the files are normalized between 0 and 0.76. The history of open times reveals how frequently the file is opened. The size of the history is configurable and set to 10 by default. Timestamps are well-known to be bad features because they continuously grow over time. Hence, instead of using them directly, we use the time difference between each consecutive pair of open timestamps as features. All time differences are normalized using min-max normalization, where max is a configurable time window. The _hotness_ of a file represents the file's access rate over a defined time interval. Intuitively, the more a file is accessed and more recently, the higher its hotness value. The file hotness \(h\) is calculated as:  
\[h(f)=\frac{\sum_{\forall t}read(f,t)\times weight(t)}{filesize(f)} \tag{6}\]  
where \(read(f,t)\) is the amount of data read from file \(f\) during time period \(t\), \(weight(t)\) is the weight of \(t\), which is inversely proportional to the time difference between the current time and the beginning of \(t\), and \(filesize(f)\) is the size of \(f\). **File Access Patterns Features:** The current temporal, spatial, and length patterns are maintained as a class with a specific number representation (e.g., mild temporal is 0 and intense temporal is 1). In addition to the pattern itself, we also maintain the _pattern frequency_, which represents the frequency of the intense, sequential, and uniform identification of the temporal, spatial, and length patterns, respectively. The pattern frequencies are normalized with the following:  
\[n_{pattern\_frequency}(c)=1-\frac{1}{\max\left(\log\left(c\right),1\right)} \tag{7}\]  
where \(c\) is the frequency of the intense, sequential, or uniform pattern. Finally, three more features are extracted: the _file access frequency_, _file open frequency_, and _fully read frequency_, which aim to quantify the frequency of access. These are normalized using min-max normalization. **Directory and File Format Features:** These features are similar and hold an aggregated view of the access patterns for each directory and each file format. We hypothesize that there is often a predominant pattern in the directory or file format, and that this pattern would help the model in its predictions. Each directory and each file format maintains the same set of nine features: an \(ID\) representing the directory or file format; the _ratio_ of each pattern identified over all files under the specified directory or file format; the _files count_, representing the total number of distinct files under the same directory or with the same file format; and the _access frequency_, counting the total access frequency of the files under the same directory or with the same file format. The file, directory, and file format IDs are normalized by  
\[n_{mod}(x,M)=\frac{|x|\%M}{M} \tag{8}\]where \(x\) is the hash value of the file name, directory, or file format and \(M\) is a configurable max number of files, root directories, and file formats, respectively. We set \(M\) = 1,000,000 for file IDs and \(M=100\) for directories and file formats. The counts and access frequency features are normalized by the fifth root, as was done for the file size discussed above.