## 2 Related Works  
Most document-based attacks often include a scripting code to execute malicious code activities. Common scripting languages like JavaScript and VBScript are frequently used to exploit document vulnerabilities through API calls, ActionScript code, or file embedding [14]. In our review of previous works focusing on static analysis, particularly for document-based malware, we found a range of models spanning from machine learning to deep learning techniques. [PERSON] and [PERSON] (2011) developed a detection model specialized in malicious JavaScript-bearing PDF documents [15]. Their static analysis involves examining lexical features in JavaScript. They achieved an 85.17% true-positive rate on large datasets from the VirusTotal portal. [PERSON] et al. (2012) [16] analyzed the existence of obfuscated JavaScript (JS) codes. They developed a JS obfuscation detector utilizing an abstract syntax tree for a context-based feature extraction. Their datasets were from real-world web pages. Results showed that the RedJsod detector achieved 97.5% accuracy. [PERSON] et al. (2019) developed the ScriptNet system to detect malicious JavaScript [17]. Their model combines convolutional neural networks (CNNs) and long short-term memory (LSTM), capable of processing JavaScript files as byte sequences. This model achieved an 81.66% true-positive rate and a false-positive rate of 0.50%. [PERSON], [PERSON], and [PERSON] (2019) proposed using CNN to detect malware in bytes of PDF files [18]. Instead of feature engineering on malware documents, the proposed model transforms a stream into byte sequences, making it applicable to any document. [PERSON] et al. also proposed HWP malware detection using spatial pyramid average pooling (SPAP) in the CNN model [19]. They experimented with various pooling layers in the CNN model and proposed an appropriate pooling method for the byte stream, achieving a 99% accuracy for approximately 6000 samples. [PERSON], [PERSON], and [PERSON] proposed byte level-malware detection using CNN models and SPAP [20]. They used an aggregate function to digest the outputs of stream-level results to obtain file-level results. Table 1 provides a summary of related studies, highlighting their features and algorithms. Attention-based models have gained widespread usage across various domains. [PERSON] et al. (2020) applied the attention mechanism to extract and highlight features from PE files automatically [23]. Compared with CNN and skip-connected LSTM models, the attention model achieved an accuracy of 96.27%. In [PERSON]'s paper [24], bidirectional LSTM (BiLSTM) and generative pre-trained Transformer 2 (GPT-2) were used to study and detect a malicious code in assembly instructions extracted from PE files. The study aimed to detect malware by classifying assembly instructions as benign or malicious at the sentence and document levels. Additionally, the authors trained the GPT-2-based model, observing an improvement in detection performance. [PERSON] and [PERSON] used bidirectional encoder representations from Transformer (BERT) to detect malware from the source code of Android applications [25]. In addition to binary classification for malware detection, multi-classification was performed to classify the dataset into 11 different malware families. In comparison with other pre-trained language models, the BERT model demonstrated superior accuracy in both binary and cross-category classification tasks. [PERSON] and [PERSON] used an oversampling method on an unbalanced dataset and proposed Doc2 Vec to detect malicious JavaScript using [22]. Features that extracted using Doc2 Vec and the attention model were fed to a support vector machine (SVM). Results showed that while the attention model exhibited an excellent overall detection rate, the Doc2 Vec model achieved the highest precision rate of 0.989 and the highest recall rate of 0.722. Our study [26] investigated HWP malware detection using a Transformer model, achieving an F1 score of 82.90%.