<figure><img src="image_1.png" /></figure>

HAL
open science

# Multi-Taxonomy Vulnerability Classification with Hierarchically Finetuned Language Models

Franco Terranova, Sana Rekbi, Abdelkader Lahmadi, Isabelle Chrisment

► To cite this version:

Franco Terranova, Sana Rekbi, Abdelkader Lahmadi, Isabelle Chrisment. Multi-Taxonomy Vulnerability Classification with Hierarchically Finetuned Language Models. To appear at DIMVA 2026 - Conference on Detection of Intrusions and Malware and Vulnerability Assessment, Jul 2026, Chania, Crete, Greece. hal-05500820

HAL Id: hal-05500820

https://hal.science/hal-05500820v1

Submitted on 9 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
**Accepted paper.** This paper has been accepted for publication at *DIMVA 2026*. If you wish to cite this work, please use the following reference:

Franco Terranova, Sana Rekbi, Abdelkader Lahmadi, and Isabelle Chrisment. "Multi-Taxonomy Vulnerability Classification with Hierarchically Finetuned Language Models," To appear in the Proceedings of the 23rd Conference on Detection of Intrusions and Malware and Vulnerability Assessment (DIMVA 2026).

# Multi-Taxonomy Vulnerability Classification with Hierarchically Finetuned Language Models

Franco Terranova¹[0009-0001-6989-2655], Sana Rekbi¹[0009-0003-7588-3223], Abdelkader Lahmadi¹[0000-0003-3882-1560], and Isabelle Chrisment¹[0000-0002-8474-0019]

¹Université de Lorraine, CNRS, Inria, LORIA, Nancy, France
{name.surname}@inria.fr

**Abstract.** The number of newly disclosed vulnerabilities continues to increase rapidly each year. When disclosed, vulnerability entries typically include an initial free-form textual description that provides a basic overview of the vulnerability. Mapping these descriptions to structured taxonomies such as MITRE ATT&CK, CWE, and CAPEC enhances contextual understanding and supports prioritization from the moment of disclosure, but manual mapping is complex and time-consuming. We present a unified framework for automated multi-label vulnerability classification with Language Models (LMs), implemented in our open-source library *CVE-LMTune*. It combines: (1) a multi-stage pipeline for building an up-to-date, multi-taxonomy labeled dataset; (2) a standardized fine-tuning and evaluation protocol addressing extreme multi-label imbalance; and (3) a hierarchical cascade that decomposes large class spaces of these frameworks into smaller, tractable sub-problems. Our experiments indicate that fine-tuned encoder-only LMs outperform text-generation models, which struggle in handling this high-cardinality, imbalanced task in the token space. Applying our hierarchical approach to the best model, SecureBERT, consistently improves weighted-F1 over the flat version: +12% on CWE (90%), +8% on CAPEC (92%), and +12% on MITRE ATT&CK (93%). We further propose a shared-embedding strategy that cuts hierarchical inference costs toward flat-model efficiency and demonstrate its robustness on newly disclosed vulnerabilities.

**Keywords:** Vulnerability Taxonomies · Multi-Label Classification · Hierarchical Classification · Language Models

## 1 Introduction

The rapid rise in cybersecurity threats and newly disclosed vulnerabilities has made manual analysis of Common Vulnerabilities and Exposures (CVEs) [25] increasingly impractical [24]. The volume of CVE disclosures has grown sharply, from 1,438 in 2000 to 40,077 in 2024—a 22.1% annual growth rate over 25 years. Structured taxonomies such as MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) [28], Common Weakness Enumeration (CWE)
[26], and Common Attack Pattern Enumeration and Classification (CAPEC) [27] standardize vulnerability and threat categorization, but mapping CVE descriptions to these taxonomies remains complex and time-consuming. This classification process requires specialized knowledge and a thorough manual examination of vulnerability details, often resulting in delays in assigning accurate labels [14] [35] [15]. Consequently, security professionals face an increased workload in correctly categorizing vulnerabilities, not only due to the volume of cases but also because this task requires a deep understanding of each taxonomy [4,17]. Efficient and accurate classification is critical for prioritizing vulnerabilities, as organizations cannot patch all issues disclosed immediately without disrupting operations. Consequently, system administrators must strategically prioritize the vulnerabilities to be remediated first [35] [17]. Leveraging structured taxonomies that offer detailed information on the potential impact of vulnerabilities can greatly enhance this prioritization process. In fact, when a vulnerability is mapped to these taxonomies, it can be analyzed from an attacker's perspective. For instance, the ATT&CK taxonomy provides insight into how an attacker might exploit the vulnerability, the potential objectives or outcomes of the exploitation, but also can inform on the corresponding mitigation and detection strategies that can be employed to defend against such attacks. Identifying these mappings is also gaining attention in various applications, including proactive cybersecurity, where simulators leverage ATT&CK-mapped vulnerabilities to predict multi-stage attack paths [37]. The benefits of this structured categorization, coupled with the growing volume of vulnerability disclosures, underscore the urgent need for automated classification to complement human expertise [4]. The information commonly available at disclosure time is a free-form textual description of the vulnerability in natural language [17] [1], making them well-suited for automated mapping using pre-trained language models (LMs). These models can be further fine-tuned for multi-label classification [23] [18] to align vulnerability descriptions with these established taxonomies. However, fine-tuning large LMs to handle a classification towards the extensive class sets of these taxonomies requires substantial labeled data, and no standardized benchmark dataset exists for multi-taxonomy mapping [16]. Existing studies are also fragmented, as each focuses on a single, distinct taxonomy and relies on inconsistent datasets, protocols, and metrics, thereby hindering comparison. To address these gaps, this paper makes three contributions:

- We propose a strategy to scrape a continuously updated **dataset**, containing approximately 228k CVEs as of submission—the largest multi-taxonomy, multi-label benchmark to date, consistently surpassing the largest previously reported labeled dataset of 82k samples, which focused on the single taxonomy of CWEs [10].

- We present a novel unified **framework** for fine-tuning diverse LMs on CWE, CAPEC, and ATT&CK across all levels of granularity, offering standardized strategies and metrics that account for multi-label scenarios and class imbalance. The framework is released alongside this paper as the open-source
library *CVE-LMTune*<fnref n="1" />. In addition, we provide the dataset, the trained models, and the configurations used to produce the experimental results<fnref n="2" />.

- We propose a **hierarchical classifier** [5] that decomposes the multi-label classification problems into stages aligned with the taxonomy hierarchies, while keeping computational overhead low by proposing a shared-embedding strategy to reduce inference time.

The paper is organized as follows: Section 2 provides background information, Section 3 reviews related work, Section 4 presents the methodology, Section 5 discusses the experiments, and Section 6 concludes the paper.

## 2 Background

The following section provides background on the target taxonomies, the process of vulnerability disclosure, as well as LMs and multi-label classification.

### 2.1 Vulnerability Disclosure

Vulnerability disclosure is the structured process of reporting, documenting, and communicating vulnerabilities. After identification, a vulnerability may be publicly disclosed or shared privately with the affected vendor for coordinated mitigation. In most cases, the reporter submits the vulnerability, along with an initial textual description outlining its nature and potential impact, to a central authority such as MITRE, which then assigns a CVE identifier. Following this initial disclosure, additional metadata is typically added after a subsequent manual analysis by experts. This typically includes their categorization into established taxonomies such as CWE, which can then be linked to CAPEC or ATT&CK techniques, thereby providing a richer contextual understanding of vulnerabilities. This enriched contextual information can help security professionals understand not only the nature of the vulnerability but also how attackers might exploit it in real-world scenarios, enabling more effective prioritization.

### 2.2 Existing taxonomies

Several widely adopted taxonomies provide standardized ways to categorize vulnerabilities, each featuring hierarchical structures that facilitate analysis at varying levels of detail.

- **CWEs**: A community-developed list of software and hardware weakness types. Each CWE entry describes a specific type of flaw and helps users navigate from broad categories of weaknesses to specific flaw types.

- **Hierarchy**: A *Pillar*, always at the top level, represents a broad family of related weaknesses. Beneath it, a flexible, multi-level hierarchy is formed by nesting elements from the set {Class, Base, Variant, Compound}. This structure has no fixed depth, allowing for varying numbers of layers.

<footnote n="1">¹ *CVE-LMTune*: https://github.com/terranovafr/CVE-LMTune</footnote>

<footnote n="2">² *Zenodo Data Repository*: https://doi.org/10.5281/zenodo.16936476</footnote>
**Example:** CWE-284: Improper Access Control (Pillar) → CWE-285: Improper Authorization (Class) → CWE-552: Files or Directories Accessible to External Parties (Base) → CWE-219: Storage of File with Sensitive Data Under Web Root (Variant)

- **CAPEC:** CAPEC builds upon CWE by providing detailed descriptions of attack patterns that exploit known weaknesses, organized from broad strategies to specific technical mechanisms.

**Hierarchy:** CAPEC organizes attack knowledge starting from high-level *Categories*, and subsequently grouping attack patterns recursively. The classic structure is the following: *Category* → *Meta Attack Patterns* → *Standard Attack Patterns* → *Detailed Attack Patterns*

**Example:** CAPEC-152: Inject Unexpected Items (Category) → CAPEC-137: Parameter Injection (Meta Attack Pattern) → CAPEC-6: Argument Injection (Standard Attack Pattern)

- **MITRE ATT&CK:** The ATT&CK taxonomy is a comprehensive knowledge base of adversary behaviors derived from real-world threat intelligence and incident reports, structured into domain-specific matrices representing each a specific operational environment: *Enterprise* (IT environments), *Mobile* (mobile devices oriented), and *ICS* (industrial control systems).

**Hierarchy:** Each ATT&CK matrix presents the same hierarchy structure but different elements inside each level. The hierarchy structure is as follows: *Tactics* (adversary goals) → *Techniques* (methods to achieve goals) → *Sub-techniques* (detailed variants).

**Example — Enterprise:** Credential Access (Tactic) → T1003 — OS Credential Dumping (Technique) → T1003.001 — LSASS Memory (Subtechnique)

Mapping vulnerabilities to these taxonomies provides complementary insights: CWE enables understanding of the underlying weaknesses, CAPEC reveals how these weaknesses can be exploited through known attack patterns, and ATT&CK connects them to real-world adversarial behaviors—together enabling more comprehensive reasoning over vulnerabilities.

## 2.3 Language Models for Multi-Label Classification

Pre-trained LMs have demonstrated remarkable capabilities in understanding and generating natural language [11]. These models produce rich, contextualized embeddings (i.e., vector representations) that capture semantic and syntactic information. Such embeddings can be leveraged directly or transformed into task-specific representations for a variety of downstream applications, including *multi-label classification*, where each instance may be associated with multiple relevant categories. Formally, let $\mathcal{X}$ denote the input space and $\mathcal{Y}$ =
$\{\lambda_1, \lambda_2, \dots, \lambda_C\}$ the class set containing $C$ possible classes. The goal is to learn a function: $f: \mathcal{X} \to \{0, 1\}^C$, which maps an input $x \in \mathcal{X}$ to a binary vector $y = (y_1, y_2, \dots, y_C)$, where $y_i = 1$ indicates the presence of label $y_i$. Multi-label classification is inherently more complex than single-label, as arbitrary label co-occurrences cause an exponential number of possible label sets. Datasets typically follow a long-tail distribution, with a few common combinations and many rare or unseen ones, making it hard to cover all cases during training. To adapt LMs for multi-label classification, two main strategies are commonly explored:

- **Text-generation models:** In this approach, the classification task is framed as a mapping from token space to token space, and the model is prompted to generate the labels as text, making classification indirect. Typical models include decoder-only or encoder-decoder architectures (e.g., GPT [7]). While flexible and capable of handling multiple tasks with a single model, these approaches must produce the correct sequence of tokens to represent the target classes—a task particularly challenging in high-cardinality settings.

- **Fine-tuned encoder-only models:** Encoder-only models (e.g., BERT [11]) map input text into a continuous embedding space that captures semantic content. A dedicated classification head is then attached and trained to map embeddings directly to the class space, producing class probabilities via an appropriate activation function for multi-label tasks. Fine-tuning strategies vary depending on dataset size and domain similarity, from updating only the classification head to a subset of the encoder layers. This approach yields direct probability outputs, eliminating the need to interpret generated text.

# 3 Related Work

This section surveys related work, beginning with knowledge bases constructed through manual mappings and moving to recent automated approaches, with a discussion of the limitations present in existing works.

## 3.1 Limitations of Manual Mappings

Several manual approaches have been proposed to map vulnerabilities along the pipeline CVE → CWE → CAPEC → ATT&CK using predefined knowledge bases manually constructed [3] [36]. These approaches assume the availability of a CVE identifier manually assigned to each CVE. Once the CVE is available, predefined mapping tables are used to link the CVE to CAPEC and subsequently to ATT&CK. However, this pipeline suffers from several limitations:

1. **Manual CWE Annotation:** A CVE must first be manually analyzed and assigned a CWE by security experts. This step is both time-consuming and prone to delays from the moment of disclosure. Furthermore, many CVEs remain unlabeled, such as those marked NVD-CWE-noinfo in the National Vulnerability Database (NVD) [29], indicating that sometimes no CWE can
be assigned due to insufficient information. This lack of labeling blocks down-
stream mapping [36], even when the relevant attack patterns and techniques
can be associated with these vulnerabilities. To ensure comprehensive cov-
erage and prevent such bottlenecks, it is therefore essential to also explore
direct mappings from CVE to CAPEC and to ATT&CK.

2. **Maintenance of Knowledge Bases:** The mappings between CWE → CAPEC and CAPEC → ATT&CK must be carefully curated and updated as these taxonomies continuously expand their structure, defining new elements in their hierarchy. For instance, the ATT&CK matrices are updated approximately every six months; the latest update (ATT&CK v18, October 2025) introduced 12 new Enterprise techniques and 2 new Mobile techniques.

3. Incomplete Coverage Tables: Even with continuous updates, not all CWEs are mapped to CAPECs or ATT&CK techniques, leaving gaps in the pipeline [36] that hinder the classification process with knowledge bases.

<figure><img src="image_3.png" /><figcaption>Fig. 1: Overview of CVE linkage and CWE assignment delay trends across years.</figcaption></figure>

To illustrate the current state of knowledge bases, we refer to the widely
used *CVE2CAPEC* [13] project. From 1999 to 2025, CVEs of that year with
manually linked to at least one CWE, CAPEC, or ATT&CK element increased
from 6%, 5%, and 4% to 92%, 91%, and 89%, respectively, reflecting active
maintenance in manual mappings. Despite this, CWE assignment remains a
bottleneck: NVD statistics—calculated by measuring the time difference between
the publication date of a vulnerability and the point at which it reaches the
*Analysis* phase—when a CWE is assigned—show an average delay of 24 days,
median of 11 days, and maximum of 170 days in 2025 (Figures 1b and 1c).
Average and median CWE assignment delays (Figure 1c) show no consistent
decrease over time, and although the maximum delay has dropped, at least one
vulnerability can remain uncategorized for over five months even in recent years.
These findings also underscore the limitations of relying on CWE assignment as
a prerequisite for initiating manual mapping to higher-level taxonomies such
as CAPEC and ATT&CK. Moreover, there is a clear need for new automated
approaches that can help streamline these mappings.
<table><caption>Table 1: Classification of existing automated mapping approaches. A dash ("-") indicates missing information.</caption><thead><tr><th>Taxonomy</th><th>Study</th><th>||D||</th><th>Level</th><th>||C||</th><th>ML LMs</th><th>MS</th><th>SS</th><th>ML CI</th><th>H</th></tr></thead><tbody><tr><td rowspan="3">CWE</td><td>[10]</td><td>82k</td><td>Full</td><td>124</td><td>1</td><td>2</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>[4]</td><td>29k</td><td>Base CWE</td><td>19</td><td>6</td><td>0</td><td>✓</td><td></td><td>✓</td></tr><tr><td>[16]</td><td>4k</td><td>Full</td><td>25</td><td>1</td><td>4</td><td></td><td>✓</td><td>✓</td></tr><tr><td>CAPEC</td><td>[8]</td><td>&lt;1k</td><td>-</td><td>-</td><td>0</td><td>2</td><td></td><td>✓</td><td></td></tr><tr><td rowspan="6">ATT&amp;CK</td><td>[12]</td><td>8k</td><td>Techniques</td><td>50</td><td>4</td><td>0</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>[24]</td><td>8k</td><td>Techniques</td><td>52</td><td>7</td><td>0</td><td>✓</td><td>✓</td><td></td></tr><tr><td>[20]</td><td>62k</td><td>Techniques</td><td>37</td><td>4</td><td>0</td><td></td><td>✓</td><td>✓</td></tr><tr><td>[3]</td><td>25k</td><td>Tactics</td><td>10</td><td>10</td><td>4</td><td>✓</td><td></td><td></td></tr><tr><td>[1]</td><td>2k</td><td>Techniques</td><td>185</td><td>0</td><td>4</td><td></td><td>✓</td><td>✓</td></tr><tr><td>[23]</td><td>11k</td><td>Techniques</td><td>-</td><td>0</td><td>16</td><td></td><td></td><td></td></tr><tr><td>CWE</td><td></td><td></td><td>Full</td><td>680</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>CAPEC</td><td></td><td></td><td>Full</td><td>442</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>ATT&amp;CK</td><td>Ours</td><td>228k<sup>3</sup></td><td>Subtechniques</td><td>173</td><td>0</td><td>12</td><td>✓</td><td>✓</td><td>✓</td></tr></tbody></table>

MS: Multiple Splits, SS: Stratified Splits, MM: Multi-label Metrics,

CI: Class Imbalance, H: Hyperparams Optimization or Exploration

## 3.2 Automated Mapping Strategies

A first drawback of existing automated vulnerability classification studies is their focus on a single taxonomy. Table 1 categorizes the studies according to the classification taxonomy used (CWE, CAPEC, or ATT&CK) and provides additional details, including dataset size (||D||), the granularity level within the taxonomy (e.g., techniques in ATT&CK), and the number of classes (||C||) considered at that level. This distinction is crucial, as not all datasets contain sufficient samples to cover each class with at least one labeled sample. The table also reports the number of traditional Machine Learning (ML) methods or LMs evaluated in each study. Many studies, however, omit standard ML practices appropriate for this task, such as multiple or stratified data splits, multi-label evaluation metrics, strategies to handle class imbalance, and hyperparameter optimization. This fragmentation, combined with the lack of large-scale reference datasets and the resulting variability in class counts across studies, complicates direct comparison. In contrast, our study consistently integrates all existing taxonomies, utilizes the largest available dataset and class set to date, performs classification at the most granular levels, and follows best practices for multi-label imbalanced learning.

<footnote n="3">³ The dataset supports continuous scraping, allowing its size to increase dynamically.</footnote>
# 4 Methodology

This section presents the methodology adopted in this study, covering the entire pipeline from data collection to hierarchical model design, showing how the classification task is decomposed into subproblems that align with the taxonomies.

## 4.1 Data Collection

We leverage historical CVEs that have been manually labeled using CWE, CAPEC, and ATT&CK labels to construct a dataset suitable for fine-tuning and evaluating LMs as multi-label classifiers. The proposed multi-step pipeline operates as follows:

1. **Data Collection and Label Mapping:** CVEs that are already labeled with CWEs are scraped from the NVD. Subsequently, using mapping tables—particularly CVE2CAPEC—CVEs are mapped to the corresponding CAPEC and ATT&CK identifiers, which are stored as labels for each CVE. This ensures consistent and propagated labeling across all frameworks.

2. **Hierarchical Labeling:** The directed acyclic graph (DAG) structure of MITRE taxonomies, along with parent-child relationships between categories, is exploited to generate hierarchical labels for each instance. Storing these relationships for each instance enables classification at multiple levels of granularity and allows the option to disregard labels below a selected level.

3. **Cross-Matrix Adaptation:** Mapping tables (again CVE2CAPEC) are employed to translate Enterprise-standard ATT&CK labels to ICS and Mobile matrices, facilitating classification across diverse operational domains. To the best of our knowledge, this results in the first dataset explicitly covering ICS and Mobile perspectives, providing a broader operational view.

All stages of the pipeline are integrated in the released library CVE-LMTune. By reconstructing live mappings, it dynamically updates class sets as underlying taxonomies evolve and newly labeled CVEs as they become available.

## 4.2 Hierarchical Classification

The CWE, CAPEC, and ATT&CK taxonomies define large and evolving class spaces, requiring models that (1) handle large-scale classification and (2) adapt to changes in the class space efficiently. Let the taxonomy be represented as a DAG with *L* levels (e.g., *L* = 3 for ATT&CK: Tactics → Techniques → Sub-techniques). At level 1, a root classifier performs multi-label classification to select relevant classes for the next level. At subsequent levels, each node *j* at level *i* has a dedicated classifier $f_j^{(i)}$ responsible for a subset of classes $C_j^{(i)} \subset C^{(i)}$, corresponding to the descendants reachable from that node in the DAG. Inference proceeds top-down along the DAG edges, where predictions at one level activate child classifiers connected to the predicted nodes:
<figure><img src="image_4.png" /><figcaption>Fig. 2: Representation of hierarchical classification with ATT&CK as an example.</figcaption></figure>

$$ \hat{y}^{(1)} = f_{1}^{(1)}(x) \text{ (root classifier)} $$

$$ \hat{y}_{j}^{(2)} = f_{j}^{(2)}(x) \text{ for all } j \text{ reachable from } \hat{y}^{(1)} $$

$$ \hat{y}_{k}^{(L)} = f_{k}^{(L)}(x) \text{ for all } k \text{ reachable from } \hat{y}^{(L-1)} $$

This approach replaces the traditional flat classifier—commonly used in prior studies, where one classifier must select from the full set of possible classes (e.g., all sub-techniques)—with a cascade of smaller, specialized classifiers, thereby decomposing the task into a sequence of more manageable sub-problems (Figure 2). For instance, a CVE exhibiting **Persistence** and **Privilege Escalation** activates only the technique-level classifiers corresponding to those tactics, thereby substantially reducing the number of classes considered at each stage and enabling selection among relevant subsets rather than across a flat set of all classes.

**Advantages.** The proposed solution offers several benefits:

+ **Inference-Time Pruning:** Entire branches can be skipped if parent nodes are not activated, allowing the model to traverse only the relevant subset of classes, thereby reducing unnecessary computations.

+ **Multi-Granular Classification:** Unlike previous solutions with specialized classifiers for tactics, techniques, and sub-techniques, the DAG-based architecture naturally performs classification at all levels simultaneously. If predictions are needed at a coarser level, lower levels can simply be ignored.

+ **Flexible Level Prediction:** The model can stop at any level if no child label is predicted, supporting predictions at non-leaf nodes. For example, a vulnerability may only be associated with a Base CVE because no more specific weakness has been identified.

+ **Better Performance on Imbalanced Data:** Decomposing the task into smaller subproblems helps mitigate class imbalance, a common challenge in CVE classification [4]. By distributing the classification process across multiple decision stages, the effect of imbalance at each stage is reduced [5].

+ **Modularity and Adaptability:** When taxonomies evolve—for example, with new techniques or sub-techniques—only the classifiers in the affected
sub-branches need retraining. This modular approach scales far better than flat classifiers, which require retraining on the whole class space. Retraining a smaller class subset also reduces the data and time required, lowering the effort for each update.

*Challenges.* The hierarchical approach also introduces some limitations:

- **Increased Inference Overhead:** A key limitation of the hierarchical classifier approach is the increased inference time. Unlike a flat classifier, this method may require executing multiple LM-based classifiers per input, potentially many also at each hierarchical level, leading to a higher computational cost during inference.

*Mitigation:* Under the encoder-only strategy, when all classifiers in the hierarchy share the same encoder-only LM backbone and fine-tuning updates only a few layers (e.g., the classification head), a frozen LM can be used to extract a shared embedding once per input. The lightweight, task-specific layers are then applied at each level, reducing total inference time by reusing the shared embedding.

- **Error Propagation:** Misclassifications at higher levels (e.g., a misclassified tactic) can propagate, causing all predictions in the next branches to be incorrect. For example, if *Persistence* is mispredicted, all techniques associated with that tactic will inevitably be misclassified.

*Mitigation:* One possible mitigation is to assign higher priority to fine-tuning higher-level classifiers, especially the root, using more epochs or data. This improves accuracy at critical decision points, potentially reducing the likelihood of error propagation.

# 5 Experiments

This section describes the experimental setup and reports the results obtained across different classification taxonomies, employing various LMs. We highlight the empirical improvements achieved by our hierarchical approach and examine multiple evaluation strategies.

## 5.1 Experimental Setup

This section describes the dataset and metrics used in the experiments.

**Dataset.** Collected as described in Section 4.1, and last updated on September 19, 2025, the dataset comprises 228,479 CVEs. This is fewer than the total number of CVEs published to date, as many older entries lack a CWE and were therefore excluded. Each taxonomy covers a distinct subset of CVEs, since some manual mappings are not available, as discussed in Section 3.1. The following are the dataset statistics for the three classification tasks:

- **CWE:** 680 labels; 228,479 samples (avg. 3.33 labels/sample)
- CAPEC: 442 labels; 210,001 samples (avg. 32.57 labels/sample)

- ATT&CK: 173 labels; 201,562 samples (avg. 12.83 labels/sample)

For ATT&CK, the Enterprise matrix is used as a representative example due to its broad adoption, extensive class coverage, and large number of samples. CWE classification constitutes the primary task, as mapping a CVE to a CWE enables further linkage to CAPEC and ATT&CK through existing manual mappings. However, since some CWE-CAPEC and CWE-ATT&CK mappings are incomplete (see Section 3.1), jointly fine-tuning models across all three classification tasks provides an effective way to address these gaps. Figure 3 shows class

<figure><img src="image_5.png" /><figcaption>Fig. 3: Distribution of labels and label co-occurrence across the taxonomies.</figcaption></figure>

distributions and top 10 label co-occurrences, highlighting that CAPEC and ATT&CK have more labels per sample and more frequent label combinations.

**Evaluation Metrics.** To evaluate the models on the multi-label classification task, we adopted a carefully selected set of metrics recommended in the literature [41] for multi-label classification. The specific subset was strategically selected to include metrics addressing class imbalance, while ensuring a non-redundant set derived from correlation analyses between performance measures [31].

- *Micro-averaged F-measure* (Micro-F1): Captures overall multi-label prediction quality by treating all classes equally, regardless of frequency.

- *Weighted F1-Score* (Weighted-F1):Computes the F1 score for each label and averages them weighted by label frequency, providing a version of F1 better suited for class-imbalanced datasets.

- *Subset Accuracy* (Subset-Acc): Measures the proportion of instances where the model predicts the exact set of labels correctly. This is a strict metric as any mismatch counts as an error, emphasizing precise label set prediction.
- Mean Reciprocal Rank (MRR): Evaluates the ranking quality of the predicted labels by measuring the position of the first correct label in the predictions.

## 5.2 Experimental Results

This section presents our experimental results, including a comparison of text-generation models and encoder-only models on multi-label classification across taxonomies, improvements from hierarchical LM classifiers versus a flat classifier, and an analysis of the temporal dataset's impact on performance.

**Text-Generation Large Language Models.** Instruction-tuned Large LMs (LLMs) excel at a wide range of generative and reasoning tasks, raising the question of whether they can be used zero-shot [39] for multi-label classification of CVE descriptions. Zero-shot inference is the only practical approach in our setting, as few-shot methods are infeasible: providing correctly labeled examples for a sufficient number of combinations of classes would require a large number of samples, far exceeding the context window of current models.

*Setup.* We benchmark four state-of-the-art text-generation models—GPT-4.1, GPT-5.1, LLaMA-3.1-8b-instant [38], and DeepSeek-V3.1 [21]—prompting them to output CVE labels given the CVE description. We conducted our experiments on a representative subset of the dataset by selecting the 1,000 oldest CVEs, whose disclosure information has been publicly available for the longest time. This choice represents the most relevant and easiest evaluation scenario, since the models have likely seen these entries during pretraining. We also varied the temperature parameter in the range {0, 0.5, 1.0} for each CVE and taxonomy, to study the behavior of the model from deterministic to more creative output [30], averaging results in all three settings. The details of the prompting strategy are included in Appendix A.

<table><caption>Table 2: Zero-shot performance of text-generation LLMs across the three classification tasks on the oldest 1,000 CVEs.</caption><thead><tr><th rowspan="2">Model</th><th colspan="3">CWE</th><th colspan="3">CAPEC</th><th colspan="3">MITRE ATT&CK</th></tr><tr><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th></tr></thead><tbody><tr><td>GPT-4.1</td><td><strong>0.29</strong></td><td><strong>0.22</strong></td><td><strong>0.00</strong></td><td><strong>0.52</strong></td><td>0.03</td><td>0.02</td><td>0.00</td><td>0.38</td><td><strong>0.01</strong></td><td><strong>0.01</strong></td><td><strong>0.02</strong></td><td><strong>0.11</strong></td></tr><tr><td>GPT-5.1</td><td>0.27</td><td>0.21</td><td><strong>0.00</strong></td><td><strong>0.52</strong></td><td>0.04</td><td>0.03</td><td>0.00</td><td><strong>0.45</strong></td><td><strong>0.01</strong></td><td><strong>0.01</strong></td><td><strong>0.02</strong></td><td>0.10</td></tr><tr><td>DeepSeek-V3.1</td><td>0.28</td><td>0.21</td><td><strong>0.00</strong></td><td>0.49</td><td><strong>0.07</strong></td><td><strong>0.06</strong></td><td><strong>0.01</strong></td><td>0.44</td><td><strong>0.01</strong></td><td><strong>0.01</strong></td><td>0.00</td><td>0.09</td></tr><tr><td>LLaMA-3.1</td><td>0.19</td><td>0.13</td><td><strong>0.00</strong></td><td>0.31</td><td>0.02</td><td>0.01</td><td>0.00</td><td>0.18</td><td><strong>0.01</strong></td><td>0.00</td><td>0.01</td><td>0.05</td></tr></tbody></table>
*Discussion.* Results indicate that zero-shot predictions yield the highest performance when applied to the CWE taxonomy, likely because CVE-to-CWE mappings are often publicly available on NVD pages and other online resources. However, even in this favorable scenario, performance remains far from operationally sufficient, with CAPEC and ATT&CK sub-techniques receiving near-zero scores. Another major challenge encountered is the sensitivity to the temperature hyperparameter with these models. Predictions vary significantly with different temperature settings, leading to inconsistent outputs for the same input. This instability was quantified by measuring agreement across varying temperature values, computed as the average Jaccard overlap between predictions, highlighting the greater consistency of GPT models with respect to the others: GPT-4.1 (0.68), GPT-5.1 (0.64), DeepSeek-V3.1 (0.45), and LLAMA-3.1 (0.22).

## Takeaway #1

Even under favorable conditions, general-purpose text-generation LLMs cannot reliably perform multi-label CVE classification in a zero-shot setting. Specialized models and fine-tuning strategies are required for operational performance in these tasks.

## Fine-Tuned Encoder-Only Language Models.
Encoder-only LMs are the standard literature choice when fine-tuning LMs for multi-label text classification. We follow this strategy by attaching a classification head to the pretrained encoders and fine-tuning the models to learn a direct mapping from the embedding space to the class space in a *flat* setting. This section investigates whether existing pretrained encoder-only LMs—general-purpose or cybersecurity-specific —can be effectively fine-tuned for the multi-label prediction tasks.

### Setup.
We used a 5-fold nested stratified CV with 60%/20%/20% splits between train-validation-test, conducting an initial study that ignores CVE temporal order and focuses on class distributions. Early stopping is applied on the validation fold. Due to strong label imbalance, we adopt an adaptive label-preserving stratified algorithm [34] to ensure every label occurs at least once in the training data. All experiments use fixed random seeds, and results are averaged across the five held-out test folds. We benchmark the two widely used encoder-only baselines: BERT, ELECTRA [9], and variants including DistilBERT [33] and RoBERTa [22]. We also evaluate four cybersecurity-oriented models: SecureBERT [2], SecRoBERTa [2], SecBERT [19], and CySecBERT [6]. Notably, these cybersecurity-specific models are essentially BERT-family architectures that have already been fine-tuned once on various cybersecurity corpora to produce more domain-adapted embeddings. The fine-tuned layers consist of the classification head and the last encoder layer, which we found empirically to be the minimum required for competitive performance, and as fine-tuning additional layers yielded negligible improvement. The main models' hyperparameters, including class-imbalance-aware loss functions, were optimized using a
<table><caption>Table 3: Comparison of ELECTRA and BERT models (and variants) across the three classification tasks based on the mean scores of a 5-fold CV.</caption><thead><tr><th rowspan="2">Model</th><th colspan="4">CWE</th><th colspan="4">CAPEC</th><th colspan="4">MITRE ATT&CK</th></tr><tr><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>MRR</th><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>MRR</th><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>MRR</th></tr></thead><tbody><tr><td colspan="14" style="text-align: center; font-weight: bold;">General-purpose models</td></tr><tr><td>ELECTRA</td><td>0.76</td><td>0.71</td><td>0.51</td><td>0.86</td><td>0.83</td><td>0.81</td><td>0.49</td><td>0.89</td><td>0.79</td><td>0.78</td><td>0.65</td><td>0.94</td></tr><tr><td>BERT</td><td>0.76</td><td>0.70</td><td>0.48</td><td>0.86</td><td>0.83</td><td>0.81</td><td>0.45</td><td>0.89</td><td>0.80</td><td>0.78</td><td>0.63</td><td>0.94</td></tr><tr><td>DistilBERT</td><td>0.78</td><td>0.74</td><td>0.53</td><td>0.87</td><td>0.84</td><td>0.82</td><td>0.51</td><td>0.89</td><td>0.80</td><td>0.79</td><td>0.66</td><td>0.94</td></tr><tr><td>RoBERTa</td><td>0.80</td><td>0.76</td><td>0.56</td><td>0.88</td><td>0.84</td><td>0.83</td><td>0.52</td><td>0.91</td><td>0.81</td><td>0.80</td><td>0.67</td><td><strong>0.95</strong></td></tr><tr><td colspan="14" style="text-align: center; font-weight: bold;">Cyber-security specific models</td></tr><tr><td>SecRoBERTa</td><td>0.77</td><td>0.73</td><td>0.52</td><td>0.86</td><td>0.83</td><td>0.81</td><td>0.49</td><td>0.90</td><td>0.79</td><td>0.78</td><td>0.64</td><td>0.94</td></tr><tr><td>SecBERT</td><td>0.78</td><td>0.74</td><td>0.54</td><td>0.87</td><td>0.84</td><td>0.82</td><td>0.52</td><td>0.90</td><td>0.80</td><td>0.79</td><td>0.66</td><td>0.94</td></tr><tr><td>CySecBERT</td><td>0.80</td><td>0.77</td><td>0.57</td><td>0.88</td><td>0.85</td><td><strong>0.84</strong></td><td>0.52</td><td><strong>0.92</strong></td><td><strong>0.82</strong></td><td><strong>0.81</strong></td><td><strong>0.68</strong></td><td><strong>0.95</strong></td></tr><tr><td>SecureBERT</td><td><strong>0.81</strong></td><td><strong>0.78</strong></td><td><strong>0.58</strong></td><td><strong>0.89</strong></td><td><strong>0.86</strong></td><td><strong>0.84</strong></td><td><strong>0.55</strong></td><td><strong>0.92</strong></td><td><strong>0.82</strong></td><td><strong>0.81</strong></td><td><strong>0.68</strong></td><td><strong>0.95</strong></td></tr></tbody></table>

25-trial Tree-structured Parzen Estimator (TPE) search [40], with further details on both optimized and manually set hyperparameters provided in Appendix B.

*Discussion.* Results in Table 3 show that encoder-only models can be fine-tuned effectively for these large multi-label classification tasks, achieving strong performance across all taxonomies. Although a performance ranking is visible, the gains between models are generally modest. Cybersecurity-specific variants do not consistently outperform general-purpose LMs—RoBERTa, for example, surpasses several of these domain-adapted models. This suggests that, given a sufficiently large task-specific dataset, a single fine-tuning largely eliminates any advantage gained from the prior cybersecurity-oriented pretraining. Consequently, the additional computational cost of creating such domain-specific variants appears hard to justify for tasks where large amounts of labeled data are available.

## Takeaway #2

With a sufficiently large dataset, fine-tuned encoder-only LMs perform similarly on our tasks, regardless of whether they are general-purpose models or have undergone domain-specific pretraining.

**Flat vs. Hierarchical Classification.** In this section, we present the empirical results of the hierarchical classification strategy proposed in this work, combined with the strongest baseline identified earlier—fine-tuned encoder-only LMs. The
approach exploits the inherent structure of the taxonomies by decomposing each prediction task into smaller sub-problems handled by a hierarchy of LMs.

*Setup.* To evaluate hierarchical classification, we compare SecureBERT (the best-performing model found in Section 5.2) under flat and hierarchical setups, using a maximum hierarchy depth of *L* = 3 across all taxonomies. Deeper levels are flattened, yielding a hybrid flat-hierarchical design. At each level, we fine-tune a separate instance of SecureBERT, all sharing the same pretrained backbone but trained on different label spaces. The experimental protocol from the previous section is maintained, with additional strategies introduced to handle hierarchy-specific challenges: (1) *Varying dataset sizes:* Each hierarchical node classifier is trained only on samples containing its corresponding classes, resulting in datasets of different sizes. To ensure balanced training, the number of epochs is scaled inversely with dataset size relative to the root model (which sees all data), allowing smaller datasets proportionally more training epochs; (2) *Shared children:* When a class appears under multiple parents, we assign it to a single parent to enforce a disjoint partition of the class set. This ensures that each class is handled by exactly one model and that no two classifiers can predict the same class (converting the DAG into a tree), thereby preventing prediction conflicts or overlaps; (3) *Sparse or small datasets:* Nodes with small datasets (fewer than 50 samples) or containing only one child class are merged with their parent to avoid undertrained or irrelevant models.

<table><caption>Table 4: Comparison of hierarchical and flat SecureBERT classifiers, based on the mean scores from a 5-fold CV across the three classification tasks.</caption><thead><tr><th rowspan="2">Model</th><th colspan="4">CWE</th><th colspan="4">CAPEC</th><th colspan="4">MITRE ATT&CK</th></tr><tr><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>MRR</th><th># Models</th><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>MRR</th><th># Models</th><th>Micro-F1</th><th>Weighted-F1</th><th>Subset-Acc</th><th>MRR</th><th># Models</th></tr></thead><tbody><tr><td>Flat</td><td>0.81</td><td>0.78</td><td>0.58</td><td>0.89</td><td>1</td><td>0.86</td><td>0.84</td><td>0.55</td><td>0.92</td><td>1</td><td>0.82</td><td>0.81</td><td>0.68</td><td>0.95</td><td>1</td></tr><tr><td>L = 2</td><td>0.88</td><td>0.87</td><td>0.67</td><td>0.96</td><td>11</td><td>0.92</td><td>0.91</td><td>0.54</td><td>0.97</td><td>10</td><td>0.91</td><td>0.90</td><td>0.71</td><td>1.00</td><td>11</td></tr><tr><td>L = 3</td><td>0.90</td><td>0.90</td><td>0.71</td><td>0.97</td><td>47</td><td>0.94</td><td>0.92</td><td>0.07</td><td>1.00</td><td>28</td><td>0.93</td><td>0.93</td><td>0.72</td><td>1.00</td><td>34</td></tr></tbody></table>

*Discussion.* The resulting improvements reach 12% for CWE, 8% for CAPEC, and 12% for ATT&CK, using weighted-F1 score as a reference metric for *L* = 3. Table 4 also reports the number of models trained for each taxonomy, illustrating that while increasing *L* enables finer-grained classification, it also increases computational cost (more models). An interesting pattern emerges for CAPEC, the dataset with the most labels per sample: at *L* = 3, subset accuracy drops sharply while weighted-F1 improves, showing that the model performs better on
average but struggles with exact matches—a single missing or extra label can
cause the mismatch. This also illustrates how hierarchical depth can shift the
balance between strict and average performance metrics.

*Inference Time.* The main drawback of the hierarchical solution is the need to invoke multiple LMs for each CVE classification—one for each activated node from root to leaf—which naturally increases inference time. One potential use case for these LMs is as real-time assistants while reporters write vulnerability descriptions, dynamically suggesting expected classes based on the text entered so far. This scenario requires low-latency processing, raising a natural question: *Do the performance gains justify the additional computational cost?*

The shared-embedding strategy described in Section 4.2 mitigates this issue by computing a single transformer embedding once—since the same LM is used across the hierarchy—and applying only the task-specific components at each level (in our setup, the final encoder layer and classification head). To assess the impact of this design, we benchmark mean inference time on 10k CVEs using: (i) a flat SecureBERT model, (ii) a hierarchical SecureBERT with $L = 3$, and (iii) its shared-embedding variant, running all experiments on a reference machine<fnref n="4" />. Figure 4 shows that, across all taxonomies, the hierarchical setup is slower than the flat model. However, the shared-embedding variant significantly mitigates this overhead, recovering roughly 72% (CWE), 81% (CAPEC), and 92% (ATT&CK) of the gap relative to the flat baseline.

Fig. 4: Inference time on 10k CVEs.

<figure><img src="image_6.png" /></figure>

## Takeaway #3

Hierarchical classification boosts the weighted-F1 score of the best model by up to 12% with $L = 3$, achieving state-of-the-art performance on these large class spaces (up to 93%). The shared-embedding strategy addresses its main drawback—higher inference time—recovering up to 92% of the speed of the corresponding flat model.

**Temporal Generalization Study.** Traditional ML evaluation methods like standard K-Fold CV or stratified splits ensure robustness but ignore the temporal nature of CVE disclosures, limiting their relevance for real-world forecasting. To better capture this, we adopt a temporal evaluation: models are trained

<footnote n="4">⁴ x86_64 AMD Ryzen 9 5900X CPU, 125 GB RAM, NVIDIA GeForce RTX 3090 Ti 24 GB, Ubuntu 22.04.1</footnote>
on past CVEs and tested on future ones. This setup answers the key question—*can a model trained up to year X generalize to CVEs disclosed after X?* (e.g., 1999→2000+, 1999-2000→2001+, etc.)—highlighting whether new CVEs introduce unseen descriptions or label distributions hindering generalization.

<figure><img src="image_7.png" /><figcaption>Fig. 5: Training set coverage, unique label coverage, and performance trends under recursive temporal splitting using the flat SecureBERT classifier.</figcaption></figure>

*Setup.* We employ a temporal K-Fold CV strategy [32] in which models are iteratively retrained on all CVEs disclosed up to year *X* and evaluated on CVEs disclosed from *X* + 1 onward. Note that the model may be inherently unable to predict the right classes for all samples, as classes introduced after year *X* fall outside its class space if no samples existed before that year. To ensure a fair comparison, the number of training epochs is inversely scaled according to the current size of the training set, following a procedure similar to the hierarchical experiment, so that earlier years with fewer samples are trained proportionally longer on the smaller dataset. We also report: (i) the % of CVEs included in the training set relative to the full dataset, indicating how much of the total labeled data available today was available at each year; (ii) the % of classes observed until each year relative to now, reflecting when certain labels first appeared; and (iii) the change in each metric resulting from the addition of new data.

*Discussion.* Figure 5 reports recursive temporal results revealing key insights:

- **Label coverage.** CAPEC and ATT&CK already cover most labels early on (~60% of those available now by 1999), while CWE labels are expanding mainly in more recent years, reflecting its evolving taxonomy: CVEs with many new weaknesses are appearing more recently. Under the assumption that no new labels emerge, models may not require further adaptation if they continue to perform well on future data. However, as soon as a single new label appears, the classifier must be fine-tuned to accommodate the new class in the output layer. As new labels continue to appear—very important especially for CWE—hierarchical models offer flexibility, requiring
fine-tuning only for the relevant sub-classifier instead of retraining a large
flat classifier over the entire label space.

- **Dataset size.** Although data growth is more concentrated in recent years, the largest performance gains occur in earlier years, suggesting that data quality benefits more than data quantity. Notably, a particularly large performance gain is observed in 2002, with absolute improvements of 23% (CWE), 41% (CAPEC), and 12% (ATT&CK) on the weighted-F1 score—despite the dataset represented less than 1% of the data available today.

- **Performance.** On average, model performance improves steadily over time across all metrics. Nonetheless, we observe occasional dips, where adding a single year of training data slightly deteriorates some performance indicators in the future. Taking the weighted-F1 score as reference, the average yearly improvement is of 3.0% (CWE), 3.0% (CAPEC), and 2.6% (ATT&CK).

Takeaway #4

For these classification tasks, data availability is no longer the pri-
mary bottleneck, as most gains were achieved early. The field needs new
methodologies (e.g., our hierarchical approach) to efficiently incorporate
emerging classes without full model fine-tuning.

6 Conclusions & Future Work

This paper presents a unified study of multi-label vulnerability classification
across the three most widely used taxonomies at their most granular level, lever-
aging the largest up-to-date dataset (228k samples). Unlike prior work, our ap-
proach standardizes classification across all taxonomies using multiple stratified
splits, multi-label and class-imbalance metrics, and hyperparameter optimization
over several encoder-only models. Exploiting the hierarchical structure boosts
performance, with weighted-F1 gains of 12% (CWE), 8% (CAPEC), and 12%
(ATT&CK subtechniques). To address the main drawback—increased inference
time—we introduce a shared-embedding strategy. Temporal splits demonstrate
the model’s ability to generalize to future CVEs, while the evolving class space
underscores the need for continuous fine-tuning—a challenge well addressed by
the proposed hierarchical approach. Future work could explore interpretability
techniques and incorporate additional disclosure features in the input space,
such as the exploit availability or reporter/vendor information. The primary
limitation of the hierarchical approach is its increased training cost, as it re-
quires fine-tuning the same LM multiple times on different subsets of classes. All
experiments are fully reproducible using the CVE-LMTune library.

Acknowledgment

This work has been partially supported by the French National Research Agency under the France 2030 label (Superviz ANR-22-PECY-0008). The views reflected herein do not necessarily reflect the opinion of the French government.
References

1. Abdeen, B., Al-Shaer, E., Singhal, A., Khan, L., Hamlen, K.: Smet: Semantic mapping of cve to att&ck and its application to cybersecurity. In: Data and Applications Security and Privacy XXXVII: 37th Annual IFIP WG 11.3 Conference, DBSec 2023 (2023). https://doi.org/10.1007/978-3-031-37586-6_15

2. Aghaei, E., Niu, X., Shadid, W., Al-Shaer, E.: Securebert: A domain-specific language model for cybersecurity. In: Li, F., Liang, K., Lin, Z., Katsikas, S.K. (eds.) Security and Privacy in Communication Networks (2023). https://doi.org/10.1007/978-3-031-25538-0_3

3. Ampel, B., Samtani, S., Ullman, S., Chen, H.: Linking common vulnerabilities and exposures to the mitre att&ck framework: A self-distillation approach. In: Proceedings of 2021 ACM Conference Knowledge Discovery and Data Mining (KDD’ 21) Workshop on AI-enabled Cybersecurity Analytics (2021). https://doi.org/10.48550/arXiv.2108.01696

4. Aota, M., Kanehara, H., Kubo, M., Murata, N., Sun, B., Takahashi, T.: Automation of vulnerability classification from its description using machine learning. In: 2020 IEEE Symposium on Computers and Communications (2020). https://doi.org/10.1109/ISCC50000.2020.9219568

5. Babbar, R., Partalas, I., Gaussier, E., Amini, M.R.: On flat versus hierarchical classification in large-scale taxonomies. In: Proceedings of the 27th International Conference on Neural Information Processing Systems (2013)

6. Bayer, M., Kuehn, P., Shanehsaz, R., Reuter, C.: Cysecbert: A domain-adapted language model for the cybersecurity domain. ACM Trans. Priv. Secur. (2024). https://doi.org/10.1145/3652594

7. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Nee-lakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems (2020). https://doi.org/10.48550/arXiv.2005.14165

8. Ciavotta, A., Palma, A., Lenti, S., Bonomi, S.: Threatlinker: An nlp-based methodology to automatically estimate cve relevance for capec attack patterns (2026). https://doi.org/10.48550/arXiv.2501.07131, to appear in the 21st European Dependable Computing Conference (EDCC 2026)

9. Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: Electra: Pre-training text encoders as discriminators rather than generators. In: Proceedings of the International Conference on Learning Representations (ICLR) (2020). https://doi.org/10.48550/arXiv.2003.10555

10. Das, S.S., Serra, E., Halappanavar, M., Pothen, A., Al-Shaer, E.: V2w-bert: A framework for effective hierarchical multiclass classification of software vulnerabilities. In: 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (2021). https://doi.org/10.1109/DSAA53316.2021.9564227

11. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2019). https://doi.org/10.18653/v1/N19-1423
12. El Jaouhari, S., Tamani, N., Jacob, R.I.: Guardlink: Dynamic linking of cve to mitre att&ck techniques using machine learning. In: GLOBECOM 2024 - 2024 IEEE Global Communications Conference (2024). https://doi.org/10.1109/GLOBECOM52923.2024.10900988

13. Galeax: CVE2CAPEC: Automated pipeline linking cves, cwe, capec, and mitre att&ck. https://github.com/Galeax/CVE2CAPEC

14. Gawron, M., Cheng, F., Meinel, C.: Automatic vulnerability classification using machine learning. In: Risks and Security of Internet and Systems (2018). https://doi.org/10.1007/978-3-319-76687-4_1

15. Giannakopoulos, T., Maliatsos, K.: On the usage of nlp on cve descriptions for calculating risk. In: Computer Security. ESORICS 2023 International Workshops (2024). https://doi.org/10.1007/978-3-031-54204-6_6

16. Haddad, A., Aaraj, N., Nakov, P., Mare, S.F.: Automated mapping of cve vulnerability records to mitre cwe weaknesses (2023). https://doi.org/10.48550/arXiv.2304.11130

17. Han, Z., Li, X., Xing, Z., Liu, H., Feng, Z.: Learning to predict severity of software vulnerability using only vulnerability description. In: 2017 IEEE International Conference on Software Maintenance and Evolution (2017). https://doi.org/10.1109/ICSME.2017.52

18. Kumarasinghe, U., Lekssays, A., Sencar, H.T., Boughorbel, S., Elvitigala, C., Nakov, P.: Semantic ranking for automated adversarial technique annotation in security text. In: Proceedings of the 19th ACM Asia Conference on Computer and Communications Security (2024). https://doi.org/10.1145/3634737.3645000

19. Kun: Secbert: A pretrained language model for cyber security text (2024), https://github.com/jackaduma/SecBERT

20. Kuppa, A., Aouad, L., Le-Khac, N.A.: Linking cve's to mitre att&ck techniques. In: Proceedings of the 16th International Conference on Availability, Reliability and Security. ARES '21 (2021). https://doi.org/10.1145/3465481.3465758

21. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report (2024). https://doi.org/10.48550/arXiv.2412.19437

22. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach (2019). https://doi.org/10.48550/arXiv.1907.11692

23. Liu, Z., Shi, J., Buford, J.F.: Cyberbench: A multi-task benchmark for evaluating large language models in cybersecurity. AAAI-24 Workshop on Artificial Intelligence for Cyber Security (AICS) (2024)

24. Mendsaikhan, O., Hasegawa, H., Yamaguchi, Y., Shimada, H.: Automatic mapping of vulnerability information to adversary techniques. In: Proceedings of the Fourteenth International Conference on Emerging Security Information, Systems and Technologies (SECURWARE) (2020)

25. MITRE Corporation: Common Vulnerabilities and Exposures (CVE). https://cve.mitre.org/ (1999)

26. MITRE Corporation: Common Weakness Enumeration (CWE). https://cwe.mitre.org/ (2006)

27. MITRE Corporation: Common Attack Pattern Enumeration and Classification (CAPEC). https://capec.mitre.org/ (2007)

28. MITRE Corporation: MITRE ATT&CK: Adversarial Tactics, Techniques, and Common Knowledge. https://attack.mitre.org/ (2013)

29. National Institute of Standards and Technology (NIST): National Vulnerability Database (NVD). https://nvd.nist.gov/ (2005)
30. Peeperkorn, M., Kouwenhoven, T., Brown, D., Jordanous, A.: Is temperature the creativity parameter of large language models? (2024). https://doi.org/10.48550/arXiv.2405.00492

31. Pereira, R.B., Plastino, A., Zadrozny, B., Merschmann, L.H.: Correlation analysis of performance measures for multi-label classification. Information Processing Management (2018). https://doi.org/10.1016/j.ipm.2018.01.002

32. Roberts, D.R., Bahn, V., Ciuti, S., Boyce, M.S., Elith, J., Guillera-Arroita, G., Hauenstein, S., Lahoz-Monfort, J.J., Schröder, B., Thuiller, W., Warton, D.I., Wintle, B.A., Hartig, F., Dormann, C.F.: Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography (2017). https://doi.org/10.1111/ecog.02881

33. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter (2020). https://doi.org/10.48550/arXiv.1910.01108

34. Sechidis, K., Tsoumakas, G., Vlahavas, I.: On the stratification of multi-label data. In: Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2011. (2011). https://doi.org/10.1007/978-3-642-23808-6_10

35. Shahid, M.R., Debar, H.: Cvss-bert: Explainable natural language processing to determine the severity of a computer security vulnerability from its description. In: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA) (2021). https://doi.org/10.1109/ICMLA52953.2021.00256

36. Simonetto, S., Bosch, P.: Comprehensive threat analysis and systematic mapping of CVEs to MITRE framework. In: Proceedings of the First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS) (2024)

37. Terranova, F., Lahmadi, A., Chrisment, I.: Scalable and generalizable rl agents for attack path discovery via continuous invariant spaces. In: 2025 28th International Symposium on Research in Attacks, Intrusions and Defenses (RAID) (2025). https://doi.org/10.1109/RAID67961.2025.00029

38. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al.: Llama: Open and efficient foundation language models (2023). https://doi.org/10.48550/arXiv.2302.13971

39. Wang, Z., Pang, Y., Lin, Y.: Large language models are zero-shot text classifiers (2023). https://doi.org/10.48550/arXiv.2312.01044

40. Watanabe, S.: Tree-structured parzen estimator: Understanding its algorithm components and their roles for better empirical performance (2023). https://doi.org/10.48550/arXiv.2304.11127

41. Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering (2014). https://doi.org/10.1109/TKDE.2013.39
## A Prompt

### Prompt Template

You are a classification assistant in the cybersecurity domain. Only return valid lists of IDs as requested (e.g., ["LABEL1", "LABEL2", ...]). Do NOT include any explanations, commentary, or extra text.

CVE ID: {ID}

Description: {DESCRIPTION}

Task: Perform a multi-label classification to {TAXONOMY} for this CVE.

To fairly evaluate the zero-shot capabilities of LLMs in all taxonomies, the above prompt template was used, each time substituting the CVE identifier, description, and taxonomy name (CWEs, CAPECs, ATT&CK subtechniques). Including the full set of class labels directly in the prompt is infeasible in our setting. The taxonomies contain hundreds of classes—for instance, the CWE taxonomy in our dataset has more than 680 entries—and a meaningful prompt would require not only the identifiers but also their full human-readable names to avoid ambiguity. This would produce an excessively large prompt, far beyond the context-window limits of current LLMs, while also introducing substantial computational overhead and distorting any comparison with encoder-based fine-tuning. For these reasons, we adopted the compact prompt formulation shown above.

## B Hyperparameters

For hyperparameter optimization, we employed TPE with 25 trials per LM (BERT and ELECTRA). To reduce the computational overhead of K-fold cross-validation during hyperparameter search, we used a single train-validation-test split. The hyper-parameter ranges reported in Table 5 were explored.

<table><caption>Table 5: Search space for hyperparameters and loss functions (bold = best).</caption><thead><tr><th>Alg.</th><th>Batch Size</th><th>LR</th><th>MGN</th><th>WU%</th><th>Losses</th></tr></thead><tbody><tr><td>BERT</td><td>8, 16, 32, 64</td><td>1e-5, 2e-5, 5e-5</td><td>0.25, 0.5, 0.75</td><td>1, 5, 10</td><td>ASL, FL, Poly, DB, DB-Focal, BCE, WBCE</td></tr><tr><td>ELECTRA</td><td>8, 16, 32, 64</td><td>1e-5, 2e-5, 5e-5</td><td>0.25, 0.5, 0.75</td><td>1, 5, 10</td><td>ASL, FL, Poly, DB, DB-Focal, BCE, WBCE</td></tr></tbody></table>
The optimization objective was to maximize the validation micro-F1 score, using CWE classification as the reference task for determining the optimal hyperparameters. CWE was prioritized because correctly identifying a CWE may allow manual mapping to CAPEC and ATT&CK techniques, making it the most relevant classification task. Standard hyperparameters—including batch size, learning rate (LR), maximum gradient norm (MGN), and warm-up steps percentage (WU%)—were explored, alongside several loss functions including both standard objectives and imbalance-aware variants: Asymmetric Loss (ASL), Focal Loss (FL), PolyLoss (Poly), Distribution-Balanced Loss (DB), Distribution-Balanced Focal Loss (DB-Focal), Binary Cross-Entropy (BCE), and Weighted Binary Cross-Entropy (WBCE). The hyperparameters identified for each model were applied consistently across all classification taxonomies. Moreover, the hyperparameters found for BERT were used for all its variants, ensuring a fair comparison among them. Other, less critical hyperparameters were kept fixed across all experiments to ensure consistency and reproducibility:

- *Gradient Accumulation Steps*: 1 (updates occur after every batch)

- *Learning Rate Scheduler*: constant

- *Weight Decay*: 0.01, to gently regularize the model

- *Number of Epochs*: 1 (proportionally higher for slave models or smaller temporal splits with a maximum possible of 30; see Section 5.2)

- *Early Stopping Patience*: 3 (stop training when no improvement on validation is observed)

- *Random Seed*: 42, to allow exact reproducibility

- *Sigmoid Threshold*: 0.5 (standard cut-off for multi-label classification)

- *Do Lowercase*: True / *Padding*: max length / *Truncation*: True

- *Maximum Sequence Length*: 512 tokens

All other hyperparameters were left at their library default values.