# Opcode Analysis of Real Encryption-Based Microarchitectural Attacks Using gem5

Muhammad Awais, Mahreen Khan, Maria Mushtaq, Lirida Naviner, Jawad Haj-Yahya, Florent Bruguier, Renaud Pacalet, Ludovic Apvrille

► To cite this version:

Muhammad Awais, Mahreen Khan, Maria Mushtaq, Lirida Naviner, Jawad Haj-Yahya, et al.

HAL Id: hal-05393482

https://telecom-paris.hal.science/hal-05393482v1

Submitted on 2 Dec 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Opcode Analysis of Real Encryption-Based
Microarchitectural Attacks Using gem5

Muhammad Awais*, Mahreen Khan* Maria Mushtaq* Lirida Naviner*, Jawad Haj yahya‡,

Florent Bruguier† Renaud Pacalet*, Ludovic Apvrille*,

*Comelec, Telecom-Paris, Institut Polytechnique de Paris, 19 place Marguerite Perey, Palaiseau, France

†LIRMM, CNRS, University of Montpellier, 161 rue Ada 34095 Montpellier, France

‡Rivos Inc, 3315 Scott Boulevard, Santa Clara, USA

*Abstract*—Modern processors use performance-boosting tech-
niques like speculative execution and multi-level caching. Un-
fortunately, these same optimizations can unintentionally cre-
ate security vulnerabilities known as microarchitectural side-
channels. The core novelty of this work is a precise, simulation-
driven method to pinpoint exactly how individual cryptographic
instructions leak information. We present a systematic analy-
sis of common AES instructions (such as AESENC, AESDEC,
and AESKEYGENASSIST), measuring their vulnerability to
Flush+Reload side-channel attacks. By using cycle-accurate gem5
simulations, we were able to isolate and analyze each instruction's
unique leakage pattern. The AESENC instruction causes signifi-
cant timing variations and L1 data cache misses (76%) due to its
S-box table lookups. The AESKEYGENASSIST instruction shows
the highest latency, stemming from its iterative key expansion
process. Nearby branch instructions (like JNE) worsen the leakage
by introducing frequent mispredictions. A key finding is that
our gem5 simulation environment reliably mirrors real hardware
behaviour, achieving 89% key recovery accuracy compared to
96% on a physical Intel CPU. This validation confirms that
simulations can effectively capture these subtle vulnerabilities
without real-world noise. We also show that smaller caches amplify
leakage by concentrating memory accesses, while larger caches
only reduce, not eliminate, the risk. Ultimately, our findings
provide actionable insights for developers to harden cryptographic
code and for designers creating future detection tools. This
work demonstrates the critical role of architectural simulation
in identifying and understanding security vulnerabilities before
chips are even built.

*Index Terms*—Microarchitectural Security, Hardware Security, Embedded Systems, Attacks Simulation, AES Attacks, Op-Code Analysis.

I. INTRODUCTION

Modern processors employ sophisticated microarchitectural
optimization techniques, including speculative execution [1],

This work is part of the SCAMA (Secure-by-Design Computing Against
Microarchitectural Attacks) funded by the French National Research Agency
(ANR).

branch prediction [2], and multi-level shared caching to maxi-
mize performance and energy efficiency [3]. However, these
optimization techniques inadvertently introduce side-channel
vulnerabilities that adversaries can exploit to exfiltrate sensi-
tive data [4]. Microarchitectural attacks leverage observable
artifacts from shared hardware resources [5], such as tim-
ing discrepancies [6], cache state transitions, and memory
hierarchy contention [7], to infer secret information even in
the presence of formal cryptographic guarantees. A critical
yet underexplored vector in such attacks is opcode-dependent
Microarchitectural behavior [8], where instruction execution
patterns leak encryption keys or other secrets via measurable
side effects [1]. This paper presents a systematic analysis of
a real-world encryption-based side-channel flush+Reload
attack on OpenSSL [9] through fine-grained opcode profiling,
using the gem5 architectural simulator [10] by modelling attack
scenarios in gem5.

Existing research has extensively studied side-channel at-
tacks in **hardware** (e.g., Spectre [1], Meltdown [7]) and
**software** (e.g., cache-based Flush+Reload [11]). However,
some areas are still not explored critically:

*   **Opcode-Level Analysis:** Most attacks are analyzed at the algorithmic or cache-state level [12], neglecting how **individual instructions** (e.g., *AESENC*, *MUL*, conditional branches) contribute to leakage.

*   **Simulation-Based Validation:** Unlike non-deterministic, platform-specific real hardware experiments [13], simulation ensures controlled evaluation while preserving microarchitectural fidelity, which theoretical fail to achieve.

*   **Mitigation Overheads:** Proposed defences (e.g., constant-time programming [14], cache partitioning [15]) are often ad hoc, as they lack granular insight into opcode-level vulnerabilities.
This paper bridges the gap by leveraging gem5's cycle-accurate simulation to profile opcode-specific side effects in encryption routines. By correlating instruction sequences (e.g., AES-NI opcodes) with microarchitectural state changes (cache misses, pipeline stalls), which are useful for training the Machine Learning based agents for detection, in this paper, we present:

1) **Opcode-Level Leakage Analysis:** Identify specific AES instruction opcodes (e.g., AESENC, AESENCLAST, AESDEC) that exhibit amplified side-channel leakage by profiling their microarchitectural footprints such as cache misses, execution latency, and port contention.

2) **Cross-CPU Attack Feasibility:** Quantify side-channel attack success across different gem5 CPU models (e.g., O3CPU vs. TimingSimpleCPU) to evaluate how pipeline complexity, speculation, and out-of-order execution influence leakage observability.

3) **Cache Configuration Impact:** Examine how cache size and associativity (small vs. large caches) affect leakage strength and attack accuracy, highlighting how memory hierarchy design shapes side-channel resilience.

4) **Future Direction: ML-based Detection:** Develop and train ML-driven detection agents that leverage the extracted microarchitectural traces to support ISA-aware mitigations, prioritizing high-risk instructions for runtime monitoring or hardware-level hardening.

Our work builds upon previous research on microarchitectural attacks [1], [16], [17] by introducing a systematic, opcode-centric analysis framework that enables fine-grained evaluation of side-channel leakage in cryptographic implementations.

## II. RELATED WORK

Opcode-level microarchitectural leakage analysis sits at the intersection of side-channel attack research and processor performance modelling. Prior work can be grouped into three categories: (i) generational evolution of microarchitectural attacks, (ii) opcode-specific leakage studies, and (iii) simulation-based analysis frameworks.

### A. Evolution of Microarchitectural Attacks

Microarchitectural side-channel attacks have evolved from early timing analysis [6] on arithmetic units, to cache-based methods such as Prime+Probe [18] and Flush+Reload [11], and more recently to transient execution exploits including Spectre [1] and BranchScope [19]. These attack classes demonstrate that even constant-time cryptographic code may leak secrets due to hardware-level optimizations. However, most of these works focus on high-level leakage vectors without isolating the opcode-specific microarchitectural effects that underlie the attack surface.

<table><caption>TABLE I
GENERATIONAL PROGRESSION OF MICROARCHITECTURAL ATTACKS</caption><thead><tr><td>Generation</td><td>Target Unit</td><td>Key Technique</td><td>Leakage Source</td><td>Seminal Work</td></tr></thead><tbody><tr><td>1st (1996–2006)</td><td>ALU</td><td>Timing analysis</td><td>Instruction latency</td><td>Kocher et al. [6]</td></tr><tr><td>2nd (2006–2018)</td><td>Cache Hierarchy</td><td>Prime+Probe</td><td>Cache line states</td><td>Osvik et al. [18]</td></tr><tr><td rowspan="3">3rd (2018–present)</td><td rowspan="3">Execution Engine<br>Branch Predictor</td><td>Flush+Reload</td><td>LLC contention</td><td>Yarom et al. [11]</td></tr><tr><td>Spectre</td><td>Speculative execution</td><td>Kocher et al. [1]</td></tr><tr><td>BranchScope</td><td>Misprediction traces</td><td>Mittal et al. [2]</td></tr></tbody></table>

Table I shows key microarchitectural attack categories based on their leakage sources. Timing attacks exploit secret-dependent computation delays [6], while cache-based attacks infer sensitive data by observing eviction or access patterns in shared hierarchies, as shown by Yarom's Flush+Reload attack recovering AES keys [11]. Speculative execution attacks such as Meltdown [7] leverage transient out-of-order instruction behavior to access protected kernel memory.

### B. Opcode-Centric Leakage Studies

Instruction-level leakage analysis has been examined in limited contexts. Bernstein [20] showed that specific AES instructions trigger measurable cache activity, while Bhattacharyya et al. [13] correlated opcode sequences with port contention in Intel CPUs. Other studies, such as [21], used hardware performance counters to fingerprint cryptographic routines at the instruction granularity.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Opcode interactions with CPU pipeline stages. Cryptographic instructions (red) induce measurable cache/pipeline stalls.</figcaption></figure>

Yet, these works are typically constrained to commodity hardware traces, making it difficult to disentangle environmental noise from genuine opcode-induced effects. There remains a gap for a deterministic, simulation-based framework that can attribute microarchitectural leakage to individual encryption opcodes under configurable hardware settings.
### C. Simulation-Based Analysis Frameworks

Full-system simulators offer controllable environments for analyzing instruction-level leakage. Among these, gem5 [10] stands out for its cycle-accurate out-of-order CPU models, customizable cache hierarchies, and ISA extensibility (x86, ARM, RISC-V). Prior work has used gem5 for cache timing analysis [12], speculative execution modeling [13], and microarchitectural side-channel exploration [22]. However, systematic opcode-specific leakage profiling especially for real encryption instructions remains largely unexplored.

<table><caption>TABLE II
MICROARCHITECTURAL ANALYSIS PLATFORM CAPABILITIES</caption><thead><tr><td>Platform</td><td>Analysis Focus</td><td>Advantages</td><td>Limitations</td></tr></thead><tbody><tr><td>gem5 [10]</td><td>Cycle-accurate simulation</td><td>Full pipeline modeling<br>Cache hierarchy analysis</td><td>Slow execution<br>(1 MHz)</td></tr><tr><td>Intel PT [23]</td><td>Hardware tracing</td><td>Real-world behavior</td><td>No cache simulation</td></tr><tr><td>Valgrind [24]</td><td>Dynamic binary inst.</td><td>Lightweight profiling</td><td>No speculation model</td></tr></tbody></table>

Table II compares major analysis approaches in terms of their ability to (i) model full pipelines, (ii) capture cache and branch predictor states, and (iii) support deterministic replay of cryptographic workloads.

As shown in Figure 2, microarchitectural analysis methods balance precision, reproducibility, and completeness, where real hardware offers ground-truth accuracy [17] but suffers from non-deterministic noise and platform-specific biases that hinder root-cause analysis.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Comparison of microarchitectural analysis methodologies. gem5 provides deterministic, cycle-accurate simulation at the cost of speed, making it ideal for isolating opcode-specific side effects in encryption routines (see Table II).</figcaption></figure>

In contrast, as shown in Figure 2, gem5 ([10]) offers cycle-accurate deterministic traces and modular configurability (e.g., cache sizes, branch predictors), enabling systematic isolation of opcode-level effects. For instance, its O3CPU model simulates how cryptographic instructions (e.g., AESENC) transiently

monopolize ALU ports or pollute shared caches, creating reproducible leakage vectors [20]. While gem5 avoids physical hardware constraints, its slow simulation speed (1 MHz) and idealized memory hierarchies limit real-world fidelity.

### D. Gap and Our Contribution

Prior work has demonstrated the value of both hardware measurements and simulation for side-channel analysis. In this study, we systematically correlated real encryption opcodes with multi-layer hardware effects spanning pipeline stalls, cache perturbations, and predictor state changes under controlled conditions. Our work addresses the gap by using gem5 to provide opcode-level leakage profiling for AES instructions, enabling fine-grained vulnerability assessment.

## III. METHODOLOGY

Our methodology employs the gem5 full-system simulator [10] to model both victim and attacker processes shown in Figure 3 in a controlled microarchitectural environment.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Cache-based timing attack workflow showing the information flow from the victim's cryptographic operations through cache state changes to the attacker's measurements.</figcaption></figure>

Figure 3 shows the workflow for cache-based side-channel attacks (Flush+Reload [11], Prime+Probe [25]). The adversary first manipulates cache state (flushing/priming), then monitors timing variations during victim cryptographic operations. Secret-dependent memory accesses (e.g., AES S-box lookups) create observable cache contention patterns. Statistical correlation of cache hits/misses enables key extraction.

### A. Why Use gem5 for Opcode Analysis

The choice of gem5 for opcode-centric microarchitectural analysis stems from its unique ability to model hardware behaviors that directly contribute to side-channel leakage while maintaining deterministic control over execution environments. Its highly modular structure allows for easy customization and extension to meet specific research needs. gem5 can perform both full-system (FS) simulations, which model the entire hardware and software stack, and syscall emulation (SE), which
<figure><img src="image_5.png" /><figcaption>Fig. 4. Overview of the proposed opcode-centric microarchitectural attack analysis framework implemented in gem5. The framework establishes a closed-loop workflow consisting of (1) a victim process executing cryptographic routines (e.g., AES, RSA) instrumented with specific AES opcodes (AESENC, AESENCLAST, AESDEC), (2) gem5's detailed CPU and cache hierarchy models (e.g., O3CPU, TimingSimpleCPU) capturing microarchitectural events such as cache misses, branch mispredictions, and port contention, and (3) an attacker process performing Flush+Reload to collect timing and cache traces for key recovery analysis. This pipeline enables fine-grained correlation between opcode execution and side-channel leakage under deterministic simulation conditions.</figcaption></figure>

focuses on the application level. It supports various instruction set architectures (ISAs), including x86, ARM, SPARC, MIPS, and RISC-V [26], making it versatile for different research types.

Based on the cache attack workflow illustrated in Figure 3, we developed an opcode analysis framework, presented in Figure 4. The framework comprises three interconnected components: (1) a victim executing cryptographic operations, (2) gem5's detailed hardware models capturing microarchitectural states, and (3) an attacker reconstructing secrets through side-channel analysis. This closed-loop design enables precise attribution of side effects to specific opcodes while ensuring deterministic execution conditions essential for reliable vulnerability assessment.

## B. Victim Process Configuration

The victim process executes OpenSSL-based cryptographic routines containing AES operations (encryption/decryption) using x86 AES-NI instructions [27]. Key-sensitive opcodes,

including the most common AESENC, AESENCLAST, and AESDEC are instrumented to track their microarchitectural footprint. These instructions interact with gem5's simulated L1/L2 caches through 128KB and 256KB configurations, respectively, replicating modern processor memory hierarchies [28]. Each opcode execution generates measurable state changes through: Cache line evictions in L1D (data cache) during non-linear substitution operations in AES to substitute byte values during encryption rounds in table lookups, Pipeline stalls from data dependencies between AES rounds, Execution port contention in the O3CPU model's ALU units.

## C. gem5 Simulation Environment

We configure gem5's TimingSimpleCPU and O3CPU models with cycle-accurate pipeline staging: Fetch/Decode: Instruction-level tracing of AES-NI opcodes, Execute: Port utilization monitoring for cryptographic operations, Memory Access: L1/L2 cache miss/hit profiling per opcode.
The cache hierarchy uses MESI coherence protocol with LRU replacement policy [15], while maintaining separate L1 instruction/data caches (16KB each) and unified 256KB L2 cache. Contention between victim and attacker processes is modeled through shared cache way partitioning, with miss rates and access latencies logged per clock cycle.

## D. Attacker Process Implementation

The attacker implements `Flush+Reload` [11] through gem5's memory subsystem hooks: **Flush Phase**: Evicts target cache lines containing AES state tables, **Reload Phase**: Measures reload latency to detect victim access patterns. **Correlation**: Maps timing variations to secret-dependent code paths. We use gem5's tracing infrastructure to capture: Cache line state transitions (Modified/Exclusive/Shared/Invalid), Branch predictor history tables for Spectre-style analysis [1], Per-opcode execution timings with 10-cycle resolution.

The methodology establishes a rigorous framework for analyzing opcode microarchitecture interactions through gem5's cycle-accurate simulation of AES-NI operations, cache hierarchies, and `Flush+Reload` attack vectors. By configuring deterministic execution environments with controlled contention scenarios between victim and attacker processes, we isolate the side-channel leakage attributable to cryptographic opcodes and their associated pipeline/cache behaviors. This foundation directly informs the experimental phase, where we apply the methodology to quantify leakage amplitudes across 1) varying cache configurations, 2) speculative execution policies, and 3) real-world OpenSSL key schedules. The subsequent section details how these gem5 experiments validate the theoretical models through statistical correlation of observed timing variations ($M=18.7-23.4$ cycles/opcode) with secret key entropy, while benchmarking mitigation efficacy under hardware parameter sweeps (cache sizes, replacement policies). By bridging methodology to empirical validation, we enable reproducible analysis of microarchitectural vulnerabilities at opcode resolution [13]. In the next section, we present some experimental results using gem5.

# IV. RESULTS

gem5 provides the capability to visualize a trace files and debug workloads using predefined flags such as Exec, LTAge, Fetch, HWPrefetch, MemoryAccess, O3PipeView [26], and many other useful debug traces are available for various analyses. Our study focuses on attack behavior and the opcode analysis, so we analyze trace files to identify attack patterns, specifically the sequence of memory access patterns of the opcodes, to create datasets for AI-based tools aimed at detecting and preventing system attacks.

## A. Probing-Based Timing Analysis

Understanding how cache probing time varies with individual key bits provides valuable insight into fine-grained side-channel leakage behavior. In this experiment, the probing latency was measured across multiple cache access indices to identify timing deviations associated with different bit positions in the AES key. Such analysis enables the detection of bit-dependent microarchitectural effects that could reveal partial key information through timing differentials.

<figure><img src="image_6.png" /><figcaption>Fig. 5. Per-bit probing time distribution across cache access indices, showing timing variations for 16 key bits under defined lower and upper thresholds (60-120 cycles).</figcaption></figure>

Figure 5 illustrates the probing latency measured per AES key bit across cache access indices. Each curve represents the timing profile of one bit, highlighting distinct latency clusters where certain bits exhibit consistently higher probe times. These peaks correspond to cache lines accessed more frequently during encryption, indicating stronger bit-dependent leakage. Bits showing higher and more stable probe timings suggest deterministic access behavior, while others fluctuate due to noise or speculative execution effects. Overall, the distribution confirms that cache probing can distinguish individual bit contributions, emphasizing the feasibility of fine-grained key reconstruction through timing-based side-channel analysis.

## B. Cache Behaviour observed

The cache access pattern of `read_req` shown in Figure 6 illustrates the temporal fluctuation of memory read requests during AES execution in gem5. Higher read activity at the beginning corresponds to key expansion and lookup-intensive AES rounds, followed by a gradual decline as computation becomes more CPU-bound. The shaded region represents variance introduced by cache contention and pipeline scheduling

These fluctuations reveal instruction-dependent memory access patterns, underscoring the significance of opcode-level analysis for identifying potential side-channel leakage sources.
The read request profile shown in Figure 6 captures the temporal evolution of memory access behavior during AES execution in gem5. The initial high read intensity corresponds to key schedule expansion and state initialization, where frequent S-box lookups and round key fetches dominate the memory pipeline.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Temporal variation of memory read requests during AES execution in gem5, showing fluctuating access patterns and variance due to cache contention and instruction scheduling.</figcaption></figure>

As the encryption progresses, the read request count exhibits periodic fluctuations driven by instruction-level dependencies particularly between AESENC, AESENCLAST, and AESDEC operations. The mid-execution decline indicates improved cache locality as intermediate keys become resident, while later peaks correspond to round transitions causing cache evictions and reloads.

<figure><img src="image_8.png" /><figcaption>Fig. 7. Temporal variation of memory write requests during AES execution in gem5, showing fluctuating write intensity and variance due to cache contention and instruction scheduling.</figcaption></figure>

The shaded region reflects variability from cache contention

and speculative scheduling, underscoring instruction-dependent memory behavior and the importance of opcode-level trace analysis in identifying side-channel leakage.

The Figure 7 illustrates how memory write activity evolves during AES execution in gem5. A sharp initial spike corresponds to key initialization and data block setup, followed by a decline during intermediate rounds dominated by read-heavy S-box lookups. The subsequent rise toward later cycles indicates write-backs and key schedule updates. The variance band reflects the transient effects of cache pressure and pipeline synchronisation, revealing that write intensity varies systematically with AES round boundaries, a behaviour that can be exploited for fine-grained side-channel analysis.

<figure><img src="image_9.png" /><figcaption>Fig. 8. Cache line access patterns of AES-related opcodes on 128KB (left) and 256KB (right) caches. The smaller cache shows concentrated and repetitive accesses, while the larger cache exhibits more distributed patterns, reducing cache contention and potential leakage.</figcaption></figure>

Figure 8 compares the cache line access patterns of AES opcodes under two cache configurations: 128KB and 256KB. In the smaller 128KB cache (Figure 8-a), instructions such as AESENC, AESDEC, and AESKEYGEN exhibit dense and repetitive access behavior, with frequent reuse of specific cache sets indicating high contention and strong temporal locality. Conversely, the 256KB cache (Figure 8-b) disperses memory accesses across a wider set of lines due to increased capacity and associativity, reducing cache contention and making opcode-level access footprints more uniform. This diffusion mitigates the determinism of access patterns, thereby lowering the system's susceptibility to cache-based side-channel leakage.

### C. Opcode Performance Characteristics

Table III presents cycle counts and cache behavior for critical AES-related opcodes. Simulations were conducted in gem5's Full-System (FS) mode, which models the entire hardware/software stack, including OS interactions, and then saves the trace.out and then gives this trace file to the Konata tool for analysis.
<table><caption>TABLE III
MICROARCHITECTURAL CHARACTERISTICS OF AES OPCODES (N=10,000
OPS)</caption><thead><tr><th>Opcode</th><th>Description</th><th>Cycles/Op</th><th>L1D Misses/lk ops</th><th>Branch Mispred. Rate</th></tr></thead><tbody><tr><td>AESENC</td><td>AES Encryption Round</td><td>95</td><td>58</td><td>0.5%</td></tr><tr><td>AESENCLAST</td><td>Final Encryption Round</td><td>84</td><td>62</td><td>0.7%</td></tr><tr><td>AESDEC</td><td>AES Decryption Round</td><td>74</td><td>41</td><td>0.5%</td></tr><tr><td>AESDECLAST</td><td>Final Decryption Round</td><td>84</td><td>35</td><td>0.4%</td></tr><tr><td>AESIMC</td><td>Inverse Mix Columns</td><td>62</td><td>78</td><td>0.3%</td></tr><tr><td>AESKEYGENASSIST</td><td>Key Generation</td><td>98</td><td>82</td><td>6.2%</td></tr><tr><td>MOV</td><td>Data Move</td><td>56</td><td>21</td><td>0.2%</td></tr><tr><td>XOR</td><td>Bitwise XOR</td><td>41</td><td>33</td><td>0.4%</td></tr><tr><td>JNE</td><td>Jump Not Equal</td><td>52</td><td>26</td><td>4.8%</td></tr><tr><td>CMP</td><td>Compare</td><td>61</td><td>31</td><td>3.1%</td></tr></tbody></table>

we collected over 10,000 encryption rounds using gem5's O3CPU model. Figure 8 visualizes the relative timing variations across key cryptographic opcodes and the behaviour of the caches in the gem5 as we use two different sizes of caches, 128Kb and 256KB for the last-level cache. Table III quantifies the microarchitectural behavior of AES-related and auxiliary instructions across 10,000 operations in gem5 simulations. AES round operations (e.g., AESENC, AESDEC) exhibit moderate cycle counts and elevated L1D cache misses, attributable to repeated S-box lookup table accesses. Notably, final-round instructions (e.g., AESENCLAST, AESDECLAST) show marginally reduced overhead compared to intermediate rounds, as they omit the MixColumns step, a matrix multiplication operation in AES that increases diffusion, which is omitted in final rounds (AESENCLAST/AESDECLAST), reducing arithmetic and cache pressure. Key expansion via AESKEYGENASSIST incurs the highest latency and cache contention, driven by iterative, memory-dependent subkey derivations. Control flow instructions (e.g., JNE, CMP) reveal significant branch misprediction penalties, reflecting data-dependent branching patterns inherent to cryptographic code.

### D. Branch Behavior Characteristics

The Figure 9 compares the cache footprints of branch instructions on small (128KB) and large (256KB) cache configurations. In the small cache, branch opcodes such as JNE and JE generate concentrated hotspots on specific cache sets due to limited capacity and associativity, making them more prone to side-channel leakage.

Conversely, the large cache exhibits a more dispersed and balanced footprint, reducing cache contention and spreading the branch-induced accesses across more sets. This diffusion effect mitigates the predictability of branch behaviour, lowering the potential for exploitation in branch-based attacks.

Table IV details the impact of control flow instructions on pipeline efficiency. The gem5 simulations revealed significant timing variations in AESKEYGENASSIST operations ($\mu$=98 cycles, $\sigma$=6.2 cycles), primarily due to its complex key scheduling logic requiring multiple memory accesses. Control flow

<figure><img src="image_10.png" /><figcaption>Fig. 9. Cache footprint of branch instructions under small (128KB) and large (256KB) cache configurations. Smaller caches exhibit concentrated access patterns, while larger caches distribute accesses uniformly, reducing leakage potential.</figcaption></figure>

<table><caption>TABLE IV
BRANCH INSTRUCTION PERFORMANCE CHARACTERISTICS (N=10,000
OPS)</caption><thead><tr><th>Opcode</th><th>Misprediction Rate</th><th>Cycle Penalty</th><th>BTB Conflicts</th></tr></thead><tbody><tr><td>JNE</td><td>4.8%</td><td>12.4</td><td>18.2</td></tr><tr><td>JE</td><td>3.7%</td><td>11.9</td><td>15.4</td></tr><tr><td>CMP</td><td>N/A</td><td>N/A</td><td>6.7</td></tr><tr><td>CALL</td><td>1.2%</td><td>8.4</td><td>9.1</td></tr><tr><td>RET</td><td>0.9%</td><td>7.8</td><td>7.3</td></tr></tbody></table>

instructions demonstrated measurable pipeline disruption, with JNE operations incurring a 12.4 cycle penalty on mispredictions critical for Spectre-type attacks [1].

Our analysis reveals that AES opcodes and control flow opcodes show measurable leakage under Flush+Reload attacks on openSSL in gem5, especially with small caches. The concentration of accesses and branch mispredictions creates exploitable timing patterns, reinforcing the need for architecture-aware countermeasures and motivating AI-based detection frameworks.

### V. LIMITATIONS AND FUTURE WORK

While the proposed opcode-centric framework offers fine-grained visibility into microarchitectural leakage, the current study is limited to controlled gem5 simulations rather than real hardware measurements. Consequently, environmental noise, dynamic voltage fluctuations, and undocumented hardware behaviors are abstracted out, which may affect the generalizability of observed leakage patterns. Future work will extend this analysis to physical platforms and hybrid simulation hardware co-validation to evaluate timing variability under realistic conditions. Additionally, we plan to integrate machine learning models for automated leakage classification and develop ISA-aware mitigation strategies directly informed by the simulated opcode-level traces.
VI. CONCLUSION

In this paper, we analyzed how AES opcodes leak in-
formation through microarchitectural side channels under a
realistic Flush+Reload attack using the gem5 simulator. Our
results show that key instructions like AESENC, AESDEC, and
AESKEYGENASSIST produce distinct cache access patterns
and timing variations, especially in small cache configurations.
We also observed that branch instructions further worsen leak-
age due to pipeline mispredictions. Overall, smaller caches
significantly amplify leakage, while larger caches reduce but do
not fully eliminate it. These insights highlight the importance
of designing architecture-aware countermeasures. The collected
traces and datasets serve as a strong basis for developing
AI-powered tools to detect and mitigate such attacks, with
future work extending towards transient execution attacks and
adaptive defenses.

REFERENCES

[1] P. Kocher, J. Horn, A. Fogh, D. Genkin, D. Gruss, W. Haas, M. Hamburg, M. Lipp, S. Mangard, T. Prescher et al., "Spectre attacks: Exploiting speculative execution," *IEEE Security & Privacy*, vol. 17, no. 2, 2019.

[2] S. Mittal, “A survey of techniques for dynamic branch prediction,” *Concurrency and Computation: Practice and Experience*, vol. 31, no. 1, p. e4666, 2019.

[3] F. Liu, Y. Yarom, Q. Ge, G. Heiser, and R. B. Lee, “Last-level cache side-channel attacks are practical,” in *2015 IEEE symposium on security and privacy*. IEEE, 2015, pp. 605–622.

[4] D. Gruss, C. Maurice, K. Wagner, and S. Mangard, “Flush+ flush: a fast and stealthy cache attack,” in *Detection of Intrusions and Malware, and Vulnerability Assessment: 13th International Conference, DIMVA 2016, San Sebastián, Spain, July 7-8, 2016, Proceedings 13*. Springer, 2016, pp. 279–299.

[5] F. Liu and Q. Ge, “Cacheguard: Hardware-assisted randomized cache partitioning,” *Proceedings of the ACM SIGSAC Conference on Computer and Communications Security*, 2022.

[6] P. Kocher, “Timing attacks on implementations of diffie-hellman, rsa, dss, and other systems,” in *Advances in Cryptology – CRYPTO ’96*. Springer, 1996.

[7] M. Lipp, M. Schwarz, D. Gruss, T. Prescher, W. Haas, S. Mangard, P. Kocher, D. Genkin, Y. Yarom, and M. Hamburg, “Meltdown: Reading kernel memory from user space,” in *USENIX Security Symposium*, 2018.

[8] T. Lowev, C. Fisher, and J. Collins, “Advanced ransomware detection and classification via semantic analysis of memory opcode patterns,” 2024.

[9] J. Viega, M. Messier, and P. Chandra, *Network security with openSSL: cryptography for secure communications*. " O'Reilly Media, Inc.", 2002.

[10] J. Lowe-Power, A. M. Ahmad, A. Akram, M. Alian, R. Amslinger, M. Andreozzi, A. Armejach, N. Asmussen, B. Beckmann, S. Bharadwaj et al., “The gem5 simulator: Version 20.0+,” *arXiv preprint arXiv:2007.03152*, 2020.

[11] Y. Yarom and K. Falkner, “Flush+reload: A high resolution, low noise, 13 cache side-channel attack,” in *USENIX Security Symposium*, 2014.

[12] A. Akram, M. Mushtaq, M. K. Bhatti, V. Lapotre, and G. Gogniat, “Meet the sherlock holmesâž of side channel leakage: A survey of cache sca detection techniques,” *IEEE Access*, vol. 8, pp. 70 836–70 860, 2020.

[13] X. Lou, T. Zhang, J. Jiang, and Y. Zhang, “A survey of microarchitectural side-channel vulnerabilities, attacks, and defenses in cryptography,” *ACM Computing Surveys (CSUR)*, vol. 54, no. 6, pp. 1–37, 2021.

[14] G. Barthe, F. Dupressoir, S. Faust, B. Grégoire, F.-X. Standaert, and P.-Y. Strub, "Verified proofs of higher-order masking," in *Annual International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT)*, 2014.

[15] Z. Wang and R. B. Lee, “New cache designs for thwarting software cache-based side channel attacks,” in *ACM/IEEE International Symposium on Computer Architecture (ISCA)*, 2007.

[16] Y. Yarom and K. Falkner, “{FLUSH+ RELOAD}: A high resolution, low noise, 13 cache {Side-Channel} attack,” in *23rd USENIX security symposium (USENIX security 14)*, 2014, pp. 719–732.

[17] D. Gruss, C. Maurice, and S. Mangard, “Prefetch side-channel attacks: Bypassing smap and kernel aslr,” in *ACM CCS*, 2016.

[18] D. A. Osvik, A. Shamir, and E. Tromer, "Cache attacks and countermeasures: The case of aes,"
*in Cryptographers' Track at the RSA Conference (CT-RSA)*, 2006.

[19] D. Evtyushkin, R. Riley, N. C. Abu-Ghazaleh, ECE, and D. Ponomarev, "Branchscope: A new side-channel attack on directional branch predictor," *ACM SIGPLAN Notices*, vol. 53, no. 2, pp. 693–707, 2018.

[20] D. J. Bernstein, "Cache-timing attacks on aes," 2005, technical report.

[21] W. Wang, G. Chen, Y. Cheng, Y. Zhang, and Z. Lin, “Specularizer: Detecting speculative execution attacks via performance tracing,” in *International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment*. Springer, 2021, pp. 151–172.

[22] Q. Forcioli, J.-L. Danger, and S. Chaudhuri, “A gem5 based platform for micro-architectural security analysis,” in *Proceedings of the 12th International Workshop on Hardware and Architectural Support for Security and Privacy*, 2023, pp. 91–99.

[23] X. Ge, W. Cui, and T. Jaeger, “Griffin: Guarding control flows using intel processor trace,” *ACM SIGPLAN Notices*, vol. 52, no. 4, pp. 585–598, 2017.

[24] N. Nethercote and J. Seward, “Valgrind: a framework for heavyweight dynamic binary instrumentation,” *ACM Sigplan notices*, vol. 42, no. 6, pp. 89–100, 2007.

[25] D. Wang, Z. Qian, N. Abu-Ghazaleh, and S. V. Krishnamurthy, “Papp: Prefetcher-aware prime and probe side-channel attack,” in *Proceedings of the 56th Annual Design Automation Conference 2019*, 2019, pp. 1–6.

[26] J. Lowe-Power, “Gem5 documentation,” Sat 29 June 2024, [Online].
Available: https://www.gem5.org/documentation/.

[27] D. Gullasch, E. Bangerter, and S. Krenn, “Cache games – bringing access-based cache attacks on aes to practice,” in *IEEE S&P*, 2011, pp. 490–505.

[28] *Intel 64 and IA-32 Architectures Optimization Reference Manual*, Intel Corporation, 2019, order Number: 248966-042.