# Aspycot: A Spike and Python Co-simulation Testbench for Hardware Monitoring IPs

Téo Biton, Sébastien Pillement, Olivier Gilles, Nikolai Kosmatov, Daniel

Gracia Pérez

► To cite this version:

Téo Biton, Sébastien Pillement, Olivier Gilles, Nikolai Kosmatov, Daniel Gracia Pérez. Aspycot: A Spike and Python Co-simulation Testbench for Hardware Monitoring IPs. Rapid Simulation and Performance Evaluation for Design Optimization: Methods and Tools (RAPIDO), Jan 2025, Barcelona, Spain. pp.17-23, 10.1145/3721848.3721851. hal-05440882

HAL Id: hal-05440882

https://hal.science/hal-05440882v1

Submitted on 7 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

Latest updates: https://dl.acm.org/doi/10.1145/3721848.3721851

RESEARCH-ARTICLE

**Aspycot: A Spike and Python Co-simulation Testbench for Hardware Monitoring IPs**

TÉO BITON, Nantes University, Nantes, Pays-de-la-Loire, France

SÉBASTIEN PILLEMENT, Nantes University, Nantes, Pays-de-la-Loire, France

OLIVIER GILLES, Thales Research and Technology, Palaiseau, France

NIKOLAI KOSMATOV, Thales Research and Technology, Palaiseau, France

DANIEL GRACIA PÉREZ, Thales Research and Technology, Palaiseau, France

Open Access Support provided by:

Thales Research and Technology

Nantes University

<figure><img src="image_3.png" /></figure>

Published: 21 January 2025

Citation in BibTeX format

RAPIDO '25: Rapid Simulation and Performance Evaluation for Design

January 21, 2025

Barcelona, Spain
# Aspycot: A Spike and Python Co-simulation Testbench for Hardware Monitoring IPs

Téo Biton

Thales Research and Technology
Palaiseau, France
Nantes Université, CNRS, IETR UMR
6164
Nantes, France
teobiton@thalesgroup.com

Sébastien Pillement

Nantes Université, CNRS, IETR UMR
6164
Nantes, France
Sebastien.Pillement@univ-nantes.fr

Olivier Gilles

Thales Research and Technology
Palaiseau, France
olivier.gilles@thalesgroup.com

Nikolai Kosmatov

Thales Research and Technology
Palaiseau, France
nikolai.kosmatov@thalesgroup.com

Daniel Gracia Pérez

Thales Research and Technology
Palaiseau, France
daniel.gracia-
perez@thalesgroup.com

## Abstract

As systems-on-chip (SoCs) scale in complexity, monitoring Intellectual Properties (IPs) such as trace components, performance monitoring units (PMUs), and security-related IPs have an essential role in providing insights on the system real-time behavior, allowing for more efficient debugging, performance analysis, and security assurance. In this work, we present a methodology to rapidly prototype and validate hardware monitoring IPs at the pre-silicon stage, by abstracting the rest of the system. The methodology relies on an instruction set simulator (ISS) to generate trace of executed applications and on tools to simulate these traces on the IPs. We propose a platform incorporating this methodology, called Aspycot. It leverages established projects from the open-source hardware community and is designed to be as modular and straightforward as possible. To illustrate the interest and efficiency of Aspycot, we present a case study related to control-flow monitoring to a single core architecture.

## CCS Concepts

* Hardware → Simulation and emulation; * Security and privacy → Hardware security implementation.

## Keywords

Hardware verification, co-simulation, validation methodology, open-source tool, control-flow integrity.

## ACM Reference Format:

Téo Biton, Sébastien Pillement, Olivier Gilles, Nikolai Kosmatov, and Daniel Gracia Pérez. 2025. Aspycot: A Spike and Python Co-simulation Testbench

<figure><img src="image_4.png" /></figure>

This work is licensed under a Creative Commons Attribution International 4.0 License.

RAPIDO '25, Barcelona, Spain
© 2025 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-1471-9/25/01
https://doi.org/10.1145/3721848.3721851

for Hardware Monitoring IPs. In *Rapid Simulation and Performance Evaluation for Design (RAPIDO '25), January 21, 2025, Barcelona, Spain*. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3721848.3721851

## 1 Introduction

Digital hardware technology researchers frequently need to conceive and design innovative solutions through either modeling components with high-level software languages or developing equivalent solutions directly using hardware description languages (HDLs). With increasing complexity and a demand for higher reliability in hardware systems, verification becomes an essential step in the design process to ensure functionality, performance, and security. Traditional hardware verification methods often fall short due to their time-consuming nature and limited adaptability [5], necessitating innovative approaches that can address modern design challenges. Open-source simulation and synthesis tools have emerged as a powerful alternative, facilitating collaboration and innovation within the hardware design community.

In recent years, several platforms and tools have been proposed to help researchers and engineers validate their solutions. For large-scale projects, full-system simulators are preferred for their performance and scalability, and many have been developed with their own specificities [2, 12, 20]. This leaves a gap in the validation of lower-level monitoring Intellectual Properties (IPs) that are critical to the data flow of the chip. Such IPs, while smaller and less complex than a complete multicore system, need to be tested close to the execution platform. Integrating the IP into the system and simulating the entire platform can be time-consuming both during design and test phases. Moreover, unless the core is already verified, its bugs can propagate to the monitoring IP and affect its verification process. Cores themselves are often verified using instruction set simulators (ISSs) as golden models [4], in a co-simulation [17] setting. We believe this methodology can be applied to other hardware peripherals.

In this context, we propose Aspycot, a novel testbench methodology based on an ISS, providing an efficient co-simulation environment specially for hardware monitoring IPs, e.g. trace components,
performance monitoring units, and security-related IPs. By combining the strengths of open-source tools, Aspycot aims to enhance prototyping efficiency, reduce verification time, and deliver comprehensive insights into hardware design behavior. It is written in Python to enhance productivity and to benefit from the large library ecosystem of the language, as done by other projects [8, 9, 11, 22]. The platform is built to be reused to simulate different IPs, with minimum effort required to support them. This provides an interesting insight to compare different solutions and to observe how they perform in the same test conditions. While conceived for a particular context, i.e. monitoring IPs, the proposed methodology proves effective to efficiently prototype and tune such IPs. To evaluate the methodology, a case study related to a control-flow monitoring IP is presented, and we show how the platform is used to quickly test different configurations of this IP.

The rest of this paper provides some context on open-source tools and related work in Section 2. Section 3 presents an overview of the proposed methodology and its implementation details. Section 4 shows the methodology applied to a particular use-case and how Aspycot boosts prototyping. Section 5 comments on extending the platform and improve its productivity. Finally, Section 6 gives conclusion and perspectives regarding the methodology.

# 2 Background & Related Work

## 2.1 Open-source Ecosystem

The open-source ecosystem of hardware design and development is thriving, with several accessible HDL simulators, synthesis tools and cell libraries. Through the RISC-V [15] instruction set architecture (ISA), there are a lot of open-source cores available, and associated verification tools, like RISC-V random instruction generator<fnref n="1" /> and Spike<fnref n="2" />. Open-source tools are gaining popularity in modern hardware design and verification, offering customizable, and community-driven alternatives to proprietary solutions. Simulators like Verilator<fnref n="3" /> or Icarus Verilog<fnref n="4" />, are widely used for fast, cycle-accurate simulation, while cocotb (Coroutine based co-simulation testbench)<fnref n="5" /> provides a Python-based framework for writing testbenches in high-level languages. For processor and ISA simulation, Spike and QEMU [2] offer robust environments for early-stage software development and hardware verification. Yosys<fnref n="6" />, an open-source synthesis tool, in conjunction with nextpnr<fnref n="7" />, supports FPGA and ASIC synthesis and place-and-route, contributing to a fully open-source hardware design flow [19]. These tools provide a flexible and collaborative ecosystem that lowers entry barriers to hardware design, accelerates verification, and fosters innovation in academia and industry.

## 2.2 Existing Frameworks

Generic and reusable frameworks have already been presented to enhance productivity of hardware designers. SystemC [14] is

a system design and verification language based on C++ that enables modeling, partitioning, and evaluating hardware/software implementations. Although very powerful, it is less productive than high-level languages and as such does not fit for the purpose of our methodology. BlueCheck [13] is a generic testbench, with a different approach: it requires a set of properties for the design under test (DUT) and tries to find counter-examples during testing. However, it is written solely for the BlueSpec language, preventing its use with other languages such as Verilog or VHDL.

In recent years, numerous modeling and simulation frameworks have favored Python as a productivity-oriented programming language [8, 9, 11, 22], over performance-oriented languages such as C/C++ or SystemC. Some methodologies integrate cocotb as a component to interface with the DUT both in the literature [21] or within the open-source community with the port of universal verification methodology (UVM) [1] to Python<fnref n="8" />. Relevant Python-based frameworks perform more than simulation and allow for modeling or generating RTL, like PyMTL3 [9]. ERASim [22] is a simpler framework to model and simulate architectures in the Python language, however it does not offer simulation of hardware IPs. Our methodology and platform solely focus on simulation of HDL IPs.

Several platforms to test full-systems have been proposed, each of them with their own particularities [2, 12, 20]. They focus on high-performance simulations, however often at the cost of accuracy and they do not deal with deeply embedded components, such as monitoring IPs. Finally, it can be hard to integrate new components in these platforms, due to their complexity. The purpose of our methodology is to be lightweight and focused on productivity, to obtain fast results.

# 3 Fast Prototyping with Aspycot

Hardware monitoring IPs play a critical role in modern systems-on-chip (SoCs) to report events happening in the lower-levels of the platform; typically, at processor scale. This is especially true for debug and trace IPs, e.g. ARM Embedded Trace Macrocell (ETM)<fnref n="9" /> or RISC-V Trace Encoder [16]. In fact, we consider that any unit that uses internal processor signals is a monitoring IP and can be prototyped and validated using our methodology. In this section, we present the methodology workflow and its implementation details through Aspycot.

## 3.1 Methodology

Aspycot relies on an ISS to emulate applications. ISSs allow developers to simulate hardware behavior without needing physical processors. Abstracting the processor has the following advantages to verify hardware monitoring IPs:

* **Lower design effort before integration:** while the time required to design the IP itself principally depends on the complexity of its specifications, integrating it to a larger platform is a time-consuming effort and not necessarily trivial.

* **Lower simulation time:** testing the monitoring IP in stand-alone will reduce simulation time, in comparison to testing the whole platform or at least the processor.

<footnote n="1">¹https://github.com/chipsalliance/riscv-dv</footnote>

<footnote n="2">²https://github.com/riscv-software-src/riscv-isa-sim</footnote>

<footnote n="3">³https://www.veripool.org/verilator/</footnote>

<footnote n="4">⁴https://github.com/steveicarus/iverilog</footnote>

<footnote n="5">⁵https://www.cocotb.org/</footnote>

<footnote n="6">⁶https://yosyshq.net/yosys/</footnote>

<footnote n="7">⁷https://github.com/YosysHQ/nextpnr</footnote>

<footnote n="8">⁸https://github.com/pyuvm/pyuvm</footnote>

<footnote n="9">⁹https://developer.arm.com/documentation/ihi0014/latest/</footnote>
* **Lower software effort:** the software toolchain and flow of tools to support the ISS are simpler to set up.

* **Golden model:** in the case where the processor is also under development, its bugs can send wrong signals to the monitoring IP. Using a golden model in the form of an ISS reduces this risk.

An overview of the workflow of the methodology is shown in Figure 1. The testbench is built to analyze the ISS executed instruction trace and to simulate the IP accordingly. Its purpose is simply to parse the list of instructions and advance simulation time after providing the signals generated by each instruction to the IP under test. The number of scheduled tests depends on the number of applications given as input to the testbench. The test is agnostic of the IP, to limit the required effort to support new IPs. The testbench defines an abstract class which contains a representation of the IP with its objects and methods to initialize the IP, reset its internal signals, affect the correct values on its inputs for each instruction and scan its output conditions to end the test. In the context of security related IPs for example, this condition could represent whether the IP has detected wrongful behavior or not. Finally, if applicable to the IP under test, sets of parameters can be specified to test different configurations of the IP.

Another aspect of the testbench is its knowledge of the application's type, through an oracle. Indeed, certain applications are simple benchmarks to evaluate the behavior of the IP in usual conditions. Alternatively, some applications can represent specialized software. They could be generated binaries that contain a high number of a particular operations, e.g. control-flow or data-flow instructions. They could also be malicious applications, with respect to a certain threat model. Either way, the oracle is the component that is able to extract the expected result from the application's type, and to compare it with the IP behavior during simulation. We present a case study related to control-flow monitoring in Section 4, however other threat models related to code or data integrity could also be evaluated. Using this knowledge of the application currently being run, the test is able to report both false positives and false negatives.

## 3.2 Implementation Details

The discussed methodology can be implemented using a variety of tools. The scope of each mentioned tools is specified in Figure 1 by highlighting whether a tool is used to implement of particular step of the methodology. In our case, we favored tools from the open-source ecosystem and a productive high-level software language, Python, to build Aspycot. We chose Spike ISS, a RISC-V ISA simulator. It made sense to us as both the RISC-V ISA and Spike are open-source, allowing complete ownership and extensibility of the ISS.

To interface with the hardware IPs, we needed efficient and productive tools. Cocotb, as a Python library, appeared to be an optimal choice for interfacing with HDL simulators. Another advantage of cocotb is its support for a variety of HDL simulators, both licensed and open-source options. Notable among these is Verilator for Verilog and SystemVerilog IPs. However, cocotb testbenches are simulator-agnostic and it also supports VHDL simulators, making it suitable to build a reusable test environment.

A lot of features from the chosen tools and languages favor the methodology. For example, the abstract classes needed to support multiple IPs can be done through polymorphism of a base class, which Python supports. Also, a recent cocotb feature that allows users to run simulations using only Python code allows us to use pytest<fnref n="10" />, a full-featured tool for testing Python programs. We can then use pytest's parameterization features to run the test for different configurations of the IP if it uses generic high-level parameters. Aspycot integrates these tools together with Python scripts that ensures reusabily and modularity. Its major original components are the transformation of Spike's logs to stimuli when running the tests, as well as the abstract interface to the IPs.

In the current implementation of the platform, the steps required to add a new IP are minimum:

(1) extend the abstract class for the new IP;

(2) declare the high-level test function with the path to HDL files;

(3) if applicable, declare the different values to test for the high-level parameters.

In addition, Python's support for automation enables the creation of scripts to automate test execution, result collection and reporting. This reduces manual intervention and allows developers to focus on more critical design tasks. In addition, component reuse and automation of the co-simulation workflow leads to faster validation cycles, making it easier to integrate new features or enhancements into the existing architecture. We discuss these potential features and extensions in Section 5.

# 4 Case Study: Control-Flow Monitoring

We want to cover the following Research Question (RQ):

**RQ1** How efficient is the platform, in terms of productivity and performance, to prototype a control-flow monitoring IP?

To that end, we propose a relevant use-case to demonstrate the efficiency of the method. The studied example is a security-related component which dynamically monitors the control-flow of applications running on a single core. The goal of this component is to detect control-flow anomalies resulting from the execution of a code-reuse attack (CRA). We evaluate an implementation of JOP-alarm [23], a simple jump-oriented programming (JOP) detection algorithm from the literature.

## 4.1 Context: Code-reuse Attacks

Code-reuse attacks are a form of exploitation where the attacker gains control of an application's execution by leveraging existing functions and system calls within the target software. A CRA manipulates the control flow to execute legitimate pieces of the application's own code in unintended ways. These attacks usually rely on prior memory corruption vulnerabilities, often on the stack or heap, which allow the attacker to redirect the execution flow to specific sequences of instructions within the program, called *gadgets* [18]. A common example of CRA is the return-to-libC attack [6], where execution is diverted with a single jump to a standard library function after corrupting a pointer. Although CRA does not introduce new code, it can be used to orchestrate complex malicious

<footnote n="10">¹⁰https://docs.pytest.org/</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 1: Aspycot's workflow. The different steps of the methodology are represented as well as the tools proposed to implement them. User inputs are also represented, with the IPs to test and the sets of parameters to use during the tests.</figcaption></figure>

behaviors by chaining the execution of pre-existing code, in the form of a *gadget chain*.

Several types of CRAs, far more complex than return-to-libC, have been introduced based on the basic principles highlighted above. Jump-oriented programming (JOP) [3] is a more advanced type of code-reuse attack that uses indirect jumps or calls to transfer execution from one gadget to the next. Each gadget ends with an indirect jump, which relies on manipulating the program's control data, such as function pointers or jump tables, rather than the stack. JOP is a powerful technique for executing arbitrary sequences of instructions while maintaining the appearance of legitimate program behavior.

## 4.2 Jump-Oriented Programming Detection Algorithm

Researchers have proposed various countermeasures against CRAs and especially JOP attacks. Detection algorithms that can be implemented as hardware components were introduced [10, 23]. Aspycot is especially useful to prototype and validate this type of heuristic detection mechanisms. The methodology could also be applied to more complex approaches that enhance trace components to enforce control-flow rules in the system [24]. In this case study, we will focus on a simplified version of JOP-alarm [23].

This algorithm detects JOP attacks based on the frequency of indirect jumps or calls, which is supposedly higher when executing an attack in comparison to a legitimate application. JOP-alarm also considers the distance between base addresses of gadgets as a valuable metric, but for the sake of simplicity, we will ignore this parameter in the case study. It maintains a *score*, that is incremented by a *step_up_value* for each indirect jump and decreased by a *step_down_value* for every other instruction. Should the score be higher than a *jop_threshold* value, an alarm is raised, meaning a potential JOP attack was detected. The algorithm is hence based on three parameters that have a direct impact on the detection scheme and are empirically defined:

<figure><img src="image_6.png" /><figcaption>Figure 2: Block diagram of a control-flow monitoring IP implementing a simplified version of JOP-alarm [23] algorithm. The inputs of the IP are internal states of the processor.</figcaption></figure>

* *jop_threshold*

* *step_up_value*

* *step_down_value*

The algorithm is shown in Algorithm 1. Defining a set of correct values for its parameters without an appropriate testbench can be time-consuming: it either requires significant micro-architecture modifications to access the monitoring IP and write the values with software, or to rebuild the whole platform for each value of HDL parameters. The interested reader can refer to [23] for more information.

## 4.3 Tuning the Algorithm

In this section, we show how to use Aspycot to quickly test different configurations of the detection algorithm. First, we implement Algorithm 1 in SystemVerilog. The IP has the same clock domain as the processor it is connected to and receives its internal signals from the commit stage, i.e. it only monitors executed instructions. Figure 2 shows a block diagram of the IP under development.
**Algorithm 1** Simple parameter-based detection algorithm of potential JOP attacks based on [23]

```python
{Initialize score value to 0}
score ← 0
```

```python
{Threshold to detect potential JOP attack}
jop_threshold ← parameter-defined
```

```python
{Step values of the counter}
step_up_value ← parameter-defined
step_down_value ← parameter-defined
```

```python
for each indirect jump or indirect call instruction do
    score ← score + step_up_value
end for
```

```python
for each sequential instruction do
    score ← score - step_down_value
end for
```

```python
{Potential JOP attack detected}
if score ≥ jop_threshold then
    raise alarm
end if
```

**Experimental setting.** To answer RQ1, we ran the experiments on a laptop equipped with 12th Gen Intel® Core™ i5-12450H × 12, 16.0 GiB RAM capacity and Ubuntu 24.04.1. The versions of the major components are the following: Python version 3.12.3, cocotb version 1.9.1, Verilator version 5.024, Spike development version<fnref n="11" />. To stress the IP and have meaningful results, we ran 10 legitimate applications from UCB's RISC-V benchmark<fnref n="12" /> and 10 generated malicious applications that reproduce the runtime signature of a JOP attack. As explained in Section 3.1, the test has knowledge of legitimate applications and malicious applications. We evaluate the precision of a configuration by measuring its $F^*$ score [7], representing the proportion of relevant classifications of applications by the IP that are correct, i.e.:

$$F^* = \frac{T_p}{N - T_n}$$

With $N$ the total number of applications, $T_p$ the number of true positives and $T_n$ the number of true negatives. In this case study, $N = 20$. We define a set of values, based on the actual values presented in [23], for each parameter of the algorithm:

* $jop_threshold \rightarrow \{100, 120\}$.

* $step_up_value \rightarrow \{10, 20\}$.

* $step_down_value \rightarrow \{1, 4\}$.

This gives a total of 8 possible configurations.

**Results.** To answer RQ1, we are going to focus on the simulation time, derived from the clock-frequency and number of clock events happened in the simulation, and real time (wall-clock time) with respect to the total number of processed instructions. The testbench

is simply running the applications, with one test for each application: one simulation event corresponds to an instruction analyzed by the monitoring IP. A test stops if the IP suspects an attack, i.e. an alarm is raised. Table 1 reports the $F^*$ score of the algorithm and the number of processed instructions for each configuration, compared to the simulation time and real time. All these results can be retrieved by running a single command in the framework, by providing the list of applications and the IP to test.

The total time required to get these results for the control-flow monitoring IP is of approximately 15 minutes.

**Conclusion.** In this experiment, we ran a set of 20 applications in 8 different configurations, in a single run. Based on the results of this run, we are able to draw conclusions on each configuration. The results for each configuration give designers a quick feedback on their relevance to the purpose of the IP, identifying false positives and false negatives. The run lasted approximately 15 minutes on a small development machine, showing the potential of the platform to prototype and validate algorithms and their parameters. This considerably enhances productivity in comparison to recompiling a whole platform for each configuration. PyMTL3 demonstrates better performance for the same case study (3 minutes) because it executes a model described in Python. However, it is complementary to Aspycot: both frameworks could be used to validate a model in high-level language and then its hardware implementation in VHDL or SystemVerilog. In the context of this simple case study, the most accurate configuration was {120, 20, 1}. It had no false positives and no false negatives, and it processed fewer instructions than {100, 20, 4}, so it detected the attacks faster.

# 5 Extending Aspycot & Reusing Components

The co-simulation testbench links Spike, as the hardware simulation backend, with Python, which provides a high-level interface for managing simulation tasks, configuring tests, and analyzing results. This structure allows developers to rapidly prototype, simulate, and validate design changes with minimal effort. However, Aspycot could be extended with modular components that give additional context to the traces of executed instructions. This modularity is achieved by separating the models from test definitions. We focus on three major extensions:

**Branch prediction models.** Branch prediction is a critical aspect of modern processor architectures that aims to improve the efficiency of the instruction pipeline by speculating on the outcome of conditional branches or indirect jumps. Extending the co-simulation testbench with generic branch prediction models allows developers to use their results as inputs to the hardware monitoring IP under test, enabling finer-grained algorithms that look at mispredicted branch rates.

**Pseudo-multithreading.** Pseudo-multithreading introduces the ability to simulate multiple applications on the hardware monitoring IP by interleaving instruction streams from different contexts. This approach is particularly useful for evaluating how thread-level parallelism can affect the behavior of the IP, especially if it is supposed to monitor events that happen at the scale of specific instructions.

<footnote n="11">¹¹1.1.1-dev 4e17969c</footnote>

<footnote n="12">¹²https://github.com/ucb-bar/riscv-benchmarks</footnote>
<table><caption>Table 1: Algorithm F* score, simulation time and real time of the control-flow monitoring IP for the different configurations. The configuration column gives the set of parameters in the following order: {jop_threshold, step_up_value, step_down_value}.</caption><thead><tr><td>Configuration</td><td>F* score</td><td>Processed instructions count</td><td>Simulation time (ns)</td><td>Real time (s)</td></tr></thead><tbody><tr><td>{100, 10, 1}</td><td>0.4</td><td>1,314,726</td><td>13,148,770</td><td>109.98</td></tr><tr><td>{100, 10, 4}</td><td>0.0</td><td>1,314,826</td><td>13,149,770</td><td>113.73</td></tr><tr><td>{100, 20, 1}</td><td>0.59</td><td>1,312,186</td><td>13,123,370</td><td>114.39</td></tr><tr><td>{100, 20, 4}</td><td>1.0</td><td>1,314,526</td><td>13,146,770</td><td>116.03</td></tr><tr><td>{120, 10, 1}</td><td>0.4</td><td>1,314,750</td><td>13,149,010</td><td>121.02</td></tr><tr><td>{120, 10, 4}</td><td>0.0</td><td>1,314,826</td><td>13,149,770</td><td>121.11</td></tr><tr><td>{120, 20, 1}</td><td>1.0</td><td>1,314,482</td><td>13,146,330</td><td>126.08</td></tr><tr><td>{120, 20, 4}</td><td>0.4</td><td>1,314,642</td><td>13,147,930</td><td>133.23</td></tr></tbody></table>

Other architectures. Expanding the co-simulation framework to support additional architectures broadens its applicability across different processor designs. At the moment, since the platform uses Spike, it is only able to analyze RISC-V instructions. Pydgin [11] is an ISS generator that supports several architectures and as a Python project, it could prove an efficient alternative. This could allow developers to simulate and validate their designs on multiple ISAs, facilitating cross-architecture comparisons and validation of the IPs in different contexts.

These extensions outline the focus areas for enhancing the co-simulation testbench, making it more versatile for performance evaluation, concurrency studies, and cross-architecture validation. Also, even though the case study presented in Section 4 is related to security, the methodology is applicable to other monitoring scenarios.

## 6 Conclusion

The methodology introduced in this paper allows for quick prototyping and validation of hardware components, for hardware monitoring. By leveraging the capabilities of open-source tools like Spike and Verilator, and the Python ecosystem through cocotb, this methodology not only streamlines the prototyping process but also enables license-free academic exploration. The integration of these tools facilitates a more efficient workflow, enabling designers to validate complex hardware designs in a timely manner.

As demonstrated in the case study, Aspycot allows for modularity and rapid evaluation of different hardware configurations. Minimal effort is required to automate the simulation process with different high-level parameters and to collect the simulation results. Future work may include incorporating additional models, such as branch prediction algorithms, and extending the framework to accommodate a broader range of hardware verification scenarios. Aspycot source code is available on GitHub<fnref n="13" /> along with the case study presented in this paper.

## References

[1] Accellera. 2014. Universal Verification Methodology (UVM) 1.2 Class Reference.

[2] Fabrice Bellard. 2005. QEMU, a fast and portable dynamic translator. In Proceedings of the Annual Conference on USENIX Annual Technical Conference (Anaheim, CA) (ATEC '05). USENIX Association, USA, 41.

[3] Tyler Bletsch, Xuxian Jiang, Vince W. Freeh, and Zhenkai Liang. 2011. Jump-oriented programming: a new class of code-reuse attack. In Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security (Hong Kong, China) (ASIACCS '11). Association for Computing Machinery, New York, NY, USA, 30–40. doi:10.1145/1966913.1966919

[4] Niklas Bruns, Vladimir Herdt, and Rolf Drechsler. 2023. Processor Verification using Symbolic Execution: A RISC-V Case-Study. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE). 1–6. doi:10.23919/DATE56975. 2023.10137202

[5] Wen Chen, Sandip Ray, Jayanta Bhadra, Magdy Abadir, and Li-C Wang. 2017. Challenges and Trends in Modern SoC Design Verification. IEEE Design & Test 34, 5 (2017), 7–22. doi:10.1109/MDAT.2017.2735383

[6] Solar Designer. 1997. "return-to-libc" attack. Bugtraq, Aug (1997).

[7] David J Hand, Peter Christen, and Nishadi Kirielle. 2021. F*: an interpretable transformation of the F-measure. Machine Learning 110, 3 (2021), 451–456.

[8] Shunning Jiang, Yanghui Ou, Peitian Pan, Kaishuo Cheng, Yixiao Zhang, and Christopher Batten. 2021. PyH2: Using PyMTL3 to Create Productive and Open-Source Hardware Testing Methodologies. IEEE Design & Test 38, 2 (2021), 53–61. doi:10.1109/MDAT.2020.3024144

[9] Shunning Jiang, Peitian Pan, Yanghui Ou, and Christopher Batten. 2020. PyMTL3: A Python Framework for Open-Source Hardware Modeling, Generation, Simulation, and Verification. IEEE Micro 40, 4 (2020), 58–66. doi:10.1109/MM.2020. 2997638

[10] Mehmet Kayaalp, Timothy Schmitt, Junaid Nomani, Dmitry Ponomarev, and Nael Abu-Ghazaleh. 2013. SCRAP: Architecture for signature-based protection from Code Reuse Attacks. In 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA). 258–269. doi:10.1109/HPCA.2013. 6522324

[11] Derek Lockhart, Berkin Ilbeyi, and Christopher Batten. 2015. Pydgin: generating fast instruction set simulators from simple architecture descriptions with meta-tracing JIT compilers. In 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 256–267. doi:10.1109/ISPASS.2015. 7095811

[12] Jason E. Miller, Harshad Kasture, George Kurian, Charles Gruenwald, Nathan Beckmann, Christopher Celio, Jonathan Eastep, and Anant Agarwal. 2010. Graphite: A distributed parallel simulator for multicore. In HPCA - 16 2010 The Sixteenth International Symposium on High-Performance Computer Architecture. 1–12. doi:10.1109/HPCA.2010.5416635

[13] Matthew Naylor and Simon Moore. 2015. A generic synthesisable test bench. In 2015 ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE). 128–137. doi:10.1109/MEMCOD.2015.7340479

[14] Preeti Ranjan Panda. 2001. SystemC: a modeling platform supporting multiple design abstractions. In Proceedings of the 14th International Symposium on Systems Synthesis (Montréal, P.Q., Canada) (ISS '01). Association for Computing Machinery, New York, NY, USA, 75–80. doi:10.1145/500001.5000018

[15] RISC-V Foundation 2019. The RISC-V Instruction Set Manual, Volume I: User-Level ISA, Document Version 20191213. RISC-V Foundation.

[16] RISC-V N-Trace Task Group 2024. RISC-V N-Trace (Nexus-based Trace) Specification, Document Version 1.0_rc51. RISC-V N-Trace Task Group.

[17] James A Rowson. 1994. Hardware/software co-simulation. In Proceedings of the 31st Annual Design Automation Conference. 439–440.

[18] Hovav Shacham. 2007. The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86). In Proceedings of the 14th ACM Conference on Computer and Communications Security (Alexandria, Virginia, USA) (CCS '07). Association for Computing Machinery, New York, NY, USA, 552–561. doi:10.1145/1315245.1315313

[19] David Shah, Eddie Hung, Clifford Wolf, Serge Bazanski, Dan Gisselquist, and Miodrag Milanovic. 2019. Yosys+nextpnr: An Open Source Framework from Verilog to Bitstream for Commercial FPGAs. In 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 1–4.

<footnote n="13">¹³Under https://github.com/ThalesGroup/aspycot</footnote>
doi:10.1109/FCCM.2019.00010

[20] Karthik Tanikella, Yohan Koy, Reiley Jeyapaul, Kyoungwoo Lee, and Aviral Shri-vastava. 2016. gemV: A validated toolset for the early exploration of system reliability. 159–163. doi:10.1109/ASAP.2016.7760786

[21] Matias Trapaglia, Ricardo Cayssials, Lorenzo De Pasquale, and Edgardo Ferro. 2019. Flexible Software to Hardware Migration Methodology for FPGA Design and Verification. In *2019 X Southern Conference on Programmable Logic (SPL)*. 39–44. doi:10.1109/SPL.2019.8714377

[22] Saud Wasly. 2021. ERAsim: A Flexible Python-based Architectural Modeling and Simulation Framework. In *2020 8th International Conference on Intelligent and*

*Advanced Systems (ICIAS)*, 1–6. doi:10.1109/ICIAS49414.2021.9642593

[23] Fan Yao, Jie Chen, and Guru Venkataramani. 2013. JOP-alarm: Detecting jump-oriented programming-based anomalies in applications. In *2013 IEEE 31st International Conference on Computer Design (ICCD)*. 467–470. doi:10.1109/ICCD.2013.6657084

[24] Anthony Zgheib, Olivier Potin, Jean-Baptiste Rigaud, and Jean-Max Dutertre. 2022. A CFI Verification System based on the RISC-V Instruction Trace Encoder. In *2022 25th Euromicro Conference on Digital System Design (DSD)*. 456–463. doi:10.1109/DSD57027.2022.00067