DOI: 10.5281/zenodo.15073441
title: Supplementary Material for paper "Shaking Up Quantum Simulators with Fuzzing and Rigour"

description: This repository provides tools for constructing quantum circuits and validating quantum validators using the Alloy modeling language to encode standard quantum gates. 

We include our Alloy model along with setup and usage instructions. Additionally, the repository contains:



xml_seeds/ – XML-based test seeds generated with Alloy, used to obtain the results presented in our paper.

execution_logs/ – Logs from simulating the test cases.

post-processing/ – Data visualised in our paper, organised as follows:



chi_js/ – Chi-square and Jensen-Shannon Divergence analysis (Fig. 6–7).

memory/ – Seeds and post-processing data for plotting memory-related results (Fig. 8).

shots/ – Data for analysing the optimal number of shots (Fig. 6).

invariants/ – Demonstrations of invariants discussed in Section 5.




For detailed instructions, see the corresponding folders.

 

Repository Structure

.├── Alloy-model│   ├── README.md│   └── standard_gate_library.als├── README.md├── execution_logs├── fuzzQ.py├── post-processing│   ├── chi_js│   ├── invariants│   │   ├── GHZ│   │   ├── Involution│   │   └── Symmetry│   ├── memory│   │   └── xml_instances│   ├── shots│   └── state_vectors_mismatch_Accumulation_of_Numerical_Errors│       └── 3q15g_noSwap└── xml_seeds

 

Parsing Standard Gates

The Alloy model, Alloy-model/standard_gate_library.als, encodes standard quantum gates such as XGate, HGate, and CXGate. Using the Alloy tool, we generate XML-based test seeds representing valid quantum circuits that adhere to the model’s constraints.  

The Python script fuzzQ.py processes these XML instances, converting them into executable Qiskit quantum circuits. It then simulates the circuits across various backends, providing detailed measurement results (outcome probabilities) to facilitate the assessment and validation of quantum simulators. 



 



Requirements

Using a Virtual Environment (Recommended)

While not mandatory, using a virtual environment ensures that your Python packages are isolated from your system’s Python installation. This prevents any version conflicts and keeps your environment clean and manageable.

Steps to Set Up:





Create a Virtual Environment

Open your terminal and run:



python3 -m venv myenv


This will create a virtual environment named myenv in your current directory.




Activate the Virtual Environment

Once the environment is created, activate it:



source myenv/bin/activate  # On macOS/Linux
myenv\Scripts\activate     # On Windows


Once activated, your terminal prompt should display (myenv).




Install Dependencies

The fuzzQ.py script is designed to automatically install the necessary libraries. However, if you prefer to have full control over the installation process, you can do so manually by following these steps:





Install pip (if not already installed)



sudo apt install python3-pip





Other Dependencies

With the environment activated, install the required packages:



pip install argparse qiskit qiskit-aer cirq quimb numpy seaborn matplotlib prettytable scipy tqdm psutil







Run the Script

Execute the program while the virtual environment is active:



python fuzzQ.py <path_to_xml_file>





Deactivate the Virtual Environment

When done, deactivate the virtual environment:



deactivate


Note: You only need to create the virtual environment and install the dependencies once. After that, you can reactivate the environment whenever you want to run the program.



 

Usage

Run the script with:



python fuzzQ.py <path_to_xml_file>


The XML file should be an Alloy-generated instance representing a quantum circuit.

Configuration

These variables can be modified inside fuzzQ.py to customise execution settings, including iteration count, circuit scaling, and backend parameters.



shots: Sets the number of times the quantum backend repeats the execution of the circuit per submission. Defaults to 10,000 but can be adjusted as needed.

num_iterations: Specifies how many times the script submits the circuit to the backend. This acts as an external wrapper, repeatedly passing the same circuit for execution.

multiply_circuit: Controls how many times the gate sequence is concatenated, effectively repeating the circuit’s operations and increasing its depth.

Test Mode: If no XML file is provided, the script runs in test mode. It will call test_circuits(), allowing you to set up an ad-hoc circuit instead of passing one via an XML file.


  

Simulation Metrics Output

The fuzzQ.py script generates detailed simulation metrics and stores them in the simulation_results.csv file. Below is an example of the output format:




Metric
Value




num_qubits
110


num_gates
10


depth
6


num_iterations
1


shots
1000


alpha
0.05


chi_reject_qiskit
1


chi_reject_cirq
0


chi_qiskit
[0.0381]


chi_cirq
[0.2087]


js_divergences
[0.0195]


mem[MiB]
513.15625




 

Metrics Explanation:



num_qubits: Number of qubits in the quantum circuit.

num_gates: Total number of quantum gates used in the circuit.

depth: The depth of the circuit, representing the number of layers of operations.

num_iterations: The number of times the simulation was repeated.

shots: Number of measurement shots performed to get statistical data.

alpha: The significance level used in statistical tests for the simulation.

chi_reject_qiskit/chi_reject_cirq: Chi-square test result for rejecting the null hypothesis, specific to Qiskit or Cirq.

chi_qiskit/chi_cirq: The calculated chi-square values for the simulation results from Qiskit and Cirq, respectively.

js_divergences: The Jensen-Shannon divergence between Qiskit and Cirq outputs, used to compare distributions.

mem[MiB]: The memory usage in MiB during the simulation.


These metrics are essential for analysing the performance, statistical significance, and resource consumption of quantum circuits, providing insights into simulations across different platforms.

 

Batch Processing of Quantum Circuits

While the fuzzQ.py script is designed to parse and simulate/measure one quantum circuit at a time, the process_xml.sh script offers a streamlined way to process multiple circuits. It scans all .xml files within the folder specified as an argument and runs simulations/measures for each file.

Example Usage



./process_xml.sh <path_to_xml_folder>