SevdanurGenc/Hybrid_QCNN_Controlled_Experiment

---

license: mit language: - en tags: - quantum-machine-learning - variational-quantum-circuit - hybrid-quantum-classical - qcnn - barren-plateau - image-classification - reproducibility - controlled-experiment - pennylane - pytorch datasets: - mnist

Does the Quantum Layer Actually Learn? A Controlled Benchmark for Hybrid QCNNs

A single-file, reproducible experimental pipeline that isolates what the quantum layer actually contributes in hybrid quantum–classical convolutional networks.

Most hybrid QCNN papers compare one hybrid model against one classical baseline and report accuracy. That design cannot tell you whether the performance comes from the quantum circuit learning or from the circuit acting as a fixed random non-linear feature map. This repository provides the control experiment that separates the two.

Headline result: across 330 training runs on two datasets, the trainable variational quantum circuit (VQC) ranked last among six models. The quantum block provides no benefit even when frozen, and training its parameters makes things measurably worse.

---

Key findings

All comparisons are paired (dataset × training size × seed), tested with the Wilcoxon signed-rank test, with bootstrap 95% confidence intervals over 10,000 resamples.

Comparison (55 paired runs)	Δ macro-F1	95% CI	p	Cohen's d
Trainable VQC − Frozen VQC	−0.073	[−0.100, −0.046]	5.2×10⁻⁶	−0.71
Trainable VQC − Parameter-matched MLP	−0.099	[−0.130, −0.067]	1.0×10⁻⁶	−0.81
Trainable VQC − Width-preserving MLP	−0.114	[−0.148, −0.080]	2.7×10⁻⁷	−0.88
Trainable VQC − Block-free baseline	−0.109	[−0.138, −0.081]	1.4×10⁻⁸	−0.98
Frozen VQC − Block-free baseline	−0.036	[−0.060, −0.014]	0.018	—

The result is two-level:

1. The quantum block does not help even with frozen parameters — removing it entirely is better.
2. Training the block parameters degrades performance further.

Mean rank over six models (1 = best): Classical CNN 1.25 · Width-preserving MLP 3.18 · Block-free baseline 3.56 · Parameter-matched MLP 3.65 · Frozen VQC 4.05 · Trainable hybrid VQC 5.29

There is a trainability window

Training does help below a certain circuit size. Ablation over qubit count (SLD, N = 100/class):

Qubits	Δ macro-F1 (trainable − frozen)	95% CI	Seeds won
2	+0.057	[+0.016, +0.127]	5/5
4	+0.115	[+0.016, +0.213]	4/5
6	−0.066	[−0.107, −0.034]	0/5
8	−0.051	[−0.100, −0.003]	2/5

Linear interpolation places the sign change at ≈ 5.3 qubits. This tracks the gradient variance, which decays exponentially with qubit count:

Var[∂⟨Z₀⟩/∂ϑ] ≈ V₀·exp(−βq) with β = 0.613 / 0.656 / 0.664 for L = 2 / 4 / 6 and R² ≥ 0.9997.

Secondary findings

• Calibration degrades. Mean ECE 0.218 (trainable) vs 0.159 (frozen), p = 5.4×10⁻⁸. The trainable circuit is systematically overconfident on its errors.
• Cost. The hybrid model trains ≈ 5.3× slower than its classical equivalents while scoring lower.
• Not an encoding artefact. Angle, angle + data re-uploading, and IQP encodings all show the same direction.
• Depth is a separate effect. Circuit depth hurts performance (Δ = −0.114 at L = 6) but does not change gradient variance — this remains unexplained and is stated as an open question.

---

What makes this a controlled comparison

Five models share the same convolutional backbone, the same bottleneck width, the same optimiser budget, and the same augmentation. They differ only in one block:

Model	Block	Role
hybrid_vqc	Trainable VQC (54 params)	Main model
ctrl_vqc_frozen	Same circuit, frozen at random init	Control A — does training help?
ctrl_mlp_matched	Linear(d→4)–Tanh–Linear(4→d) (58 params)	Control B — same budget, classical
ctrl_mlp_wide	Linear(d→d)–Tanh–Linear(d→d) (84 params)	Control B2 — no inner compression
ctrl_bottleneck	Identity	Control C — is a block needed at all?

Plus two deliberately unmatched references: classical_full (a normal CNN head) and pure_vqc (no convolutional backbone, to show the information-bottleneck cost).

Protocol decisions that matter

Each of these corresponds to a common failure mode in the literature:

• The test set is never used for model selection. Validation is carved out of the training pool; the test set is read exactly once, after training completes.
• Fixed test set across all runs. Drawn with a fixed seed, identical for every training size and seed — this is what makes paired statistics legitimate.
• Fixed-size validation set (20/class) at every training size, so early stopping is equally reliable in the low-data regime.
• Equal data. All models train on the same N. (Training the classical model on 60k and the hybrid on 10k invalidates the comparison.)
• Equal bottleneck width. Squeezing the quantum model to 6 dimensions while leaving the classical head at 256 is a structural handicap, not a fair test.
• Constant optimiser budget (~1,200 steps) across training sizes.
• Residual gate, z ← z + γ·B(z) with learnable γ initialised to 1, so a block can never hurt merely by existing. The comparison measures what the block adds, not whether adding a block hurts.
• 5 seeds, paired non-parametric tests, effect sizes and CIs — not single-run point estimates.

---

Repository contents

Hybrid_QCNN_Controlled_Experiment.ipynb   # the full pipeline (setup → experiments → figures)

The notebook is self-contained and runs end to end in Google Colab.

---

Quick start (Google Colab)

1. Open the notebook in Colab and select a T4 GPU runtime.
2. Run Cell 1 (installation), then Runtime → Restart session.
3. Run Cell 1b — it verifies package versions and executes a 2-qubit sanity circuit.
4. Run cells 2–13 in order (definitions only).
5. Run Cell 14 — a 3–5 minute smoke test. Do not skip this.
6. Run Cell 15 for a runtime estimate, then the experiment stages.

Point RESULTS_DIR at Google Drive. Every finished run is appended to runs.jsonl and completed runs are skipped on restart, so a dropped Colab session costs you nothing — just re-run the same cell.

Pinned dependencies

pip install "pennylane==0.41.1" "pennylane-lightning==0.41.1"
pip install "autoray==0.7.2"          # must come AFTER pennylane
pip install torch torchvision scikit-learn scipy matplotlib pandas pillow
pip uninstall -y jax jaxlib           # last step

Two pins are not optional:

• autoray < 0.8 — PennyLane 0.41 uses autoray.autoray.NumpyMimic, removed in autoray 0.8.0. Without the pin, import pennylane raises AttributeError: module 'autoray.autoray' has no attribute 'NumpyMimic'. Tested boundary: present up to 0.7.2, gone from 0.8.0.
• Remove JAX. Colab preinstalls it; PennyLane 0.41 tries to import it and fails on JAX ≥ 0.6.0 (jax.core.Primitive was moved). This work uses the PyTorch interface, so JAX is simply removed rather than downgraded. Do it last — earlier pip install steps can pull it back in.

Cell 1b checks both conditions and prints the fix if either is violated.

---

Datasets

Dataset	Size	Role
Sign Language Digits (Mavi, 2020)	2,062 images, 10 classes	Primary — small enough that the low-data regime is natural, not simulated
MNIST	70,000 images, 10 classes	Calibration baseline

Both go through the identical preprocessing path: grayscale → bilinear resize to 28×28 → scale to [0, 1].

Two gotchas in the SLD loader, both handled in the notebook:

• Images are not uniformly sized: 2,059 are 100×100 but three are 3,024×3,024. Each image is resized before stacking; stacking first raises ValueError: all input arrays must have the same shape.
• The GitHub release has no X.npy/Y.npy (those are in the Kaggle version). Data lives in Dataset/<digit>/*.JPG, so folder name = true digit label. The loader supports both layouts.

Verified class distribution: [205 206 206 206 207 207 207 206 208 204].

---

Compute requirements

Measured on a Colab T4: ~115 ms/step for quantum models, ~6 ms/step for classical ones.

Stage	Approx. wall-clock
Smoke test	3–5 min
Main stage (330 runs)	~3 h with early stopping (6.3 h upper bound)
Ablations (depth, qubits, encoding)	~2.5 h
Gradient variance probe	~8 min (CPU)

Expect the main stage to span two free-tier sessions. The resume mechanism handles this.

---

Outputs

Running the analysis stage writes to RESULTS_DIR:

• runs.jsonl — one JSON record per run (metrics, per-epoch history, confusion matrix, config)
• summary.csv, statistical_tests.csv
• figures/ — data-efficiency curves, net-contribution plot, training dynamics, trainability window, gradient variance, confusion matrices, per-class F1, calibration curves

---

Limitations

Read the headline result narrowly. What was measured is the behaviour of one architecture family at one scale.

• Noiseless state-vector simulation — no shot noise, no decoherence. Real hardware may differ.
• One ansatz family: Rot gates with ring-topology CNOTs (hardware-efficient). Problem-adapted or symmetry-preserving ansätze may behave differently.
• ≤ 10 qubits, ≤ 6 layers.
• ≤ 125 samples/class (SLD), ≤ 500 (MNIST).
• No systematic hyperparameter search beyond the quantum learning rate.

This is not evidence that quantum machine learning does not work. It is evidence that, in this regime, the frozen-circuit control is necessary before any claim about the quantum layer's contribution can be made.

---

Related work

The direction is consistent with — and extends — Bowles, Ahmed & Schuld (arXiv:2403.07059), who benchmarked 12 quantum models across 160 datasets and found that removing entanglement usually did not degrade performance. Here, removing the quantum block entirely is shown to be better, under matched conditions on two independent datasets.

Mechanism follows McClean et al., Barren plateaus in quantum neural network training landscapes (Nat. Commun. 9:4812).

---

Citation

@article{qcnn_frozen_control,
  title   = {Does the Quantum Layer Actually Learn in Hybrid Quantum--Classical
             Convolutional Networks? A Comparative Evaluation Using a Frozen-Circuit Control},
  author  = {Author Name},
  journal = {Journal of Engineering Sciences and Research},
  year    = {2026},
  note    = {Code and results: https://huggingface.co/<user>/<repo>}
}

---

License

MIT for the code in this repository. The Sign Language Digits dataset and MNIST retain their own original licences.