Classical optimizer comparison for VQE on H2 under finite shot noise
Domain: quantum (variational-quantum-eigensolver, quantum-chemistry, classical-optimization)
Topic: Benchmarking classical optimizers (SPSA, COBYLA, L-BFGS-B, ADAM) for VQE convergence on H2 under finite shot noise across multiple shot budgets and bond lengths
Venue: ARC-Bench 2026
Headline metric: shots_to_chemical_accuracy (minimize)

Research briefing:
The Variational Quantum Eigensolver (VQE) is the canonical algorithm
for near-term quantum chemistry. The quality of a VQE result depends
on the ansatz, but it also critically depends on the classical
optimizer that updates variational parameters using noisy energy
estimates from shot-based measurement. Different optimizers exploit
gradient information very differently. Gradient-free methods such as
COBYLA and Nelder-Mead do not need explicit gradients and may be
robust to shot noise but converge slowly in high dimensions.
Simultaneous Perturbation Stochastic Approximation (SPSA) approximates
the gradient using just two function evaluations regardless of
parameter count, trading gradient quality for evaluation count and
often dominating in low-shot regimes. Gradient-based methods such as
L-BFGS-B and ADAM need explicit or finite-difference gradients, which
are expensive under shot noise because each partial derivative is
estimated from a noisy function value.

The choice of optimizer is rarely studied head-to-head under a fixed
shot budget. Most VQE papers fix the optimizer and tune everything
else around it. Here we hold ansatz (hardware-efficient EfficientSU2
reps=2), Hamiltonian (H2 in STO-3G basis), and initial parameters
(small random Gaussian) fixed, and vary only the optimizer and the
shot budget per energy evaluation. The diagnostic question is which
optimizer reaches chemical accuracy (within 1.6 mHa of the FCI
reference) using the fewest cumulative shots.

A credible study uses at least two molecular geometries to avoid
overfitting to a single energy landscape. We use H2 at two bond
lengths: the equilibrium distance (0.74 angstrom) where the ground
state is near Hartree-Fock and the optimization landscape is well
behaved, and a stretched geometry (1.5 angstrom) where multireference
character makes the landscape harder and tests optimizer robustness.
Each optimizer is run at a SINGLE fixed shot budget of 1024 shots per
energy evaluation. This is chosen as a representative middle-ground
value: low enough that shot noise is non-trivial (per-eval std ~15 mHa,
well above the 1.6 mHa chemical accuracy threshold, so the running-mean
convergence criterion matters), but high enough that some optimizers
can plausibly converge within their iteration budgets. All cells use
3 random seeds per (optimizer, geometry) cell, giving 4 optimizers x
1 shot budget x 2 geometries x 3 seeds = 24 total cells. Cumulative
shots is the sum of shots used across all energy evaluations
(including finite-difference gradient evaluations where applicable).
The Hamiltonian itself is constructed via
qiskit_nature.second_q.drivers.PySCFDriver (qiskit_nature and pyscf
are required dependencies for this topic; hardcoded Pauli string
fallbacks are NOT accepted).

Implementation contract (see quantum-qiskit skill for the reference
code). In qiskit 2.x, `qiskit_algorithms.VQE` and
`qiskit_nature.second_q.algorithms` are NOT importable (they depend on
the removed `qiskit.primitives.BaseEstimator` V1 interface). VQE MUST
be implemented as a manual optimization loop:
energy(theta) = estimator.run([(bound_ansatz, qubit_op)]).result()[0].data.evs + e_nuclear,
using `qiskit.primitives.StatevectorEstimator` (the V2 primitive that
works in qiskit 2.x) for the energy evaluation, and one of the four
optimizer classes from `qiskit_algorithms.optimizers` (SPSA, COBYLA,
L_BFGS_B, ADAM) for the parameter update, called via
`optimizer.minimize(energy, initial_point)`. Each call to `energy`
counts as one evaluation; cumulative shots is
`n_evaluations * shots_per_eval`. The Hamiltonian itself is constructed
via qiskit_nature.second_q.drivers.PySCFDriver +
qiskit_nature.second_q.mappers.ParityMapper (these submodules ARE safe
to import in qiskit 2.x — only the qiskit_nature.second_q.algorithms
module is broken). With ParityMapper and 2-qubit reduction
(num_particles=(1,1)), the H2/STO-3G Hamiltonian is 2 qubits, not 4.
Implement shot noise by adding Gaussian noise to each energy value
with sigma = ||H||_1 / sqrt(shots_per_eval). The convergence criterion
for shots_to_chemical_accuracy is the cumulative shots when a 5-eval
running mean energy stays within 1.6 mHa of E_FCI for 5 consecutive
evaluations; if not reached, report None or a sentinel distinct from
the upper bound so analysis can flag the cell as "did not converge"
rather than treating the budget cap as a measurement.

Research question: *Under finite shot noise, which classical optimizer
(SPSA, COBYLA, L-BFGS-B, ADAM) reaches VQE chemical accuracy using the
fewest cumulative shots, and how does the answer depend on shot budget
per evaluation and on Hamiltonian difficulty (equilibrium vs stretched
H2)?*

Hypotheses:
statement: At 1024 shots per energy evaluation, SPSA reaches chemical accuracy (|E_running_mean - E_FCI| < 1.6 mHa for 5 consecutive 5-eval windows) using fewer cumulative shots than L-BFGS-B (3-seed median), on at least 1 of 2 H2 geometries., statement: At 1024 shots per energy evaluation, gradient-free methods (SPSA and COBYLA, 6 seed-runs combined across both geometries) achieve chemical accuracy with success rate at least 4/6, while gradient-based methods (L-BFGS-B and ADAM, 6 seed-runs combined) achieve it with success rate at most 3/6. This isolates the shot-noise sensitivity of finite-difference gradient estimation., statement: Optimizer ranking by shots_to_chemical_accuracy changes between H2 equilibrium and H2 stretched geometries: at least one optimizer pair (e.g. SPSA vs COBYLA, or SPSA vs L-BFGS-B) swaps relative ranking between the two geometries, indicating that Hamiltonian difficulty changes optimizer choice.

Experiment design:
research_question: Under finite shot noise, which classical optimizer reaches VQE chemical accuracy in the fewest cumulative shots, and how does the answer depend on shot budget per evaluation and on Hamiltonian difficulty?
conditions: name: spsa
description: qiskit_algorithms.optimizers.SPSA(maxiter=200, learning_rate=0.05, perturbation=0.1). Fixed shot budget: 1024 shots per energy evaluation. Each iteration costs 2 energy evaluations., name: cobyla
description: qiskit_algorithms.optimizers.COBYLA(maxiter=200, rhobeg=0.1, tol=1e-4). 1024 shots per energy evaluation. Each iteration costs 1 energy evaluation., name: lbfgsb
description: qiskit_algorithms.optimizers.L_BFGS_B(maxiter=100, ftol=1e-6) with finite-difference gradients (epsilon=1e-3). 1024 shots per energy evaluation. Each iteration costs ~2*num_params energy evaluations for the gradient plus 1 for the function value (so ~25 evals per iter for EfficientSU2 reps=2 on 4 qubits)., name: adam
description: qiskit_algorithms.optimizers.ADAM(maxiter=200, lr=0.05, beta_1=0.9, beta_2=0.999) with finite-difference gradients (epsilon=1e-3). 1024 shots per energy evaluation. Same per-iter cost as L-BFGS-B.
baselines: Cumulative shots vs. FCI energy reference is the within-method benchmark, Hartree-Fock energy at each geometry is the trivial classical baseline below which any optimizer should clear
metrics: name: shots_to_chemical_accuracy
direction: minimize
description: Cumulative shots until |E_running_mean - E_FCI| stays below 1.6 mHa for 5 consecutive evaluations. If not reached within the maxiter budget, report the total cumulative shots used (sentinel = max-shots) and flag success=False., name: final_energy_error_hartree
direction: minimize
description: |E_final - E_FCI| in Hartree at the end of optimization (after all maxiter iterations)., name: success_rate
direction: maximize
description: Fraction of seeds (out of 3 per condition x geometry) where the run reached chemical accuracy by the maxiter budget.
datasets: name: h2_equilibrium
source: qiskit_nature.second_q.drivers.PySCFDriver (REQUIRED dependency)
description: H2 in STO-3G basis at bond length 0.74 angstrom. Build the FermionicOp via PySCFDriver, apply ParityMapper with 2-qubit reduction to obtain a 4-qubit qubit Hamiltonian (alternatively JordanWignerMapper + tapering). FCI reference: E_FCI = -1.137283 Ha. Hardcoded Pauli strings are NOT an acceptable substitute for this benchmark; both qiskit_nature and pyscf must be installed and used., name: h2_stretched
source: qiskit_nature.second_q.drivers.PySCFDriver (REQUIRED dependency)
description: H2 in STO-3G basis at bond length 1.5 angstrom (multireference / strong-correlation regime). 4 qubits via the same mapping pipeline as h2_equilibrium. FCI reference: E_FCI ~ -1.001 Ha. Hardcoded Pauli strings NOT accepted.
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 1200

Requirements:

Scoring rubric:
requirements: A credible VQE optimizer comparison on H2 in STO-3G basis at two bond lengths (0.74 A equilibrium, 1.5 A stretched), at a fixed shot budget of 1024 shots per energy evaluation. The agent must (a) construct the 4-qubit H2 Hamiltonian at each geometry via qiskit_nature.PySCFDriver, (b) implement VQE via qiskit_algorithms.VQE.compute_minimum_eigenvalue() with a hardware-efficient EfficientSU2 reps=2 ansatz, (c) run 4 optimizers (SPSA, COBYLA, L-BFGS-B, ADAM) x 2 geometries x 3 seeds = 24 cells, (d) log cumulative shots and energy trajectories per cell, and (e) score H1/H2/H3 with numerical evidence keyed by (optimizer, geometry).
judging_note: Optimizer comparisons are scored on (i) correctness of the Hamiltonian (FCI reference within 1 mHa of -1.137 Ha for H2 equilibrium), (ii) consistent ansatz / initial parameters across cells so only the optimizer and shot budget vary, (iii) accurate accounting of cumulative shots including gradient evaluations for L-BFGS-B and ADAM, and (iv) numerical evidence backing H1/H2/H3. qiskit_nature.second_q.drivers.PySCFDriver MUST be used to build the Hamiltonian. Hardcoded Pauli strings are NOT acceptable and a submission that uses them fails q03-code-hamiltonians.
sub_tasks: requirements: Code-development bucket: VQE pipeline correctly implements 4 optimizers, both H2 geometries, and accurate cumulative-shot tracking.
sub_tasks: requirements: VQE is implemented using qiskit_algorithms.VQE or an equivalent custom loop. The ansatz is EfficientSU2(num_qubits=4, reps=2, entanglement='linear'). Initial parameters are sampled from N(0, 0.1) with the same seed across all 4 optimizers for a given (shot_budget, geometry, seed) triple. All 4 optimizers are correctly instantiated from qiskit_algorithms.optimizers (SPSA, COBYLA, L_BFGS_B, ADAM) with the maxiter and learning rates listed in the manifest.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: Both H2 Hamiltonians are constructed at the correct bond lengths (0.74 and 1.5 angstrom) in STO-3G basis using qiskit_nature.second_q.drivers.PySCFDriver (REQUIRED) and either ParityMapper with 2-qubit reduction or JordanWignerMapper with tapering. The diagonalized Hamiltonian (numpy eigvalsh of the matrix form) gives FCI energies within 1 mHa of -1.137 Ha (equilibrium) and -1.001 Ha (stretched). Hardcoded Pauli strings as a substitute for PySCFDriver are NOT accepted and fail this leaf.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: Cumulative shots is tracked correctly per run. For SPSA and COBYLA, each iteration adds shots_per_eval * num_evaluations_per_iter (2 for SPSA, 1 for COBYLA). For L-BFGS-B and ADAM with finite-difference gradients, each iteration adds shots_per_eval * (2 * num_params + 1) so the gradient evaluations are accounted for. The shots_to_chemical_accuracy metric is computed using the running mean energy (over the last 5 evaluations) and reports the cumulative shot count at the first evaluation where the mean stays within 1.6 mHa of E_FCI for 5 consecutive evaluations.
task_category: Code Development
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Execution-validity bucket: all cells ran with valid energies and shot accounting.
sub_tasks: requirements: At least 22 cells out of 24 expected (4 optimizers x 1 shot budget (1024) x 2 geometries x 3 seeds = 24) completed and produced a final_energy_error_hartree value. If any cells are omitted (e.g. ADAM ran out of budget), they must be documented with a recorded cause. Missing more than 2 cells (10 percent) without justification fails this leaf. The condition_summaries MUST be keyed by (optimizer, geometry) so each of the 8 condition cells has 3 seed entries — collapsing across the geometry axis (e.g. reporting only per-optimizer aggregates) fails this leaf because it makes H1 and H3 unevaluable.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Numerical validity: every final energy is finite (no NaN/Inf), every final_energy_error_hartree >= 0, every success_rate in [0, 1], cumulative shots is monotone increasing within a run. At least one cell across the entire sweep reaches chemical accuracy (otherwise the rubric thresholds are uncalibrated for this setup).
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Results bucket: quantitative tests of H1/H2/H3 plus a per-hypothesis writeup.
sub_tasks: requirements: Quantitative test of H1. On at least 1 of 2 geometries at 1024 shots/eval, does SPSA reach chemical accuracy with fewer median cumulative shots than L-BFGS-B (over 3 seeds)? 100% if SPSA wins on at least 1/2 geometries with a >=20 percent shot reduction, 67% if SPSA wins on at least 1/2 with any margin, 33% if SPSA and L-BFGS-B are within 10 percent of each other, 0% if L-BFGS-B is consistently faster.
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Quantitative test of H2. At 1024 shots/eval, pooling across both geometries: do gradient-free methods (SPSA + COBYLA, total of 6 seed-runs = 2 methods x 3 seeds) have a combined chemical-accuracy success rate of at least 4/6, while gradient-based methods (L-BFGS-B + ADAM, 6 seed-runs) have at most 3/6? 100% if cleanly met, 67% if the gradient-free success rate exceeds gradient-based by at least 30 percent, 33% if gradient-free exceeds gradient-based at all, 0% if gradient-based methods are equal or better.
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Quantitative test of H3. Does the optimizer ranking by median shots_to_chemical_accuracy change between H2 equilibrium and H2 stretched geometries? Specifically: take the 4-element ranking of optimizers (by 3-seed-median shots) on each geometry; H3 is supported if at least one pair swaps relative order between the two rankings (Kendall tau distance >= 1 between the two rankings). 100% if at least one swap detected with clear evidence, 67% if one optimizer's rank differs by exactly 1 position, 33% if rankings are similar but values differ, 0% if rankings are identical or H3 is unevaluable (e.g. due to collapsed condition_summaries that drop the geometry axis).
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Writeup of at least 200 words (in submission/README.md ## Agent-produced writeup section) with explicit supported / refuted / inconclusive verdict for each of H1, H2, H3 with numerical evidence (specific median cumulative shots, success rates, and final energy errors per optimizer x shot budget). Identifies a dominant systematic uncertainty (seed variance, finite-difference epsilon, optimizer hyperparameter sensitivity, shot-noise variance at low budgets). Discusses the difference between equilibrium and stretched H2 in terms of optimizer ranking.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation