Package: qcd-gym
Quantum Circuit DesignerDescriptionThis repository contains the Quantum Circuit Designer, a genericgymnasiumenvironment to build quantum circuits gate-by-gate usingpennylane, revealing current challenges regarding:State Preparation (SP): Find a gate sequence that turns some initial state into the target quantum state.Unitary Composition (UC): Find a gate sequence that constructs an arbitrary quantum operator.ObservationsThe observation is defined by the full complex vector representation of the state of the current circuit: $s = \ket{\boldsymbol{\Psi}}\in\mathbb{C}^{2^\eta}$.
While this information is only available in quantum circuit simulators efficiently (on real hardware, $\mathcal{O}(2^\eta)$ measurements would be needed), it depicts a starting point for RL from which future work should extract a sufficient, efficiently obtainable, subset of information.
This $2^\eta$-dimensional state representation is sufficient for the definition of an MDP-compliant environment, as operations on this state are required to be reversible.ActionsWe use a $4$-dimensionalBoxaction space $\langle o, q, c, \Phi \rangle = a \in \mathcal{A} = {\Gamma \times \Omega \times \Theta}$ with the following elements:NameParameterTypeDescriptionOperation$o \in \Gamma$intspecifying operation (see next table)Qubit$q \in[0, \eta)$intspecifying qubit to apply the operationControl$c \in[0, \eta)$intspecifying a control qubitParameter$\Phi \in[- \pi,\pi]$floatcontinuous parameterThe operations $\Gamma$ are defined as:oOperationConditionTypeArgumentsComments0$\mathbb{M}$Meassurement$q$Control and Parameter omitted1$\mathbb{Z}$$q = c$PhaseShift$q,\Phi$Control omitted1$\mathbb{Z}$$q \neq c$ControlledPhaseShift$q,c,\Phi$-2$\mathbb{X}$$q = c$X-Rotation$q,\Phi$Control omitted2$\mathbb{X}$$q \neq c$CNOT$q,c$Parameter omitted3$\mathbb{T}$TerminateAll agruments omittedWith operations according to the following unversal gate set:CNOT: $$CX_{q,c} = \ket{0}\bra{0}\otimes I + \ket{1}\bra{1}\otimes X$$X-Rotation: $$RX(\Phi) = \exp\left(-i \frac{\Phi}{2} X\right)$$PhaseShift: $$P(\Phi) =  \exp\left(i\frac{\Phi}{2}\right) \cdot \exp\left(-i\frac{\Phi}{2} Z\right)$$ControlledPhaseShift: $$CP(\Phi) = I \otimes \ket{0} \bra{0} + P(\Phi) \otimes \ket{1} \bra{1}$$RewardThe reward is kept $0$ until the end of an episode is reached (either by truncation or termination).
To incentivize the use of few operations, a step-cost $\mathcal{C}_t$ is applied when exceeding two-thirds of the available operations $\sigma$:
$$\mathcal{C}_t=\max\left(0,\frac{3}{2\sigma}\left(t-\frac{\sigma}{3}\right)\right)$$Suitable task reward functions $\mathcal{R}^{}\in[0,1]$ are defined, s.t.: $\mathcal{R}=\mathcal{R}^{}(s_t,a_t)-C_t$, according to the following challenges:ChallengesState PreparationThe objective of this challenge is to construct a quantum circuit that generates a desired quantum state.
The reward is based on thefidelity$\mathcal{F} = |\braket{\psi_{\text{env}}|\psi_{\text{target}}}|^2 \in [0,1]$ between the target an the final state:
$$\mathcal{R}^{SP}(s_t,a_t) = \begin{dcases} F(s_t, \Psi), &  \text{if $t$ is terminal.}  \0, & \text{otherwise.} \end{dcases}$$
Currently, the following states are defined:'SP-random'(a random state overmax_qubits)'SP-bell'(the 2-qubit Bell state)'SP-ghz<N>'(the<N>qubit GHZ state)Unitary CompositionThe objective of this challenge is to construct a quantum circuit that implements a desired unitary operation.
The reward is based on theFrobenius norm$D = |U - V(\Sigma_t)|_2$ between the taget unitary $U$ and the final unitary $V$ based on the sequence of operations $\Sigma_t = \langle a_0, \dots, a_t \rangle$:$$ R^{UC}(s_t,a_t) = \begin{dcases} 1 - \arctan (D), & \text{if $t$ is terminal.} \0, & \text{otherwise.} \end{dcases}$$The following unitaries are currently available for this challenge:'UC-random'(a random unitary operation onmax_qubits)'UC-hadamard'(the single qubit Hadamard gate)'UC-toffoli'(the 3-qubit Toffoli gate)SeeOutlookfor more challenges to come.Further ObjectivesThe goal of this implementation is to not only construct any circuit that fulfills a specific challenge but to also make this circuit optimal, that is to give the environment further objectives, such as optimizing:Circuit DepthQubit CountGate Count (or: 2-qubit Gate Count)Parameter CountQubit-ConnectivityThese circuit optimization objectives can be switched on by the parameterpunishwhen initializing a new environment.Currently, the only further objective implemented in this environment is thecircuit depth, as this is one of the most important features to restrict for NISQ (noisy, intermediate-scale, quantum) devices. This metric already includes gate count and parameter count to some extent. However, further objectives can easily be added within theRewardclass of this environment (seeOutlook).SetupInstall the quantum circuit designer environmentpipinstallqcd-gymThe environment can be set up as:importgymnasiumasgymenv=gym.make("CircuitDesigner-v0",max_qubits=2,max_depth=10,challenge='SP-bell',render_mode='text',verbose=True)observation,info=env.reset(seed=42);env.action_space.seed(42)for_inrange(9):action=env.action_space.sample()# this is where you would insert your policyobservation,reward,terminated,truncated,info=env.step(action)ifterminatedortruncated:observation,info=env.reset()env.close()The relevant parameters for setting up the environment are:ParameterTypeExplanationmax_qubits $\eta$intmaximal number of qubits availablemax_depth $\delta$intmaximal circuit depth allowed (= truncation criterion)challengestrRL challenge for which the circuit is to be built (seeChallenges)punishboolspecifier for turning on multi-objectives (seeFurther Objectives)Running benchmarksRunning benchmark experiments requires a full installation including baseline algorithms extendingstable_baselines3and a plotting framework extendingplotly:
This can be achieved by:gitclonehttps://github.com/philippaltmann/QCD.git
pipinstall-e'.[all]'Specify the intended <Challenge> as: "challenge-qmax_qubits-dmax_depth":# Run a specific algoritm and challenge (requires `pip install -e '.[train]'`)python-mtrain[A2C|PPO|SAC|TD3]-e<Challenge># Generate plots from the `results` folder (requires `pip install -e '.[plot]'`)python-mplotresults# To train the provided baseline algorithms, use (pip install -e .[all])./run# Test the circuit designer (requires `pip install -e '.[test]'`)python-mcircuit_designer.test