Lecture note on “Modern Physics and Machine Learning”
Please note that this lecture note is constantly updated.
last update: 2024/9/24
contact: Yuto Ashida (The University of Tokyo)

Contents
Chapter 1.

Introduction

7

Chapter 2. Quantum mechanics review
2.1. Fundamental concepts
2.1.1. Quantum states
2.1.2. Observables
2.1.3. Time evolution
2.1.4. Measurement
2.1.5. Tensor product2.2. Ensembles
2.2.1. Density operators
2.3. Distance measures
2.4. Example: Qubit
2.4.1. Bloch sphere2.4.2. Unitary operator
2.4.3. Density operator
2.5. Example: Harmonic oscillator
2.5.1. Fock state
2.5.2. Coherent state
2.5.3. Squeezed state

10
10
10
10
11
11
12
12
14
15
16
17
17
18
18
18
19
20

Chapter 3. Theory of quantum measurement and open systems
3.1. Introduction: indirect measurement in qubit-boson model
3.2. Positive operator-valued measure
3.3. Completely positive trace-preserving (CPTP) map
3.4. Kraus operators and the equivalence to indirect measurement model
3.5. Bayesian inference and quantum measurement
3.5.1. Diagonal POVM and QND measurement
3.5.1.1. Convergence to measurement basis: wavefunction collapse
3.5.1.2. Born rule of collapsed states
3.5.2. Example: photon QND measurement in cavity QED
3.6. Continuous quantum measurement
3.6.1. Quantum trajectories
3.6.2. Quantum master equation

23
23
26
26
27
28
29
31
32
32
34
34
37

3

CONTENTS

3.6.3. Measurement dynamics vs Dissipation
3.6.4. Brief remark on non-Hermitian physics
3.6.5. Physical examples
Example 1: Photon emission
Example 2: Damped harmonic oscillator
Example 3: Lossy many-body systems
3.6.6. Diffusive limit
3.6.6.1. Wiener process
3.6.6.2. Stochastic Schrödinger equation
3.6.6.3. Example: continuous position measurement
3.7. Master equation from Born-Markov approximations
3.8. Short remark on the validity of master equation approaches
3.9. Exercises
Exercise 3.1 (Continuous position measurement: 2 points)
Exercise 3.2 (Continuous measurement of indistinguishable particles: 2 points)
Exercise 3.3 (Completely positivity: 1 point)
Chapter 4. Foundations of quantum optics
4.1. Introduction
4.2. Quantization of the electromagnetic field
4.2.1. Classical electromagnetism review
4.2.2. Quantization of electromagnetic fields
4.2.3. Commutation relations of quantized fields
4.3. Bosonic Gaussian states
4.3.1. Introduction: single mode
4.3.2. Multiple modes
Characteristic function
Bosonic Gaussian states
Gaussian unitary operation
Example: Gaussian pure state
4.3.3. Application to weakly interacting BEC
4.4. Fermionic Gaussian states
4.4.1. Introduction: single mode
4.4.2. Multiple modes
Gaussian unitary operation
Example: Fermionic Gaussian pure state
Fermionic coherent states and characteristic function
4.4.3. Application to BCS superconductors
4.5. Variational principle
4.5.1. Introduction
4.5.2. Complex-valued variational manifold

4

38
39
39
40
40
41
42
42
42
44
45
48
50
50
50
50
51
51
51
51
53
55
56
56
58
58
59
61
62
63
65
65
66
68
70
71
75
77
77
78

CONTENTS

5

4.5.3. Real-valued variational manifold
4.5.4. Imaginary time evolution
4.5.5. Application to Gaussian states
4.6. Superconducting qubits
4.6.1. LC circuits
4.6.2. Josephson junction
4.6.3. Superconducting qubits: realizing effectively two-level systems
4.6.4. Light-matter interaction: circuit QED
4.7. Exercises
Exercise 4.1 (Field commutation relations: 1 point)
Exercise 4.2 (Single-mode bosonic pure Gaussian state: 1 point)
Exercise 4.3 (McCoy’s formula: 1 point)
Exercise 4.4 (Bosonic Gaussian states: 2 points)
Exercise 4.5 (Generalized uncertainty relation: 1 point)
Exercise 4.6 (Fermionic Gaussian states: 2 points)
Exercise 4.7 (Conservation laws in variational analysis: 1 point)
Exercise 4.8 (Variational time-evolution equations: 1 point)

79
80
82
84
84
85
87
88
90
90
90
90
90
91
91
91
91

Chapter 5. Quantum light-matter interaction
5.1. Classical electrodynamics review
5.1.1. Equations of motion
5.1.2. Redundancy in the dynamical variables and the Coulomb gauge
5.2. Quantized electrodynamics Hamiltonians
5.2.1. Quantum electrodynamics in the Coulomb gauge
5.2.2. Unitary transformation
5.2.3. Gauge transformation
5.3. Long wavelength approximation
5.3.1. Power-Zienau-Woolley transformation and dipole interaction
5.3.2. Matrix elements in different gauges and TRK sum rule
5.3.3. Unitary transformations beyond gauge transformations
5.3.3.1. Pauli-Fierz-Kramers transformation
5.3.3.2. Asymptotically decoupling transformation
5.3.4. Spontaneous emission: Wigner-Weisskopf theory
5.4. Brief introduction to Cavity/Waveguide QED
5.4.1. Two-level approximation: quantum Rabi model and Dicke model
5.4.2. Rotating wave approximation: Jaynes-Cummings model
5.4.3. Strong-coupling physics: asymptotic freedom
5.4.4. Waveguide QED Hamiltonian and Spin-Boson model
5.4.5. Circuit realizations
5.5. Exercises
Exercise 5.1 (PZW gauge without long-wave approximation: 2 points)

92
92
92
94
96
96
97
98
99
99
100
102
102
103
104
106
106
108
108
109
110
112
112

CONTENTS

6

Exercise 5.2 (Applications of TRK sum rule: 2 points)
Exercise 5.3 (Mass renormalization in the asymptotically decoupled frame: 2 points)
Exercise 5.4 (Spontaneous emission: 1 point)

113
113
114

Chapter 6. Machine learning and quantum science
6.1. Introduction
6.2. Basic concepts
6.2.1. Supervised learning
6.2.1.1. Function approximation, cost function, and training
Example 1: Kernel method, support vector machine, and variational quantum circuits
Example 2: Neural network and classifying phases of matter
6.2.2. Unsupervised learning
Example 3: Autoencoder
Example 4: Restricted Boltzmann machine and neural network states
6.3. Black-box optimization
Random search
Evolutionary strategy
Genetic algorithm
Differential evolution
6.4. Exercises
Exercise 6.1 (Gaussian kernel method: 1 point)
Exercise 6.2 (Global optimization: 1 point)

115
115
116
116
116
117
119
120
120
121
122
123
123
124
124
127
127
127

Chapter 7. Reinforcement learning
7.1. Introduction
7.2. Motivating example
7.3. Formalism: Markov decision process
7.4. Value-based search
7.4.1. Bellman equation
7.4.2. Q learning
7.4.3. Deep Q-learning
Fixed target network
Experience replay
7.4.4. Physics application: quantum control of continuously monitored systems
7.5. Policy-based search
7.6. Black-box optimization in deep RL
7.7. Some tips on implementation
7.8. Exercise
Exercise 7.1 (Mastering video game “Pong” via Black box optimization: 2 points)

128
128
128
130
131
131
132
133
134
134
136
136
138
138
139
139

CHAPTER 1

Introduction
In this course, we aim to learn some of the fundamental topics in modern quantum science, at theintersection of quantum optics, quantum information, and many-body physics. We will also try to coverelements of machine learning methods related to those topics, especially reinforcement learning that is
gaining importance in the context of quantum control and other related fields. As I am theorist, the lectures will be given from a theoretical perspective, while I shall try to connect them with actual quantum
experiments in laboratory when appropriate. Below let me provide a brief overview and background of
this course.
It is a wonderful time for studying quantum science; one can now routinely perform microscopic
observations and manipulations of genuine quantum systems in the laboratory. This has allowed us to
study and understand quantum physics in a highly controlled and clean/coherent manner, paving the way
toward realizing future quantum devices, such as quantum computers or quantum simulators. At the sametime, such development necessarily requires us to understand physics of open systems as no physicalsystems can be completely isolated from their environments.
The system-environment interaction, or said differently, measurement backaction in general causes
wavefunction collapse and is often detrimental to performing operations in quantum devices that are supposed to be coherent. Besides the role as the main obstacle for quantum technologies, however, suchcoupling to external degrees of freedom has often offered a new possibility of realizing some unique andexotic phenomena that are otherwise very difficult to realize in isolated systems. It is thus of fundamental
importance to understand basics of open quantum systems, and it is this topic we aim to cover in Chapters
2 and 3.
We will also cover foundations of quantum optics and quantum light-matter interactions in Chapters 4
and 5. These topics will not only serve as a perfect example for a rather abstract theory of open quantum
systems presented in the previous Chapters, but also play central roles in the modern quantum technologies. For example, superconducting qubits and circuit/cavity QED systems discussed there are currently
one of the most promising building blocks toward realizing quantum computers. Moreover, theoreticalmethods we introduce there, including variational principles and bosonic/fermionic Gaussian states, arein fact useful theoretical tools to understand quantum many-body physics in condensed matter systems.
On another front, recent years have witnessed the emergence of a new powerful computational technique, that is, machine learning. In particular, deep artificial neural networks are now routinely used toclassify images, translate languages, control robots, and even play video games at superhuman levels. In
the context of physics, machine learning has found applications to predicting certain material properties,
71. INTRODUCTION

8classifying phases of matter, and controlling quantum or classical physical systems. This is a rapidly
growing area of research and it is of course impossible to cover all the topics in the emerging field of “machine learning and physics” in the present course. Rather we here try to especially focus on one importantbranch of machine learning, namely, reinforcement learning.
Compared to a more elementary topic like supervised learning, reinforcement learning can be considered as advanced concept since it requires neither teacher training a student nor a priori knowledge about aproblem at hand. In this sense, reinforcement learning can potentially find a novel solution to the problem
beyond human abilities; said differently, machine can have kind of creativity. Indeed, when combinedwith deep neural networks, it has achieved superhuman performance in the game of Go. We will also see
an illustrative example which indicates that the reinforcement learning holds great promise for controllingopen quantum systems. We will learn about some basic concepts of machine learning methods including
reinforcement learning in Chapters 6 and 7. Nevertheless, I would not pretend to teach specific techniques
for numerical implementations, such as how to use existing libraries etc., but rather the main goal of thispart is to help you to gain intuition/key ideas behind the machine learning algorithms. I believe that theproper understanding of those algorithms is more important than learning about specific numerical techniques, as you should then be able to straightforwardly generalize/apply existing codes to problems at
your hand.

1. INTRODUCTION

Summary of Chapter 1
This course attempts to cover the following topics:• Review of (mostly) undergrad quantum mechanics (Chap. 2)• General theory of quantum measurement and Markovian open quantum systems (Chap. 3)• Brief introduction to non-Hermitian physics (Chap. 3)• Quantization of the electromagnetic field (Chap. 4)• General theory of bosonic and fermionic Gaussian states (Chap. 4)• Basics of time-dependent variational principles (Chap. 4)• Introduction to circuit/cavity/waveguide quantum electrodynamics (Chap. 4 and 5)• General theory of quantum light-matter interaction (Chap. 5)• Brief review of basic concepts in machine learning (Chap. 6)• Basics of (deep) reinforcement learning (Chap. 7)• Black-box optimization and its application to deep reinforcement learning (Chap. 6 and 7)• Deep reinforcement learning of a simple quantum control task (Chap. 7)
What is not included in this course:• Non-Markovian or nonperturbative open quantum systems• Theory of non-Gaussian many-body states• Details about supervised/unsupervised learning• Model-based reinforcement learning algorithms• Details about policy-based reinforcement learning algorithms• Details about implementations for machine learning practitioners• and many other important topics...

9

CHAPTER 2

Quantum mechanics review
2.1. Fundamental concepts
2.1.1. Quantum states. Mathematically, a quantum state |ψ⟩ is described as a vector in a Hilbert
space H:
Hilbert space. Hilbert space H is a vector space over the complex numbers C. It associates with an
inner product ⟨ψ|ϕ⟩ of an ordered pair of vectors |ψ⟩, |ϕ⟩, which satisfies• Conjugate symmetry: ⟨ϕ|ψ⟩ = ⟨ψ|ϕ⟩∗ .
• Linearity: ⟨ψ|(a|ϕ⟩ + b|φ⟩) = a⟨ψ|ϕ⟩ + b⟨ψ|φ⟩.
p̸ 0.
• Positivity: ∥|ψ⟩∥ ≡ ⟨ψ|ψ⟩ > 0 for |ψ⟩ =It is also a complete metric space with respect to the distance defined by the norm d(|ψ1 ⟩, |ψ2 ⟩) =∥|ψ1 ⟩ − |ψ2 ⟩∥.
Precisely speaking, we should consider a quantum state as a ray, which is an equivalent class of vectors that differ only by a nonzero complex constant factor. Since we are interested in a vector that has theunit norm, ⟨ψ|ψ⟩ = 1, this just means that its overall phase factor is physically irrelevant, i.e., it does notaffect an expectation value of a physical observable as we see below.
2.1.2. Observables. An observable is a physical quantity that can be measured and mathematicallydescribed as a self-adjoint operator. To see this, let us first introduce the notion of a Hermitian operator.
Hermiticity. A linear map Ô is a Hermitian operator if and only if ⟨ϕ|Ô|ψ⟩ = ⟨ψ|Ô|ϕ⟩∗ for ∀|ψ⟩, |ϕ⟩ ∈ H.
We can also express this condition as the self-adjoint condition Ô = Ô† , where we define the adjointÔ† of Ô by1(2.1.1)⟨ϕ|Ôψ⟩ = ⟨Ô† ϕ|ψ⟩.

It is known that a self-adjoint operator has a spectral decomposition, i.e., its eigenstates form a complete
orthonormal basis in H:
1Strictly speaking, for infinite-dimensional case, the self-adjointness not only requires the Hermiticity, but also the condition thatthe domains of Ô and Ô† are the same. Yet, this subtle difference does not cause problems at least in this course, so we shall use
the terms Hermitian or self-adjoint operators interchangeably.
102.1. FUNDAMENTAL CONCEPTS

Ô =(2.1.2)X

11

λn P̂n ,nwhere λn is an eigenvalue and P̂n is the projection operator onto the corresponding eigenspace satisfying(2.1.3)P̂n P̂m = δnm P̂n(2.1.4)P̂n = P̂n† .

These projectors form a complete set such that they span the entire Hilbert spaceX(2.1.5)ˆP̂n = I,n

where Iˆ is the identity operator.
2.1.3. Time evolution. Time evolution of a quantum state is given by the Schrödinger equation (weset ℏ = 1):(2.1.6)id|ψ(t)⟩ = Ĥ|ψ(t)⟩,dtwhere Ĥ is a self-adjoint operator called the Hamiltonian. Its solution is easily obtained as(2.1.7)|ψ(t)⟩ = Û (t)|ψ(0)⟩,(2.1.8)Û (t) = e−iĤt .

Here Û (t) is the unitary operator, i.e., Û (t)Û † (t) = 1. As a result, the norm of a quantum state ispreserved:(2.1.9)⟨ψ(0)|ψ(0)⟩ = ⟨ψ(t)|ψ(t)⟩ = 1.

2.1.4. Measurement. A measurement of an observable Ô for a state |ψ⟩ gives one of the eigenvalues
λn with probability(2.1.10)pn = ⟨ψ|P̂n |ψ⟩and prepares the corresponding eigenstate:(2.1.11)P̂n |ψ⟩√ .
pnSuppose that we perform the same measurement processes for the same state many times; then the expectation value ⟨Ô⟩ of the measurement outcomes is given by2.2. ENSEMBLES(2.1.12)⟨Ô⟩ ≡

X

λn pn =X12

λn ⟨ψ|P̂n |ψ⟩ = ⟨ψ|Ô|ψ⟩,nnwhere we used the spectral decomposition of Ô. Note that the phase factor of a vector |ψ⟩ is irrelevant to
the expectation value as we mentioned above.
2.1.5. Tensor product. Consider two systems, A and B, whose Hilbert space is denoted by HA and
HB , respectively. Then the composite system A + B is given by the tensor product HA ⊗ HB , i.e., it is
now spanned by a quantum state |ψ⟩A ⊗ |ϕ⟩B with |ψ⟩A ∈ HA and |ϕ⟩B ∈ HB .
To understand the notion of tensor product, it is useful to consider a simple example of a pair of twolevel systems. Let us describe a quantum state |ψ⟩ of a two-level system spanned by {|0⟩, |1⟩} as a vector
representation!
a.
(2.1.13)|ψ⟩ = a|0⟩ + b|1⟩ ⇐⇒ |ψ⟩ =bThen suppose that we have the following states in each of the two-level systems|ψ⟩A =(2.1.14)uv!
, |ϕ⟩B =xy!
.

The resulting composite state is a four-dimensional vector given byux
 uy 
|ψ⟩A ⊗ |ϕ⟩B = .
 vx 


(2.1.15)vy
Similarly, if we have the operators(2.1.16)

Â =

a b
c d

!

e f
g h

, B̂ =!
,

the composite operator is a 4 × 4 matrix given by(2.1.17)ae ag

Â ⊗ B̂ = 
 ce
cg

af
ah
cf
ch


be bf
bg bh 
.
de df dg dh

2.2. Ensembles
The formulation of quantum mechanics in the previous section is provided for an ideal situation,in which all the microscopic quantum degrees of freedom in a system of interest are accessible to an2.2. ENSEMBLES

13

observer; in other words, this formulation is supposed to deal with a completely isolated (or closed)
quantum system. However, in real world, what we can observe is only a small part of an entire quantum
system and this small quantum system interacts with external degrees of freedom that one cannot handle,
i.e., a real quantum system necessarily behaves as an open system.
To motivate this, consider the following quantum state in the composite space of a pair of two-levelsystems:(2.2.1)1|ψ⟩AB = √ (|0⟩A |0⟩B + |1⟩A |1⟩B ) .
2

Suppose that we have access to a two-level system A, but not for B and thus we know nothing about astate of B. In this case, only an observable acting on A is of interest and we expect that there exists a selfconsistent theoretical description within A. To see this, consider the expectation value of an observableÔA that (nontrivially) acts only on A:⟨ÔA ⟩ ≡AB ⟨ψ|ÔA ⊗ IˆB |ψ⟩AB1
(2.2.3)=A ⟨0|ÔA |0⟩A + A ⟨1|ÔA |1⟩A .
2
We now see that the expectation value can be obtained by solely dealing with a quantum state in A.
However, this value is different from the usual expression for the expectation value with respect to a purestate, but can be represented as the trace over the following density operator ρ̂A :(2.2.2)(2.2.4)⟨ÔA ⟩ = TrA [ρ̂A ÔA ] ≡X

A ⟨i|ρ̂A ÔA |i⟩A

i=0,1

with
1
(|0⟩A A ⟨0| + |1⟩A A ⟨1|) .
2
We should interpret ρ̂A as an ensemble of possible quantum states occurring with the equal probability.
This means that the expectation value is equal to the ensemble average (or incoherent mixture) over thecorresponding expectation values with respect to two states:(2.2.5)(2.2.6)

ρ̂A ≡ TrB [|ψ⟩AB AB ⟨ψ|] =⟨ÔA ⟩ =1 X
A ⟨i|ÔA |i⟩A .
2
i=0,1

In other words, an initially pure quantum state should in general be described as an incoherent mixture ofdifferent pure states after it interacts with unknown external degrees of freedom. For instance, the relative
phase in the original pure state |ψ⟩AB is no longer accessible and there will be no interference effects inρ̂A . The partial trace over B above corresponds to such procedure of discarding knowledge about external
degrees of freedom; since an observer cannot access to environment degrees of freedom, the best one can2.2. ENSEMBLES

14

do is to take the unbiased summation over quantum states in B. As we see later, this process lies at theheart of important phenomenon called decoherence.
2.2.1. Density operators. In general, an operator ρ̂ acting on a Hilbert space H is called a density
operator if it satisfies the following properties:• ρ̂ = ρ̂†• ρ̂ ≥ 0 ⇐⇒ ⟨ψ|ρ̂|ψ⟩ ≥ 0 ∀|ψ⟩ ∈ H• Tr(ρ̂) = 1
If a quantum state is a pure state |ψ⟩, the corresponding density operator is given byρ̂ = |ψ⟩⟨ψ|(2.2.7)and satisfies the following conditionρ̂2 = ρ̂.

(2.2.8)It is often useful to note the fact that there is an orthonormal basis that diagonalizes ρ̂ with nonnegativeand real eigenvalues whose summation gives one:ρ̂ =(2.2.9)X

pi |i⟩⟨i|, ⟨i|j⟩ = δij ,iXpi = 1.

i

The coefficients pi can naturally be interpreted as the probability weight associated with ρ̂, which is theincoherent mixture of pure states {|i⟩}.
It is also worthwhile to note that a density operator satisfies the convexity; using two density matrices
ρ̂1,2 , one can construct another density matrix by a linear combination:ρ̂(λ) = λρ̂1 + (1 − λ)ρ̂2 , 0 ≤ λ ≤ 1.

(2.2.10)

Indeed, it obviously satisfies the self-adjoint condition ρ̂(λ) = ρ̂† (λ) and the normalization Tr[ρ̂(λ)] = 1.
The positivity is also satisfied because(2.2.11)⟨ψ|ρ̂(λ)|ψ⟩ = λ⟨ψ|ρ̂1 |ψ⟩ + (1 − λ)⟨ψ|ρ̂2 |ψ⟩ ≥ 0, ∀|ψ⟩ ∈ H

We note that for any density operator ρ̂A one can perform its purification, that is, one can construct acomposite pure state |Ψ⟩AB that gives ρ̂A upon doing a partial trace over degrees of freedom in B. Using
the expression (2.2.9), one possible purification is given by(2.2.12)|Ψ⟩AB =

X√iwith orthonormal states in B

pi |i⟩A ⊗ |ϕi ⟩B

2.3. DISTANCE MEASURES15

B ⟨ϕi |ϕj ⟩B = δij .

(2.2.13)
Indeed, it leads to the required relationρ̂A = TrB [|Ψ⟩AB AB ⟨Ψ|].

(2.2.14)2.3. Distance measures
Suppose that we have two density operators ρ̂ and σ̂. Since a density operator satisfies the positivity,√√
there exists the square-root operator ρ̂ following from the condition ( ρ̂)2 = ρ̂. For instance, in the
diagonal basis, it is given byX√
p(2.3.1)ρ̂ =pi |i⟩⟨i|.
i

Building on this, one can quantify the distinguishability of two ensembles via the following measure
known as the fidelity:
 q
2
p p
F (ρ̂, σ̂) ≡ Trρ̂σ̂ ρ̂.

(2.3.2)

Alternatively, one can represent the fidelity as the L1 norm as follows:(2.3.3)F (ρ̂, σ̂) =

√ p 2
σ̂ ρ̂

≡ Tr

i
hp
Â† Â .

1

where we define the L1 norm byÂ

(2.3.4)1

A useful expression of the L1 norm is(2.3.5)Â

1hi= maxÛ Tr Û † Â ,where Û is unitary. One can check this expression by using the singular value decomposition of Â.
It satisfies the following properties• F (ρ̂, σ̂) = F (σ̂, ρ̂)• 0 ≤ F (ρ̂, σ̂) ≤ 1, F (ρ̂, ρ̂) = 1
2P √• F (ρ̂, σ̂) =if [ρ̂, σ̂] = 0i pi qi
where pi and qi are eigenvalues of ρ̂ and σ̂, respectively, and we define the commutator by(2.3.6)[Â, B̂] ≡ ÂB̂ − B̂ Â.

2.4. EXAMPLE: QUBIT

16

Note that the last relation for commuting density operators is nothing but the usual fidelity between (classical) probability distributions. The upper bound in the second relation F ≤ 1 follows from the relation

F (ρ̂, σ̂) ≤ Tr(ρ̂)Tr(σ̂) = 1,

(2.3.7)

which can be shown by using the relation (2.3.5) and the Cauchy-Schwarz inequality:(2.3.8)F (ρ̂, σ̂) = Tr


p

 

† √  2p † p 
√ √ 
Tr
ρ̂Û
σ̂
≤ Tr
ρ̂Û
ρ̂Û
σ̂ σ̂ = Tr(ρ̂)Tr(σ̂).

If one of the density operators is pure ρ̂ = |ψ⟩⟨ψ|, then the fidelity reduces to(2.3.9)F (ρ̂, σ̂) = ⟨ψ|σ̂|ψ⟩.

In particular, when the other one is also pure σ̂ = |ϕ⟩⟨ϕ|, it simplifies to the overlap(2.3.10)F (ρ̂, σ̂) = |⟨ψ|ϕ⟩|2 .

Finally, there exists a useful general relation between the fidelity and the trace norm:1|ρ̂ − σ̂|21 .
4
It will be a good exercise for you to check this inequality (hint: you may use the triangle inequality for the
trace norm, the Cauchy-Schwarz inequality, and the relation (2.3.5)).
(2.3.11)F (ρ̂, σ̂) ≤ 1 −2.4. Example: Qubit
Let us illustrate the fundamental notions introduced above by discussing the minimal example: atwo-dimensional Hilbert space spanned by orthonormal quantum states {|0⟩, |1⟩}. Since an overall phasefactor is irrelevant, a generic normalized state in this space is described as(2.4.1)

 
 
θ
θ
iϕ
|ψ(θ, ϕ)⟩ = cos|0⟩ + e sin
|1⟩ =
2
2

 !
cos 2θ
 .
eiϕ sin 2θ

The angles θ and ϕ characterize the probability weights for each state and the relative phase between two
states, respectively.
This represents a quantum counterpart of classical bit, namely, qubit. For instance, the spin-1/2 degreeof freedom of electron or two internal states in photon polarizations exactly behave as the qubit. However,as we will see in later Chapters, qubits in many of state-of-the-art quantum computers, such as superconducting qubits, actually use two lowest levels in a larger Hilbert space. Thus, it is important to keep inmind that they do not perfectly operate as two-level systems in strict sense, but rather effectively play therole of qubits with a certain (usually high) fidelity.

2.4. EXAMPLE: QUBIT

17

2.4.1. Bloch sphere. It is useful to consider the basis |0⟩, |1⟩ as the spin-up and down states along
the z axis, |0⟩ = | ↑⟩, |1⟩ = | ↓⟩. A general qubit state |ψ(θ, ϕ)⟩ can then be interpreted as a spin vector
on the Bloch sphere pointing in the direction corresponding to the anglen = (sin θ cos ϕ, sin θ sin ϕ, cos θ)T .

(2.4.2)To see this, let us introduce the Pauli operators(2.4.3)

σ̂1 =

0 1
1 0

!
, σ̂2 =

0 −i
i 0

!
, σ̂3 =

1 0
0 −1

!
.

(We also often interchangeably use the notation σ̂1,2,3 = σ̂ x,y,z later.) These operators satisfy the commutation and anticommutation relations[σ̂i , σ̂j ] = σ̂i σ̂j − σ̂j σ̂i = 2i

(2.4.4)3X

ϵijk σ̂kk=1{σ̂i , σ̂j } = σ̂i σ̂j + σ̂j σ̂i = 2δij Iˆ

(2.4.5)where ϵijk is the completely antisymmetric tensor. One can check that a state |ψ(θ, ϕ)⟩ is an eigenstate of!
cos θe−iϕ sin θ(2.4.6)n · σ̂ =eiϕ sin θ − cos θand has an eigenvalue +1.

2.4.2. Unitary operator. The most general unitary operator acting on the two-dimensional Hilbert
space is expressed by(2.4.7)Û (θ, ϕ) = !
−e−iϕ sin 2θ
cos 2θ
.
eiϕ sin 2θ
cos 2θ

In the language of the Bloch sphere, this corresponds to the counterclockwise rotation by angle θ aroundthe axis n0 :(2.4.8)iθÛ (θ, ϕ) = exp − n0 · σ̂ , n0 = (− sin ϕ, cos ϕ, 0)T .
2

A generic qubit state |ψ(θ, ϕ)⟩ can be obtained by acting this unitary operator on |0⟩ = (1, 0)T :(2.4.9)|ψ(θ, ϕ)⟩ = Û (θ, ϕ)|0⟩.

This is as it should be, since |0⟩ corresponds to spin pointing in positive z direction and Û (θ, ϕ) rotates this
into the direction n = (sin θ cos ϕ, sin θ sin ϕ, cos θ)T that corresponds to a state |ψ(θ, ϕ)⟩ as mentioned
earlier.

2.5. EXAMPLE: HARMONIC OSCILLATOR

18

2.4.3. Density operator. We next discuss a density operator of a qubit. The general 2 × 2 Hermitianmatrix is parameterized by 4 real parameters. Since a density operator satisfies the normalization condition
Tr[ρ̂] = 1, it is characterized by 3 real parameters. As Pauli operators are linearly independent andtraceless, we conclude that the most general form of a density matrix is given by(2.4.10)

1 + v3 v1 − iv2v1 + iv2 1 − v3

11
ρ̂(v) = (Iˆ + v · σ̂) =22!
, v ∈ R3 , |v| ≤ 1.

Its determinant is1
1 − v2 .
4Thus, the condition |v| ≤ 1 ensures that ρ̂ only has nonnegative eigenvalues.
The condition for a pure state ρ̂2 = ρ̂ can be read asdetρ̂ =

(2.4.11)(2.4.12)(v · σ̂)2 = Iˆ ⇐⇒ |v| = 1.

Thus, using a unit-norm vector n, we can represent a density operator for a generic pure state as1ρ̂(n) = (Iˆ + n · σ̂).
2(2.4.13)

It is easy to check that this density operator with n = (sin θ cos ϕ, sin θ sin ϕ, cos θ)T in fact corresponds
to the above pure state |ψ(θ, ϕ)⟩:(2.4.14)ρ̂(n) = |ψ(θ, ϕ)⟩⟨ψ(θ, ϕ)|.
2.5. Example: Harmonic oscillator

2.5.1. Fock state. We next consider another elementary example - a harmonic oscillator2:(2.5.1)(2.5.2)

Ĥ =




1
ω 2
p̂ + x̂2 = ω â† â +42x̂ = â + â† , p̂ = i(â† − â), [x̂, p̂] = 2i.

Its spectrum is obtained by noting the fact that the annihilation â and creation â† operators satisfy thecommutation relation(2.5.3)[â, â† ] = 1,

and introducing the Fock states {|n⟩} with n = 0, 1, 2 . . . as
2There are several possible choices for position/momentum operators; another common convention is x̂ = (â + â† )/√2,√p̂ = i(â† − â)/ 2. The choice (2.5.2) is often used in quantum optics, so we shall follow this notation.

2.5. EXAMPLE: HARMONIC OSCILLATORâ|n⟩ =

(2.5.4)√n|n − 1⟩, â† |n⟩ =√19

n + 1|n + 1⟩,â† â|n⟩ = n|n⟩.

(2.5.5)

In particular, for the ground state with n = 0, also called as vacuum state, we getâ|0⟩ = 0.

(2.5.6)The corresponding eigenvalue is1
En = ω n +2(2.5.7).

2.5.2. Coherent state. Fock states are eigenstates of the number operator n̂ ≡ â† â, while one canalso construct an eigenstate of the annihilation operator â, the coherent state:|α⟩ ≡(2.5.8)∞X
αn2√ e−|α| /2 |n⟩, α ∈ Cn!
n=0â|α⟩ = α|α⟩.

(2.5.9)The occupation probabilities in the Fock basis obey the Poisson distribution with mean n̄ ≡ |α|2 :P (n) =

(2.5.10)n̄n −n̄e .
n!

Note that |α⟩ is not an eigenstate of the creation operator â† , which acts asâ† |α⟩ =(2.5.11)∂+ α∗ |α⟩.
∂α

The vacuum state |0⟩ is a special example of a coherent state with α = 0. In fact, a general coherent
state can be obtained by acting the following displacement operator on the vacuum:
†

∗

2

†

∗

2

∗

†

(2.5.12)D̂(α) = eαâ −α â = e−|α| /2 eαâ e−α â = e|α| /2 e−α â eαâ ,(2.5.13)D̂(α)|0⟩ = |α⟩.

The displacement operator satisfies the following properties• D̂† (α) = D−1 (α) = D(−α)• D̂† (α)âD̂(α) = â + α• D̂† (α)â† D̂(α) = â† + α∗∗∗∗∗
• D̂(α + β) = e−(αβ −α β)/2 D̂(α)D̂(β) = e(αβ −α β)/2 D̂(β)D̂(α)

2.5. EXAMPLE: HARMONIC OSCILLATOR

20

The last relation follows from the Baker-Campbell-Hausdorff (BCH) formula:(2.5.14)1

1eÂ+B̂ = eÂ eB̂ e− 2 [Â,B̂] = eB̂ eÂ e 2 [Â,B̂] if [Â, [Â, B̂]] = [B̂, [Â, B̂]] = 0.

In particular, note that two different coherent states are not orthogonal:
2|⟨β|α⟩| = e−|α−β| /2 .

(2.5.15)

2.5.3. Squeezed state. To motivate the notion of a squeezed state, consider the following Hamiltonian that includes the so-called Â2 term on top of the usual harmonic oscillator3:
g2 2
Â , Â ≡ â† + â.
2ω
Since this Hamiltonian is still quadratic in terms of creation or annihilation operators, one can diagonalize
it by using the unitary transformation often called as the Bogoliubov transformation:(2.5.16)Ĥ = ωâ† â +(2.5.17)b̂ = cosh râ + sinh râ† ≡ Ûr† âÛr ,r(2.5.18)

r

e =

p
r
†2
2
Ω
, Ω = ω 2 + 2g 2 , Ûr = e 2 (â −â ) .
ωNote that the new annihilation/creation operators b̂, b̂† satisfy the canonical commutation relation(2.5.19)[b̂, b̂† ] = 1.

Thus, the above Hamiltonian is diagonalized as(2.5.20)Ĥ = Ωb̂† b̂ + const.,

where we use â = cosh rb̂ − sinh rb̂† . The unitary operator Û is an example of the squeezing operator,
and the transformed ground state is given by the squeezed vacuum(2.5.21)

r

2

†2

ir

|0⟩b ≡ Ûr† |0⟩ = e 2 (â −â ) |0⟩ = e 4 (x̂p̂+p̂x̂) |0⟩.

To see the reason why it represents “squeezing”, it is useful to rewrite the Hamiltonian in terms of
position and momentum operators as (aside constant)(2.5.22)

 g2 2 ωω 2Ĥ =p̂ + x̂2 +
x̂ =
4
2ω
4



Ω2 2
2
p̂ + 2 x̂ .
ω

3We will later see that this Hamiltonian is relevant to the minimal setup of cavity quantum electrodynamics in the Coulomb

gauge.

2.5. EXAMPLE: HARMONIC OSCILLATOR

21This means that the above Hamiltonian is nothing but the usual harmonic oscillator, but with a tighter
potential with Ω2 /ω 2 ≥ 1. Consequently, the ground state |0⟩b is more strongly confined in the x direction
compared with the original vacuum |0⟩. Due to the uncertainty relation, this results in an inevitable“expansion” in the p direction. More explicitly, we get(2.5.23)

Ûr

x̂
p̂

!
Ûr† =

e−r x̂
er p̂

!
.

Because position and momentum operators are interchangeable in harmonic oscillators, one can simiarly consider a squeezing in a general direction on the x-p plane. A general squeezing operator is givenby(2.5.24)†2∗ 2

S(ζ) ≡ e(ζâ −ζ â )/2 , ζ = reiθ ∈ C,

which transforms the annihilation and creation operators by(2.5.25)

Ŝ † (ζ)âŜ(ζ) = cosh râ + eiθ sinh râ† .

2.5. EXAMPLE: HARMONIC OSCILLATOR

22

Summary of Chapter 2
Section 2.1 Fundamental concepts• A quantum state is described as a vector (or precisely speaking, ray) in a Hilbert space.
• An observable is described by a self-adjoint operator on a Hilbert space.
• Time evolution of a quantum state is given by the Schrödinger equation governed by the selfadjoin operator called the Hamiltonian.
• A measurement process is described as an orthogonal projection onto the corresponding
eigenspace.
• A composite system is described by the tensor product.
Section 2.2 Ensembles• When only a small part of an entire quantum system is accessible, a quantum state is describedby an ensemble called density operator.
• A density operator is self-adjoint, positive, and satisfies the convexity.
• Any density operator can be purified, i.e., one can construct a composite pure state that gives
a density operator of interest upon doing a partial trace.
Section 2.3 Distance measures• The distinguishability of two quantum states can be quantified by the fidelity.
• The fidelity can be related to other distance measures such as the L1 norm through the(in)equalities.
Section 2.4 Qubit• A general qubit pure state can be represented as a unit-norm vector on the Bloch sphere.
• Similarly, a density operator of a qubit can be represented as unnormalized vector v in 3D
space with |v| ≤ 1.
Section 2.5• Eigenspectrum of a harmonic oscillator is given by using the annihilation and creation operators.
• An energy eigenstate is given by a Fock state.
• An eigenstate of the annihilation operator is a coherent state.
• When a quadratic term is added to the Hamiltonian, eigenstate is in general squeezed along acertain direction on the position-momentum space.

CHAPTER 3

Theory of quantum measurement and open systems
3.1. Introduction: indirect measurement in qubit-boson model
In this Chapter, we give an introduction to theory of quantum measurement and open systems. Before
proceeding to a general formalism, let us first consider a simple motivating example, where a system
of interest is a single qubit and it interacts with a “meter” degree of freedom modeled by a harmonic
oscillator. Physically, this may be considered as a toy model for spontaneous emission process, in which
the two-level system (qubit) can be considered as an atom that can emit a photon into a simplified singlemode electromagnetic mode playing the role of the meter.
The model Hamiltonian is(3.1.1)

ĤJC =ℏ∆ zσ̂ + ℏωc b̂† b̂ + ℏγ(σ̂ − b̂† + σ̂ + b̂),2σ̂ x ± iσ̂ y, [b̂, b̂† ] = 1,2
which is known as the Jaynes-Cummings model. In fact, this model can be microscopically derived fromthe fundamental light-matter Hamiltonian after performing certain approximations, as we will see in later
Chapters. A notable feature of this Hamiltonian is that it conserves the excitation number(3.1.2)σ̂ ± =

1
Nex = (σ̂ z + 1) + b̂† b̂.
2
To be concrete, suppose that the initial state is the product state of the excited qubit and the photon
vacuum
(3.1.3)(3.1.4)ρ̂tot (0) = |e⟩⟨e| ⊗ |0⟩⟨0|,which belongs to the excitation manifold with Nex = 1. The evolved state is(3.1.5)ρ̂tot (t) = Û ρ̂tot (0)Û † , Û = e−iĤJC t .

We then perform a projection measurement on the meter degree of freedom; since we consider thesector with Nex = 1, there are only two possible outcomes corresponding to the following projection
operators(3.1.6)P̂0 = |0⟩⟨0|, P̂1 = |1⟩⟨1|.

23

3.1. INTRODUCTION: INDIRECT MEASUREMENT IN QUBIT-BOSON MODEL

24

They satisfy the completeness relation (within the Hilbert space of interest here):IˆA =(3.1.7)

1
X

P̂m .

m=0

The postmeasurement (unnormalized) state with outcome m ∈ {0, 1} is given by(3.1.8)Em (ρ̂S (0)) ≡ TrA

h



i
IˆS ⊗ P̂m ρ̂tot (t) IˆS ⊗ P̂m ,where we note that the trace TrA is taken only over the meter degree of freedom, and Em is defined by thelinear map acting on the system initial state:ρ̂S (0) = |e⟩⟨e|.

(3.1.9)Physically, the linear map Em represents the nonunitary evolution of the system conditioned on measurement outcome m. The outcome occurs with the probability(3.1.10)pm = TrS [Em (ρ̂S (0))] .

The normalized postmeasurement state isρ̂S,m =

(3.1.11)

Em (ρ̂S (0)).
pm

We can rewrite this nonunitary evolution in a more convenient way. To this end, we introduce theKraus operators M̂m by
M̂m = ⟨m|Û |0⟩,(3.1.12)

where note that the inner product is taken only over the meter degree of freedom and thus M̂m is anoperator that acts on the system Hilbert space. This leads to the expression of the postmeasurement state

i
h

(3.1.13)
Em (ρ̂S (0)) = TrA IˆS ⊗ P̂m Û ρ̂S (0) ⊗ |0⟩⟨0|Û † IˆS ⊗ P̂m(3.1.14)= ⟨m|Û |0⟩ρ̂S (0)⟨0|Û † |m⟩(3.1.15)†= M̂m ρ̂S (0)M̂m.

The probability is then expressed by the positive operator-valued measure (POVM) Êm as follows(3.1.16)hi†pm = TrS ρ̂S (0)Êm , Êm = M̂mM̂m .

Importantly, Êm are in general nonorthogonal and thus can be considered as a generalization of projection
measurement operators P̂m . The Kraus operators and the POVMs satisfy the completeness condition:(3.1.17)

1
X
m=0

Êm =

1
X
m=0

†
M̂m
M̂m = IˆS ,3.1. INTRODUCTION: INDIRECT MEASUREMENT IN QUBIT-BOSON MODEL

25

leading to1X(3.1.18)

pm = 1.

m=0

So far, we discuss the time evolution that is conditioned on a certain measurement outcome. Anotherimportant concept is the nonunitary evolution that is unconditioned on measurement outcomes, often
called the nonselective evolution defined by(3.1.19)

E(ρ̂S (0)) ≡1X

Em (ρ̂S (0)) =

m=0

1
X

†
M̂m ρ̂S (0)M̂m.

m=0This mapping E satisfies the linearity, positivity1, and trace-preserving properties Tr[E(ρ̂)] = 1; we will
generalize its notion as the completely positive trace preserving (CPTP) map later.
As shown below, it is often useful to consider the so-called continuous measurement limit, for which
we take γ → ∞ and τ → 0 while keeping γ 2 τ finite. Here, we denote τ as the (short) time duration
for a single whole process of the indirect measurement. In this limit, at the leading order of γτ ≪ 1, thepostmeasurement state when a photon is detected is given by(3.1.20)

ρ̂S,1 =

τ L̂ρ̂S (τ )L̂†
,
p1

where
(3.1.21)zρ̂S (τ ) = ÛS |e⟩⟨e|ÛS† , ÛS = e−i∆σ̂ τ /2 ,and we define the so-called jump operator by(3.1.22)L̂ ≡p
γ 2 τ σ̂ − .

In this limit, the Kraus operator corresponding to photon detection (m = 1) is given by√(3.1.23)M̂1 ≃ τ L̂
while the probability is(3.1.24)hip1 = τ TrS L̂ρ̂S (τ )L̂† .

This nonunitary evolution (often called quantum jump) corresponds to the process that, due to the photon
detection, an observer acquires the information that the system (two-level atom) de-excites and is thus
projected onto the ground state, i.e., ρ̂S,1 = |g⟩⟨g|.
Meanwhile, when a photon is not detected during the time interval τ , the postmeasurement state isgiven by
oi
1 h
τn †
(3.1.25)
ρ̂S,0 =
ρ̂S (τ ) −L̂ L̂, ρ̂S (τ ) ,p0
2

1A linear operator Ô ∈ L(H) acting on the Hilbert space H is positive, Ô ≥ 0, iff ⟨ψ|Ô|ψ⟩ ≥ 0 for ∀|ψ⟩ ∈ H. A linear

mapping E : L(H) → L(H) is then said to be positive, E ≥ 0, iff E(Ô) ≥ 0 for any positive operator Ô ≥ 0.

3.3. COMPLETELY POSITIVE TRACE-PRESERVING (CPTP) MAP

26

and the probability iship0 = 1 − τ TrS L̂ρ̂S (τ )L̂† .

(3.1.26)
The corresponding Kraus operator isM̂0 = 1 − iĤeff τ /ℏ

(3.1.27)

where we introduce the non-Hermitian effective Hamiltonian byℏ∆ z i †ℏ∆ z iγ 2 τσ̂ − L̂ L̂ =σ̂ −|e⟩⟨e|.
22
2
2
It is important to note that, even in this no-photon-counting process, an observer actually gains the information about the system in the sense that she/he knows that no spontaneous emission has occurred during
the time interval τ . This information acquisition indicates that the system is more likely to be in the groundstate than in the excited state. It is this tendency that is ascribed to the measurement backaction, which is
captured by the non-Hermitian term in Ĥeff .
(3.1.28)Ĥeff =

3.2. Positive operator-valued measure
From now on we present a general theory of quantum measurement by considering each notion introduced in the previous section in a more abstract manner. To begin with, motivated by the above simple
example, we characterize the POVMs Êm by the following properties†• Êm = Êm• Êm ≥ 0Pˆ•m Êm = I

Recall that they are not orthogonal in general. Using the POVMs, the probability of observing measurement outcome m is given by(3.2.1)

pm = Tr[Êm ρ̂].

We note that the Hermiticity and positivity of POVMs ensures that pm are real and nonnegative, while thePcompleteness relation ensures the normalization m pm = 1; altogether, pm can indeed be interpreted asprobabilities.

3.3. Completely positive trace-preserving (CPTP) map
We next introduce the notion of a completely positive trace-preserving (CPTP) map E in a generalway. We shall first give a formal definition of the CPTP map E. Consider a Hilbert space H and a set of
linear operators L(H) acting on H. Since a linear map E : L(H) → L(H) is supposed to represent certain
time evolution, it should map a density operator ρ̂ ∈ L(H) to another density operator in the same Hilbert
space. This means that E must be (at least) positive E ≥ 0. Since the system of interest is in practicejust part of the whole world (i.e., ultimately there always exist external degrees of freedom other than thesystem), we must also ensure that E is positive even when it acts on part of a larger Hilbert space. Thus,3.4. KRAUS OPERATORS AND THE EQUIVALENCE TO INDIRECT MEASUREMENT MODEL27

it is in fact legitimate to require a stronger condition known as the completely positivity2. Altogether, we
define the CPTP map by the following conditions:• E(aρ̂ + bσ̂) = aE(ρ̂) + bE(σ̂)• E(ρ̂) is a density operator, i.e., it is self-adjoint, positive, and satisfies Tr[E(ρ̂)] = Tr[ρ̂] = 1.
• E is completely positive, i.e., a linear map E ⊗ I : L(H ⊗ HA ) → L(H ⊗ HA ) is positive forarbitrary (additional) Hilbert space HA . Here, I is the identity operator acting on L(HA ).
It is worthwhile to note that, if the completely positivity is not satisfied, an initially entangled state in
H ⊗ HA can lead to negative probabilities after the mapping E ⊗ I (see e.g., Peres, PRL 77, 1413 (1996)).
In this sense, the completely positivity is necessary to ensure that E indeed gives physically reasonable
time evolution even when the initial state is correlated with external world.
We recall that the simple example discussed before indicates that E has an operator-sum representation (3.1.19) with the Kraus operators M̂m . The map E then physically represents the nonselective (orunconditional) nonunitary evolution that is ensemble averaged over all the possible measurement outcomes m. These expressions follow from the originally unitary evolution in the larger Hilbert space. Thisobservation naturally motivates the question: given a general CPTP map E acting on a Hilbert space H,
is it possible to embed H into a larger Hilbert space and to construct a unitary evolution on this extended
system that reduces to E upon tracing out the degrees of freedom outside H? One can answer this questionin the affirmative way, as we briefly explain in the next section.
3.4. Kraus operators and the equivalence to indirect measurement model
There is an equivalence between a CPTP map and an indirect measurement model in an extended
Hilbert space (exemplified by the simple model above). This means that any nonunitary dynamics ofopen quantum systems can be considered as a part of a larger closed system equipped with orthogonal
projection measurements.
To state this in a more formal manner, let {Em }m∈M be a set of linear mappings acting on a vector
space spanned by a linear operator L(H). Suppose that they satisfy the following conditions:P• E ≡ m Em is trace preserving, i.e., Tr[E(Ô)] = Tr[Ô] for ∀Ô ∈ L(H).
• Em is completely positive for ∀m ∈ M .
Then, one can show that [Hellwig & Kraus, Comm. Math. Phys. 16, 142 (1970); Kraus, Ann. Phys. 64,
311 (1971); Stinespring, Proc. Amer. Math. Soc. 6, 211 (1955)], if and only if {Enm }m∈M satisfies
n theoabove conditions, there is the following indirect measurement model with some set of HA , ρ̂A , Û , P̂m(often called Stinespring representation):(3.4.1)

Em (ρ̂) = TrHA

m∈M

h


i
Iˆ ⊗ P̂m Û (ρ̂ ⊗ ρ̂A ) Û † Iˆ ⊗ P̂m ,where HA is anHilberto space and ρ̂A is a density operator for HA , Û is a unitary operator acting on
H ⊗ HA , and P̂mis a set of projection operators on the subspace of HA corresponding to eachm∈Mmeasurement outcome m.
2We remark that positive maps are not necessarily completely positive; we leave constructing such a map to Exercise of this

Chapter.

o3.5. BAYESIAN INFERENCE AND QUANTUM MEASUREMENT

28

P
For a given CPTP map E = m Em , this statement guarantees the existence of a meter A and a unitary
operator, which will reproduce E upon tracing out the meter degrees of freedom. However, the constructed
meter and the unitary operator are still rather mathematical objects; to deal with a real physical system, ingeneral we need to perform careful analysis of the interaction between the measured system and the meterin light of each physical situation. We will demonstrate such an analysis for a simple example later.
As shown above, the most general measurement process is described by a set of the completely posiP
tive maps Em whose nonselective mapping E = m Em is trace reserving. Then, it is useful to note thatthe measurement process always permits the following representation (often called Kraus representation):Em (ρ̂) =(3.4.2)

X†
M̂m,k ρ̂M̂m,k.

k

To see this, it suffices to employ the spectral decomposition(3.4.3)

ρ̂A =X

ci |ψi ⟩AA ⟨ψi |, P̂m =X

i|ϕm,j ⟩AA ⟨ϕm,j |,jwhere {|ψi ⟩A } and {|ϕm,j ⟩A } are orthonormal basis in HA . Then, introducing the Kraus operators by(3.4.4)M̂m,k ≡ M̂m,(i,j) =√ci A ⟨ϕm,j |Û |ψi ⟩A ,one can show that the (unnormalized) postmeasurement state in Eq. (3.4.1) can be written as in Eq. (3.4.2).
The Kraus representation allows us to express the POVMs by the operator-sum form as(3.4.5)

Êm =

X

†
M̂m,k
M̂m,k ,XˆÊm = I.

mk

The normalized postmeasurement state is given by(3.4.6)ρ̂m =Em (ρ̂)pm

with the probability of observing measurement outcome m being(3.4.7)

pm = Tr[ρ̂Êm ].

Conversely, one can show that any Kraus representation can be transformed to some Stinespring representation.
3.5. Bayesian inference and quantum measurement
Bayesian inference is usually discussed in the context of classical estimation theory. In this section,
we see that a certain class of quantum measurement processes is in fact completely equivalent to Bayesianinference. A prominent example includes the quantum nondemolition (QND) measurement that has beenrealized in a beautiful cavity QED experiment (2012 Nobel Prize), as we briefly discuss below.

3.5. BAYESIAN INFERENCE AND QUANTUM MEASUREMENT

29

3.5.1. Diagonal POVM and QND measurement. To motivate the notion of QND measurement,
consider a measurement process whose POVM and Kraus operators can be diagonally decomposed into

(3.5.1)

Êm =X

cm,j |ψj ⟩⟨ψj |, M̂m,k =jXc′m,k,j |ψj ⟩⟨ψj |j

by using the common orthogonal basis {|ψj ⟩} with some coefficients cm,j =the probabilities associated with the projection operators by′2
k |cm,k,j | . Let us denotePP (j) = Tr[P̂j ρ̂], P̂j = |ψj ⟩⟨ψj |(3.5.2)and write the conditional probability distribution of measurement outcomes for |ψj ⟩ as

P (m|j) = Tr[Êm ρ̂j ], ρ̂j =(3.5.3)P̂j ρ̂P̂j.
P (j)Similarly, let P (j|m) be the conditional probability distribution of finding a quantum state in |ψj ⟩ after
obtaining measurement outcome m:
P (j|m) = Tr[P̂j ρ̂m ], ρ̂m =(3.5.4)

Em (ρ̂).
P (m)Then, it satisfies Bayes’ theorem(3.5.5)

P (j|m) =X
P (m|j)P (j), P (m) =P (m|i)P (i).
P (m)iThis follows from the relation(3.5.6)

P (j|m) =Tr[P̂j Em (ρ̂)]
Tr[Em (P (j)ρ̂j )]Tr[Êm ρ̂j ]
P (m|j)P (j)==P (j) =
.
P (m)
P (m)
P (m)
P (m)

If we perform measurement process n times whose measurement outcomes are m = {m1 , m2 , . . . , mn },we obtainQn(3.5.7)Pn (j|m) =l=1 P (ml |j)P0 (j)P (m), P (m) =n
XYi

P (ml |i)P0 (i).

l=1This is nothing but the usual chain rule of Bayes’ relation. Hence, we can reconstruct the probability
distribution Pn (j|m) of the postmeasurement quantum state in the {|ψj ⟩} basis along with the same lineof classical estimation theory.
A quantum nondemolition (QND) measurement is an important class of quantum measurements thatsatisfies the above diagonal condition (3.5.1) and thus essentially reduces to the problem of Bayesian inference. To introduce the notion of QND measurement, consider a measurement process whose nonselective
evolution is3.5. BAYESIAN INFERENCE AND QUANTUM MEASUREMENT

30ih
E(ρ̂) = TrHA Û (ρ̂ ⊗ ρ̂A )Û † ,(3.5.8)where the unitary operator Û acts on the composite system consisting of the system and the meter, andwe assume that the meter initial state is pure ρ̂A = |ψA ⟩⟨ψA |. We also consider a Hermitian operator X̂
acting on the system. Then, the measurement is called QND measurement of X̂ if the following conditionis satisfied:[X̂, Û ]|ψA ⟩ = 0.

(3.5.9)A sufficient condition for this isX̂ = Û † X̂ Û ,(3.5.10)

meaning that X̂ is the conserved quantity. The name, QND, comes from the fact that such measurement
process does not alter the statistics of a certain observable (X̂ in our notation) after the measurement, i.e.,
the following relation holds true for all eigenvalues x of X̂ (note that P̂x is the corresponding projection
operator):

Tr[P̂x ρ̂] = Tr[P̂x E(ρ̂)].

(3.5.11)

Said differently, the probability distribution of the observable X̂ remains the same even after the measurement process.
Using the diagonal representation, one can obtain a simplified expression of a QND measurement
operator. To do so, let us use the spectral decomposition of X̂,X̂ =(3.5.12)X

x|x⟩⟨x|,x

as well as the Kraus representationE(ρ̂) =

(3.5.13)X†
M̂m,k ρ̂M̂m,k,m,kand the QND condition⟨x|E(ρ̂)|x⟩ = ⟨x|ρ̂|x⟩, ∀x, ρ̂.

(3.5.14)It then follows that

(3.5.15)

Xm,k†
M̂m,k |x⟩⟨x|M̂m,k= |x⟩⟨x|, ∀x.

3.5. BAYESIAN INFERENCE AND QUANTUM MEASUREMENT

31

The expectation value of this operator for |x′ ⟩ with x ̸= x′ gives(3.5.16)

X|⟨x|M̂m,k |x′ ⟩|2 = 0,m,kleading to(3.5.17)⟨x|M̂m,k |x′ ⟩ = 0, ∀x ̸= x′ .

Thus, the Kraus operator and the POVMs are diagonal in the x basis,(3.5.18)

X

M̂m,k =

c′m,k,x |x⟩⟨x|,x(3.5.19)

Êm =X

cm,x |x⟩⟨x|,x

which are nothing but the diagonal condition in Eq. (3.5.1). A QND measurement thus becomes equivalent to the Bayesian inference in the above sense. Building on this fact, below we shall summarize several
important properties of QND measurements.
3.5.1.1. Convergence to measurement basis: wavefunction collapse. We consider a measurement
process satisfying the diagonal condition (3.5.1) in the previous section and call {|ψi ⟩} as the measurement basis. Suppose that we have performed the measurement n times and obtained the corresponding nmeasurement outcomes {mk }nk=1 . Then, suppose that the measurement outcome mn+1 is obtained as aconsequence of the n + 1-th measurement process.
As noted earlier, the probability distribution of the n + 1-th postmeasurement state is calculated by
the Bayes formula(3.5.20)nPn+1 (j| {mk }n+1k=1 ) = Pn (j| {mk }k=1 )P (mn+1 |j).
Pn (mn+1 )To discuss the probability distribution of the quantum state after an infinite number of measurements, let
us denote P∞ (j| {mk }∞k=1 ) as the probability of finding the quantum state to be j after an infinite numberof measurements:(3.5.21)∞
P∞ (j| {mk }∞k=1 ) = P∞ (j| {mk }k=1 )P (m|j).
P∞ (m)

Only the state j for which P∞ (j| {mk }∞k=1 ) ̸= 0 leads to a nontrivial condition and let us restrict to suchj:(3.5.22)

P (m|j) = P∞ (m) =Xi

P (m|i)P∞ (i| {mk }∞k=1 ).

3.5. BAYESIAN INFERENCE AND QUANTUM MEASUREMENT

32

This relation can be satisfied by(3.5.23)

P∞ (i| {mk }∞k=1 ) = δij ,which means that, in the infinite number limit of measurement processes, the measurement induces theconvergence (or wavefunction collapse) of a quantum state into a single particular measurement basis|ψj ⟩3.
3.5.1.2. Born rule of collapsed states. We next show that the distribution of those collapsed states
reproduces the initial distribution P0 (j). To this end, we regard Pn+1 (j| {mk }n+1k=1 ) as the random variable
and consider the average of its value over the measurement outcome mn+1 :(3.5.24)

E[Pn+1 (j| {mk }n+1k=1 )] =X

Pn (mn+1 )Pn+1 (j| {mk }n+1
k=1 )

mn+1

(3.5.25)=X

P (mn+1 |j)Pn (j| {mk }nk=1 )mn+1(3.5.26)= Pn (j| {mk }nk=1 ).

Thus, the expectation value of the next random variable Pn+1 is equal to the previous one Pn for ∀j.
It follows then from the convergence theorem of classical probability theory that the infinite limit of arandom variable Pn (j| {mk }nk=1 ) converges almost surely in the sense of stochastic convergence, and its
expectation value is equal to the initial distribution,(3.5.27)

E[P∞ (j| {mk }∞k=1 )] = P0 (j), ∀j.

Physically, this represents the Born rule, i.e., the wavefunction collapse occurs according to the probability
distribution determined by the initial wavefunction.
3.5.2. Example: photon QND measurement in cavity QED. To illustrate these general properties,
we here consider the minimal example consisting of a single atom system interacting with a cavity photon.
We again start from the Jaynes-Cummings-type interaction,(3.5.28)

Ĥint = ℏγ(|e⟩⟨f |b̂† + |f ⟩⟨e|b̂),

where |f ⟩ is another atomic excited state above |e⟩. At this time we consider a cavity photon as a system
of interest while a two-level system is a meter degree of freedom.
To ensure the QND condition and simplify the Hamiltonian, we assume the off-resonant condition|δ| ≫ γ for the detuning δ = ωc − ωf e . After adiabatically eliminating the |f ⟩ state, at the leading order
one gets the interaction Hamiltonian(3.5.29)
3Strictly speaking,Ĥint ≃

ℏγ 2 †b̂ b̂|e⟩⟨e|δthere may exist other solution of P∞ (i| {mk }∞k=1 ) satisfying Eq.
{pj ; (pj )m = P (m|j)} are not linearly independent.

(3.5.22) when vectors

3.5. BAYESIAN INFERENCE AND QUANTUM MEASUREMENT

33

and the total effective Hamiltonian(3.5.30)Ĥeff = ℏωc n̂ + ℏΩn̂|e⟩⟨e|, Ω =γ2,δwhere n̂ = b̂† b̂ is the photon-number operator. This observable commutes with the Hamiltonian,[Ĥeff , n̂] = 0,

(3.5.31)

and thus ensures the QND condition (3.5.9).
We assume that the initial meter state is prepared in the excited state:ρ̂A = |e⟩⟨e|.

(3.5.32)

To be specific, we consider the following unitary operator as realized in the actual experiment [Gleyzes etal., Nature 446, 297 (2007)]:Û = ÛR2 Ûc ÛR1 ,(3.5.33)

where ÛR1,2 correspond to the interferometric operations known as Ramsey interferometry:
!
!
1 1
1
e−iϕ
1
1
, ÛR2 = √,(3.5.34)
ÛR1 = √
−1 1
−eiϕ
1
2
2
while Ûc is the light-matter interaction governed by Ĥeff :(3.5.35)Ûc = e−iĤeff T=e−iωc T n̂

e−iΩT n̂ 0
0
1

!
.

After the unitary operation, we perform the projection measurement on the two-level system. The resulting
measurement outcome is either m = e or m = g depending on the excited or ground state of two-levelsystems. When m = e, the Kraus operator is(3.5.36)e−iωc T n̂ e−iΩT n̂ − e−iϕM̂e = ⟨e|Û |e⟩ =2while for m = g we get−e−iωc T n̂ e−iϕ−iΩT n̂ + 1(3.5.37)M̂g = ⟨g|Û |e⟩ =.
2
The corresponding POVMs are given by(3.5.38)Êe = M̂e† M̂e =1 − cos (ΩT n̂ − ϕ),2

1 + cos (ΩT n̂ − ϕ),2which are not orthogonal in general, but satisfy the completeness relation(3.5.39)Êg = M̂g† M̂g =(3.5.40)

X
m=e,g

ˆ
Êm = I.

3.6. CONTINUOUS QUANTUM MEASUREMENT

34

Following the general theory formulated above, we can write the postmeasurement state and the probability distribution as(3.5.41)ρ̂m =†M̂m ρ̂M̂m, pm = Tr[ρ̂Êm ], m ∈ {e, g} .
pm

Building on our discussion of QND measurement in the previous section, we can immediately obtain
the evolution of the postmeasurement probability distribution of the cavity photon number using the Bayes
formula(3.5.42)

P (n|m) =P (m|n)P0 (n)pmwhere P0 (n) = ρnn is the initial distribution in the Fock basis and P (m|n) is the conditional probability
distribution defined byhi  1−cos(ΩT n−ϕ) m = e2(3.5.43)
P (m|n) ≡ Tr Êm P̂n = 1+cos(ΩT
.
n−ϕ)

m=g
2

Here, P̂n = |n⟩⟨n| is the projection operator on the Fock state with n.
Suppose that we perform the measurement process N times. As shown before, the resulting postmeasurement distribution in the Fock basis is obtained by the chain rule:(3.5.44)

PN (n| {mi }) = N

NY

P (mj |n)P0 (n).

j=1Note that the eigenstates of the observable n̂ constitute the complete set of the quantum state of interest.
From our general arguments before, we can thus interpret the present measurement process as the “wavefunction collapse” of the cavity photon caused by the observation. More specifically, we conclude that aquantum state eventually converges to a particular Fock state, and the distribution of collapsed states obey
the Born rule, i.e., it matches with the initial distribution P0 (n) in the Fock basis.

3.6. Continuous quantum measurement
3.6.1. Quantum trajectories. We here focus on a class of quantum measurements known as continuous quantum measurement, which is in fact relevant to many physical systems and lies at the heartof theory of Markovian open quantum systems. Consider the indirect measurement model in which thesystem in the Hilbert space HS repeatedly interacts with the meter in HM during a short time interval τ(see Fig. 3.6.1). After each interaction, one performs a projective measurement on the meter, obtains ameasurement outcome, and resets it to the common state |ψ0 ⟩M ; the latter process ensures that the meterretains no memory about previous measurement outcomes. An observer thus obtains information about
the measured system through a sequence of measurement outcomes {m1 , m2 , . . .}. We assume that eachmeasurement outcome takes a discrete value from mi ∈ {0, 1, 2, . . . , M } (i = 1, 2, . . .).
We start from the system-meter product state3.6. CONTINUOUS QUANTUM MEASUREMENT35

F IGURE 3.6.1. Repeated indirect measurement model.

(3.6.1)ρ̂tot (0) = ρ̂S (0) ⊗ P̂0 , P̂0 = |ψ0 ⟩M M ⟨ψ0 |,and assume that the dynamics of the total system is governed by a general Hamiltonian(3.6.2)Ĥ = ĤS + V̂ , V̂ = γMXÂm ⊗ B̂m + H.c., γ ∈ R,

m=1

where Âm and B̂m act on HS and HM , respectively. The time scale of ĤS is assumed to be much longer
than τ . We further assume that B̂m changes the meter state into the subspace of HM corresponding to ameasurement outcome m, i.e. we impose the relations(3.6.3)P̂m′ B̂m = δm′ m B̂m (m′ = 0, 1, . . . , M ; m = 1, 2, . . . , M ),where P̂m′ is a projector onto the subspace of HM corresponding to a measurement outcome m′ . Notethat the reset state is labelled by m′ = 0. A set of P̂m′ satisfies the completeness condition(3.6.4)MX

ˆP̂m′ = I.

m′ =0For each indirect measurement process, there are two possibilities, i.e., either (i) one observes a change
in the meter state from the reset state by obtaining an outcome m = 1, 2, . . . , M or (ii) one observes nochange in the state of the meter from the reset state and thus obtains the outcome 0.
In the first case (i), which is often called as the quantum jump process, the nonunitary mapping Em ofthe system ρ̂S corresponding to obtaining an outcome m = 1, 2, . . . , M is given byh
i(3.6.5)Em (ρ̂S ) = TrM P̂m Û (τ ) ρ̂S ⊗ P̂0 Û † (τ )P̂m ≃ τ L̂m ρ̂S L̂†m ,where we define the total unitary operator Û (t) = e−iĤt , and TrM denotes the trace over the meter. To
simplify the expression in the last equality, we here assume the continuous measurement limit:(3.6.6)γτ ≪ 1 while keeping γ 2 τ finite.

3.6. CONTINUOUS QUANTUM MEASUREMENT36

The operator L̂m is often called the jump or Lindblad operator and defined byq†(3.6.7)L̂m = γ 2 τ ⟨B̂m
B̂m ⟩0 Âmwhere ⟨· · · ⟩0 is an expectation value with respect to |ψ0 ⟩M .
In contrast, in the second case (ii) for which one observes no change of the meter state corresponding
to the outcome 0, the nonunitary mapping E0 can be simplified ash
i(3.6.8)E0 (ρ̂S ) = TrM P̂0 Û (τ ) ρ̂S ⊗ P̂0 Û † (τ )P̂0
( M
)
τ X †
(3.6.9)L̂m L̂m , ρ̂S≃ ρ̂S − iτ [ĤS , ρ̂S ] −2
m=1

P
Clearly, these mappings satisfy the trace-preserving property TrS [ M
m′ =0 Em′ (ρ̂S )] = 1.
We next want to formulate this continuous measurement process in the language of stochastic differential equation. To do so, we consider the evolution during a time interval dt = N τ , which is still much
shorter than the time scale of the internal dynamics ĤS but still contains a large number of repetitive
interactions with the meter, i.e., N ≫ 1. The probabilities of observing a change of the meter state scaleas(3.6.10)pm = TrS [Em (ρ̂S )] = τ TrS [L̂m ρ̂S L̂†m ] ∼ O((γτ )2 ) ≪ 1,

which is negligibly small. Thus, one can assume that such a jump event occurs at most once during eachinterval dt. Under these conditions, the nonunitary dynamical mappings of the system during the interval
dt can be described as(3.6.11)Φm=1,...,M (ρ̂S ) ≡

N 
X


†
+ O(dt2 )E0N −i ◦ Em ◦ E0i−1 (ρ̂S ) = M̂m ρ̂S M̂m

i=1(3.6.12)Φ0 (ρ̂S ) ≡ E0N (ρ̂S ) = M̂0 ρ̂S M̂0† + O(dt2 )Here, we introduce the Kraus operators as(3.6.13)√M̂m = L̂m dt (m = 1, 2, . . . , M ),(3.6.14)M̂0 = 1 − iĤeff dt,

where Ĥeff is an effective non-Hermitian HamiltonianM(3.6.15)i X †Ĥeff = ĤS −L̂m L̂m .
2
m=1

An operator M̂m acts on a quantum state if the jump event with outcome m is observed, which occurs†
with probability TrS [M̂m ρ̂S M̂m]. The operator M0 acts on the state if no jumps are observed during atime interval [t, t + dt], and this no-count process occurs with the probability TrS [M̂0 ρ̂S M̂0† ]. Note that

3.6. CONTINUOUS QUANTUM MEASUREMENT

37

these operators fulfill the completeness condition (aside negligible contributions on the order of O(dt2 )):M
X(3.6.16)†ˆM̂m′ M̂m′ = IS .

m′ =0For the sake of simplicity, suppose that the initial state is pure and so is the state |ψ⟩S in the course oftime evolution. Due to the probabilistic nature of quantum measurement, an appropriate way to formulate
the dynamics under measurement is to use a stochastic differential time-evolution equation. We thenintroduce a discrete random variable dNm with m = 1, . . . , M , whose mean value is given by(3.6.17)†
E [dNm ] = ⟨M̂m
M̂m ⟩S = ⟨L̂†m L̂m ⟩S dt,

where E[·] represents the ensemble average over the stochastic process and ⟨· · · ⟩S denotes an expectation
value with respect to a quantum state |ψ⟩S of the system. The random variables dNm satisfy the following
stochastic calculus:dNm dNn = δmn dNm ,(3.6.18)

meaning that dN takes either a value of 0 or 1 and thus at most a single jump event can be observed ateach time interval dt4. The stochastic change of a quantum state |ψ⟩S during the time interval [t, t + dt]
can now be obtained as
!
M
M
X
X
M̂m |ψ⟩S
M̂0 |ψ⟩S+
dNm q.
(3.6.19)|ψ⟩S → |ψ⟩S + d|ψ⟩S = 1−
dNm q
†
m=1
⟨M̂0† M̂0 ⟩S m=1
M̂m ⟩S⟨M̂m
Physically, the first term on the rightmost side describes the no-count process occurring with the probabilP
ity 1− M
m=1 E[dNm ] while the second term describes the detection of a jump process m occurring with aprobability E[dNm ]. We note that the denominator in each term is introduced to ensure the normalizationof the state vector. We can further rewrite it as(3.6.20)d|ψ⟩S = −iĤeff +

M
1X

2

!
⟨L̂†m L̂m ⟩S

m=1

dt|ψ⟩S +

M
X





 qL̂m |ψ⟩S − |ψ⟩S  dNm .
m=1⟨L̂†m L̂m ⟩S

The first term on the right-hand side describes the non-Hermitian time evolution, in which the factor
PM†
m=1 ⟨L̂m L̂m ⟩S /2 plays the role of maintaining the normalization of the state vector. The second term
describes the state change for the jump event m, where L̂m acts on the quantum state and causes its
discontinuous change. An individual realization of this stochastic differential equation corresponding to atime sequence of specific outcomes is often referred to as the quantum trajectory.
3.6.2. Quantum master equation. Taking the ensemble average over all the possible trajectories,
one can obtain the differential equation for the nonselective (or unconditional) nonunitary evolution of
4Technically, we remark that the stochastic process dN

m is not a simple Poisson process because its intensity depends on astochastic vector |ψ⟩S ; it is known as a marked point process in the field of stochastic analysis.

3.6. CONTINUOUS QUANTUM MEASUREMENT

38

density operator. This is known as the master equation in the Gorini-Kossakowski-Sudarshan-Lindblad(GKSL) form. To see this explicitly, let us rewrite the time-evolution equation in the density-operator
form ρ̂S = |ψ⟩S S ⟨ψ|:

(3.6.21)dρ̂S = −i†Ĥeff ρ̂S − ρ̂S Ĥeff



M
X

M
X
dt +
⟨L̂†m L̂m ⟩S ρ̂S dt +m=1
m=1

L̂m ρ̂S L̂†m
⟨L̂†m L̂m ⟩S

!
− ρ̂S

dNm .

where we take the leading order of O(dt) and use the stochastic calculus. We note that this equation
remains valid for a generic density matrix ρ̂S that is not necessarily pure. Introducing the ensembleaveraged density matrix E[ρ̂S ] and taking the average over measurement outcomes, one arrives at the
GKSL master equation:M(3.6.22) X
d E[ρ̂S ]†= −i Ĥeff E[ρ̂S ] − E[ρ̂S ]Ĥeff+
L̂m E[ρ̂S ]L̂†m .
dt
m=1

The master equation describes the dynamics when no information about measurement outcomes isavailable to an observer. It can also be applied to analyze dissipative dynamics of a quantum system
coupled to a Markovian environment. Roughly speaking, this is because one cannot keep track of all theenvironmental degrees of freedom and, to describe dissipative dynamics, the best one can do would be
to take the trace over them (corresponding to the summation over measurement outcomes above). One
can then simplify the dissipative evolution equation of the reduced density matrix ρ̂S by assuming certain
conditions; we will later see sufficient conditions to justify the use of the Markovian description. We
note that such conditions are essentially equivalent to the assumptions made in deriving the continuous
measurement limit above.

3.6.3. Measurement dynamics vs Dissipation. We have seen that there are two types of dynamics
in open quantum systems, namely, conditional or unconditional evolution (also often called selective or
nonselective evolution). The conditional dynamics describes the evolution of a quantum system undermeasurement in the cases when an observer can access information about measurement outcome in someway. In contrast, if measurement outcome is completely unknown, one must take the ensemble averageover measurement outcomes, and the dynamics becomes the dissipative one described by the unconditional, Markovian master equation. It is relatively recent development that we have learned that these two
types of dynamics can indeed be distinct at both qualitative and quantitative levels, especially in manybody setups, and lead to rich phenomenology in conditional evolution beyond conventional dissipative
dynamics.
Here we shall briefly exemplify several classes of such conditional dynamics of continuously measured quantum systems. First of all, we emphasize that an appropriate theoretical description of open
quantum dynamics depends on how much information about measurement outcomes is available to an
observer. For instance, if one can access the complete information about measurement outcomes, i.e. all
the times and types of quantum jumps, the dynamics is described by a single realization of the quantum
trajectory:

3.6. CONTINUOUS QUANTUM MEASUREMENT|ψtraj ⟩S =(3.6.23)

n h
Y

39

i
Ûeff (∆tk )L̂mk Ûeff (t1 )|ψ0 ⟩S /∥ · ∥,k=1

where 0 < t1 < · · ·< tn < t are the times of quantum jumps whose types are {m1 , m2 , . . . , mn }, ∆tk =
tk+1 − tk is the time difference with tn+1 ≡ t, and Ûeff (t) = e−iĤeff t is the non-Hermitian time evolution
operator. In the context of dissipative systems, this situation may be considered as an ideal case in which
one can keep track of all the environmental degrees of freedom.
Meanwhile, even when one cannot access such complete information, we may still have the ability toaccess partial information about measurement outcomes. One natural situation, which is relevant to certain
experiments, is that one can know about the total number of quantum jumps occurring during a certain
time interval, but not their types and occurrence times. In such a case, the quantum state is described by
the following density matrix conditioned on the number n of quantum jumps that have occurred during atime interval [0, t]:ρ̂(n) (t) ∝X|ψα ⟩S S ⟨ψα |α∈Dn(3.6.24)
=

X Z t

Z t2
dtn · · ·0{mk }n
k=1

dt1
0

n h
n h
i
i
Y
Y
†
†
(∆tk ) ,L̂†mk Ûeff(t1 )Ûeff (∆tk )L̂mk Ûeff (t1 )ρ̂S (0)Ûeff
k=1

k=1

where the ensemble average is taken over the subspace Dn spanned by all the trajectories having n quantum jumps during [0, t]. This is interpreted as coarse-grained dynamics of pure quantum trajectories,
where the number of jumps is known while the information about the times and types of jumps are averaged out. The no-jump process ρ̂(0) (t) corresponds to non-Hermitian evolution and is the simplest
example of this class of dynamics.
3.6.4. Brief remark on non-Hermitian physics. As we have learned above, dynamics of a quantum
system under continuous measurement contains two parts, non-Hermitian evolution and random quantum
jumps. Obviously, both of them can be implemented in a numerical calculation, but it is often the casethat only the former allows one to resort to analytical/exact theoretical analyses. Such non-Hermitian
description can be useful as an effective theoretical tool to understand qualitative physics of open systems.
For instance, this is the case when there only exist spontaneous decays or loss processes (but no gain
processes) for which phenomena of our interest occur in transient regimes rather than steady states5.
Otherwise, however, non-Hermitian evolution alone is in general inaccurate and quantum jumps should
be included to consistently analyze continuously measured dynamics. Interested students may see thereview paper [Adv. Phys. 69, 3 (2020)] and the references therein.
3.6.5. Physical examples.
5Technically, this is because, in these lossy systems, the analysis of the Liouvillian of the master equation can significantly be

simplified to that of the non-Hermitian operator Ĥeff , since the Liouvillian possesses the tridiagonal matrix structure and thus
shares the same spectral feature with that of Ĥeff .

3.6. CONTINUOUS QUANTUM MEASUREMENT

40

Example 1: Photon emission. Let us consider a simple example of photon emission on the basis of
the Jaynes-Cummings model (3.1.1) discussed before6. The system-meter interaction isV̂ ∝ σ̂ − b̂† + σ̂ + b̂

(3.6.25)

and we assume that the reset state is the photon vacuum |ψ0 ⟩M = |0⟩. The jump operator corresponding
to spontaneous emission is√(3.6.26)
L̂ = γ σ̂ −and the non-Hermitian effective Hamiltonian isiγ
iγ(3.6.27)Ĥeff = ĤS − σ̂ + σ̂ − = ĤS − |e⟩⟨e|,22which describes the dynamics during the no-count process. We emphasize that, even in the no-count case,an observer actually gains information about the system that no spontaneous emission has been detected.
This information acquisition indicates that the system is more likely to be in the ground state than in
the excited state. This tendency can be ascribed to the measurement backaction represented by the anti√
Hermitian term in Ĥeff . Meanwhile, when the jump process (described by L̂− = Γσ̂ − ) is detected, anobserver acquires the information that the system is projected onto the ground state.
Example 2: Damped harmonic oscillator. We next consider a harmonic oscillator as a system ofinterest, while the meter is also a harmonic oscillator with the reset state chosen to be the ground state|ψ0 ⟩M = |0⟩. The system-meter interaction isV̂ ∝ âb̂† + â† b̂.

(3.6.28)
We can calculate the jump operator as(3.6.29)

L̂ =

√

γâ,

which describes the damping of the harmonic oscillator. The corresponding effective non-Hermitian
Hamiltonian isiγ(3.6.30)Ĥeff = ĤS − â† â.
2
The unconditional master-equation dynamics is given by(3.6.31)

6


d ρ̂
γ
= −iω[â† â, ρ̂] +2âρ̂â† − â† âρ̂ − ρ̂â† â ,dt
2Strictly speaking, photon emission usually arises from interaction between atom and continuum photonic modes. We here focus
on a single photon mode for the sake of simplicity, and later revisit the related problem in the case of the continuum photon
modes.

3.6. CONTINUOUS QUANTUM MEASUREMENT

41

where we set ĤS = ωâ† â. It is a good exercise to check that the expectation value of the oscillator
amplitude then exhibits the damping:


da
dρ̂
(3.6.32)≡ Tr â→ a(t) = a(0)e−iωt−γt/2 .
dt
dt
Obviously, the steady state is the ground state ρ̂ = |0⟩⟨0| since there only exists loss process.
Example 3: Lossy many-body systems. Consider a quantum many-body system subject to particle
loss processes. The one- or two-body loss processes can effectively be described by the jump operators
L̂one (x) ∝ Ψ̂(x), L̂two (x) ∝ Ψ̂(x)Ψ̂(x).

(3.6.33)

The corresponding non-Hermitian Hamiltonian is

 2 2


Z
ℏ ∇
g − iγ †††(3.6.34) Ĥeff = dx Ψ̂ (x) −Ψ̂ (x)Ψ̂ (x)Ψ̂(x)Ψ̂(x) ,+ Vr (x) − iVi (x) Ψ̂(x) +2m2
where Ψ̂ (Ψ̂† ) is the annihilation (creation) operator of either a fermion or a boson; for the sake ofsimplicity, we omit spin degrees of freedom. A one-body loss process can be taken into account by
an imaginary potential, −iVi (x), while the two-body loss effectively changes an interaction parameterto a complex value g − iγ. Suppose that the initial state is an eigenstate of the total particle numberRN̂ = dxΨ̂† (x)Ψ̂(x). Then, its expectation value evolves asdNdTr(N̂ ρ̂)1†⟩.
== ⟨Ĥeff − ĤeffdtdtiℏNamely, imaginary parts of the eigenvalues of non-Hermitian Hamiltonian directly characterize the loss
rate of particles.
It is often useful to describe the system by using tight-binding approximation (especially in ultracold
atoms), resulting in(3.6.35)(3.6.36)

Heff = −J

N 
X



b†m bm+1 + H.c.

m=1

N

Ur − iUi X+
nm (nm − 1) ,2
m=1

where b̂m (b̂†m ) is the annihilation (creation) operator of a particle at site m and only a two-body loss
is included; an imaginary interaction parameter −iUi results from two-body loss process. While thismodel is not exactly solvable, its asymptotically exact spectrum can be obtained by the strong-couplingexpansion analysis in the limit of Ur,i ≫ J. For instance, the decay rate ΓMott of the Mott-insulator stateat unit filling ρ = 1 (i.e., the imaginary part of the corresponding complex eigenvalue) is given by(3.6.37)

ΓMott
3J 2 Ui
.
=N
4 Ur2 + Ui2

Interestingly, the loss rate is suppressed at strong Ui as(3.6.38)

ΓMott ∝

J2,
Ui

3.6. CONTINUOUS QUANTUM MEASUREMENT

42

which is known as the continuous quantum Zeno effect.

3.6.6. Diffusive limit. We have so far discussed a measurement process in which an operator L̂minduces a discontinuous change of a quantum state. Below we shall discuss yet another type of continuous
measurement associated with a diffusive stochastic process.
3.6.6.1. Wiener process. To begin with, we introduce the notion of Wiener stochastic process. Roughly
speaking, the Wiener process Wt is a continuous random walk that (i) satisfies the initial condition W0 = 0√
at t = 0 and (2) is a normally distributed random variable with zero mean and width t at t > 0, whose
probability distribution is
P (Wt , t) = √(3.6.39)

1 −Wt2 /2te.
2πt

We also often denote it by Wt ∈ N (0, t). To define its differential, let us first consider the Wiener
increment for time interval ∆t:∆W = Wt+∆t − Wt ,(3.6.40)

which has a normal distribution with the following mean and variance:
E[∆W ] = 0, E[(∆W )2 ] = ∆t.

(3.6.41)

Then, we consider its differential limit ∆t → 0, which effectively gives the time derivative of the Wiener
process that we express by∆t → dt, ∆W → dW.

(3.6.42)

We require that this Wiener differential satisfies the stochastic calculus
E[dW ] = 0, dW 2 = dt.

(3.6.43)

The latter is often called Itô rule named after Kiyoshi Itô; this condition is indeed reasonable since therandom variable (∆W )2 has the mean ∆t while its fluctuation becomes negligibly small compared toO(∆t) quantity in the limit of ∆t → 0.
3.6.6.2. Stochastic Schrödinger equation. Previously, we arrive at the following stochastic differential equation for a quantum system under measurement (see Eq. (3.6.21)):(3.6.44)dρ̂ = −i†Ĥeff ρ̂ − ρ̂Ĥeff



dt +

M
X

⟨L̂†m L̂m ⟩ρ̂dt +

m=1

M
X

L̂m ρ̂L̂†m

m=1⟨L̂†m L̂m ⟩!
− ρ̂ dNm ,3.6. CONTINUOUS QUANTUM MEASUREMENT

43

with
M

i X †
Ĥeff = Ĥ −L̂m L̂m .
2(3.6.45)m=1

We are here interested in the limit that detection rate of quantum jumps is high while an operator L̂m only
induces an infinitesimally small change of a quantum state. To this end, we assume that measurement
backaction induced by L̂m is weak in the sense that it satisfies√(3.6.46)L̂m = Γ(1 + ϵâm )for a Hermitian operator âm , where Γ characterizes the detection rate of jump events and ϵ ≪ 1 is asmall dimensionless parameter. We also assume that detections of jump events are so frequent that the
expectation value ∆Nm of the number of jump-m events being observed during a time interval ∆t is
sufficiently large. From the central-limit theorem, ∆Nm can then be approximated as(3.6.47)∆Nm ≃ Γ(1 + 2ϵ⟨âm ⟩)∆t +√Γ (1 + ϵ⟨âm ⟩) ∆Wm ,where ∆Wm ∈ N (0, ∆t) are the (mutually independent) Wiener increments each of which obeys thenormal distribution with the zero mean and the variance ∆t. We then take the diffusive measurement
limit, i.e., ϵ → 0 and Γ → ∞ while Γϵ2 is kept constant. It would be a good Exercise to check that theresulting stochastic differential equation in the limit ∆t → 0 is given by#M
1 X hˆ hˆ iidρ̂ = −i[Ĥ, ρ̂] −lm , lm , ρ̂
dt
2
"

m=1

(3.6.48)M n
Xoˆlm − ⟨ˆlm ⟩, ρ̂ dWm ,+m=1

where we introduce operators ˆlm =(3.6.49)√

Γϵ2 âm and the (differential) Wiener stochastic processes that satisfy

E[dWm ] = 0, dWm dWn = δmn dt.

When a quantum state is pure, the stochastic time-evolution equation can be rewritten as#M2
1 X ˆ
ˆlm − ⟨lm ⟩dt|ψ⟩d|ψ⟩ = −iĤ −
2
"

m=1

(3.6.50)

+

M 

X
ˆlm − ⟨ˆlm ⟩ dWm |ψ⟩.
m=1

The stochastic Schrödinger equation with the diffusive process is typically applied to analyze nonunitary
dynamics of certain quantum optical systems, such as a cavity under homodyne measurement as well as aquantum particle under continuous position measurement.

3.6. CONTINUOUS QUANTUM MEASUREMENT

44

3.6.6.3. Example: continuous position measurement. As a concrete example of a diffusive continuous measurement, we here consider a single quantum particle subject to continuous position measurement.
In this case, the effective “jump” operator corresponds to a position operator:
ˆl =(3.6.51)√

κx̂,resulting in the time-evolution equations:(3.6.52)ih√κd|ψ⟩ = 1 − iĤ − (x̂ − ⟨x̂⟩)2 dt|ψ⟩ + κ(x̂ − ⟨x̂⟩)dW |ψ⟩,
2



√
κ
dρ̂ = −i[Ĥ, ρ̂] − [x̂, [x̂, ρ̂]] dt + κ {x̂ − ⟨x̂⟩, ρ̂} dW.
2
Suppose that the unitary part is simply governed by the free-particle Hamiltonian:(3.6.53)p̂2,2m
and also that the initial state is a Gaussian state with zero mean ⟨x̂⟩ = 0. It is then a good Exercise toderive the following differential relations of the expectation values:Ĥ =(3.6.54)(3.6.56)√⟨p̂⟩
dt + 4κVx dWm√d⟨p̂⟩ = 2 κCxp dW(3.6.57)dVx =d⟨x̂⟩ =

(3.6.55)(3.6.58)

2
Cxp dt − 4κVx2 dtm2dVp = κ(1 − 4Cxp
)dt

(3.6.59)

dCxp =Vp
dt − 4κVx Cxp dt
mwhere we define the variances as VO ≡ ⟨Ô2 ⟩ − ⟨Ô⟩2 and covariance by Cxp = ⟨{x̂, p̂}⟩/2 − ⟨x̂⟩⟨p̂⟩, anduse Wick’s theorem (see also Chapter 4).
These relations explicitly show that the means and variances are decoupled from the higher-order moments. This means that, if the initial state is completely characterized by the moments up to second order,i.e., it is a Gaussian state, a quantum state remains Gaussian during time evolution under measurement. In
fact, one can show that, even if the initial state is non-Gaussian, at sufficiently long time, a quantum state
reduces to a certain Gaussian state due to the measurement backaction.
One can also calculate the heating rate, i.e., the increase rate of the kinetic energy K:(3.6.60)dK ≡ d

p̂2
2m


=



√
1
1
dVp + d(⟨p̂⟩2 ) =κdt + 4⟨p̂⟩ κCxp dW
2m2m

3.7. MASTER EQUATION FROM BORN-MARKOV APPROXIMATIONS

45

resulting inκt.
2mThis means that the position measurement continuously disturbs momentum and thus causes the inevitable
heating that leads to t linear increase of the kinetic energy. Physically, it can be interpreted as, for example, a consequence of recoil energy due to photon scattering when particle position is probed by lightscattering, like the one performed in optical microscopy.
Position measurement also induces the decoherence. For the sake of simplicity, consider the infinite
mass case m → ∞; then the nonselective evolution simplifies to(3.6.61)

E[K(t)] = K(0) +κdρ̂ = − [x̂, [x̂, ρ̂]]dt.
2
We expand the density operator in the position basis
Z(3.6.63)ρ̂ = dxdy |x⟩⟨y| ρxy , ρxy ≡ ⟨x|ρ̂|y⟩.
(3.6.62)

We then see that the position measurement causes the exponential decay of the off-diagonal elementsdρxyκ
= − (x − y)2 ρxy .
dt
2Note that the decoherence rate is proportional to the square of the distance between the two elements,
namely, macroscopic coherence is fragile against decoherence.

(3.6.64)3.7. Master equation from Born-Markov approximations
In the context of continuous quantum measurement, the dynamics is governed by the GKSL masterequation when an external observer cannot access to any information about meter degrees of freedom, i.e.,measurement outcomes. There, the best one can do is to take the (unbiased) ensemble average over all
the possible outcomes, which in turn results in the nonunitary evolution of a density operator; even if theinitial state is pure, the irreversible information leaking from the system into meter degrees of freedom(outside the system Hilbert space) results in a mixed state after nonunitary evolution.
In fact, besides quantum measurement, the master-equation description itself is applicable to a certainclass of dissipative systems, known as Markovian open systems. Roughly speaking, there one considers
a quantum system that is weakly coupled to external environments that possess negligibly small memory.
More precisely, the Markovian description is valid when the following approximations can be justified7:(1) Born approximation: System-environment interaction γ is sufficiently weak such that if the total
system has no system–environmental correlations at the initial time, an evolved state at a latertime must also remain so.
(2) Markov approximation: the environmental correlation time τenv is much shorter than the relaxation time scale τrel of the whole dynamics including both the system and the environment; τrel
7While these conditions are usually well-justified in quantum optics/AMO systems, it is important to keep in mind that this is in

general not the case in, for example, condensed matter physics.

3.7. MASTER EQUATION FROM BORN-MARKOV APPROXIMATIONS

46

can typically be estimated as τrel ∝ ∆/µ2 with ∆ being typical level spacing in the internal
dynamics of the system and µ being the strength of the system–environment coupling.
(3) “Secular” approximation: there is a time scale separation such that 1/∆ ≪ τrel , which ensuresthat the jump operators L̂m in the master equation are time independent.
Building on these approximations, below we shall re-derive the master equation, but at this time startingfrom a theory of dissipative systems rather than quantum measurement.
We assume that the initial state has no system-environmental correlations:ρ̂SE (0) = ρ̂S (0) ⊗ ρ̂E ,(3.7.1)and describe the total system-environment density operator as ρ̂SE that obeys the unitary evolution:dρ̂SE= −i[Ĥ, ρ̂SE ], Ĥ = ĤS + ĤE + ĤSE ,dt(3.7.2)where ĤS (ĤE ) acts only on system (environmental) degrees of freedom, and ĤSE is the system-environment
interaction. The reduced system density operator isρ̂(t) ≡ TrE [ρ̂SE (t)](3.7.3)and our goal is to show that its dynamics is governed by the GKSL master equation.
To begin with, it is useful to introduce the interaction picture:(3.7.4)

ρ̃ˆSE (t) = ei(ĤS +ĤE )t ρ̂SE (t)e−i(ĤS +ĤE )t(3.7.5)

ˆ (t) = ei(ĤS +ĤE )t Ĥ e−i(ĤS +ĤE )t .
H̃
SE
SE

Then, the time evolution of the whole system is given bydρ̃ˆSEˆ , ρ̃ˆ ].
= −i[H̃SE SE
dt
Integration of this equation from t to t + ∆t results in
Z t+∆tˆ (t′ ), ρ̃ˆ (t′ )].
(3.7.7)
ρ̃ˆSE (t + ∆t) = ρ̃ˆSE (t) − idt′ [H̃
SE
SE(3.7.6)t

We iterate this procedure for ρ̃ˆSE(3.7.8)ρ̃ˆSE (t+∆t) = ρ̃ˆSE (t)−iZ t+∆t(t′ ) one more time and obtain

Z t+∆tZ t′ˆ (t′ ), ρ̃ˆ (t)]−ˆ (t′ ), [H̃ˆ (t′′ ), ρ̃ˆ (t′′ )]]′dt′ [H̃dtdt′′ [H̃
SE
SE
SE
SE
SE

t

t

We now use the Born approximation, which leads to(3.7.9)ˆ ⊗ ρ̃ˆE .
ρ̃ˆSE (t) ≃ ρ̃(t)t

3.7. MASTER EQUATION FROM BORN-MARKOV APPROXIMATIONS

47

When considering the trace over the environment, this results in the following simplification
ˆ (t′ ), ρ̃ˆ (t)]] ≃ ρ̃(t)Tr
ˆ
′ ˆ
ˆ
ˆ
TrE [[H̃SESE
E [[H̃SE (t ), ρ̃E ]] = ρ̃(t)TrE [[ĤSE , ρ̂E ]] = 0.

(3.7.10)The last equality can always be attained by absorbing (if any) mean values into the definition of thesystem Hamiltonian. Physically, this means that there are fluctuations only around zero mean in thesystem-environment interaction.
To further simplify the analysis, we use the Markov approximation; this allows us to set the intermediate time scale τrel ≫ ∆t ≫ τenv and thus, at the lowest order of ∆t and together with the Bornˆ ⊗ ρ̃ˆE ), leading toapproximation (ρ̃ˆSE (t′′ ) ≃ ρ̃(t)ˆ ≃−∆ρ̃(t)(3.7.11)Z ∞Z t+∆tdτhi
ˆ (t′ ), [H̃ˆ (t′ − τ ), ρ̃(t)ˆ ⊗ ρ̃ˆE ]]dt′ TrE [H̃
SE
SE

t

0

≡ t′ − t′′ and we take the range of integration [0, t′ − t] ∼ [0, ∞] since ∆t ≫ τenv .

where τThis isknown as the Redfield equation; we note that it is known that this equation does not in general lead to thepositivity map, and the resulting density operators may be nonpositive.
To construct a physically consistent description satisfying the positivity, we employ further simplifications. To this end, we assume that the system-environment interaction satisfies the following conditions:

ĤSE =(3.7.12)X

L̂m ⊗ R̂m + H.c.,mˆ (t) = eiĤS t L̂ e−iĤS t = L̂ e−iωm t ,
L̃
m
m
m

(3.7.13)

h
i
h
i
h
i
ˆ (t )R̃ˆ † (t ) = Tr ρ̃ˆ R̃ˆ (t )R̃ˆ (t ) = Tr ρ̃ˆ R̃ˆ † (t )R̃ˆ † (t ) = 0.
TrE ρ̃ˆE R̃
m 1
2
m
1
n
2
1
2
E
E
E
E
n
m
n

(3.7.14)

These conditions are satisfied in typical situations considered in this lecture8 (see, e.g., Eq. (5.3.49)),
while a derivation in a more general setup is possible (see e.g., the textbook by H.-P. Breuer, “The Theory
of Open Quantum Systems”, Oxford University Press (2007)).
Building on these simplifications, we then use the final condition, the so-called secular condition,
which allows us to set the time scale τrel ≫ ∆t ≫ 1/(ωm − ωn ), leading toZ t+∆t(3.7.15)′dt′ ei(ωm −ωn )t ≃ ∆t δmn .

t

ˆ (t) acting on the density
This effectively eliminates the time dependence in the “jump” operators L̃moperator. Altogether, from Eq. (3.7.11), we arrive at

8Imagine, for example, a typical situation, for which there only exist energy conserving terms like L̂†m ∝ âm with R̂m ∝ bm ,†where â, b̂ are bosonic annihilation/creation operators and ρ̂E is the vacuum state. Technically, this treatment can be justified
when the rotating wave approximation is valid as we see later.

3.8. SHORT REMARK ON THE VALIDITY OF MASTER EQUATION APPROACHES(3.7.16)

48


i

Xh
ˆ
∆ρ̃(t)
ˆ L̂†m − L̂†m L̂m ρ̃(t)ˆ+ H.c. ,≃Cm L̂m ρ̃(t)∆tmwhere we defineZ ∞(3.7.17)

Cm ≡
0

h
i
ˆ † (τ )R̃ˆ (0) = γm + iκm , γ , κ ∈ R.
dτ eiωm τ TrE ρ̃ˆE R̃
m
m m
m
2

Taking the differential limit ∆t → 0, we obtain a more familiar form as follows:(3.7.18)


 X
dρ̂
†
= −i Ĥeff ρ̂(t) − ρ̂(t)Ĥeff+
γm L̂m ρ̂(t)L̂†m
dt
m

with
Ĥeff ≡ ĤS + Σ̂ −(3.7.19)iX
γm L̂†m L̂m ,2 m

where the self-energy term Σ̂ is(3.7.20)Σ̂ =1X
κm L̂†m L̂m .
2 m

This term is often called Lamb shift, since it gives an analogue of the vacuum-fluctuation induced energy
shifts in the system; we will revisit this point later when we learn about quantum theory of electrodynamics.
3.8. Short remark on the validity of master equation approaches
Before proceeding further, let us make a few remarks on the Markovian master-equation approaches.
Firstly, we note that when a system of interest gets strongly correlated with an external environment, thenonvanishing system-environment entanglement in general invalidates the Born-Markov approximations
used above. Such a situation is ubiquitous in condensed matter systems (a.k.a quantum impurity problems)and also relevant to certain state-of-the-art AMO systems, such as strongly coupled light-matter systems.
In these cases, the master equation approaches are no longer valid, and ideally, one has to explicitly take
into account both system and enviromental degrees of freedom on equal footing. Historically, this lineof studes date back to the pionering works by Feynman and Vernon or Caldeira and Leggett. For further
developments, you may refer to Sec. 4.3.3 of Adv. Phys. 69, 249 (2020) or lecture note that can be foundin this link (in Japanese).

3.8. SHORT REMARK ON THE VALIDITY OF MASTER EQUATION APPROACHES

49

Summary of Chapter 3
Section 3.1-3.4 Indirect measurement model, POVM, CPTP map• In the indirect measurement model, a projection measurementnis performed
on meter thatointeracts with a system of interest. The measurement operators M̂m,k acting on a densityP†
operator of the system, ρ̂ → k M̂m,k ρ̂M̂m,k, are in general nonorthogonal and called Kraus
operators. The probability distribution pm of measurement outcome m is characterized by theP†positive operator-valued measure (POVM): Êm = k M̂m,k
M̂m,k and pm = Tr[Êm ρ̂].
• Consider a linear mapping E between density operators on part of the whole Hilbert space. To
ensure that E gives physically reasonable time evolution (the target vector E(ρ̂) can indeed beinterpreted as a density operator), we must make it completely positive and trace-preserving(CPTP) map.
• For any CPTP map E, one can construct an indirect measurement model in such a way that its
nonselective evolution matches with E. In other words, E always permits Stinespring reprePsentation, E(ρ̂) = TrA [Û (ρ̂ ⊗ ρ̂A )Û † ], or Kraus representation E(ρ̂) = α M̂α ρ̂M̂α† .
Section 3.5 Bayesian inference and QND measurement• If a measurement process satisfies the diagonal condition (both POVM and Kraus operators
can be diagonalized by the same basis), the probability distribution of the postmeasurement
state in this basis satisfies Bayes’ relation.
• A quantum nondemolition (QND) measurement is a measurement process that does not alter
the statistics of a certain observable even after the measurement.
• Since a QND measurement satisfies the diagonal condition, the probability distribution satisfies Bayes’ theorem in the eigenbasis of the QND observable X̂. After many repetitionsof measurements, a quantum state eventually converges to a particular eigenstate X̂, and the
distribution of those “collapsed” states obey the Born rule (it is given by the initial distribution
in the eigenbasis of X̂).
Section 3.6 Continuous quantum measurement• Taking the weak and frequent limit of the repeated indirect measurement model, the dynamics
of measured system can be described by the stochastic Schrödinger equation. Each realizationof this stochastic equation is called quantum trajectory.
• The stochastic dynamics consists of two parts, deterministic non-Hermitian evolution andrandom events causing discrete state changes often called quantum jumps.
• Upon the ensemble average over all the possible trajectories, the nonselective evolution is
given by the GKSL master equation. This master equation can also be derived from the BornMarkov approximations.
• When the detection rate of quantum jumps is high but their influences are weak, one canreplace the stochastic process by the Wiener process.

3.9. EXERCISES

50

3.9. Exercises
Exercise 3.1 (Continuous position measurement: 2 points). Consider a single particle subject tocontinuous position measurement(3.9.1)ih√κd|ψ⟩ = −iĤ − (x̂ − ⟨x̂⟩)2 dt|ψ⟩ + κ(x̂ − ⟨x̂⟩)dW |ψ⟩2with the Hamiltonianp̂2+ V (x̂).
2m
Using the Itô calculus, derive the following time evolution equation for a density operatorĤ =(3.9.2)√κdρ̂ = −i[Ĥ, ρ̂] − [x̂, [x̂, ρ̂]] dt + κ {x̂ − ⟨x̂⟩, ρ̂} dW.
2
Next, suppose that the initial state is a Gaussian state (i.e., a state that is fully characterized by the moments
up to the second order) with zero mean ⟨x̂⟩ = 0. Derive the time evolution equations for the mean values⟨x̂⟩, ⟨p̂⟩ and Vx,p , Cxp in the case of V (x) = mω 2 x2 /2. Show that the state remains a Gaussian state
during the time evolution. Explain why this is in general not the case for a non-harmonic potential.
Finally, derive the time evolution equation of the following Wigner function in the case of the harmonic potential(3.9.3)(3.9.4)W (x, p) ≡

1
2π

Z ∞

dy eipy ⟨x + y/2|E[ρ̂]|x − y/2⟩,−∞

and discuss its relation to the Fokker-Planck equation.

Exercise 3.2 (Continuous measurement of indistinguishable particles: 2 points). Consider two
noninteracting particles trapped in the harmonic potential, where the single-particle Hamiltonian is given
by Ĥ = p̂2 /2m+mω 2 x2 /2. The particles are either distinguishable particles or identical bosons/fermionsof mass m. First, calculate the particle squared distance ⟨(x1 − x2 )2 ⟩ for each case. To simplify the
analysis, you may assume that two particles reside in energy eigenstates.
Second, generalize your consideration to the case of two particles under continuous position measurement, and derive the time evolution equation; do the results depend on the distinguishability of theparticles? If possible, try to numerically calculate the time evolution of the squared distance ⟨(x1 − x2 )2 ⟩by choosing a certain initial state.

Exercise 3.3 (Completely positivity: 1 point). Give an example of positive, but not completely
positive maps acting on a density operator and explain why it is so.

CHAPTER 4

Foundations of quantum optics
4.1. Introduction
Here we shall cover several fundamental concepts of quantum optics and modern quantum science.
We start from quantization of the electromagnetic field, introduce Bosonic/Fermionic Gaussian states,
learn about variational principles, and briefly review physics of superconducting qubits. These contents
not only constitute fundamental building blocks of quantum optics, but also play important roles in a
broader context of quantum science, such as condensed matter physics or recent efforts toward applying
variational algorithms to Noisy Intermediate-Scale Quantum computer (NISQ).
4.2. Quantization of the electromagnetic field
4.2.1. Classical electromagnetism review. We first review the free evolution of the classical electromagnetic field (i.e., without any couplings to matter degrees of freedom); we will quantize the fulllight-matter Hamiltonian including matter in the next Chapter.
Our starting point is the following free Maxwell’s equations
1
∂t E.
c2
To rewrite these equations, we first note the Helmholtz theorem, which states that a vector field V(r) canbe uniquely decomposed into transverse and longitudinal parts as follows:(4.2.1)∇ · E = 0, ∇ · B = 0, ∇ × E = −∂t B, ∇ × B =(4.2.2)

V(r) = V⊥ (r) + V∥ (r),(4.2.3)∇ · V⊥ = 0,

(4.2.4)∇ × V∥ = 0.

This theorem, together with the equations ∇·B = 0, ∇·E = 0, and ∇×E = −∂t B, lead to the following
expressions for some vector field A(r, t) (called the vector potential):(4.2.5)

B = B⊥ = ∇ × A(4.2.6)

E = E⊥ = −∂t A⊥ .

Decomposing the final equation ∇ × B = c12 ∂t E into transverse and longitudinal parts, we obtain(4.2.7)−∇2 A⊥ =1
1∂t E⊥ = − 2 ∂t2 A⊥
c2
c51

4.2. QUANTIZATION OF THE ELECTROMAGNETIC FIELD

52

for the transverse part, and−∇2 A∥ + ∇(∇ · A∥ ) = 0(4.2.8)for the longitudinal part. It is now obvious that the longitudinal part of the vector potential A∥ is completely decoupled from dynamical degrees of freedom. Thus, we can always ensure the condition (known
as the Coulomb gauge):A∥ = 0 ⇐⇒ ∇ · A = 0.

(4.2.9)We assume that this condition will always be satisfied in the following discussions in this Chapter. Altogether, the Maxwell’s equations now reduce to the time evolution of the transverse vector field A = A⊥(for the sake of notational simplicity, we shall omit ⊥ from now on)
1 2
∂ A,
c2 t
which can be related to the electric and magnetic fields via∇2 A =(4.2.10)B = ∇ × A, E = −∂t A.

(4.2.11)

One can check that this equation of motion also follows from the Lagrangian:ϵ0L=
2(4.2.12)Z
d3 r (∂t A)2 − c2 (∇ × A)2 .

The canonical momentum isΠ≡(4.2.13)

δL= ϵ0 ∂t A = −ϵ0 Eδ(∂t A)

while we get
δL= −ϵ0 c2 ∇ × (∇ × A) = ϵ0 c2 ∇2 A,
δA
resulting in the Euler-Lagrange equation(4.2.14)

δL
1
d δL
=
⇐⇒ ∇2 A = 2 ∂t2 A.
dt δ(∂t A)
δA
c(4.2.15)

The corresponding Hamiltonian isZ(4.2.16)H=

3d r Π · ∂t A − L =ZΠ2 ϵ0 c22d r+(∇ × A) .
2ϵ0
2
3



In terms of the electric and magnetic fields, this is nothing but the well known expression of the totalenergy of the electromagnetic field:

4.2. QUANTIZATION OF THE ELECTROMAGNETIC FIELD

Z
H=(4.2.17)53


ϵ0 2 ϵ0 c2 2
d r
E +
B .
2
2
3



Using the mode expansion, one can further rewrite this Hamiltonian (which is useful for our purposeof quantization of electromagnetic fields in the next section). To this end, we note that the equationof motion (4.2.15) can be solved by using the ansatz Aω (r, t) = Aω (r)e−iωt + c.c., resulting in theHelmholtz equation:ω2
2∇ + 2 Aω (r) = 0.
c(4.2.18)

A specific eigensolution depends on boundary conditions and let us denote a set of these eigenfrequencies
by {ωn }. We can now express a general solution as a linear combination of those eigenmodes:A(r, t) =(4.2.19)X

αn fn (r)e−iωn t + c.c.,nwhere αn are mode amplitudes and fn are the orthonormal mode functions satisfying the eigenequation:
(4.2.20)

ω2∇ + 2nc2Z
fn (r) = 0,

d3 r fn† (r)fm (r) = δnm .

It is a good exercise to show that, using these eigenmodes, the Hamiltonian can be rewritten as(4.2.21)

H=

XP2

 X
ϵ0 ωn2 2+Qn =ϵ0 ωn2 (αn∗ αn + αn αn∗ ),
2ϵ0
2
n
n

n

where we define(4.2.22)

Qn ≡ αn + αn∗ , Pn ≡ iϵ0 ωn (αn∗ − αn ).

Note that the free electromagnetic Hamiltonian is equivalent to a collection of independent harmonic
oscillators with eigenfrequencies ωn ; we may then regard Pn and Qn as effective momentum and position
variables.

4.2.2. Quantization of electromagnetic fields. The quantization of the electromagnetic field can
now be done by promoting classical variables Qn , Pn to operators Q̂n , P̂n and assuming the following
commutation relations:(4.2.23)[Q̂n , P̂m ] = iℏδnm .

This is equivalent to the following replacement of the amplitude variables by the annihilation operators
obeying the commutation relation:

4.2. QUANTIZATION OF THE ELECTROMAGNETIC FIELDr
αn →(4.2.24)54ℏ
ân , [ân , â†m ] = δnm .
2ϵ0 ωnThe quantized Hamiltonian is


1
†
Ĥ =
ℏωn ân ân +,2
nX(4.2.25)

from which it is now clear that the excitations of the quantum electromagnetic field correspond to elementary (i.e., particle-like) excitations which are called photons. The quantized vector field (in the Heisenberg
representation) is expressed by(4.2.26)Â(r, t) =

X

r

n


ℏ
fn (r)ân e−iωn t + H.c. .
2ϵ0 ωnThis is a general expression that holds true for arbitrary boundary conditions.
It is useful to consider the case of free space that corresponds to imposing the periodic boundary
conditions in a box of volume V = L3 . Since the vector potential is transverse, it contains two independent
polarization modes λ = 1, 2 other than the wavevector k and each eigenmode is labeled byn → (k, λ).

(4.2.27)

The photon operators and the mode functions thus obey

(4.2.28)[âkλ , â†k′ λ′ ] = δkk′ δλλ′ ,(4.2.29)

1
fkλ (r) = √ ϵkλ eik·r , ωk = ck,V

2πnα, nα ∈ Z, α ∈ {x, y, z} .
L
Note that the mode functions satisfy the orthonormal relation:Z†(4.2.31)dV fkλ· fk′ λ′ = δkk′ δλλ′ .
(4.2.30)k · ϵkλ = 0, kα =The vector potential (4.2.26) can then be obtained bys(4.2.32)Â(r, t) =

X
kλ



ℏ
ϵkλ âkλ eik·r−iωk t + H.c. .
2ϵ0 ωk V

Let eα be the unit vector in the direction α ∈ {x, y, z}. It is often useful to note the relation
Xkα kβ(4.2.33)(ϵkλ · eα ) (ϵkλ · eβ ) = δαβ −
,
|k|2
λ

4.2. QUANTIZATION OF THE ELECTROMAGNETIC FIELD

55

which follows from the fact that the left hand side represents the projection onto the subspace spanned by
the transverse vector fields. Another useful relation can be obtained in the thermodynamic limit V → ∞,where the summation is replaced by the integral
Z
1 X
d3 k
.
(4.2.34)
→
V
(2π)3
k

This induces the replacements of the annihilation/creation operators and their commutation relation as
follows:1(4.2.35)âkλ → √ âkλ ,V(4.2.36)[âkλ , â†k′ λ′ ] = δkk′ δλλ′ → [âkλ , â†k′ λ′ ] = (2π)3 δ(k − k′ )δλλ′ .

4.2.3. Commutation relations of quantized fields. We can use the relations in the previous section
to derive the commutation relations of quantized fields in free space. To demonstrate this, we here consider
the following commutation relation between Â and Ê:iℏ X∗((fkλ )α (fkλ)β + c.c.)
2
kλ


Z
kα kβ
iℏ
d3 k
′
=
δαβ − 2eik·(r−r ) + c.c..
3
2
(2π)
k

[Âα (r, t), −ϵ0 Êβ (r′ , t)] =(4.2.37)

Using the formula of the transverse delta function:




Z
kα kβ
3rα rβ
2
1
d3 k
ik·r
⊥
(r),
δαβ − 2e= δαβ δ(r) −
δαβ −
≡ δαβ
(4.2.38)
(2π)3
k
3
4πr3
r2
we get(4.2.39)⊥(r − r′ ).
[Âα (r, t), −ϵ0 Êβ (r′ , t)] = iℏδαβWhile these operators appear to behave as the position-like operator Â and the momentum-like operatorΠ̂ = −ϵ0 E as inferred from the Lagrangian above, we must keep in mind that the Coulomb-gauge constraint ∇ · Â = 0 restricts the vector variables into the transverse subspace. This fact leads to the above
unusual commutation relation with the modified, transverse delta function. In the wavevector space, one
can check that this relation is equivalent to the commutation relation for the transverse modes corresponding to [âkλ , â†k′ λ′ ] = δkk′ δλλ′ , while the longitudinal parts vanish due to the the Coulomb-gauge
constraint.
It is also notable that the commutator (4.2.39) is nonvanishing even at |r − r′ | ̸= 0 that lies outside
the light cone. However, in fact one can show that all the equal-time commutators among Ê and B̂ vanish
at different positions, which ensure the causality (see Exercise). This indicates that the vector potentialÂ itself is not a local observable (although its global property may lead to observable effects such as the
Aharonov-Bohm effect).

4.3. BOSONIC GAUSSIAN STATES

56

4.3. Bosonic Gaussian states
Theory of Bosonic Gaussian states allows us to fully characterize time evolution of the free quantizedelectromagnetic modes discussed in the previous section from a general perspective and thus play animportant role in quantum optics, especially in the context of photonic quantum computation. We here
introduce some basic concepts of Bosonic Gaussian states starting from a single-mode case and also
discuss its application to a quantum many-body system.
4.3.1. Introduction: single mode. To understand the notion of Gaussian states, we first consider asingle-mode case whose position and momentum operators are defined by (see also Sec. 2.5 in Chapter
2):(4.3.1)x̂ = â + â† , p̂ = i(â† − â),(4.3.2)[x̂, p̂] = 2i, [â, â† ] = 1.

The vacuum state |0⟩ of this mode is defined byâ|0⟩ = 0.

(4.3.3)

The Gaussian (pure) state is basically a state that can be obtained from the vacuum by acting the unitary
operator ÛGS (called Gaussian unitary) that is an exponential of â, â† terms up to second order:|ψ⟩ = ÛGS |0⟩.

(4.3.4)For a single-mode case, a general Gaussian unitary operator can be generated by a combination of
three unitary operators (we will later show this statement from a general argument). The first one is asqueezing operator defined by(4.3.5)r†2

2

S(r) = e 2 (â −â ) , r ∈ R,

which acts as(4.3.6)Ŝ † (r)âŜ(r) = cosh râ + sinh râ† .

As we discussed before, this corresponds to the squeezing along the x/p direction (see Eq. (2.5.23)).
The second one is a phase shift, which corresponds to time evolution governed by the harmonicoscillator Hamiltonian:(4.3.7)†R̂(θ) = eiθâ â , θ ∈ R.

The final one is the displacement operator(4.3.8)†∗D̂(α) = eαâ −α â , α ∈ C

4.3. BOSONIC GAUSSIAN STATES

57

which acts asD̂† (α)âD̂(α) = â + α.

(4.3.9)A generic single-mode Gaussian state is then given by combining the three operations as follows:(4.3.10)|ψ(α, θ, r)⟩ = D̂(α)R̂(θ)Ŝ(r)|0⟩
†

∗

r

†

†2

2

= eαâ −α â eiθâ â e 2 (â −â ) |0⟩.

(4.3.11)

As noted above, it is now clear that a Gaussian state is constructed by a unitary operator that is an exponential of the function including â, â† terms up to second order.
While this is the expression in terms of â, â† operators, we will later see that it is useful to switch
to the x̂, p̂ representation. We first note that the displacement operator in the x̂, p̂ representation can be
rewritten by using a real vector ξ ∈ R2 as
!
!#
"

0 1
ξ1
i
(4.3.12)D̂ξ ≡ exp
x̂ p̂
2
−1 0
ξ2


i T
(4.3.13)ϕ̂ σξ .
= exp
2
We here introduce the vector-valued operator ϕ̂ and the matrix σ by
!
!
x̂
0 1
y
, σ ≡ iσ =.
(4.3.14)
ϕ̂ =
p̂
−1 0
The displacement operator acts as(4.3.15)D̂ξ† ϕ̂D̂ξ = ϕ̂ + ξ

⇐⇒

D̂ξ†

x̂
p̂

!

x̂ + ξ1p̂ + ξ2

D̂ξ =

!
.

A generic single-mode Gaussian state can then be expressed by the x̂, p̂ representation as follows:
i

iθ

2

2

ir

(4.3.16)|ψ(ϕ1 , ϕ2 , θ, r)⟩ = e 2 (x̂ϕ2 −p̂ϕ1 ) e 4 (x̂ +p̂ ) e− 4 (x̂p̂+p̂x̂) |0⟩,(4.3.17)

ϕ1 = 2Re[α], ϕ2 = 2Im[α], ϕ1,2 ∈ R.

One can relate four real parameters, ϕ1 , ϕ2 , θ, r, of a single-mode Gaussian state to its first and second
moments of ϕ̂. On one hand, the mean value of ϕ̂ is simply obtained by(4.3.18)†⟨ψ|ϕ̂|ψ⟩ = ⟨0|D̂ϕ
ϕ̂D̂ϕ |0⟩ = ϕ,and thus directly related to the variables ϕ1,2 in |ψ⟩. On the other hand, the covariance matrix is definedby(4.3.19)no1(Γϕ )ξη ≡ ⟨ψ| δ ϕ̂ξ , δ ϕ̂η |ψ⟩, δ ϕ̂ = ϕ̂ − ϕ, ξ, η ∈ {1, 2}
2

4.3. BOSONIC GAUSSIAN STATES

58

and its matrix elements are the functions of the variables θ, r. It is a good Exercise to obtain the expressionof Γϕ in terms of θ, r and show the following relation:


1 T T
i T
(4.3.20)χψ (ξ) ≡ ⟨ψ|D̂ξ |ψ⟩ = exp − ξ σ Γϕ σξ + ϕ σξ , ξ ∈ R2 .
8
2Here, χψ (ξ) is known as the characteristic function and will play a central role in our discussions ofmulti-mode Gaussian states below. This equation means that the single-mode bosonic Gaussian state is
fully characterized by the Gaussian characteristic function χψ (ξ) whose first and second moments are
determined by the mean vector ϕ and covariance matrix Γϕ defined in Eqs. (4.3.18) and (4.3.19).

4.3.2. Multiple modes.
Characteristic function. Let us now formulate a general theory of Bosonic Gaussian states with multiple modes. To do so, we denote bosonic annihiliation and creation operators of mode i ∈ {1, 2, . . . , Nb }as âi and â†i , and introduce a vector of the x̂, p̂ operators by(4.3.21)ϕ̂ = (x̂1 , x̂2 , . . . , x̂Nb , p̂1 , p̂2 , . . . , p̂Nb )T ,(4.3.22)x̂i = âi + â†i , p̂i = i â†i − âi ,(4.3.23)hiâi , â†j = δij , [âi , âj ] = 0,i, j ∈ {1, 2, . . . , Nb } .

With this definition, the x̂, p̂ operators (which we represented as a component of the operator-valued vectorϕ̂) obey the commutation relation
!
h
i
0
INb
y
(4.3.24), ξ, η ∈ {1, 2, . . . , 2Nb } .
ϕ̂ξ , ϕ̂η = 2iσξη , σ ≡ iσ ⊗ INb =−INb 0The displacement operator, which is also known as the Weyl operator, is characterized by 2Nb real variables ξ:(4.3.25)iTD̂ξ = e 2 ϕ̂ σξ , D̂ξ† ϕ̂D̂ξ = ϕ̂ + ξ, ξ ∈ R2Nb .

This operator is complete in the sense that any Nb -mode bosonic many-body operator can be expressed
as a certain linear superposition of D̂ξ . To represent this more explicitly, we introduce the characteristic
function of operator Â byhi(4.3.26)
χA (ξ) ≡ Tr ÂD̂ξ .
Its inverse transformation allows us to express Â as a linear superposition of the displacement operators:
Z
d2Nb ξ
(4.3.27)Â =χ (ξ)D̂ξ† .
Nb A
2N
(4π)
R b
This formula can be shown by using the relation
h i(4.3.28)
Tr D̂ζ = (4π)Nb δ (ζ) .

4.3. BOSONIC GAUSSIAN STATES

59

Thus, there is a one-to-one correspondence between an operator Â and a complex function χA ; in thisrespect, this transformation is sort of the Fourier transformation. In analogy with the Parseval relation in
the Fourier analysis, one can show the relation:i Z
hd2Nb ξ
χ (−ξ)χB (ξ).
(4.3.29)
Tr ÂB̂ =Nb A
R2Nb (4π)
When an operator is Hermitian, the characteristic function satisfiesχ∗A (ξ) = χA (−ξ).

(4.3.30)

Using these relations, the purity of a density operator ρ̂ is expressed as
Z
Z
d2Nb ξ
d2Nb ξ
2(4.3.31)
Tr[ρ̂ ] =
χ
(−ξ)χ
(ξ)
=
|χρ (ξ)|2 .
ρ
ρ
Nb
Nb
2N
2N
(4π)
(4π)
R b
R b
In fact, from the definition of the characteristic function, one can show that any correlation functions of ρ̂
is obtained by certain derivatives of χρ :
i i
h h
qN
pN
(4.3.32)
Tr ρ̂ x̂p11 · · · x̂Nbb p̂q11 · · · p̂Nbb
S(4.3.33)

P=2

i qi +piPii qi −pi

∂ q1

∂ qNb

∂ p1

∂ pNb

· · · pN
· · · qN
p1
∂ξ1q1
∂ξNbb ∂ξ1+Nb∂ξ2Nbbχρ (ξ)
ξ=0Here, [· · · ]S is the totally symmetric ordering; for example, [x̂i p̂j ]S = (x̂i p̂j + p̂j x̂i )/2 (see Exercise for a
useful formula to simplify its calculation). It is also noteworthy that the Wigner function gives alternative
expression of the characteristic function, which is defined by the Fourier transformation as follows:Z
d2Nb ξ − i ζ T σξ(4.3.34)
W (ζ) =
e 2
χ(ξ).
2Nb
R2Nb (4π)

Bosonic Gaussian states. We call ρ̂G a Bosonic Gaussian state when it is completely characterized
by the correlation functions up to the second-order moment. In terms of the characteristic function, thisdefinition is equivalent to requiring the following (Gaussian) functional form:(4.3.35)



1 T T
i T
χρG (ξ) = exp − ξ σ Γϕ σξ + ϕ σξ ,82where χρG is the characteristic function of a Gaussian state ρ̂G :Z
d2Nb ξ
(4.3.36)ρ̂G =χ (ξ)D̂ξ† .
N b ρG2N(4π)
R bHere, a vector-valued displacement variable ϕ and a covariance matrix Γϕ characterize the first- andsecond-order moments of a Gaussian state:oE
1 Dn(4.3.37)
ϕ = ⟨ϕ̂⟩G , (Γϕ )ξη =δ ϕ̂ξ , δ ϕ̂η
,2
G(4.3.38)⟨· · · ⟩G ≡ Tr[ρ̂G · · · ], δ ϕ̂ = ϕ̂ − ϕ.

4.3. BOSONIC GAUSSIAN STATES

60

The covariance matrix Γϕ contains complete information about two-point correlation functions, such as⟨δ x̂δ p̂⟩, ⟨δ x̂δ x̂⟩ or ⟨δ p̂δ p̂⟩ type functions. From the definition, it follows that Γϕ is a real-symmetric
2Nb ×2Nb matrix. One can show that it satisfies the following generalized Heisenberg uncertainty relation:
Γϕ + iσ ≥ 0.

(4.3.39)

When we use a basis in which Γϕ is diagonal, this relation simplifies to the well-known uncertainty relationfor each mode:δ x̂2i(4.3.40)

δ p̂2i ≥ 1.

In fact, the uncertainty relation (4.3.39) itself holds true for arbitrary density operator ρ̂ besides a Gaussianstate (see Exercise).
A general Bosonic Gaussian state ρ̂G is completely characterized by a set of variables (ϕ, Γϕ ) where
ϕ is arbitrary 2Nb dimensional real vector and Γϕ is a real-symmetric 2Nb × 2Nb matrix satisfying thecondition (4.3.39). This follows from the fact that, for a given Gaussian state, the relation (4.3.39) is
equivalent to the positivity of a density operator ρ̂G ≥ 0 (see Exercise). We note that the relation (4.3.39)
automatically ensures the positivity of covariance matrix Γϕ ≥ 0, since Eq. (4.3.39) also indicates Γϕ −iσ ≥ 0 because of σ = −σ T .
It is sometimes useful to switch to the â, â† representation. To this end, we introduce the complex
variables by

λi =(4.3.41)ξi − iξi+Nb
ξi + iξi+Nb, λ∗i =.
2
2

Using the vector notation(4.3.42)λ = (λ1 , λ2 , . . . , λNb )T , â = (â1 , â2 , . . . , âNb )T ,the Weyl operator is rewritten as(4.3.43)†T∗D̂λ = eâ λ−â λ .

The x̂, p̂ and â, â† representations can in general be interchanged by using the following linear transformation:
!
!
â
INb
INb
(4.3.44)ϕ̂ = T, T ≡.
â∗−iINb iINb
This leads to the following relation that connects the two different expressions of the covariance matrix:
*
!
+


δ â
1
1
(4.3.45)
Γb ≡= I2Nb + T † Γϕ T,δ â† δ âT
∗
2
4
δ â
G

where we denote δ â = â − ⟨â⟩G and Γb is the covariance matrix in the â, â† representation and contains
complete information about ⟨δâδâ† ⟩, ⟨δâδâ⟩ or ⟨δâ† δâ† ⟩ type functions.

4.3. BOSONIC GAUSSIAN STATES

61

Gaussian unitary operation. We next characterize a set of all the possible unitary operations that map
a Gaussian state to another Gaussian state. To do so, we introduce the symplectic group that is defined bya set of real matrix S satisfying
SσS T = σ.

(4.3.46)

Physically, this transformation S represents a linear transformation that keeps the commutation relation(4.3.24). Using a certain S, we can show that the covariance matrix can be diagonalized as (called
Williamson decomposition):(4.3.47)SΓS T = diag (κ1 , . . . , κNb , κ1 , . . . , κNb )where 1 ≤ κ1 · · · ≤ κNb are known as the Williamson eigenvalues. To show this, we rewrite this equalityas
S = (D ⊕ D)1/2 OΓ−1/2(4.3.48)

where O is an orthogonal matrix and D = diag(κ1 , . . . , κNb ). Since S is symplectic, we arrive at(4.3.49)

−1/2

OΓ

σΓ

−1/2

T

O =

0
D−1
−D−1
0

!
.

This relation says nothing but there exists an orthogonal matrix O that transforms a real antisymmetricmatrix, Γ−1/2 σΓ−1/2 , to a canonical form given in the right-hand side; this is the well-known fact in linear
algebra.
Using the Williamson decomposition, we can now characterize arbitrary Gaussian unitary operations.
Consider a Gaussian unitary operation that maps a Gaussian state with the mean vector and covariancematrix (ϕ, Γϕ ) to another one with (ϕ′ , Γ′ϕ ). The Williamson decomposition ensures that there exists a
symplectic transformation S that relates two covariance matrices:(4.3.50)

Γ′ϕ = SΓϕ S T .

As inferred from the definition of Γϕ , this transformation is equivalent to inducing the symplectic transformation onto ϕ̂ as S ϕ̂. Meanwhile, the mean vectors can be mapped by simply using the displacement
ϕ → ϕ′ . Altogether, we conclude that a Gaussian operation of interest here is induced by a unitary
operator acting as(4.3.51)†
ÛS,∆
ϕ̂ÛS,∆ = S ϕ̂ + ∆, ∆ = ϕ′ − Sϕ.

Indeed, the transformed state(4.3.52)†
ρ̂′G = ÛS,∆ ρ̂G ÛS,∆

has the mean vector ϕ′ = Sϕ + ∆ and the covariance matrix Γ′ϕ = SΓϕ S T by construction.
To obtain a more explicit form of a Gaussian unitary operation ÛS,∆ , we note the fact that a symplectic
matrix can be written as4.3. BOSONIC GAUSSIAN STATES(4.3.53)62

S = eσR , R = RT .

Using this real symmetric matrix R, we can show that the following unitary operator satisfies the requiredcondition (4.3.51):
h
i
h
i
1 T
1 T
(4.3.54)
ÛS,∆ = exp iĤ∆ exp −iĤS , Ĥ∆ = ϕ̂ σ∆, ĤS = ϕ̂ Rϕ̂.
2
4
We remark that this expression can be regarded as a time evolution governed by “Hamiltonian”, Ĥ∆ ,
ĤS , that are quadratic in terms of x̂, p̂, or equivalently, â, â† operators. Conversely, the time evolution
induced by any quadratic Hamiltonian can be written in this form by using suitable R and ∆. In particular,
one can thus conclude that if an initial state is Gaussian, it remains so during the time evolution governedby a quadratic (i.e., noninteracting) Hamiltonian.
To understand physical meaning of the symplectic transformation S, we can use the Euler/BlochMessiah decomposition of S:(4.3.55)

S = KDL,

where K and L are symplectic and orthogonal real matrices, and D is the following diagonal matrix
(4.3.56)
D = diag er1 , er2 , . . . , erNb , e−r1 , e−r2 , . . . , e−rNb , ri ∈ R.
Said differently, any symplectic transformation can be decomposed into coordinate transformation (i.e.,(generalized) rotation), followed by a squeezing operation of each mode with squeezed parameters {rj },and again followed by another coordinate transformation.
Example: Gaussian pure state. Let us focus on the case of Gaussian pure states. From the puritypTr[ρ̂2G ] = 1/ det(Γϕ ) (see Exercise), a Gaussian state is pure if and only if all the Williamson eigenvalues satisfy κ1 = . . . = κNb = 1. Thus, a covariance matrix for a pure Gaussian state can be decomposedby a symplectic matrix S as

(4.3.57)

Γϕ = S T S.

It then follows that
(4.3.58)(σΓϕ )2 = −I2Nb .

Equivalently, one can consider the relation (4.3.58) as the necessary and sufficient condition for ρ̂G beingpure.
A general pure Gaussian state ρ̂G = |ψ⟩⟨ψ|, which has mean vector ϕ and covariance matrix Γϕ , isobtained by acting a Gaussian unitary operation on the vacuum as(4.3.59)

i

T

i

T

|ψ⟩ = ÛS,ϕ |0⟩ = e 2 ϕ̂ σϕ e− 4 ϕ̂ Rϕ̂ |0⟩,where S = eσR with R = RT satisfies Γϕ = SS T . Indeed, its first-order moment satisfies(4.3.60)†⟨ϕ̂⟩ = ⟨0|ÛS,ϕϕ̂ÛS,ϕ |0⟩ = ⟨0|S ϕ̂ + ϕ|0⟩ = ϕ,

4.3. BOSONIC GAUSSIAN STATES

63

and second-order moment is
  
 
oE 1  
1 Dn0 = SS T ξη = (Γϕ )ξη .
(4.3.61)0Sδ ϕ̂ , Sδ ϕ̂δ ϕ̂ξ , δ ϕ̂η
=
2
2
η
ξ
In particular, when there is only a single mode (Nb = 1), the Euler/Bloch-Messiah decomposition
leads to(4.3.62)S=

cos θ − sin θ
sin θ cos θ

!

!

er 0
0 e−r

cos ϕ − sin ϕ
sin ϕ cos ϕ!
.

The first rotation trivially acts on the vacuum, i.e., it does not alter the state. Thus, together with adisplacement, a general single-mode Gaussian state is characterized by(4.3.63)

iθ

i

2

2

ir

2

†2

|ψ(ϕ1 , ϕ2 , θ, r)⟩ = e 2 (x̂ϕ2 −p̂ϕ1 ) e 4 (x̂ +p̂ ) e− 4 (x̂p̂+p̂x̂) |0⟩or in the â, â† expression(4.3.64)

†

∗

†

r

|ψ(α, θ, r)⟩ = eαâ −α â eiθâ â e 2 (â −â ) |0⟩,

as we discussed in the introduction of this Chapter.
4.3.3. Application to weakly interacting BEC. A multi-mode bosonic Gaussian state fully characterizes energy eigenmodes of the free electromagnetic field in arbitrary geometry (i.e., under generalboundary conditions). In fact, a bosonic Gaussian state is also very useful in condensed matter physics.
As an illustrative example, here we shall apply it to the analysis of a weakly interacting Bose gas at zerotemperature. This problem is not exactly solvable and one has to resort to some approximative methods.
We review a simple variational theory using a subclass of bosonic Gaussian states, which is actually nothing but the usual Bogoliubov theory (that you may already be familiar with). One advantage of discussing
the problem in terms of a Gaussian state is that one can easily generalize the analysis to a wider variational
manifold spanned by the whole bosonic Gaussian states as discussed later.
We want to obtain the ground state of a weakly interacting Bose gas:Z
(4.3.65)Ĥ =dr ψ̂r†



ℏ2 ∇2
−
2m



U
ψ̂r +
2

Z

dr ψ̂r† ψ̂r† ψ̂r ψ̂r .

To begin with, let us consider the noninteracting case U = 0; the exact ground state is then obtained as
the Bose-Einstein condensate (BEC), in which all the particles occupy the zero momentum single-particle
states:(4.3.66)†N=0|ΨU
GS ⟩ ∝ âk=0 |vac⟩,

where N is the total number of bosons. An essential feature of this state is the presence of the off-diagonal
long-range order (ODLRO):(4.3.67)lim ⟨ψ̂r† ψ̂0 ⟩ =

|r|→∞

N
̸= 0.
V

4.3. BOSONIC GAUSSIAN STATES64

The central idea of variational analysis is to use a simpler variational state that still captures the essential
physical feature expected in the exact ground state. When considering (noninteracting) BEC, a natural
choice is the coherent state:√  †N âk=0 −âk=0=0|ΨUvar ⟩ = e

(4.3.68)|vac⟩,

which clearly has nonvanishing ODLRO and thus keeps the essential feature of the present case. An
advantage of using a coherent state is that it can often significantly simplify the theoretical analysis in
more complex problems; while this state is not an eigenstate of the particle number, one can neglect its
fluctuation in the thermodynamic limit. Even in mesoscopic regimes (say, N ∼ O(102 )), we will later seethat the expectation value of the total particle number N̂ can be exactly conserved during the variationaltime evolution. In this sense, one can consistently take into account the particle conservation into theory
even if a variational state is not a particle-number eigenstate.
Building on this observation, we next switch on a weakly repulsive interaction U > 0. We expect that=0
the ground state should be expressed by slightly modifying the noninteracting variational state |ΨU
var ⟩.
More specifically, the interaction should “excite” a certain amount of zero-momentum bosons leading
to the suppressed condensate particle number N0 < N . Because of the momentum conservation, the“excited” bosons must behave as a creation of a pair of particles with momentum k and −k. This physicalintuition leads to the following variational form for the interacting ground stateP(4.3.69)>0|ΨU
var ⟩ = e



 √ 
N0 â†k=0 −âk=0
âk â−k −â†k â†−krk
k̸=0 2e|vac⟩,

which spans a certain subspace within the whole Bosonic Gaussian states. We then consider the squeezing
parameters rk as variational parameters determined by minimizing the expectation value of total energy.
In fact, this whole discussion is equivalent to the standard Bogoliubov theory. To see this, following√
the usual mean-field treatment, we expand as âk = N0 δk,0 + δâk and neglect the higher order terms of
δâ, leading to the mean-field HamiltonianN
nU N XnU  † ††(4.3.70)
ĤMF =+(ϵk + nU ) âk âk +âk â−k + âk â−k , n = .
2
2
V
k̸=0

Since this is a quadratic Hamiltonian, we can diagonalize it by the Bogoliubov transformation(4.3.71)(4.3.72)b̂k = cosh rk âk + sinh r−k â†−k = Û † âk Û ,−Û = ePrkk̸=0 2âk â−k −â†k â†−k,where the squeezing parameters are given by(4.3.73)nUsinh 2rk = p.
ϵk (ϵk + 2nU )The resulting diagonalized mean-field Hamiltonian is
Xp(4.3.74)
ĤMF = const. +ϵk (ϵk + 2nU )b̂†k b̂k .
k̸=0

4.4. FERMIONIC GAUSSIAN STATES

65

Its ground state is given by the squeezed vacuum |vac⟩bk̸=0 = Û † |vac⟩. Together with the BEC part, we
can now write the mean-field ground state as
N0
>0
|ΨU
MF ⟩ = |vac⟩bk̸=0 |BECk=0 ⟩,(4.3.75)P=e(4.3.76)rkk̸=0 2 √ âk â−k −â†k â†−k
N0 â†k=0 −âk=0e|vac⟩,

which is nothing but the above Gaussian variational state (4.3.69).
We argued that one advantage of using variational states is that one can significantly simplify the theoretical analysis. In the present context, this can be inferred from, for example, the following factorization
when evaluating the expectation value of the interaction term:E
XD †X D † E D † † E(4.3.77)âk+q â†p−q âk âp ≃ N02 + N0
4 âk âk + âk â−k + ⟨âk â−k ⟩ .
k,p,q

k̸=0

Note that the right-hand side only contains second-order correlation functions that are completely characterized by a Gaussian state. Then, the procedure of minimizing the variational energy ⟨Ĥ⟩var with respectto {rk } can also be considered as the optimization of the covariance matrix Γϕ of a Gaussian state forminimizing a loss function ⟨Ĥ⟩var . We will develop a general variational formalism along this line later.
4.4. Fermionic Gaussian states
Coherent state, or more generally, Gaussian state is historically discussed mainly in the context ofbosonic systems. One can in fact introduce similar notions also in the case of fermionic states as we
discuss in this section. Similar to bosonic cases, the Fermionic Gaussian states lie at the heart of suchstandard mean-field theory as Hartree-Fock method or Bardeen-Cooper-Schrieffer (BCS) theory of superconductors.
4.4.1. Introduction: single mode. As an introduction, we first consider the single-mode case ofspin-1/2 fermion. The annihilation and creation operators satisfy the following anticommutation relations:(4.4.1)onĉσ , ĉ†σ′ = δσσ′ , {ĉσ , ĉσ′ } = 0, σ ∈ {↑, ↓} .

To motivate the notion of Gaussian states, consider a single-mode fermionic state characterized by thefollowing expectation values1:(4.4.2)⟨ĉ†↑ ĉ↑ ⟩θ = sin2 θ, ⟨ĉ†↑ ĉ†↓ ⟩θ = ⟨ĉ↓ ĉ↑ ⟩θ = sin θ cos θ.

Such a state is given by a superposition of the following two states:(4.4.3)ĉ†↑ ĉ†↓ |0⟩, |0⟩with coefficients(4.4.4)|Ψ(θ)⟩ = cos θ + sin θĉ†↑ ĉ†↓ |0⟩.

1As shown later, this physically implies the condensation of paring state of two fermions in BCS superconductors.

4.4. FERMIONIC GAUSSIAN STATES

66

This state can be written by using a unitary operatorÛθ = e(4.4.5)−θ ĉ†↑ ĉ†↓ +ĉ↑ ĉ↓

as follows:|Ψ(θ)⟩ = Ûθ† |0⟩ = Û−θ |0⟩.

(4.4.6)To see this, we note the relation(4.4.7)

Ûθ†

!

ĉ↑
ĉ†↓

Ûθ =

cos θ − sin θ
sin θ cos θ

!

ĉ↑
ĉ†↓

!
,

which, for example, leads to the expectation value ⟨Ψ(θ)|ĉ↓ ĉ↑ |Ψ(θ)⟩:(4.4.8)(4.4.9)(4.4.10)†⟨0|Ûθ ĉ↓ ĉ↑ Ûθ† |0⟩ = ⟨0|Û−θĉ↓ ĉ↑ Û−θ |0⟩= ⟨0| − sin θĉ†↑ + cos θĉ↓ cos θĉ↑ + sin θĉ†↓ |0⟩= sin θ cos θ.

Altogether, the simplest fermionic Gaussian state is given byθ ĉ†↑ ĉ†↓ +ĉ↑ ĉ↓† †|0⟩.
(4.4.11)|Ψ(θ)⟩ = cos θ + sin θĉ↑ ĉ↓ |0⟩ = e
For later use, let us also introduce the Majorana representation of fermionic operators:(4.4.12)ψ̂1σ = ĉσ + ĉ†σ , ψ̂2σ = i(ĉ†σ − ĉσ ),which satisfy the anticommutation relationsno(4.4.13)ψ̂ξσ , ψ̂ησ′ = 2δξη δσσ′ ξ, η ∈ {1, 2} .
†; roughly speaking, this corresponds toThis operator is “real” in the sense that it satisfies ψ̂ξσ = ψ̂ξσthe x̂, p̂ representation in the bosonic case discussed before. However, different from the bosonic x̂, p̂
2 = ψ̂ 2 = 1. The
operators, the Majorana operators either anticommute with each other or satisfy ψ̂1σ
2σ
above state can be expressed in terms of the Majorana operators as(4.4.14)θ ĉ†↑ ĉ†↓ +ĉ↑ ĉ↓|Ψ(θ)⟩ = eθ|0⟩ = e 2 (ψ̂1↑ ψ̂1↓ −ψ̂2↑ ψ̂2↓ ) |0⟩.

4.4.2. Multiple modes. We now generalize the argument to the case of Nf -mode fermionic system(for the sake of simplicity, we omit the spin degrees of freedom here). The annihilation and creation
operators for each mode satisfy(4.4.15)noĉi , ĉ†j = δij , {ĉi , ĉj } = 0,i, j ∈ {1, 2, . . . , Nf } .

In the Majorana representation, they satisfy(4.4.16)ψ̂1,i = ĉi + ĉ†i , ψ̂2,i = i ĉ†i − ĉi4.4. FERMIONIC GAUSSIAN STATES(4.4.17)noψ̂ξ , ψ̂η = 2δξη , ψ̂ξ = ψ̂ξ† ,67

ξ, η ∈ {1, 2, . . . , 2Nf } .

We then introduce the operator-valued vector in the Majorana representation by
T
,(4.4.18)ψ̂ = ψ̂1,1 , ψ̂1,2 , . . . , ψ̂1,Nf , ψ̂2,1 , ψ̂2,2 , . . . , ψ̂2,Nf
which gives an analog of ϕ̂ vector in the bosonic case above.
Similar to the bosonic case, one can construct the Fermionic Gaussian states by starting from thecorresponding Weyl operator, coherent states, and characteristic functions as detailed later. Here, however,
we shall begin with a more explicit and straightforward introduction of Fermionic Gaussian states.
Specifically, we define the Fermionic Gaussian states by a density operator described by
i T(4.4.19)ρ̂G = N exp − ψ̂ X ψ̂ ,4where X is arbitrary 2Nf × 2Nf real antisymmetric matrix and N is the normalization factor. This stateis completely characterized by the following two-point correlation function:iEi Dhψ̂ξ , ψ̂η, ⟨· · · ⟩G ≡ Tr[ρ̂G · · · ],(4.4.20)(Γψ )ξη =2
G
where the covariance matrix Γψ is also real antisymmetric matrix. Recall that the Bosonic Gaussianstates were characterized by the mean vector ϕ in addition to the covariance matrix Γϕ ; in contrast, theFermionic Gaussian states here have no mean vectors since there is no BEC. To see the relation between
X in ρ̂G and Γψ , we note that real antisymmetric matrix can be transformed to the following canonicalform by using an orthogonal matrix O:!
0D(4.4.21)
OXOT =, D = diag β1 , . . . , βNf , βi ∈ R.
−D 0
We also introduce the corresponding state vector byˆψ̃ = Oψ̂

(4.4.22)and denote the annihilation and creation operators for this as c̃ˆi and c̃ˆ†i ; note that these operators alsosatisfy the anticommutation relations since O is orthogonal. In this canonical frame, the density matrix
becomes
NfX(4.4.23)ρ̂G = N exp −
βj c̃ˆ†j c̃ˆj  .
j=1

The corresponding covariance matrix can be obtained as(4.4.24)

OΓψ OT =0− tanh(D/2)
tanh(D/2)
0!
.

Since this has pure imaginary eigenvalues ±i tanh(βj /2), the covariance matrix satisfies(4.4.25)−I2Nf ≤ iΓψ ≤ I2Nf ⇐⇒ −Γ2ψ ≤ I2Nf .

4.4. FERMIONIC GAUSSIAN STATES

68

Conversely, for a given 2Nf × 2Nf real antisymmetric matrix Γψ satisfying this condition, there exists a
Fermionic Gaussian state whose covariance matrix is given by Γψ .
You can check that ρ̂G is pure if and only if βi → ±∞ ∀i, which leads to(4.4.26)

ρ̂2G = ρ̂G ⇐⇒ Γ2ψ = −I2Nf .

More generally, the purity of ρ̂G is given by (see Exercise)(4.4.27)

Tr[ρ̂2G ] =

r
det

h

 i
I2Nf − Γ2ψ /2 .

A general multi-point correlation function is given by the certain product of matrix elements of Γψ asfollows (known as Wick’s theorem):
h


p2N i
(4.4.28)Tr ρ̂G ψ̂1p1 ψ̂2p2 · · · ψ̂2Nff = Pf (−i)Γψ |P , pξ ∈ {0, 1} ,where Γψ |P describes the submatrix that only contains the raws and columns for which pξ = 1, and Pf isthe Pfaffian which is defined for antisymmetric 2n × 2n matrix M as follows:
1 X
sgn(σ)Mσ(1)σ(2) · · · Mσ(2n−1)σ(2n) .
(4.4.29)
Pf(M ) = n
2 n!
σ∈S2n

Here, S2n is the permutation group. The Pfaffian satisfies the following important property:(4.4.30)

Pf(M )2 = det(M ).

For instance, the four-point correlation function for ξ1 < ξ2 < ξ3 < ξ4 is given byhi(4.4.31) Tr ρ̂G ψ̂ξ1 ψ̂ξ2 ψ̂ξ3 ψ̂ξ4 = − (Γψ )ξ1 ξ2 (Γψ )ξ3 ξ4 − (Γψ )ξ1 ξ3 (Γψ )ξ2 ξ4 + (Γψ )ξ1 ξ4 (Γψ )ξ2 ξ3 .
We note that, conversely, a density operator satisfying Wick’s theorem is given by a certain Fermionic
Gaussian state. We will prove Wick’s theorem later by using Fermionic coherent states.
It is also useful to employ the Nambu representation (corresponding to â, â† representation in thebosonic case):!
Tĉ
(4.4.32), ĉ = ĉ1 , ĉ2 , . . . , ĉNf ,∗ĉwhich can be related to the Majorana representation via
!
!
1 1
ĉ
(4.4.33)ψ̂ = T, T ≡⊗ INf =
−i i
ĉ∗

INf
−iINf

INf
iINf

!
.

The correlation function in the Nambu representation is given by
!
+
*


ĉ
i
1
T
†
= I2Nf − T † Γψ T.
(4.4.34)
Γf ≡
ĉ ĉ
∗
2
4
ĉ
G

Gaussian unitary operation. We next consider a unitary operation that maps a Fermionic Gaussianstate to another Gaussian state (Gaussian operation). To this end, note that the parameters {βi } in Eq.

4.4. FERMIONIC GAUSSIAN STATES

69(4.4.23) are invariant under unitary transformation. This means that, for any two Gaussian states that are
related by a unitary operation, there exists an orthogonal matrix O that relates their covariance matrices asΓ̃ψ = OΓψ OT .

(4.4.35)

Thus, a set of Gaussian unitary operations is characterized by the unitary operation ÛO that induces thischange, which in the state vector corresponds to the transformationˆψ̃ = ÛO† ψ̂ ÛO = Oψ̂.

(4.4.36)

Recall that the orthogonal group is generated by the reflections and rotations; accordingly, Gaussianunitary operations consist of those two parts. First, the reflections (corresponding to ψ̂ξ → ψ̂ξ , ψ̂η → −ψ̂η ,∀η ̸= ξ) are induced by the following unitary operator
ÛO = ψ̂ξ .
n oA set of operators given by the product of ψ̂ξ generates the reflection transformations. Second, therotations are given by the following matrix with a real antisymmetric matrix X:(4.4.37)

O = eX , X = −X T .

(4.4.38)

This rotation is induced by the following unitary operator:hii T(4.4.39)
ÛO = exp −iĤO , ĤO = ψ̂ X ψ̂.
4
These transformations correspond to the special orthogonal group and contain the identity operator in
the limit X → 0. Physically, one may consider this unitary as a time evolution governed by a quadratic
Hamiltonian. Conversely, if the initial state is Gaussian and the Hamiltonian is quadratic, the state remains
Gaussian during the unitary evolution.
In the Nambu representation, these Gaussian operations are given by
!
!
ˆc̃
ĉ
(4.4.40)
=U
,
ˆc̃∗
ĉ∗
where U ≡ T −1 OT is the unitary matrix.
Finally, it is useful to explicitly include the spin-1/2 degrees of freedom of fermions:T(4.4.41)ψ̂ = ψ̂1,1↑ , . . . , ψ̂1,Nf ↑ , ψ̂1,1↓ , . . . , ψ̂1,Nf ↓ , ψ̂2,1↑ , . . . , ψ̂2,Nf ↑ , ψ̂2,1↓ , . . . , ψ̂2,Nf ↓ .
One can show that the Gaussian unitary (corresponding to the rotations) can be written in the following
canonical form by using a suitable basis:(4.4.42)ÛO = e1 Tψ̂ X ψ̂
4

=e

P Nf

i=1 θiĉ†i↑ ĉ†i↓ +ĉi↑ ĉi↓

 PN

e

f
i=1 φiĉ†i↑ ĉi↓ +ĉi↑ ĉ†i↓, θi , φi ∈ R.

This expression is particularly useful when we consider an application to the BCS superconductors as
detailed later.

4.4. FERMIONIC GAUSSIAN STATES

70

Example: Fermionic Gaussian pure state. We here focus on a pure Fermionic Gaussian state and summarize several important properties. In the same manner as in the bosonic case, a general pure Fermionic
Gaussian state is given by acting a Gaussian unitary operator on the vacuum:|ψ⟩ = Û(4.4.43)n o  Xξη|0⟩.
ψ̂ξ , e 2 ψ̂ξ ψ̂ηHere, Û represents a set of operators generated by a product of unitary operators generating the reflection(4.4.37) and rotation (4.4.39). While this is not in general a particle-number eigenstate, it can always betaken as an eigenstate of the following parity operator:(4.4.44)

PNf

P̂ = (−1)†
i=1 ĉi ĉi

2NfNf= (−i)

ψ̂1 ψ̂2 · · · ψ̂2Nf ≡ (−i)

Nf

Y

ψ̂ξ ,

ξ=1

(4.4.45)

P̂ |ψ⟩ = P |ψ⟩, P ∈ {1, −1} .

This follows from the fact that P̂ commutes with the rotation and anticommutes with the reflection, whilethe vacuum is parity even:


o
n
Xξη
(4.4.46)P̂ , ψ̂ξ = 0, P̂ , e 2 ψ̂ξ ψ̂η = 0, P̂ |0⟩ = |0⟩.
For a given Gaussian state, its parity is given by its covariance matrix as follows:(4.4.47)

P = (−1)Nf Pf(Γψ ).

Another useful formula for pure Gaussian states is the overlap:(−2)Nf P Pf (Γ + Γ )ψ1
ψ2(4.4.48)|⟨ψ1 |ψ2 ⟩|2 =0

P = P1 = P2.

P1 ̸= P2

We will derive these formulas later by using Fermionic coherent states.
If a Gaussian state is an eigenstate of particle numbers, it reduces to the well-known Slater determinantstate:(4.4.49) p1  p2
pN
f|ψSlater ⟩ = ĉ†1ĉ†2
· · · ĉ†Nf|0⟩, pi ∈ {0, 1} .

A Gaussian unitary operation restricted to particle-number conserving subspace corresponds to a unitarythat does not mix annihilation and creation operators (i.e., a unitary operator only contains the exponentialof ĉ† ĉ terms). Specifically, it acts as(4.4.50)Nf p1  p2

pN
X
f
†
†
†
ˆ
ˆ
ˆ
ˆ
c̃2
· · · c̃Nf|0⟩, c̃i =Uij ĉj .
ÛOSlater |ψSlater ⟩ = c̃1i=1Here, Nf ×Nf unitary matrix U corresponds to a basis change. Thus, the Hartree-Fock theory is a methodthat identifies the optimal Slater-determinant state by using Uij as variational parameters. However, note

4.4. FERMIONIC GAUSSIAN STATES

71

that a general Gaussian state is wider than a simple Slater-determinant state and can, for example, be used
to analyze BCS superconductors.
To see this more explicitly, we consider a parity-even Gaussian state including the spin degrees of
freedom:1T|ψ⟩ = e 4 ψ̂ X ψ̂ |0⟩.

(4.4.51)

Here, the antisymmetric matrix X is related to the covariance matrix via
hi
i(4.4.52)(Γψ )ξη = ⟨ψ| ψ̂ξ , ψ̂η |ψ⟩ = − eX σe−X ξη ,2which can be obtained by using Eq. (4.4.35) in the case that the right hand side is the covariance matrixof the vacuum, −σ. Using the canonical form introduced above, one can express the Gaussian state as
P Nf(4.4.53)|ψ⟩ = ei=1 θiĉ†i↑ ĉ†i↓ +ĉi↑ ĉi↓

|0⟩ =

Nf 
Y


cos θi + sin θi ĉ†i↑ ĉ†i↓ |0⟩,i=1

which is nothing but the well-known BCS variational states2. Said differently, any Gaussian state written
as Eq. (4.4.51) can be expressed by this BCS form in a suitable basis. Finally, as a corollary of this fact,
we note that any pure Gaussian state is a ground state of a certain quadratic Hamiltonian. Indeed, one canconstruct such “parent Hamiltonian” as
i
i
h
h
X
(4.4.54)Ĥ0 =cos (2θi ) ĉ†i↑ ĉi↑ + ĉ†i↓ ĉi↓ − sin (2θi ) ĉ†i↑ ĉ†i↓ + ĉi↓ ĉi↑ .
i

In the simplest example of a single-mode spin-1/2 fermionic system, the canonical form is given byT(4.4.55)ψ̂ = ψ̂1↑ , ψ̂1↓ , ψ̂2↑ , ψ̂2↓
and




θ

 −θ

X=


(4.4.56)




−θ 
θ

leading to(4.4.57)|ψ⟩ = ÛO |0⟩ = e

1 Tψ̂ X ψ̂
4

θ
2



† †



θ ĉ ĉ +ĉ ĉ
|0⟩ = e (ψ̂1↑ ψ̂1↓ −ψ̂2↑ ψ̂2↓ ) |0⟩ = e ↑ ↓ ↑ ↓ |0⟩.

This is nothing but the elemental example we discussed before in the introduction of this section.
Fermionic coherent states and characteristic function. We now introduce the fermionic analog of coherent states and apply it to derive several useful formulas we introduced above. A crucial difference fromthe bosonic case is that fermionic coherent states are eigenstates of annihilation operators with eigenvalues
being given by the Grassmann number:
2Strictly speaking, the BCS states usually consist of the pairing between different modes k and −k, but one can easily change
the present expression into that form.

4.4. FERMIONIC GAUSSIAN STATES(4.4.58)72{ηi , ηj } = {ηi∗ , ηj } = ηi∗ , ηj∗ = 0, ∀i, j ∈ {1, 2, . . . , Nf } .

Here, ηi∗ is the complex conjugate of ηi . We define that the Grassmann number anticommutes withfermionic operators:no(4.4.59){ηi , ĉi } = ηi , ĉ†i = 0.
We also define the Hermitian conjugate of η such that it is given in the similar manner as in operators:(ηi ηj )† = ηj∗ ηi∗ .

(4.4.60)

The fermionic analog of the Weyl operator (a.k.a displacement operator) is then defined by(4.4.61)D̂η ≡ eĉ† η−η † ĉ




Nf 
Y
1
†
†
∗
∗
1 + ĉi ηi − ηi ĉi + ĉi ĉi −=ηi ηi .
2i=1

One can check that it indeed satisfies the propertyD̂η† ĉi D̂η = ĉi + ηi ,(4.4.62)
resulting in the fermionic coherent state:ĉi D̂η |0⟩ = ηi D̂η |0⟩.

(4.4.63)

In contrast to the bosonic case, acting the displacement operator on the fully occupied state |1⟩ ≡one can construct an eigenstate of the creation operator:ĉ†i D̂η |1⟩ = ηi∗ D̂η |1⟩.

(4.4.64)

We now switch to the Majorana representation:(4.4.65)

ξ1,i = ηi + ηi∗ , ξ2,i = i(ηi∗ − ηi ),where
ξ = (ξ 1 , ξ 2 )T

(4.4.66)

are real Grassmann numbers. The displacement operator then becomes(4.4.67)

1TD̂ξ = e 2 ψ̂ ξ , D̂ξ† ψ̂ D̂ξ = ψ̂ + ξ.

Using the Majorana representation, arbitrary Grassmann-valued function can be expanded as(4.4.68)f (ξ) =

2Nf
X

X

fi1 i2 ···in ξi1 ξi2 · · · ξin

n=0 i1 <i2 <···<in

To introduce the integration of Grassmann-valued function, we define as follows:!
Z
Z
i1 , · · · , in(4.4.69)dξi = 0,dξjn · · · dξj1 ξi1 · · · ξin = ϵ,
j1, , · · · , jnQ†i ĉi |0⟩,

4.4. FERMIONIC GAUSSIAN STATES

73

where (i1 , . . . , in ) is a set of mutually different integer numbers, and the right hand side gives zero when
it is different from another set (j1 , . . . , jn ), while provides 1 or -1 if these two sets are the same andconnected by the even or odd permutations, respectively.
As an example, let us consider the following integral:Z
ZZ−ξi ξj(4.4.70)dξi dξj e= dξi dξj (1 − ξi ξj ) = dξi dξj ξj ξi = 1.
More generally, for a given antisymmetric matrix M , one can obtain the following Gaussian formulas:


Z
Z
Z
1 T
(4.4.71)Dξ expξ M ξ = Pf (M ) ,Dξ ≡ dξ2Nf · · · dξ1 ,2
Z(4.4.72)
1
1 T −1Dξ exp ζ T ξ + ξ T M ξ = Pf(M ) expζ M ζ .
2
2

One important advantage of using the fermionic coherent states is the simplification in certain calculations, such as
Z
h
i
T
Nf
(4.4.73)
Tr ÂB̂ = (−2)
DζDξeξ ζ Aξ Bζ ,where we assign the Grassmann-valued function Aξ for each operator Â via the following replacement of
the Majorana representation:
X
X
p2N
p2N
(4.4.74)Â =Ap1 ···p2Nf ψ̂1p1 · · · ψ̂2Nff → Aξ =Ap1 ···p2Nf ξ1p1 · · · ξ2Nff
{pi }

{pi }

with
pi ∈ {0, 1} .

(4.4.75)

Using this mapping between an operator and a Grassman-valued function, a Gaussian density operator
i T(4.4.76)ρ̂G = N exp − ψ̂ X ψ̂
4corresponds to the function(4.4.77)

ρG,ξ =

1
2Nf


exp


i T
ξ Γψ ξ ,2

where Γψ is the covariance matrix of ρ̂G (see Exercise).
Similar to the bosonic case, one can introduce the fermionic analog of the characteristic function as
follows:hi(4.4.78)
χA (ξ) ≡ Tr ÂD̂ξ = 2Nf Aξ/2 .
As inferred from the last equality here, the Grassmann-valued function introduced above itself is essentially the characteristic function aside constants. In particular, the characteristic function of a Gaussian
density operator is
i T(4.4.79)χρG (ξ) = exp ξ Γψ ξ .
8

4.4. FERMIONIC GAUSSIAN STATES74

One can also use this condition as the definition of a Fermionic Gaussian state in the similar way as wehave done in the bosonic case.
Using the characteristic function, one can expand a general operator acting on fermionic states as theintegral over Grassmann variables:Z(4.4.80)Â = Dζ χA (ζ)Êζ ,where(4.4.81)Êζ = 4Nf

Z

i

T

1

T

Dλe 8 ζ σζ− 2 λ ζ D̂λ |0⟩⟨0|D̂λ .

You can check this formula by using the relationhi(4.4.82)
Tr Êζ D̂ξ = δ (ζ − ξ) ≡ (ζ1 − ξ1 ) · · · (ζ2Nf − ξ2Nf ).
Here, δ(ζ − ξ) is the delta function of Grassmann variables. It is a good Exercise to check this relationfrom the equalitiesih(4.4.83)
Tr ÔD̂ξ |0⟩⟨0|D̂ξ′ = ⟨0|D̂ξ† ′ ÔD̂ξ |0⟩,1

T

(4.4.84)

D̂ξ D̂λ = D̂ξ+λ e 4 λ ξ ,(4.4.85)⟨0|D̂λ† D̂ξ+λ |0⟩ = e 4 λ ξ− 8 ξ σξ .

1

T

i

T

As an application of Fermionic coherent representations, let us derive some formulas we used before.
First, Wick’s theorem (4.4.28) can be derived as follows:Zh
p2N
p2Nf i
i T
T
p1 p2
Nf
(4.4.86)Tr ρ̂G ψ̂1 ψ̂2 · · · ψ̂2Nf = (−1)
DζDξeξ ζ e 2 ξ Γψ ξ ζ1p1 · · · ζ2Nff
Z
p2N
i T
T
Nf
(4.4.87)= (−1)
DξDζeξ ζ ζ1p1 · · · ζ2Nff e 2 ξ Γψ ξ
Z
P
1−p2N
i T
(4.4.88)= (−1) i pi /2 Dξξ11−p1 · · · ξ2Nf f e 2 ξ Γψ ξ
Z
P


i T
pi /2
i
(4.4.89)= (−1)
Dξe 2 ξ Γψ |P ξ = Pf (−iΓψ )|P .
Q2N
Second, the parity formula (4.4.47) can be obtained by using the expression P̂ = (−i)Nf ξ=1f ψ̂ξ andapplying this formula to the case of p1 = · · · = p2Nf = 1. Finally, the inner product formula (4.4.48) is4.4. FERMIONIC GAUSSIAN STATES

75

similarly obtained by(4.4.90)(4.4.91)(4.4.92)|⟨ψ1 |ψ2 ⟩|2 = Tr[ρ̂G,1 ρ̂G,2 ]
Z
i T
i T
T
−Nf
= (−2)
DζDξeξ ζ e 2 ξ Γψ,1 ξ e 2 ζ Γψ,2 ζ
Z
−1
i T
−Nf
= (−2)
Pf (iΓψ,2 ) Dξe 2 ξ (Γψ,1 −Γψ,2 )ξ

(4.4.93)= (−2)−Nf Pf (iΓψ,2 ) Pf (i(Γψ,1 + Γψ,2 ))(4.4.94)= (−2)−Nf P Pf (Γψ,1 + Γψ,2 )Here, we used the Gaussian integration formula (4.4.72), the parity formula P = (−1)Nf Pf(Γψ,1 ), andthe relation Γ−1ψ,2 = −Γψ,2 that holds true in a pure state.
4.4.3. Application to BCS superconductors. As an application of Fermionic Gaussian states, wehere briefly review the theory of BCS superconductors. To this end, consider the BCS model consistingof attractively interacting fermions:(4.4.95)Ĥ =X

ϵk ĉ†kσ ĉkσ −

kσ

g X †
ĉ ′ .
ĉ† ĉ ′ĉΩ ′ k+q↑ −k↓ −k +q↓ k ↑kk qPHere, ϵk = ℏ2 k 2 /(2m) − µ is the single-particle energy including the chemical potential and Ω = k 1
is the total mode number. The second term describes the effective attractive interaction arising from, e.g.,phonon-mediated interaction. The essential feature of the ground state of this model is the formation andcondensation of paring between two fermions with different spins (known as the Cooper pairs). This leads
to the emergence of the ODLRO in the following sense:
 2
N
1 X
†
†
(4.4.96)⟨ĉr1 ↑ ĉr2 ↓ ĉr3 ↓ ĉr4 ↑ ⟩ → Oĉkσ eik·r ,̸= 0, ĉrσ = √2VV kwhere the limit → represents the long-distance limit between (r 1 , r 2 ) and (r 3 , r 4 ). To capture this feature, one straightforward choice would be to take a quantum state of the following form (the momentum
conservation leads to the paring between k and −k):Y † † (4.4.97)
gk ĉk↑ ĉ−k↓ |vac⟩,kwhich consists of the products of two-fermion states. Despite the conceptual simplicity, this wavefunction
is not very useful especially when one wants to analyze certain physical quantities by calculating the
expectation values with respect to this state. This motivates us to construct a more simple and tractablevariational state, which still captures the same essential physical feature (i.e., nonvanishing ODLRO).
To do so, recall that the Gaussian state has nonvanishing expectation values of ⟨ĉ†↑ ĉ†↓ ⟩ or ⟨ĉ↓ ĉ↑ ⟩, which
can lead to nonzero ODLRO. The following fermionic Gaussian state is thus a natural candidate to describe
the ground-state physics of the BCS model:


P

Y
†
†
† †
BCS
k θk ĉk↑ ĉ−k↓ +ĉk↑ ĉ−k↓(4.4.98)|Ψvar ⟩ =cos θk + sin θk ĉk↑ ĉ−k↓ |vac⟩ = e|vac⟩.
k4.4. FERMIONIC GAUSSIAN STATES76

Indeed, this state features the ODLRO(4.4.99)⟨ĉ†r↑ ĉ†r↓ ĉ0↓ ĉ0↑ ⟩ → ⟨ĉ†r↑ ĉ†r↓ ⟩⟨ĉ0↓ ĉ0↑ ⟩ =

(

 2
2
N
k sin θk cos θk )
,
∼
O
V2
V2

P

and thus captures the condensation of the Cooper pairs. While |ΨBCS
var ⟩ is not an eigenstate of the particle number, this does not cause problems in the variational analysis, since (similar to the coherent-state
description of the BEC discussed before) the fluctuations can be neglected in the thermodynamic limit.
The variational parameters θk of |ΨBCS
var ⟩ are optimized by minimizing the expectation value of thetotal energy:
BCSmin⟨ΨBCS
var |Ĥ|Ψvar ⟩.

(4.4.100)θk

Since the expectation values can be decomposed into the products of two-point correlation functions via
Wick’s theorem, one can check that this procedure is equivalent to the usual mean-field analysis. To see
this, let us follow the standard mean-field procedure; we expand the following operator around the mean
field ∆k ,
ˆ k , ∆k ≡ ⟨ĉ−k↓ ĉk↑ ⟩ ∈ R,ĉ−k↓ ĉk↑ ≡ ∆k + δ ∆(4.4.101)and neglect the higher-order terms of the fluctuations δ∆k . The resulting mean-field Hamiltonian is(4.4.102)

ĤMF =X¯ 2 − g∆¯ϵk ĉ†kσ ĉkσ − gΩ∆X † †ĉk↑ ĉ−k↓ + ĉ−k↓ ĉk↑ ,kkσwhere we define
¯ ≡ 1
∆
Ω

(4.4.103)

X

∆k .

k

One can diagonalize the quadratic Hamiltonian ĤMF by using the following unitary transformation (cf.
Eq. (4.4.7)):
!
!
!
!
ĉk↑
ĉk↑
dˆk↑
cos θk − sin θk†Û =(4.4.104)= Û
ĉ†−k↓dˆ†−k↓ĉ†−k↓sin θk cos θkwith(4.4.105)Û = e

−

P

k θk



ĉ†k↑ ĉ†−k↓ +ĉk↑ ĉ−k↓,where the parameters are defined byrrq
Ek + ϵkEk − ϵk¯ 2 + (ϵk − µ)2 .
(4.4.106)cos (θk ) =, sin (θk ) =, Ek = ∆2Ek
2Ek
The diagonalized Hamiltonian is(4.4.107)

ĤMF = const. +Xkσ

Ek dˆ†kσ dˆkσ .

4.5. VARIATIONAL PRINCIPLE

77

Thus, the mean-field ground state is given by the vacuum of dˆ operators|ΨBCS
MF ⟩ = |vac⟩d ,(4.4.108)

which corresponds to the squeezed vacuum in the original ĉ operators:(4.4.109)†|ΨBCS
MF ⟩ = |vac⟩d = Û |vac⟩
P

k θk



ĉ†k↑ ĉ†−k↓ +ĉk↑ ĉ−k↓

=e
|vac⟩

Y
=
cos θk + sin θk ĉ†k↑ ĉ†−k↓ |vac⟩(4.4.110)(4.4.111)k

Indeed, this state satisfies(4.4.112)dˆkσ |vac⟩d = Û † ĉkσ Û Û † |vac⟩ = Û † ĉkσ |vac⟩ = 0.

BCS
The mean-field ground state |ΨBCS
MF ⟩ is nothing but the the Gaussian variational state |Ψvar ⟩ in Eq.
(4.4.98).
Finally, we note that the variational parameters θk are determined by the following self-consistentcondition
X
1X
1X
¯ ≡ 1
∆k =
⟨ĉ−k↓ ĉk↑ ⟩ =sin θk cos θk ,(4.4.113)
∆
Ω
Ω
Ω
k

k

k

leading to(4.4.114)

¯ = 1
∆
Ω

X
k

¯
g∆
q
.
¯ 2 + ϵ22 ∆k¯ ̸= 0), we arrive at
In the superconducting phase (∆g
1X
q(4.4.115)1=.
Ω¯ 2 + ϵ22 ∆kkAgain, from a perspective of Gaussian states, these mean-field procedures are equivalent to finding the
optimal values of θk (or said differently, the covariance matrix) of the Gaussian variational state thatminimize the total energy. In the next section, we will develop a more systematic variational approach toperform the optimization over the entire Gaussian manifold. For this purpose, we first need to introduce
the concept of the variational principle.

4.5. Variational principle
4.5.1. Introduction. The difficulty of exactly solving quantum many-body problems arises from thevast Hilbert space that cannot (in general) be handled by classical computers. Unless a faithful quantum
computer is realized, the best we can do at this stage would be to design a wavefunction that avoids thedifficulty of exponential divergence while still captures essential physical features in a system of interest.
Suppose that we can construct such a class of variational states that possess both the efficiency andflexibility. To apply them to actual physical systems, we want to obtain the approximative solutions within

4.5. VARIATIONAL PRINCIPLE

78

this variational subspace that represent either the ground state or the time-evolved state in nonequilibrium
situations. How can one optimize the parameters of those variational states for these purposes?
The time-dependent variational principle (TDVP) provides a general and systematic framework toachieve this aim, which is the main subject of this section. The idea of the variational principle plays animportant role also in the context of certain quantum algorithms including Variational Quantum Eigensolver (VQE) on NISQ devices, and more generally, in the field of classical machine learning.
Below we will first review a general theory of the TDVP for both imaginary and real time evolutions;the former can be used to analyze the ground state while the latter is applicable to study nonequilibrium
dynamics. We then apply this formalism to Bosonic and Fermionic Gaussian states introduced above.

4.5.2. Complex-valued variational manifold. We first consider a variational state |Ψz ⟩ that is parameterized by complex parameters z. We assume that it is always normalized as ⟨Ψz |Ψz ⟩ = 1 below. If
we naively write down the time-evolution equation, we getd?
|Ψ ⟩ = −iĤ|Ψz(t) ⟩.
dt z(t)(4.5.1)d
The left-hand side is given by the derivative of the variational state, dt|Ψz(t) ⟩, and thus lies in the restrictedsubspace of the whole Hilbert space. Meanwhile, on the right hand side, the Hamiltonian in generalcontains complex interaction terms and thus the state −iĤ|Ψz(t) ⟩ can lie outside the restricted subspacecorresponding to the left hand side. This means that one cannot equate these two states in general. Said
differently, the vector on the left hand side belongs to the tangential space of the variational manifold,while that on the right hand side in general does not.
Thus, we have to approximate the time-evolution equation on the variational manifold in some way.
For this purpose, the best approximation can be made by minimizing the following quantity:d(4.5.2)min+ iĤ |Ψz(t) ⟩ .
dtz(t)From a geometric point of view, this is equivalent to the following condition:
d(4.5.3)|Ψ ⟩ = P̂∂ −iĤ |Ψz(t) ⟩dt z(t)where P̂∂ is the projection of a state vector onto the tangential space Tz(t) of the variational manifold,no∂|Ψz(t) ⟩which is a complex vector space spanned by the basis vectors. More explicitly, we can express
∂zi
it as
P̂∂ |Ψ⟩ ≡ arg min ∥|Ψ⟩ − |v⟩∥

(4.5.4)|v⟩∈Tz((4.5.5)Tz ≡|v⟩ |v⟩ =Xi∂|Ψz ⟩ci, ci ∈ C∂zi

Equation (4.5.3) is known as the McLachlan variational principle.

).

4.5. VARIATIONAL PRINCIPLE

79

Similarly, the minimization condition (4.5.2) can also be satisfied by ensuring the following equality:d|Ψ ⟩ = P̂∂ Ĥ|Ψz(t) ⟩.
dt z(t)This is known as the action variational principle. To see this, recall that the action can be defined byZd(4.5.7)
S = dt L, L = ⟨Ψz∗ (t) | i − Ĥ |Ψz(t) ⟩,
dt
P̂∂ i

(4.5.6)and consider the variation δz∗ S with respect to the change z ∗ → z ∗ + δz ∗ :X∂⟨Ψz∗ (t) |d∗δz ∗ S =δzii − Ĥ |Ψz(t) ⟩ = 0, ∀δzi∗(4.5.8)
∂zi∗
dt
i


d
(4.5.9)⇐⇒ P̂∂ i − Ĥ |Ψz(t) ⟩ = 0.
dtThis indeed gives the variational principle in Eq. (4.5.6).
We note that, in the present complex-valued case, the McLachlan and action variational principles areequivalent. The reason is that the tangential space Tz is invariant under the multiplication by i, i.e., theoperations P̂∂ and i commute with each other [P̂∂ , i] = 0.
The resulting time evolution equation of the variational parameters z(t) can be obtained by multiply∂|Ψz(t) ⟩ing ∂zi from left in Eq. (4.5.3):
X(4.5.10)gij

j(4.5.11)

gij ≡

dzj
= −ihi ,dt∂⟨Ψz∗ (t) | ∂|Ψz(t) ⟩∂⟨Ψz∗ (t) |Ĥ|Ψz(t) ⟩.
,
h
=
i
∂zi∗
∂zj
∂zi∗

4.5.3. Real-valued variational manifold. We next consider a variational state |Ψx ⟩ that is parameterized by real variables x; it turns out that one has to be careful about a choice of the variational principles
in this case. As done before, we consider the minimization problem

min(4.5.12)x(t)d+ iĤ |Ψx(t) ⟩ .
dtThis leads to the two variational principles as discussed above, which are distinct in the present casebecause the operations P̂∂ and i do not commute with each other [P̂∂ , i] ̸= 0 in general. Indeed, thetangential
nospace of real-valued variational manifold is the real vector space spanned by the basis vectors∂|Ψx(t) ⟩and thus∂xi(4.5.13)P̂∂ |Ψ⟩ = arg min ∥|Ψ⟩ − |v⟩∥|v⟩∈Tx((4.5.14)Tx ≡|v⟩ |v⟩ =Xi∂|Ψx ⟩ai, ai ∈ R∂xi)

It is clear that this tangential space Tx is not necessarily invariant under the multiplication by i, i.e., thevector i|v⟩ may lie outside of Tx .

4.5. VARIATIONAL PRINCIPLE

80

Keeping this important point in mind, let us first consider the McLachlan variational principle:
d(4.5.15)|Ψx(t) ⟩ = P̂∂ −iĤ |Ψx(t) ⟩.
dt
∂|Ψ

⟩

from the left and taking the real parts, we get the time-evolution equations of theMultiplying ∂zx(t)ivariational parameters for this variational principle:
X dxj(4.5.16)gR= Re [−ivi ] ,ijdtj(4.5.17)

R
gij
≡ Re




∂⟨Ψx(t) |∂⟨Ψx(t) | ∂|Ψx(t) ⟩, vi =
Ĥ|Ψx(t) ⟩.
∂xi
∂xj
∂xi

Meanwhile, the action variational principle leads to(4.5.18)
Multiplying(4.5.19)P̂∂ id|Ψ ⟩ = P̂∂ Ĥ|Ψx(t) ⟩.
dt x(t)∂|Ψx(t) ⟩from the left and taking the real parts, at this time we get
∂zi

X

I
gij

j

(4.5.20)

I
≡ Im
gij



dxj
= −Re[vi ],dt∂⟨Ψx(t) | ∂|Ψx(t) ⟩
.
∂xi
∂xj

Note that these two variational time-evolution equations (4.5.16) and (4.5.19) are in general inequivalent.
The two variational principles we encountered above have both advantages and shortcomings. Specifically, in the McLachlan variational principle the symmetries and the corresponding conservation laws of
the system are consistently satisfied during the variational time evolutions. To see this, suppose that wehave Ô such that [Ô, Ĥ] = 0 and Ô|ψ⟩ belongs to the tangential space. Then, its expectation value evolves
as(4.5.22)d⟨Ô⟩x(t) = ⟨Ψx(t) |(iĤ)P̂∂ Ô − ÔP̂∂ (iĤ)|Ψx(t) ⟩dt= ⟨Ψx(t) |(iĤ)ÔP̂∂ − P̂∂ Ô(iĤ)|Ψx(t) ⟩(4.5.23)= i⟨Ψx(t) |[Ĥ, Ô]|Ψx(t) ⟩ = 0,

(4.5.21)

showing that Ô is conserved during the variational evolution. However, we emphasize that one cannot ingeneral choose the Hamiltonian Ĥ itself as Ô since Ĥ may not keep the tangential space invariant (i.e.,[Ĥ, P̂∂ ] ̸= 0). This means that the McLachlan variational principle in general does not conserve the totalenergy.
In contrast, one can show that the action variational principle always conserves the total energy, while
it does not satisfy the other possible symmetries and conservation laws in general (see Exercise).
4.5.4. Imaginary time evolution. Let us next consider the imaginary-time evolution:

4.5. VARIATIONAL PRINCIPLE(4.5.24)d|Ψτ ⟩ = − Ĥ − ⟨Ĥ⟩τ |Ψτ ⟩dτ

(4.5.25)⇐⇒ |Ψτ ⟩ =e−Ĥτ |Ψτ ⟩81.

e−Ĥτ |Ψτ ⟩This is especially useful when one is interested in the ground-state physics, since the state converges to
the ground state in τ → ∞ if the initial state has nonzero overlap with the ground state.
We want to find the best approximation for the imaginary-time evolution within a certain variational
manifold. This can be achieved by the following minimization:d+ Ĥ − Evar |Ψx(τ ) ⟩(4.5.26)
min
dτ
x(τ )
(4.5.27)

Evar = ⟨Ψx(τ ) |Ĥ|Ψx(τ ) ⟩.

Note that in the present case there is no i factor (as we found for real-time evolution) and thus the McLachlan and action principles are equivalent and lead to the same results for both real-valued and complexvalued variational states.
To be concrete, here we consider the McLachlan variational principle leading tod(4.5.28)|Ψx(τ ) ⟩ = P̂∂ Evar − Ĥ |Ψx(τ ) ⟩ = −P̂∂ Ĥ|Ψx(τ ) ⟩.
dτ
Here we use P̂∂ |Ψx ⟩ = 0 that follows from the definition (4.5.13). The variational imaginary time∂|Ψ⟩evolution of the variational parameters are given by multiplying ∂xx(t)from left and taking the realiparts:
X dxj(4.5.29)gR= −Re [vi ] .
ijdτj

R , one can show that the variational energy monotonically decreases during
Because of the positivity of gij
the imaginary time evolution:(4.5.30)

dEvar
≤ 0.
dτ

R has the inverse, we can rewrite the time evolution equation as
If gij(4.5.31)

X

dxj
=−
(g R )−1 ji Re [vi ] .
dτiIt is worthwhile to note that this equation can be considered as the generalized stochastic gradient descenton the curved space whose metric is given by g R . In the fields of machine learning or information geometry, this gives an analogue of the so-called Natural Gradient descent or the Riemannian gradient descent.
For the sake of comparison, we recall that the simplest stochastic gradient method is defined by(4.5.32)

xj (τ + δτ ) = xj (τ ) − ηδτ

∂
⟨Ĥ⟩τ
∂xj

4.5. VARIATIONAL PRINCIPLE

82

which is widely used especially in the field of machine learning. When the metric is not flat, the modified
gradient descent (4.5.31) can in practice converge more efficiently than the simplest form of the stochastic
gradient descent.
We note that, in the imaginary time evolution, energy or other quantities are not conserved in generaleven if there exist certain symmetries. However, (if necessary) one can ensure a conservation of, e.g.,
particle number by further restricting the projection operation P̂∂ onto a certain subspace associated witha conserved quantity at each time step. In practice, this can be achieved by adding the penalty term to the
Hamiltonian (e.g., like λ(N̂ − N0 )2 for the particle number conservation with N0 being the initial particle
number).

4.5.5. Application to Gaussian states. It is a good demonstration to apply the time-dependent variational principle above to the case of Gaussian variational states. First, we recall that pure Fermionic
Gaussian states are in general expressed by(4.5.33)n o  Xξηψ̂ξ ψ̂η
2|0⟩, Xξη ∈ R.
|ψ⟩ = ÛG ψ̂ξ , e

In particular, if the Gaussian state is given by using only the rotations, we can express it as(4.5.34)1T|ψ⟩ = e 4 ψ̂ X ψ̂ |0⟩, X ∈ R2Nf ×2Nf , X = −X T ,whose covariance matrix is given by(4.5.35)Γψ = −eX σe−X , σ =

0
−INf

INf
0

!
.

Several remarks are in order. First, it is redundant to use X as variational parameters of Fermionic
Gaussian states. It is the covariance matrix Γψ that completely characterizes Gaussian states without unnecessary redundancies. Indeed, one can check that different choices of X can lead to the same Γψ in Eq.
(4.5.35). Second, while Gaussian variational states are parametrized by real variables, the corresponding
tangential space satisfies [P̂∂ , i] = 0 and thus the McLachlan and action variational principles are equivalent. Consequently, energy and other conserved quantities (if any) are consistently conserved during thevariational time evolutions. To see this explicitly, we can obtain the tangential space of the Gaussian states
by taking the derivative with respect to the variational parameters Γψ . One can show that the result is




Nf


X
(4.5.36)
Tf ≡ |v⟩ |v⟩ = ÛG 
cij ĉ†i ĉ†j  |0⟩, cij ∈ C ,


1≤i<j

which is invariant under the multiplication by i, leading to [P̂∂ , i] = 0. The same applies to Bosonic
Gaussian states.
Let us now derive the time-evolution equations of the variational parameters Γψ . While this can bedone by using Eq. (4.5.16), here let us follow a more direct procedure. To this end, we represent a pureFermionic Gaussian state by4.5. VARIATIONAL PRINCIPLE(4.5.37)83|ψt ⟩ = ÛG |0⟩, ÛG† ψ̂ ÛG = Oψ̂, Γψ = −OσOT , OOT = I2Nf ,and note the following relation (you can check this by expanding Ĥ in terms of ψ̂):


i T T
P̂∂ Ĥ|ψt ⟩ = P̂∂ ÛG(4.5.38)
ψ̂ O HOψ̂ + δ̂ |0⟩
4


i T T
= ÛG : ψ̂ O HOψ̂ : |0⟩,(4.5.39)4H≡4(4.5.40)

δEvarδ⟨Ĥ⟩G
=4
,
δΓψ
δΓψ

where : : represents the normal order (i.e., : Â :≡ Â − ⟨0|Â|0⟩), while δ̂ denotes higher-order terms of ψ̂that is projected out by P̂∂ . Using these equations, we obtain the time evolution equation of the covariancematrix by(4.5.41)(4.5.42)d(Γψ )ξηid|ψt ⟩= ⟨ψt |[ψ̂ξ , ψ̂η ]+ c.c.
dt
2 h
dti= Re ⟨0|ÛG† [ψ̂ξ , ψ̂η ]P̂∂ Ĥ|ψt ⟩
 

 

i T
= Re ⟨0|[ Oψ̂ , Oψ̂ ] : ψ̂ OT HOψ̂ : |0⟩
4
ξ
η
i
h
= −Re (i (1 − iΓψ ) H (1 + iΓψ ))ξη .

(4.5.43)(4.5.44)

Here we used Wick’s theorem to derive the last equality:
⟨0|ψ̂α ψ̂β |0⟩ = I2Nf + iσ αβ .

(4.5.45)
We then arrive at the final resultdΓψ= HΓψ − Γψ H
dtwhich gives the variational real time evolution of the Fermionic Gaussian states. The imaginary time
evolution can also be obtained in the similar manner, resulting in(4.5.46)dΓψ= −Im [i (1 − iΓψ ) H (1 + iΓψ )]dτ= −H − Γψ HΓψ .

(4.5.47)(4.5.48)

We next consider a pure Bosonic Gaussian state labelled byi(4.5.50)(4.5.51)TiT|ϕ⟩ = ÛG |0⟩ = e 2 ϕ̂ σ∆ e− 4 ϕ̂ Rϕ̂ |0⟩,(4.5.49)
σ=

0
−INb

INb
0

!
, ∆ ∈ R2Nb , R ∈ R2Nb ×2Nb R = RT ,Γϕ = SS T , S = eσR , SσS T = σ.

4.6. SUPERCONDUCTING QUBITS

84

Similar to the fermionic case, we have to keep in mind that the variables (∆, Γϕ ) are the correct choice asvariational parameters (while R is redundant). The tangential space is then given by




Nb
Nb


X
X
cij b̂†i b̂†j +ci b̂†i  |0⟩, cij ∈ C(4.5.52)
Tb ≡ |v⟩ |v⟩ = ÛG 


1≤i≤j

i=1

which satisfies [P̂∂ , i] = 0.
Following the similar steps above for the fermionic case, one can obtain the real time evolution equations. The results are (see Exercise):(4.5.53)(4.5.54)dΓϕd∆= σH∆ ,= σHΓ Γϕ − Γϕ HΓ σ,dtdt
H∆ ≡ 2

δEvar
δEvar, HΓ = 4.
δ∆δΓϕThe imaginary time evolution is given by(4.5.55)(4.5.56)dΓϕd∆= −Γϕ H∆ ,= −σHΓ σ − Γϕ HΓ ΓϕdτdτH∆ ≡ 2

δEvar
δEvar, HΓ = 4.
δ∆δΓϕThese sets of equations of motion are exact if the Hamiltonian is quadratic, while give approximative
solutions when there exist interaction terms. One can use the imaginary time evolution and take τ → ∞to obtain the variational ground state, while the real time evolution allows us to analyze nonequilibrium
dynamics of the system.

4.6. Superconducting qubits
4.6.1. LC circuits. We next briefly discuss the physics of superconducting qubits, one of the mostpromising building blocks toward realizing quantum computers. To begin with, imagine that we haveelectrical circuits at low temperature that is well below the critical temperature Tc of the superconductor.
Then, the resistance R goes down to zero and, as long as the circuit is well isolated from environmental
degrees of freedom, the circuit can be operated in quantum regime.
It is such situation that we are interested in here, and our starting point is the following quantized
Hamiltonian of a simple LC circuit:(4.6.1)

Ĥ =

Q̂2
1
1
+ Cω 2 Φ̂2 , ω ≡ √
,
2C
2
LC

where Q̂ is the charge on the capacitor and Φ̂ is the flux in the inductor, which are the canonically conjugate
variables satisfying(4.6.2)[Φ̂, Q̂] = iℏ.

4.6. SUPERCONDUCTING QUBITS

85

In analogy with the harmonic oscillator, Φ̂ and Q̂ can be considered as the analogues of position and momentum operators, respectively. This equivalence motivates us to introduce the annihilation and creation
operators by
r
r
ℏ
ℏωC †
(4.6.3)Φ̂ =(â + â† ), Q̂ = i(â − â),2ωC2
(4.6.4)[â, â† ] = 1.
The resulting Hamiltonian is1†Ĥ = ℏω â â +.
2

(4.6.5)

From this expression, it is now clear that an excitation of quantized LC circuit corresponds to an electromagnetic elementary excitation (a.k.a photon) of frequency ω, which is typically in the microwave
regime.
4.6.2. Josephson junction. As shown above, the LC circuit is equivalent to the harmonic oscillator
corresponding to a quadratic Hamiltonian, which means that at most one can only perform a Gaussian
operation that essentially generates a linear transformation of photon modes (see the discussions in theprevious sections). We want to introduce nonlinearity to realize circuit operations that are inaccessible
in the linear regime. In fact, it is such nonlinearity that eventually allows one to implement effectively
two-level systems, namely, qubits.
For this purpose, we shall explain about the Josephson junction that effectively replaces the linear
inductance in the circuit by the nonlinear inductance corresponding to the energy:
!
1
2π Φ̂
2 2
(4.6.6)Cω Φ̂ → −EJ cos, Φ0 = h/2e.
2Φ0
To derive this result, we recall the BCS theory of superconductors explained before; the superconducting
phase was well captured by the ground state of the following quadratic Hamiltonian:X † †X¯ ∗ ĉ−k↓ ĉk↑ ,
¯ 2−g
¯
+
∆
ĉ
∆ĉ
(4.6.7)
ĤMF =
ϵk ĉ†kσ ĉkσ − gΩ|∆|k↑ −k↓kkσwhere at this time we take the order parameter to be a generic form with a phase:¯ = |∆|e¯ iφ , φ ∈ [0, 2π).
∆(4.6.8)Its ground state is given by the vacuum of the quasiparticle operator dˆk and can be expressed as theGaussian state in terms of ĉk operators as follows:(4.6.9)|Ψφ ⟩ =Ycos θk + sin θk eiφ ĉ†k↑ ĉ†−k↓ |vac⟩,kwhich is obviously 2π-periodic(4.6.10)|Ψφ ⟩ = |Ψφ+2π ⟩.

4.6. SUPERCONDUCTING QUBITS

86

We then note that its Fourier transformation gives the projection onto the eigenspace of total number of
Cooper pairs with eigenvalue N :Z 2π
dφ −iN φ(4.6.11)|ΨN ⟩ =e|Ψφ ⟩ = P̂N |Ψφ=0 ⟩, N ∈ Z.
2π
0
We may consider the variational superconductor wavefunctions, |Ψφ ⟩ and |ΨN ⟩, as effective “position”
and “momentum” eigenstates of the one-body wavefunction on the circle φ ∈ [0, 2π). To obtain the φbasis representation of the operator N̂ , we can use the relation
Z 2π(4.6.12)

N |ΨN ⟩ =
0

dφ
2π






Z 2π
d −iN φdφ −iN φdi e|Ψφ ⟩ =
e−i |Ψφ ⟩ ,
dφ
2π
dφ
0

where we used the fact that N is an integer number and |Ψφ ⟩ is 2π periodic. One may thus identify as
N → −i(4.6.13)d.
dφPrecisely speaking, this identification is equivalent to the quantization procedure associated with the following operators
N̂ , T̂ = e−iφ ,(4.6.14)which satisfy the commutation relation[T̂ , N̂ ] = T̂ .

(4.6.15)

One can check that this leads to the N basis representation of the operator T̂ :T̂ =(4.6.16)X|N − 1⟩⟨N |.

N

Note that the phase φ itself cannot be an operator while T̂ is, since the phase must satisfy the 2π periodicity3. Physically, N̂ can be interpreted as the number of Cooper pairs in the superconductor, each of which
consists of two electrons as noted above. Then, the role of T̂ is to increase the number of Cooper pairs byone.
Suppose now that we have two BCS superconductors and they are linked by a thin insulator; then,electron pairs can tunnel across the potential barrier with certain nonvanishing probabilities. This tunneling is, at the lowest order process, described by the following effective Hamiltonian:(4.6.17)ĤJ = −


EJ  †
T̂1 T̂2 + H.c. = −EJ cos φ, φ ≡ φ1 − φ2 .
2

3Suppose that we can treat the phase as an operator φ̂ that obeys [φ̂, N̂ ] = i. However, we get ⟨Ψ |[φ̂, N̂ ]|Ψ ⟩ = (N −NNN )⟨ΨN |φ̂|ΨN ⟩ = 0, which contradicts with the commutation relation.

4.6. SUPERCONDUCTING QUBITS

87

The variable φ describes the phase difference between two conductors and we have EJ > 0 which meansthat the phase match is energetically favored. The transfer of the Cooper pairs will also lead to the nonzero
electric dipole between the superconductors that induces the electrostatic energy, which can be describedby
2ĤC = 4EC N̂ − Ng ,(4.6.18)

where EC = e2 /2C is the capacitance energy with C being the total capacitance of the circuit, the factorof 4 comes from the fact that the Cooper pair has charge 2e, and Ng is the global charge bias in the unitof 2e. The resulting total Hamiltonian of the Josephson junction is(4.6.19)

2
ĤJJ = ĤC + ĤJ = 4EC N̂ − Ng − EJ cos φ.

To make a comparison with the circuit Hamiltonian discussed before, it is also useful to express ĤJJ in
terms of the charge and flux variables as
(4.6.20)(4.6.21)ĤJJ =Q̂ − 2eNg

22CQ̂ = 2eN̂ , Φ =− EJ cos

2πΦ
Φ0


,

Φ0
ℏ
φ = φ,
2π2ewhich explains the replacement in Eq. (4.6.6).
4.6.3. Superconducting qubits: realizing effectively two-level systems. There are in fact several
ways to realize effectively two-level systems (qubits) by operating the Josephson junction in different
parameter regimes. As a first approach, we shall consider the regime in which the charge energy isdominant EC ≫ EJ (the so-called Cooper-pair-Box regime). It is then useful to work in the N basis,
leading to the Hamiltonian(4.6.22)
2 E XJĤJJ = 4EC N̂ − Ng −(|N ⟩⟨N + 1| + H.c.) .
2
N

At EJ = 0, the energy eigenstates are given by the charge eigenstates {|N ⟩} and the ground and firstexcited states correspond to the ones that provide the two lowest values of (N − Ng )2 . Without Josephsonenergy EJ , this setup merely leads to the equal energy spacing that is unfavorable for the purpose ofrealizing qubits, since the resonant operation inevitably induces the transitions between many different
levels besides the lowest two levels we are interested in. With the nonlinear term EJ > 0, however,energy levels are now inhomogeneously shifted and the energy degeneracies can be lifted, allowing one
to use the two lowest levels as effective qubits. This type of qubits is known as the charge qubits.
In practice, the charge qubit is sensitive to the environmental voltage noise that comes through thefluctuations of Ng , which ultimately leads to the decoherence. To circumvent this difficulty, one possible
way is to work in another regime EC ≪ EJ (the so-called transmon regime). In the language of the4.6. SUPERCONDUCTING QUBITS

88

one-body wavefunction, this corresponds to the deep potential (or equivalently large mass) that associateswith the small “position” fluctuations ⟨φ2 ⟩ − ⟨φ⟩2 ≪ 1. This allows us to treat the phase variable φ as
an effectively continuous variable without 2π-periodicity restriction. Expanding the cos(φ) term, we thenarrive at the effective Hamiltonian,
2 1
1(4.6.23)ĤJJ ≃ 4EC N̂ − Ng + EJ φ̂2 − EJ φ̂4 ,24!
which is equivalent to the weakly anharmonic oscillator. Due to this anharmonicity, the energy degeneracies are lifted and thus one can use the two lowest energy eigenstates as an effective qubit; a quantum
operation can be performed by using a process that is only resonant to the first excitation energy. This
type of qubits is known as the transmon qubits.
4.6.4. Light-matter interaction: circuit QED. Finally, we shall briefly mention how superconducting qubits can be electromagnetically coupled to microwave photons. To be concrete, suppose that we
capacitively couple the Josephson junction to the LC circuit discussed at the beginning of this section.
The corresponding Hamiltonian can be simply obtained by replacing the (normalized) gate voltage Ng bythat of the LC circuit N̂g , resulting in(4.6.24)
2
Ĥ = 4EC N̂ − N̂g − EJ cos φ + ℏωr â† â,

where we definer
Cg Q̂Cg ℏω †
(4.6.25)N̂g ==i(â − â).
C 2e
2e 2C
Here, we used Eq. (4.6.3) and Cg is the gate capacitance connecting the Josephson junction with the LC
circuit. While we here discuss the coupling to only a single electromagnetic mode (as realized in cavity
resonators), more generally, one can couple the Josephson junction to continuum number of electromagnetic modes by using a certain type of resonators such as waveguides.
These systems offer an ideal platform to study the physics of light-matter interaction, namely, theinteraction between collective excitations of electrons (matter) in the superconductors and quantized microwave photons (light) in the LC circuit. This subfield of quantum optics is often referred to as circuit
quantum electrodynamics (QED) together with cavity / waveguide QED; we will give a short introduction
for this topic in the next Chapter.

4.6. SUPERCONDUCTING QUBITS

89

Summary of Chapter 4
Section 4.2 Quantization of the electromagnetic field• The free quantum electromagnetic Hamiltonian is equivalent to a collection of independent harmonic oscillators with eigenfrequencies {ωn }, which are determined by solving theHelmholtz equation, (∇2 + ω 2 /c2 )Aω = 0, under certain boundary conditions. The corresponding elementary excitation is called photon.
• In nonrelativistic quantum electrodynamics, the Coulomb gauge is commonly used and leads
to the unusual commutation relation between the vector potential and electromagnetic fields
with transverse delta function, which apparently violates the locality. This indicates that thevector potential is not a local observable, but still may influence global property like in the
Aharanov-Bohm effect.
Section 4.3 Bosonic Gaussian states• A bosonic quantum many-body state is called Gaussian state when it is completely characterized by only the correlation functions up to the second-order moment. Time evolution generated by a quadratic bosonic Hamiltonian, such as the free quantum electromagnetic fields, isgiven by a bosonic Gaussian unitary.
• When a bosonic Gaussian state is used as a variational ansatz for analyzing weakly interacting
BEC, it naturally incorporates the usual treatment including Bogoliubov or Gross-Pitaevskii
theory.
Section 4.4 Fermionic Gaussian states• A fermionic Gaussian state is completely characterized by the two-point correlation function.
Examples include the Slater determinant and BCS states. Time evolution generated by aquadratic fermionic Hamiltonian is given by a fermionic Gaussian unitary.
• A coherent state can be constructed also in fermionic systems and is useful for the purpose ofobtaining several formulas of Gaussian states.
Section 4.5 Variational principle• Time-dependent variational principle (TDVP) gives an optimal way to approximate the exact
time evolution on the variational manifold at each local time. Variational imaginary time
evolution can be viewed as natural gradient descent on the variational manifold.
• When applied to Gaussian variational states, TDVP allows us to derive a set of equations of
motion that can be used to analyze ground-state properties or nonequilibrium dynamics of
many-body systems.
Section 4.6 Superconducting qubits• When two BCS superconductors are linked by a thin insulator (Josephson junction), its effective Hamiltonian essentially reduces to that of a single quantum particle subject to periodic
cosine potential.
• The nonlinear inductance in Josephson junction leads to unequal energy spacing and allows
one to use the two lowest energy eigenstates as effectively two-level systems, qubits. Depending on the parameter regimes, they are often called charge or transmon qubits.

4.7. EXERCISES

90

4.7. Exercises
Exercise 4.1 (Field commutation relations: 1 point). Calculate the following unequal-time commutation relations among Ê and B̂ in free space:(4.7.1)[Êα (r, t), Êβ (r′ , t′ )](4.7.2)[B̂α (r, t), B̂β (r′ , t′ )](4.7.3)[Êα (r, t), B̂β (r′ , t′ )]Explain how one can interpret the results in terms of the light cone at |r − r′ | = c|t − t′ |.

Exercise 4.2 (Single-mode bosonic pure Gaussian state: 1 point). Consider the following singlemode pure bosonic Gaussian state:(4.7.4)

iθ

i

2

2

ir

|ψ(ϕ1 , ϕ2 , θ, r)⟩ = e 2 (x̂ϕ2 −p̂ϕ1 ) e 4 (x̂ +p̂ ) e− 4 (x̂p̂+p̂x̂) |0⟩, ϕ1,2 , θ, r ∈ R.

Derive the expression of the covariance matrix in terms of the real parameters ϕ1,2 , θ, r:(4.7.5)no1(Γϕ )ξη ≡ ⟨ψ| δ ϕ̂ξ , δ ϕ̂η |ψ⟩, δ ϕ̂ = ϕ̂ − ϕ, ξ, η ∈ {1, 2} ,2where ϕ̂ = (x̂, p̂)T . Similarly, derive the expression of the characteristic function in terms of ϕ, θ, r, ξand check the following relation:


1 T T
i T
(4.7.6)χψ (ξ) ≡ ⟨ψ|D̂ξ |ψ⟩ = exp − ξ σ Γϕ σξ + ϕ σξ , ξ ∈ R2 .
8
2

Exercise 4.3 (McCoy’s formula: 1 point). Show McCoy’s formula for the operators x̂, p̂ satisfying[x̂, p̂] = 2i:m(4.7.7)

1 X[x̂m p̂n ]S = m
2
l=0

m
l

!
l n m−lx̂ p̂ x̂
∂ ∂= exp −ix̂m p̂n .
∂ x̂ ∂ p̂

Here, [· · · ]S represents the totally symmetric ordering (for example, [x̂i p̂j ]S = (x̂i p̂j + p̂j x̂i )/2).

Exercise 4.4 (Bosonic Gaussian states: 2 points).
• Express the Wigner function (4.3.34) as the integral over Nb real variables (cf. Eq. (3.9.4)) [youcan replace the trace Tr[· · · ] by the integral over position variables x and perform the integration
over 2Nb variables ξ].

4.7. EXERCISES

91• Derive the expression of the overlap between two Bosonic Gaussian states Tr[ρ̂1 ρ̂2 ]; you may
use the following formula for a real, symmetric, positive definite matrix A and real vector v:
sZ
1 T(2π)N 1 vT A−1 v
TdN r e− 2 r Ar+v r =(4.7.8)e2.
det(A)RN
Show that the obtained formula in particular leads to the simplified expression of the puritypTr[ρ̂2 ] = 1/ det(Γϕ ) of a Bosonic Gaussian state.
Exercise 4.5 (Generalized uncertainty relation: 1 point). Show the generalized uncertainty relation
for a general (i.e., not necessarily Gaussian) density operator ρ̂ of bosons:Γϕ + iσ ≥ 0,

(4.7.9)by using the bosonic commutation relations:(4.7.10)hiϕ̂ξ , ϕ̂η = 2iσξη , σ ≡ iσ y ⊗ INb =

0
−INb

INb
0

!
.

h n
oi
is the covariance matrix of ρ̂. In particular, when ρ̂ is a bosonicHere, (Γϕ )ξη = 12 Tr ρ̂ δ ϕ̂ξ , δ ϕ̂ηGaussian state, show that the condition (4.7.9) is equivalent to the positivity of ρ̂.

Exercise 4.6 (Fermionic Gaussian states: 2 points). Consider
a general multi-mode Fermionicr ihGaussian state ρ̂. Show that its purity is given by Tr[ρ̂2 ] = det I2Nf − Γ2ψ /2 . Show also thatthe characteristic function of a Gaussian state (4.4.76) is given by (4.4.77) [you may use the canonical2 = 1 or ξ 2 = 0].
form (4.4.21) and note the relations ψ̂1,2
Exercise 4.7 (Conservation laws in variational analysis: 1 point). Consider a real-valued variational state. Show that the action variational principle (4.5.18) ensures the energy conservation, but it does
not in general satisfy the other conservation laws.

Exercise 4.8 (Variational time-evolution equations: 1 point). Derive a set of the variational timeevolution equations (4.5.53) and (4.5.55) for Bosonic Gaussian states [you can follow the same procedure
as done for the fermionic case above].

CHAPTER 5

Quantum light-matter interaction
We here introduce a theory of nonrelativistic quantum electrodynamics (QED) by including matter
degrees of freedom in the theory of quantized electromagnetic field introduced in the previous Chapter.
We first do this within the Coulomb gauge, and introduce several unitary transformations that are useful,
e.g., to introduce simplified effective models or analyze strong light-matter interaction. We briefly mention
the relevance of these discussions to cavity/waveguide/circuit QED systems that are of current interest in
the fields of quantum optics, quantum information science, and quantum chemistry.

5.1. Classical electrodynamics review
Before developing quantum theory, we shall start from reviewing the theory of classical electrodynamics.

5.1.1. Equations of motion. The dynamics of classical electromagnetic fields interacting with classical charged particles are governed by Maxwell’s equations:(5.1.1)

∇·E=

1
1
ρ
, ∇ · B = 0, ∇ × E = −∂t B, ∇ × B = 2 ∂t E +
j.
ϵ0
c
ϵ0 c2

Here, we represent the charge density, current density, and the particle velocity as follows:X
X(5.1.2)
ρ(r) =qα δ(r − rα (t)), j(r) =qα vα δ(r − rα (t)), vα ≡ dt rα .
ααEach charged particle obeys the equation of motion:mα d2t rα = qα [E(rα (t), t) + vα × B(rα (t), t)] .

(5.1.3)

From the Helmholtz theorem, we can introduce the vector potential A and the scalar potential ϕ via
B = ∇ × A, E = −∂t A − ∇ϕ.

(5.1.4)We then write down the Lagrangian of the total system by(5.1.5)(5.1.6)

X1

Z

ϵ 


0
E2 − c2 B2 + j · A − ρϕ
2
2
α
Z
ϵ h
i

X1
0
mα (dt rα )2 + d3 r(∂t A + ∇ϕ)2 − c2 (∇ × A)2 + j · A − ρϕ .
=
2
2
α

L=

mα (dt rα )2 +

d3 r

92

5.1. CLASSICAL ELECTRODYNAMICS REVIEW93

One can check that the Euler-Lagrange equations for generalized coordinates, A, ϕ, and {rα }, reproduce
Maxwell’s equations (5.1.1) and the equation of motion (5.1.3). At first sight, it might seem that theLagrangian has the following set of dynamical variables:{A, ∂t A, ϕ, ∂t ϕ} , {rα , vα } .

(5.1.7)

As shown below, in fact this choice of dynamical variables is redundant since they are not independent
degrees of freedom. Thus, the present system is a constrained dynamical system and, to complete thecanonical quantization, we have to carefully identify the correct (nonredundant) dynamical degrees of
freedom in this system.
To this end, we introduce the Fourier transformation of arbitrary field variable byZ

d3 k
Vk eik·r ,
(2π)3

Z

d3 r V(r)e−ik·r .

V(r) =(5.1.8)Vk =(5.1.9)When it is vector-valued, we recall that the Helmholtz theorem allows us to decompose it into transverseand longitudinal parts as
V(r) = V⊥ (r) + V∥ (r),(5.1.10)where each component satisfies(5.1.11)∇ · V⊥ (r) = 0 ⇐⇒ k · Vk⊥ = 0,

(5.1.12)∇ × V∥ (r) = 0 ⇐⇒ k × Vk = 0.

∥

More specifically, the transverse part can be obtained by using the transverse delta function:(Vk⊥ )i =(5.1.13)


X
ki kj
δij −
(Vk )j .
|k|2
jIn real space, this can be read as⊥(V (r))i =(5.1.14)XZ⊥
d3 r′ δij(r − r′ )(V(r′ ))j ,j⊥ (r) is defined in Eq. (4.2.38).
where δij
Using these expressions, we can rewrite Maxwell’s equations in the Fourier space as(5.1.15)with∥

Ek = −

iρk
1
1 ⊥
∥
⊥
⊥
⊥
k, Bk = 0, ik × E⊥jk = −∂t Bk , ik × Bk = 2 ∂t Ek +ϵ0 k 2c
ϵ0 c2 k5.1. CLASSICAL ELECTRODYNAMICS REVIEW(5.1.16)

ρk =

94

X qα
X qα v αe−ik·rα , jk =e−ik·rα .
3
3
(2π)
(2π)
α
α

Note that the longitudinal part of the last equation, ∇×B = c12 ∂t E+ ϵ01c2 j, leads to the following equality(5.1.17)∥∥

0 = ∂t Ek + jk /ϵ0 ⇐⇒ 0 = ∂t ρk + ik · jk ,which merely indicates the charge conservation and does not provide additional constraints. It is now clearthat the correct choice of the independent dynamical degrees of freedom is the one that contributes to theactual time evolution in Eq. (5.1.15), that is,(5.1.18)

n
o
⊥
E⊥
,
B
k
k , {rα , vα } ,∥∥kk and Bk = 0. Said differently,
while the longitudinal fields always obey the constraints Ek = − ϵiρ ⊥ ⊥ 0 k2
it suffices to give the initial conditions of the variables Ek , Bk , {rα , vα } for the purpose of fully
predicting the dynamics of the present system; the whole time evolution is then obtained by solving Eqs.
⊥
(5.1.15) and (5.1.3) with these initial conditions. We remark that, since E⊥
k , Bk obey the first-derivative
time-evolution equation, we don’t need to specify the initial “velocity” of these variables.

5.1.2. Redundancy in the dynamical variables and the Coulomb gauge. As mentioned before,
the original Lagrangian appeared to have 8 dynamical variables for electromagnetic degrees of freedom:(5.1.19){A, ∂t A, ϕ, ∂t ϕ} .

However, as we have seen at the last part in the previous section, there should be only 4 dynamical⊥variables, which correspond to the transverse fields E⊥
k , Bk . To proceed with the quantization, we thus
must omit the redundant, extra 4 degrees of freedom from the theory.
To do so, we first note that ∂t ϕ does not appear in any equations of motion and ϕ can be related toother variables via the constraint(5.1.20)

1
ϕk = 2kρk∥ik · ∂t Ak +,ϵ0which follows from the first equality in Eq. (5.1.15). This fact means that ∂t ϕ and ϕ are not actualdynamical degrees of freedom, but can be eliminated from the theory through this constraint.
Now there are 6 variables. To identify another constraint, we recall the relations between electromagnetic fields and the potentials in the Fourier space:
∥

∥

⊥
⊥
⊥
B⊥
k = ik × Ak , Ek = −∂t Ak , Ek = −∂t Ak − ikϕk .
⊥Since we know that the variables E⊥
k , Bk constitute the independent dynamical degrees of freedom,
we can conclude that it suffices to consider only the transverse component of the vector potential A as

(5.1.21)

5.1. CLASSICAL ELECTRODYNAMICS REVIEW

95

dynamical variables. This can be achieved most easily by imposing the following gauge condition, which
is known as the Coulomb gauge:∥∇ · A(r) = 0 ⇐⇒ A∥ (r) = Ak = 0.

(5.1.22)

Altogether, we now identify that, in terms of the potentials, the 4 independent dynamical degrees of
freedom correspond to the transverse components of the vector potential and their “velocity”:
noA⊥ , ∂t A⊥ .

(5.1.23)

For the sake of notational simplicity, from now on, we shall abbreviate ⊥ in A⊥ and assume that thevector potential is always transverse.
To derive the Hamiltonian, we have to express the Lagrangian in terms of only independent dynamicalvariables {A, ∂t A} , {rα , vα }. To this end, we eliminate the longitudinal electric field E∥ and the scalarpotential ϕ in the original Lagrangian,(5.1.24)

L=

X1
α

2

2

mα (dt rα ) +

Z

d3 r

ϵ 


0
E2 − c2 B2 + j · A − ρϕ ,2

in the following way
Z(5.1.25)

d3 r

Z

1
|E∥ (r)|2 − ρϕ = −d3 r|∇ϕ(r)|2 = −VC
22
1 X qα qβ
,
VC ≡
4πϵ0
|rα − rβ |ϵ(5.1.26)0α>βwhich gives the electrostatic contributions. Using the relationE⊥ = A, B = ∇ × A,

(5.1.27)
we then arrive at(5.1.28)

L=

X1
α

2

2

mα (dt rα ) − VC +

Z

d3 r

ϵ 


0
(∂t A)2 − c2 (∇ × A)2 + j · A .
2

One can now derive the canonical momentum as(5.1.29)(5.1.30)
which leads to the Hamiltonian∂L= mdt rα + qα A(rα ),∂(dt rα )δLΠ≡= ϵ0 ∂t A,
δ(∂t A)pα ≡5.2. QUANTIZED ELECTRODYNAMICS HAMILTONIANS(5.1.31)H=X

Z
pα · dt rα +96

d3 r Π · ∂t A − L

α(5.1.32)=

X [pα − qα A(rα )]2
2mα

α

Z
+ VC +d3 r
Π2 ϵ0 c2+(∇ × A)2 .
2ϵ0
2

To simplify the expression, let us assume the periodic boundary conditions and use the mode expansion√
of A and Π in terms of mode functions fkλ (r) = ϵkλ eik·r / V (cf. Eqs. (4.2.29) and (4.2.19) in Chapter
4):

X
eik·r
αkλ ϵkλ √ + c.c. ,
V
kλ


X
eik·r
√
−iϵ0 ωk αkλ ϵkλΠ(r) =+ c.c.
V
kλA(r) =(5.1.33)(5.1.34)
with the dispersion relation

ωk = ck.

(5.1.35)

Then, the electromagnetic part of the total Hamiltonian can be rewritten asZ(5.1.36) X 2Pkλ ϵ0 ωk2 2
Π2 ϵ0 c22d r+(∇ × A) =
+
Qkλ ,
2ϵ0
2
2ϵ0
2
3



kλ

where we define the generalized coordinates and momentum variables by(5.1.37)∗∗− αkλ ).
, Pkλ ≡ iϵ0 ωk (αkλQkλ ≡ αkλ + αkλ5.2. Quantized electrodynamics Hamiltonians
5.2.1. Quantum electrodynamics in the Coulomb gauge. It is now straightforward to quantize theclassical electrodynamic Hamiltonian in the Coulomb gauge obtained above:X [pα − qα A(rα )]2X P2ϵ0 ωk2 2kλHC =(5.2.1)+ VC +
+
Qkλ ,
2mα
2ϵ0
2
α
kλ

X
eik·r
(5.2.2)
A(r) =
αkλ ϵkλ √ + c.c. .
VkλSpecifically, we require that the particle coordinate and momentum obey the canonical commutation relations(5.2.3)[(r̂α )i , (p̂β )j ] = iℏδαβ δij ,while the same also applies to the generalized coordinates and momentum for electromagnetic degrees of
freedom:5.2. QUANTIZED ELECTRODYNAMICS HAMILTONIANS

97[Q̂kλ , P̂k′ λ′ ] = iℏδkk′ δλλ′ .

(5.2.4)In terms of normal variables αkλ , the latter is equivalent to the following replacement:s
αkλ →(5.2.5)ℏâkλ
2ϵ0 ωkwhere âkλ is the bosonic annihilation operator satisfying the commutation relation:[âkλ , â†k′ λ′ ] = δkk′ δλλ′ .

(5.2.6)

Summarizing, the quantized electrodynamic Hamiltonian is finally given by(5.2.7)

ĤC =

h
i2
X p̂α − qα Â(r̂α )2mαα+ VC +Xℏωk â†kλ âkλkλwith the vector potential beings(5.2.8)Â(r) =

X
kλ



ℏ
âkλ ϵkλ eik·r + H.c. .
2ϵ0 ωk V

Note that the present formulation in the Coulomb gauge remains valid as far as we are interested in
low-energy behavior in the sense ℏω ≪ mc2 . To describe high-energy process at ℏω ≥ mc2 , such as
creation of electron-positron pairs, we have to treat also matter degrees of freedom as excitations of field
operators on equal footing. There, rather than the Coulomb gauge, it is more natural to use the gauge
condition that is manifestly relativistic invariant.
5.2.2. Unitary transformation. One can employ a unitary transformation to analyze the obtained
QED Hamiltonian in another reference of frame other than the Coulomb gauge. The most general form
for such unitary transformation is1

o
n
i 
†
(5.2.9)Û = exp f {r̂α , p̂α } ; âkλ , âkλ,ℏnowhere f is some function of particle and photon operators, {r̂α , p̂α } and âkλ , â†kλ . In the transformed
frame, all the observables Ô (including the Hamiltonian, position operators, field amplitudes, etc) are
related to the original expressions in the Coulomb gauge via(5.2.10)

ˆ = Û † ÔÛ ,Õ

while a quantum state is transformed to be(5.2.11)|Ψ̃⟩ = Û † |Ψ⟩.

1In principle, one can also include manifestly time-dependent variables, which may be useful when there exists classical external

drivings.

5.2. QUANTIZED ELECTRODYNAMICS HAMILTONIANS98

Of course, all the frames must ultimately give the same physical results. In practice, however, one cannot
exactly solve the exact QED Hamiltonian and usually resort to a variety of approximations as discussed
below. After performing such approximations, different references of frames in general lead to different predictions since the validity of making certain approximations can depend on a choice of frames.
This means that some frames may significantly simplify the analysis compared to other frames, and thus
it is often crucial how to choose a correct frame of reference depending on different regimes/physicalphenomena at hand.

5.2.3. Gauge transformation. One particular class of unitary transformations corresponds to merely
adding the time-derivative term to the Lagrangian, which is nothing but the gauge transformation. Specifically, the corresponding unitary transformation takes the formi 
(5.2.12)Û = exp f {r̂α } ; Â ,ℏwhich means that f is a function of only generalized coordinates r̂α and Â; the latter means that it includesâ, â† operators only through the vector potential Â.
To see that this indeed corresponds to the gauge transformation, consider adding the time-derivative
term to the classical Lagrangian:(5.2.13)L̃({rα , dt rα } ; A, ∂t A) = L({rα , dt rα } ; A, ∂t A) +(5.2.14)= L({rα , dt rα } ; A, ∂t A) +

df
dt
X
α

dt rα

∂f
∂f
+ ∂t A.
∂rα∂A

The new canonical momentum variables arep̃α ≡(5.2.15)

∂ L̃
∂f
∂f
= mdt rα − eA(rα ) += pα +∂(dt rα )
∂rα
∂rα

and
Π̃ ≡

(5.2.16)δ L̃
∂f= ϵ0 ∂t A +.
δ(∂t A)∂A

The corresponding new Hamiltonian is(5.2.17)

H̃ =

X

Z
p̃α · dt rα +d3 r Π̃ · ∂t A − L̃

α(5.2.18)=

X [p̃α − qα A(rα )]2
α

2mα

Z
+ VC +"#
Π̃2 ϵ0 c22
d r+(∇ × A) .
2ϵ0
23This means that the form of the Hamiltonian is invariant under the gauge transformation, while one has toreplace the canonical momentum by the transformed ones via(5.2.19)pα → p̃α , Π → Π̃.

5.3. LONG WAVELENGTH APPROXIMATION99

It is now clear that, in the language of quantum mechanics, the above unitary transformation (5.2.12)
indeed induces such replacement2:
ˆ α = Û † p̂α Û = p̂α + ∂fp̃∂r̂α
ˆ = Û † Π̂Û = Π̂ + ∂f .
Π̃
(5.2.21)∂ Â
We note that gauge transformation is just a special subclass among all the possible unitary transformations, and there still exists a much wider class of unitary transformations that do not reduce to the gaugetransformations; we will see such examples later.

(5.2.20)5.3. Long wavelength approximation
Consider a globally charge-neutral system (such as a neutral atom) that consists of nuclei and localized
electrons, whose center of mass is positioned at r = 0. If the dynamics of those particles are well localized
in the length scales that are much shorter than typical wavelengths of photons, we can perform the longwavelength approximation:Â(r̂α ) ≃ Â(0).

(5.3.1)

From now on, we impose this condition and abbreviate the position variable as Â ≡ Â(0). For the sakeof simplicity, suppose that there is essentially a single valence electron in a neutral system that contributes
to the electrodynamics, and its motion around the nuclei can be modeled by using a simplified effective
potential V (r). We then arrive at the following QED Hamiltonian in the Coulomb gauge:h(5.3.2)(5.3.3)p̂ + eÂ

ĤC =

Â =

2me
s
X
kλ

i2
+ V (r̂) +Xℏωk â†kλ âkλ ,kλℏ(âkλ ϵkλ + H.c.) .
2ϵ0 ωk V

Below we shall illustrate the applications of several unitary transformations to this type of Hamiltonians
under the long-wavelength approximation.
5.3.1. Power-Zienau-Woolley transformation and dipole interaction. Let us first consider a concrete example of the gauge transformation known as the Power-Zienau-Woolley (PZW) transformation:
(5.3.4)i Û = exp f r̂; Â , f ≡ −er̂ · Â,ℏ

2Strictly speaking, the condition (5.2.21) may include redundant degrees of freedom depending on a choice of the gauge con-ditions in the original Hamiltonian. In that case, it must be modified such that it is expressed by only using the independent
dynamical degrees of freedom.

5.3. LONG WAVELENGTH APPROXIMATION

100

which transforms the particle momentum asˆ = Û † p̂Û = p̂ + ∂f = p̂ − eÂ.
p̃∂r̂

(5.3.5)

To determine how the electromagnetic canonical momentum Π̂ changes under this transformation, werecall that in the Coulomb gauge it is related to the transverse electric field via Π̂ = −ϵ0 Ê⊥ . We thus
obtainˆ ⊥ (r)) = Û † (Ê⊥ (r)) Û(Ẽ
ii(5.3.6)iie hr̂ · Â, (Ê⊥ (r))i
ℏ
e X ⊥
⊥
= (Ê (r))i +δij (r)(r̂)jϵ0= (Ê⊥ (r))i +(5.3.7)(5.3.8)jwhere we used the commutation relation (4.2.39). One can check that the magnetic field B̂ remains
invariant under the transformation. Using the relation(5.3.9)Xkλϵ0ℏωk â†kλ âkλ + const. =
2

Z

i
h
d3 r Ê⊥2 (r) + c2 B̂2 (r) ,we then arrive at the transformed Hamiltonian in the PZW gauge:(5.3.10)ĤPZW = Û † ĤC Û

(5.3.11)=Xp̂2ℏωk â†kλ âkλ ,+ V (r̂) + er̂ · Ê⊥ + DP 2 (r̂) +2mekλwhere we recall that we abbreviate the position variables in the electromagnetic fields: er̂ · Ê⊥ ≡ er̂ ·Ê⊥ (0). This r · E term describes the light-matter interaction in this frame, which is called the dipole
interaction term. There is additional term corresponding to the dipole self-energy:
Z
X
e2
⊥
⊥
DP 2 (r̂) ≡d3 r(5.3.12)r̂i r̂j δim(r)δjm
(r).
2ϵ0
ijm



Z
kj km
d3 k Xe2
ki km(5.3.13)
=
r̂
r̂
δ
−
δ
−
.
i j
jm
im
2ϵ0
(2π)3
k2
k2
ijm

While this term appears to be divergent, it is in fact nondiverging since we must restrict the k integration
up to a certain momentum cutoff, above which the long wavelength approximation becomes inapplicable.
At the cost of introducing the dipole self-energy, we note that the PZW Hamiltonian no longer possessesÂ2 term that was present in the Coulomb gauge.

5.3.2. Matrix elements in different gauges and TRK sum rule. Since the long wavelength approximation should remain valid in both Coulomb and PZW gauges (at least in the weak light-matter coupling
regimes), we expect that the different gauges must provide the same physical predictions, despite theapparently different expressions of the Hamiltonians.

5.3. LONG WAVELENGTH APPROXIMATION

101

To explicitly demonstrate this equivalence for one- and two-photon processes, consider an external
driving by a weak monochromatic classical field at frequency ω:(5.3.14)A(r = 0, t) = Aϵ cos(ωt)(5.3.15)E(r = 0, t) = ωAϵ sin(ωt)

with ϵ = (1, 0, 0)T . At the leading order, we can neglect the Â2 term in ĤC and the DP 2 term in ĤPZW .
In the Coulomb gauge, the first-order time-dependent perturbation theory with respect to the interactioneterm Ĥint,C = mp̂ · A in ĤC leads to the transition rate
(5.3.16)

w1C =

πA2
|fnm |2 δ(ωnm − ω),2ℏ2where fnm ise⟨n|p̂|m⟩, p̂ = ϵ · p̂,mwhich includes the matrix elements of the momentum operator between two internal energy eigenstatesof the neutral atom, having an energy difference ℏωnm ≡ En − Em .
In the PZW gauge, the similar perturbation calculation with respect to the interaction term Ĥint,PZW =er̂ · E in ĤPZW results in the transition rate
(5.3.17)fnm ≡(5.3.18)

w1PZW =

πA2 ˜ 2
|fnm | δ(ωnm − ω),2ℏ2where f˜nm is given by(5.3.19)f˜nm ≡ ieω⟨n|r̂|m⟩, r̂ = ϵ · r̂,

which, in this case, includes the matrix elements of the position operator.
We want to check that these two different gauges predict the same transition rate, namely,(5.3.20)

fnm = f˜nm ⇒ w1C = w1PZW .

This equivalence follows from the relation between matrix elements of momentum and position operatorsin the energy basis:m⟨n|[r̂, Ĥ0 ]|m⟩ = imωnm ⟨n|r̂|m⟩,ℏ
where we define the single-particle Hamiltonian by(5.3.21)(5.3.22)⟨n|p̂|m⟩ = −i

Ĥ0 =

p̂2
+ V (r̂)2meand use Ĥ0 |n⟩ = En |n⟩.
Similarly, one can also check the gauge equivalence for two-photon transition processes. The secondorder time-dependent perturbation theory in the Coulomb gauge leads to5.3. LONG WAVELENGTH APPROXIMATION

w2C =(5.3.23)102

πA4|gnm |2 δ(ωnm − 2ω),
8ℏ2

where
gnm ≡

(5.3.24) e 2 X ⟨n|p̂|l⟩⟨l|p̂|m⟩mlω − ωlm.

The transition rate in the PZW gauge is

w2PZW =

(5.3.25)πA4|g̃nm |2 δ(ωnm − 2ω)
8ℏ2

with
g̃nm ≡ −ω 2 e2

(5.3.26)

X ⟨n|r̂|l⟩⟨l|r̂|m⟩lω − ωlm.

One can check that the equivalence between these results,

gnm = g̃nm ⇒ w2C = w2PZW ,(5.3.27)

directly follows from the Thomas-Reiche-Kuhn (TRK) sum rule (see Exercise):Xℏ(5.3.28)(ωln + ωlm ) ⟨n|r̂|l⟩⟨l|r̂|m⟩ = δnm .
ml

The TRK sum rule can be shown as follows:(5.3.29)Xl(ωln + ωlm ) ⟨n|r̂|l⟩⟨l|r̂|m⟩ = −i X(⟨n|r̂|l⟩⟨l|p̂|m⟩ − ⟨n|p̂|l⟩⟨l|r̂|m⟩)mlℏ
i(5.3.30)= − ⟨n|[r̂, p̂]|m⟩ = δnm ,mmwhere we used Eq. (5.3.21) to obtain the first equality.

5.3.3. Unitary transformations beyond gauge transformations. As noted earlier, there also exist
unitary transformations that do not reduce to the gauge transformations. Some of them are useful for the
purpose of analyzing quantum light-matter systems in certain regimes; let us briefly look at such examples
below.
5.3.3.1. Pauli-Fierz-Kramers transformation. The first example is the Pauli-Fierz-Kramers transformation, by which the light-matter interaction can be absorbed by the field-induced shift of the particle
position together with the fluctuation in the particle potential. To simplify the argument, we consider 1D
case as follows:

5.3. LONG WAVELENGTH APPROXIMATIONh(5.3.31)(5.3.32)p̂ + eÂ

ĤC =

103

i2
+ V (x̂) +

2me

X 
fk âk + â†k .
Â =Xℏωk â†k âk ,kkStarting from this Coulomb-gauge Hamiltonian, we define the unitary transformation



X
i
fk  †(5.3.33)
Û = exp − p̂Ẑ , Ẑ ≡(−ie)
âk − âk ,
ℏ
me ωk
k

which includes the particle momentum p̂, and the field operator Ẑ that does not reduce to Â; thus it cannot
be described as a gauge transformation.
The unitary transformation acts on each operator as(5.3.34)

Û † x̂Û = x̂ + Ẑ

(5.3.35)Û † âk Û = âk −efkp̂.
me ℏωkThe coefficients in Ẑ are chosen in such a way that the p̂ · Â term in the Coulomb gauge is cancelled by
the term arising from the shift of the â† â term. The resulting transformed Hamiltonian is(5.3.36) ĤPFK = Û † ĤC Û
hP e2 f 2 i!2
1 − 2 k me ℏωk k
2X
X 2ef 2ek=(5.3.37)ℏωk â†k âkÂ −p̂ + V (x̂ + Ẑ) +
p̂2 +
2me
2me
me ℏωkkkX
1 2(5.3.38)≃p̂ + V (x̂ + Ẑ) +ℏωk â†k âk ,2mekwhere the last expression holds true only for a weak light-matter coupling. An advantage of this expression is that, at the lowest order of e, the light-matter interaction is incorporated only through the potential
term V , and this feature can simplify calculations of transition amplitudes for certain scattering processes.
5.3.3.2. Asymptotically decoupling transformation. The Pauli-Fierz-Kramers transformation introduced above is not very satisfactory in the sense that, at strong light-matter coupling, the higher-orderterms in Eq. (5.3.37) must contribute to dynamics in a rather complicated manner.
To circumvent this difficulty, one can introduce the asymptotically decoupling (AD) unitary transformation, which can completely disentangle light and matter degrees of freedom in the strong-coupling
limit. To do so, we first note that one can diagonalize the quadratic photon part in ĤC as(5.3.39)X
e2 Â2 X+ℏωk â†k âk =ℏΩn b̂†n b̂n + const.
2menkThis is always possible by using a certain bosonic Gaussian transformation we studied in the previous
Chapter. In terms of new photon operators b̂n , b̂†n , one can show that the Hamiltonian takes the following

5.3. LONG WAVELENGTH APPROXIMATION

104

form (see Exercise):(5.3.40)

ĤC =

X
X
p̂2
+ V (x̂) − p̂ζn (b̂n + b̂†n ) +ℏΩn b̂†n b̂n .
2menn

We then define the unitary transformationX ζn 
i(5.3.41)
Û = exp − p̂Ξ̂ , Ξ̂ ≡ib̂†n − b̂n ,ℏΩnkwhich again does not reduce to a gauge transformation since it includes the operators p̂ and Ξ̂ ̸= Â. Thistransformation acts on each operator as(5.3.42)Û † x̂Û = x̂ + Ξ̂,(5.3.43)Û † b̂n Û = b̂n +

ζn
p̂.
ℏΩn

P
Note that the coefficients in Ξ̂ are chosen in such a way that, at this time, the interaction term −p̂ n ζn (b̂n +b̂†n ) in ĤC is cancelled by the term arising from the displacement of b̂† b̂ term. After some calculations
which we leave to Exercise, the transformed Hamiltonian is given by a very simplified form:(5.3.44)ĤAD = Û † ĤC Û

(5.3.45)=

Xp̂2+ V (x̂ + Ξ̂) +ℏΩn b̂†n b̂n
2meffnwith the mass being renormalized tomeff ≡ me(5.3.46)

X e2 f 2k1+2
me ℏωk!
.

kThis expression of the Hamiltonian may look similar to the Pauli-Fierz-Kramers form (5.3.38), however,the crucial point is that the present one (5.3.45) is exact and valid for arbitrary light-matter couplingstrengths (as far as the long wavelength approximation is meaningful). As we will revisit later, an advantage of this frame is that the effective light-matter coupling strength (characterized by ζn /Ωn ) asymptotically vanishes in the strong coupling limit e → ∞. This “asymptotic freedom” can simplify the analysisof strong coupling physics in quantum light-matter systems.
5.3.4. Spontaneous emission: Wigner-Weisskopf theory. We here illustrate an application of nonrelativistic QED to yet another elemental process, namely, spontaneous emission of a photon from an
excited atom. Suppose that the emission is induced by the light-matter interaction between an atomic
dipole, which can be modeled by an effectively two-level system, and vacuum electromagnetic field in
free space. Projecting the PZW-gauge Hamiltonian (5.3.11) onto the two-level manifold, the total Hamiltonian is obtained by(5.3.47)ĤPZW = ℏωaσ̂ z + 12− σ̂ xXkλ(gk âkλ + H.c.) +Xkλℏωk â†kλ âkλ ,5.3. LONG WAVELENGTH APPROXIMATION

105

where we introduce the coupling strengths and the dipole matrix element byr
ωk(5.3.48)
gk =d · ϵkλ eik·r , d ≡ ⟨e|(−er̂)|g⟩.
2ℏϵ0 VTo further simplify the analysis, one can perform the rotating wave approximation and the Born-Markov
approximation, which can be justified when the light-matter interaction is sufficiently weak. The former
leads to the interaction term
!
X
X
+
−
∗ †
(5.3.49)
ĤSE = − σ̂
gk âkλ + σ̂
gk âkλ ,kλkλwhile the latter allows us to treat the problem of spontaneous emission within the framework of Markovian open systems by integrating out the electromagnetic field. Following the general argument given in
Chapter 3, we obtain the total decay rate (see Eq. (3.7.17)):

Γ=2(5.3.50)

XZ ∞
kλ

Z ∞
=

(5.3.51)

0=(5.3.52)0dτ |gk |2 eiωa τ ⟨0|âk e−iωk τ â†k |0⟩

2k 2 dk ck |d|2πδ(ωa − ck)
π 2 2ℏϵ0 3

ωa3 |d|23πϵ0 ℏc3

where we assume the isotropic atom
1|d · ϵkλ |2 = |d|2
3

(5.3.53)
and used the relations

1 X
→
V

(5.3.54)k

Z ∞(5.3.55)
0Z

d3 k, ωk = ck,(2π)3dτ ei(ωa −ωk )τ = πδ(ωa − ωk ) + P.v.

i.
ωa − ωk

The resulting Markovian master equation for the spontaneous emission is (see Eq. (3.7.18))(5.3.56)(5.3.57)


dρ̂
i
†
=−
Ĥeff ρ̂ − ρ̂Ĥeff+ Γσ̂ − ρ̂σ̂ + ,
dt
ℏ


 z

σ̂ z + 1iℏΓ + −iΓσ̂ + 1Ĥeff ≡ ℏωa−σ̂ σ̂ = ℏ ωa −
.
2
2
2
2

If an observer can access information about electromagnetic modes (by, for example, photodetectors)
and knows that no photon is emitted, the initial state then evolves under the effective non-Hermitian
Hamiltonian which is characterized by a complex energy ℏ(ωa − iΓ/2). In contrast, if an observer does
not know about when a photon is emitted, the correct description is the master equation and the initiallypure state evolves into a mixed density matrix.

5.4. BRIEF INTRODUCTION TO CAVITY/WAVEGUIDE QED106

To check that the obtained decay rate is indeed small as appropriate for validating the master-equation
description, we can rewrite(5.3.58)

Γ=

4αQED ωa3 |d|2
e2
1
,
α
≡
≃
.
QED
3c2 e24πϵ0 ℏc
137

Thus, we have Γ/ωa ∼ αQED (ra /λa )2 ≪ 1 where ra is the size of the atomic dipole ra = |d|/e and λais the wavelength for atomic transition and we used the long-wavelength condition ra /λa ≪ 1.
5.4. Brief introduction to Cavity/Waveguide QEDAs we have seen in the previous section, the light-matter interaction between a quantum emitter andquantum electromagnetic field in free space is very weak, which ultimately has its origin in the smallnessof the fine structure constant αQED . This can be problematic especially in the fields of quantum information technologies or quantum optics, since one needs strong light-matter interaction to efficiently andfaithfully operate, convert, and propagate quantum information in-between (artificial) atoms and photons.
One way to circumvent this difficulty is to confine electromagnetic fields into a tiny box, namely,
cavity. This confinement effectively enhances the quantum light-matter interaction since the coupling√strength is proportional to 1/ V with V being the mode volume. The resulting strong quantum lightmatter coupling allows one to study the interaction between atoms and photons in (typically a few) discrete electromagnetic modes at the fundamental level of single quanta. This field, which studies quantum
electrodynamics of atoms and photons confined in cavity, is known as cavity QED. More recently, similarstrong coupling is also achieved for (artificial) atoms coupled to photons in continuum modes; this emerging field is coined as waveguide QED, since continuum electromagnetic modes are typically realized incertain waveguide resonators. Below we shall briefly address some of basics in these fields.
5.4.1. Two-level approximation: quantum Rabi model and Dicke model. To introduce a simplified model for describing cavity QED systems, we typically assume that only a single electromagnetic
mode in a cavity is dominantly coupled to an atomic dipole. We then start from the effective QED Hamiltonian in the Coulomb gauge:hi2p̂ + eÂ
(5.4.1)ĤC =+ V (x̂) + ℏωc â† â, Â = f â + â† ,2mewhere f is the mode amplitude depending on the cavity geometry and ωc is the cavity frequency. A
standard choice of a potential V (x) for modeling an atomic dipole is the double well potential,x2
V (x) = v 1 − 2D

(5.4.2)

2,which has two degenerate minima at x = ±D; then, it can represent the spectrum of atomic dipole ofsize ≃ −eD. The two lowest energy levels play the role of the ground and first excited states, and wemay project the Hamiltonian onto such two-level manifold (i.e., the two-level approximation) to further
simplify the analysis.

5.4. BRIEF INTRODUCTION TO CAVITY/WAVEGUIDE QED

107

For the purpose of performing the two-level approximation, it is advantageous to switch to the PZWgauge:(5.4.3)ĤPZW = Û † ĤC Û ,(5.4.4)=pp̂2+ V (x̂) + mg 2 x̂2 + ig mℏωc x̂(â† − â) + ℏωc â† â,
2me

where the PZW transformation is given byÛ = e(5.4.5)−iex̂Âℏr, g ≡ −efωc.
mℏWhile the Coulomb and PZW gauges must provide the same results if no approximation is made, thetwo-level approximation can in fact make differences between them. In other words, the validity of the
two-level approximation in fact depends on a choice of reference frames. It is now understood that thePZW gauge is well suited for obtaining a two-level description up to the regimes in which the lightmatter coupling strength reaches to an order of elementary excitation energies ℏωa,c (known as ultrastrong
coupling regimes).
To derive the two-level model in the PZW gauge, we consider the two lowest levels of the renormalizedp̂2
single-particle Hamiltonian Ĥ0 = 2m
+ V (x̂) + mg 2 x̂2 in Eq .(5.4.4); note that the last term, mg 2 x̂2 ,erepresents the dipole self-energy. We denote the two-level manifold of Ĥ0 by {|g⟩, |e⟩} and introduce thePauli operators as(5.4.6)σ̂ z = |e⟩⟨e| − |g⟩⟨g|, σ̂ x = |e⟩⟨g| + |g⟩⟨e|.

Projecting the matter part of ĤPZW onto this manifold, we obtain the two-level effective model, often
called the quantum Rabi model, as follows:


ℏωa z
σ̂ + ℏλσ̂ x â† + â + ℏωc â† â,2where λ ≡ f ωc d/ℏ with d = ⟨e|(−ex̂)|g⟩ and we replaced â → iâ.
These arguments can straightforwardly be generalized to the case of N atomic dipoles coupled to acavity electromagnetic mode. Specifically, we start from the Coulomb-gauge Hamiltonian(5.4.7)(5.4.8)ĤRabi =

ĤC =

N
X
i=1


i2
p̂
+
e
Â
 i

+ V (x̂i ) + ℏωc â† â,

2me
h

h P
i
−ie i x̂i Â
and perform the multi-dipole PZW transformation Û = exp, after which we project theℏtransformed Hamiltonian onto the two-level manifold, and obtain5.4. BRIEF INTRODUCTION TO CAVITY/WAVEGUIDE QED(5.4.9)NN
Xℏωa X zℏλ2
ĤDicke =σ̂i + ℏλσ̂ix â† + â + ℏωc â† â +
2
ωc
i=1

i=1

108

N
X

!2
σ̂ix

.

i=1

The last term arises from the dipole self-energy of multiple atoms, which can also be interpreted as thecavity-mediated dipole-dipole coupling. Strictly speaking, this Hamiltonian should be referred to as thegeneralized Dicke model, since the original Dicke model usually refers to the model without the selfenergy term. In fact, the original Dicke model eventually becomes invalid at large λ and is known togive rise to qualitatively inaccurate predictions, such as an occurrence of the (equilibrium) superradianttransition at sufficiently large λ (see Exercise).
5.4.2. Rotating wave approximation: Jaynes-Cummings model. Consider a single atomic dipole
coupled to a cavity electromagnetic mode. If the atomic frequency is near resonant and the light-matterinteraction is weak in the sense |ωa − ωc |, λ ≪ ωa,c , we can neglect the contributions from the counterrotating terms σ̂ + â† and σ̂ − â in the Rabi model (5.4.7), which are off-resonant and thus rotate rapidly inthe rotating wave frame. This allows us to further simplify the two-level model (5.4.7) as follows:


ℏωa z
σ̂ + ℏλ â† σ̂ − + âσ̂ + + ℏωc â† â.
2This Hamiltonian is known as the Jaynes-Cummings model and has a conserved quantity(5.4.10)

ĤJC =σ̂ z + 1.
2
The ground state with Nex = 0 is unique and given by |g⟩|0⟩, while other sector at nonzero Nex is spannedby two states |e⟩|Nex −1⟩ and |g⟩|Nex ⟩. The spectrum is then obtained by just solving the 2×2 eigenvalue
problem. For instance, at the resonant condition ωa = ωc , excitation energies are given by(5.4.11)(5.4.12)N̂ex = â† â +pENex ,± = Nex ωa ± λ Nex .

The corresponding energy eigenstates hybridize light and matter degrees of freedom and are often called
as dressed states.
√
The energy splittings 2λ Nex depend on the excitation number Nex , which have interesting physicalconsequences. To see this, consider exciting an atom by a low-photon external pumping at frequency ω.
In the limit of λ → 0, all the transition processes become resonant at ω = ωa . At λ ̸= 0, however, the Nexdependence of energy splittings allow one to choose ω such that only a specific process is resonant. Thus,in the ideal case where the linewidth is sufficiently small compared to λ, one can realize an effectively
nonlinear optical medium. For instance, if one chooses ω = ωa − λ, only a single photon can be excitedsince two (and higher) photon excitation processes are off-resonant and suppressed, leading to effective“photon blockade”.
5.4.3. Strong-coupling physics: asymptotic freedom. The two-level description discussed above
ultimately breaks down when the coupling strength reaches and surpasses elementary excitation energies
ωa,c . Physically, this is because at such strong couplings the photon and atoms are strongly entangled and5.4. BRIEF INTRODUCTION TO CAVITY/WAVEGUIDE QED

109

thus they form hybridized states consisting of superposition of high-lying energy modes in both photon
and atomic levels. It is this strong entanglement that invalidates the level truncations necessary to derive
the two-level models.
To circumvent this difficulty, we can perform the asymptotically decoupling unitary transformationthat we introduced in the previous section. To see this, we first rewrite the Coulomb-gauge Hamiltonian(5.4.1) asrℏp̂2+ V (x̂) − g(5.4.13)p̂(b̂ + b̂† ) + ℏΩb̂† b̂,
ĤC =
2me
mΩ
p
(5.4.14)Ω ≡ ωc2 + 2g 2 ,pwhere we perform the Bogoliubov transformation b̂ + b̂† = Ω/ωc (â + â† ). We then use the unitarytransformation to get(5.4.15)ĤAD = Û † ĤC Û

(5.4.16)=p̂2+ V (x̂ + Ξ̂) + ℏΩb̂† b̂
2meffwhere we introduce(5.4.17)(5.4.18)

meff = me (1 + 2(g/ωc )2 ), Ξ̂ = iξ(b̂† − b̂),
"
#
ip̂Ξ̂
Û = exp −.
ℏHere, the length scale ξ characterizes the effective light-matter coupling strength in the transformed frameand defined byr(5.4.19)

ξ≡g

s
ℏ
=
mΩ3

ℏg 2
.
m(ωc2 + 2g 2 )3/2

Note that ξ converges as ξ ∝ g −1/2 → 0 in the strong-coupling limit g ≫ ωc . This means that atoms
and photons are asymptotically decoupled in the strong coupling regimes. One can then easily study
strong-coupling physics of light-matter systems in this frame, which is otherwise difficult to analyze ingeneral.

5.4.4. Waveguide QED Hamiltonian and Spin-Boson model. We have so far assumed that atoms
are coupled to only a single electromagnetic mode. If we couple atoms to electromagnetic modes ofwaveguide resonators instead of cavity, atoms in general couple to continuum electromagnetic modes. The
discussions above for cavity QED can be extended to the case of such waveguide QED setups possessing
multiple continuum photon modes.
A particularly well-known model in this context is the continuum-mode counterpart of the Rabi model,
which is also known as the spin-boson model:

5.4. BRIEF INTRODUCTION TO CAVITY/WAVEGUIDE QED(5.4.20)

ĤSB =

110


 X
X
ℏωa z
σ̂ + σ̂ xℏλk â† + â +ℏωk â† â.
2kk

The spin-boson model has historically been extensively studied in the context of open quantum systems,
as it can describe dissipative dynamics of a spin-1/2 coupled to bosonic baths. Unless λk is sufficiently
weak, the master equation approach is in general insufficient and thus this model belongs to a class of
non-Markovian (or nonperturbative) open quantum systems. It is interesting to note that its ground state
can exhibit the quantum phase transition of the Berezinskii-Kosterlitz-Thouless (BKT) type. Under certain conditions, this transition can be identified as the antiferromagnetic-to-ferromagnetic transition in theanisotropic Kondo model. However, to apply the spin-boson description to waveguide QED systems, wehave to keep in mind that this two-level description can be a good effective model only when a number ofassumptions used above are plausible. When some of those assumptions (such as the two-level approximation etc) break down (which might be the case especially at strong light-matter couplings), we have togo back to the original QED Hamiltonian and reexamine the validity of the analysis.
5.4.5. Circuit realizations. As we briefly mentioned in the previous Chapter, one can use superconducting circuits to study the physics of quantum light-matter interaction; this field is often coined as circuit
QED. To complete the analogy between circuit QED and cavity QED, let us recall the Hamiltonians ineach case:(5.4.21)(5.4.22)2
Ĥcircuit = 4EC N̂ − N̂g − EJ cos φ + ℏωr â† â, N̂g ∝ â† + â,hi2p̂ + eÂ+ V (x̂) + ℏωc â† â, Â ∝ â + â† .
Ĥcavity =2me

It is now clear that these two models are essentially equivalent if we identify the potential in the cavity
QED Hamiltonian as the periodic potential V (x) ∝ cos(2πx/λ).
In circuit QED, the Josephson junction plays the role of an artificial atom which is coupled to microwave photons in LC circuit. If we couple a Josephson junction to a more complex circuit with infinite
length, this artificial atom will interact with continuum of microwave photons, thus offering an ideal platform to study physics of waveguide QED3. Owing to low frequency of photons in circuit setups (typicallyin microwave regimes), it is usually easier to attain strong light-matter couplings in circuit setups thanconventional cavity QED setups (typically in optical regimes). Circuit QED systems are thus also one of
the most ideal experimental platforms for exploring physics of strong quantum light-matter interaction.

3Strictly speaking, the analogy with the waveguide QED Hamiltonian may not be perfect in this case, since the Â2 may in facttake a slightly different form depending on circuit geometry in circuit QED Hamiltonian.

5.4. BRIEF INTRODUCTION TO CAVITY/WAVEGUIDE QED111

Summary of Chapter 5
Section 5.1-5.3 Quantum electrodynamics in the Coulomb gauge and unitary transformations• To eliminate the redundant degrees of freedom, one can impose the Coulomb gauge condition, after which the standard canonical quantization procedure allows us to obtain the QED
Hamiltonian (5.2.7).
• The unitary transformation that only includes the particle positions r̂α and the vector potentialÂ generates a gauge transformation. One example is the Power-Zienau-Woolley transformation, after which the interaction is described by the r · E term at the leading order.
• There also exist unitary transformations that go beyond gauge transformations. Examples
include the Pauli-Fierz-Kramers or asymptotically decoupling transformations, which can beparticularly useful in strong light-matter coupling regimes.
Section 5.4 Brief introduction to cavity/waveguide QED• If a dipole is dominantly coupled to only a single electromagnetic mode, we can use the twolevel effective model called the quantum Rabi model. This model can be faithfully derived
from the QED Hamiltonian in the PZW gauge up to ultrastrong coupling regimes. Its N dipole generalization gives the (generalized) Dicke model.
• When the rotating wave approximation is valid, one can further simplify the model to the
Jaynes-Cummings Hamiltonian.
• A single two-level system coupled to continuum electromagnetic modes can be modeled by
the spin-boson model.
• When light-matter coupling strength reaches/surpasses energy scales of elementary excitations, both the two-level and rotating wave approximations break down in general. In such
regimes, one may use the asymptotically decoupling transformation to simplify the analysis.

5.5. EXERCISES

112

5.5. Exercises
Exercise 5.1 (PZW gauge without long-wave approximation: 2 points). Introduce the atomic polarization density by
Z 1
du δ(r − ur̂).

P̂(r) ≡ −er̂

(5.5.1)0Consider the gauge transformation (called the PZW transformation) corresponding to the following unitary operator:
 Z

i
3
Û = exp
d r P̂(r) · Â(r) .
ℏ(5.5.2)Check that this reduces to Eq. (5.3.4) after performing the long-wave approximation.
We here aim to derive the transformed Hamiltonian without resorting to the long-wave approximation.
To do so, consider the exact Coulomb-gauge Hamiltonianhi2p̂ + eÂ(r̂)
ĤC =(5.5.3)+ V (r̂) +2meXℏωk â†kλ âkλ ,

kλ

with
s
Â(r) =

(5.5.4)

X
kλ



ℏ
âkλ ϵkλ eik·r + H.c. .
2ϵ0 ωk V

Show that the PZW transformation above leads to the following transformed Hamiltonian:(5.5.5)(5.5.6)ĤPZW = Û † ĤC Û
2

R1
Z
p̂ − er̂ × 0 du uB̂(ur̂)=+ V (r̂) − d3 r P̂⊥ (r) · Ê⊥ (r)
2me
Z
X
1
+
d3 r P̂⊥2 (r) +ℏωk â†kλ âkλ .
2ϵ0
kλYou may use the relations(5.5.7)X
kλ(5.5.8)ℏωk â†kλ âkλ + const. =

ϵ0
2

Z

h
i
d3 r Ê⊥2 (r) + c2 B̂2 (r) ,⊥[Âα (r, t), −ϵ0 Êβ (r′ , t)] = iℏδαβ(r − r′ ).

5.5. EXERCISES

113

Exercise 5.2 (Applications of TRK sum rule: 2 points). (a) Show the equivalence between expressions of matrix elements for two-photon scattering process in different gauges:
gnm = g̃nm

(5.5.9)where(5.5.10)gnm ≡ e 2 X ⟨n|p̂|l⟩⟨l|p̂|m⟩m

ω − ωlml, g̃nm ≡ −ω 2 e2

X ⟨n|r̂|l⟩⟨l|r̂|m⟩lω − ωlmwith the two-photon resonant frequency
ω=(5.5.11)ωnm.
2

You may use the TRK sum rule:Xℏ(5.5.12)(ωln + ωlm ) ⟨n|r̂|l⟩⟨l|r̂|m⟩ = δnm .
ml(b) Consider the Dicke model (5.4.9) at large, but finite number N of electric dipoles. Use the TRK sum
rule to show that the energy difference between the first excited state and the ground state does not vanish
at any finite light-matter coupling strength4. What would happen if the last term (self-energy term) in Eq.
(5.4.9) is neglected? You may use the Holstein-Primakoff approximation:(5.5.13)NX

σ̂i+ ≃√

N b̂† ,i=1

NX

σ̂ z ≃ −N + 2b̂† b̂,

i=1

where b̂, b̂† are bosonic operators obeying [b̂, b̂† ] = 1.

Exercise 5.3 (Mass renormalization in the asymptotically decoupled frame: 2 points). Consider
the Coulomb-gauge Hamiltonian in the long-wave approximation:h(5.5.14)(5.5.15)p̂ + eÂĤC =i2+ V (x̂) +

2me

X 
Â =
fk âk + â†k .

Xℏωk â†k âk ,kk(a) The quadratic photon part can be diagonalized as(5.5.16)X
e2 Â X+ℏωk â†k âk =ℏΩn b̂†n b̂n + const.
2menk

4This argument essentially forbids realization of the equilibrium superradiant transition in cavity QED systems with a single

electromagnetic mode, which is often called “no-go theorem”. Note, however, that possibility of realizing superradiant transition
is not excluded in general multi-mode cases.

5.5. EXERCISES

114

via the canonical transformation(5.5.17)

âk =

h
i
X
(O)kn cosh(rnk )b̂n − sinh(rnk )b̂†n ,nwhere rnk ∈ R are squeezing parameters while O is an orthogonal matrix. Determine the expression of
rnk and derive the relation satisfied by O.
(b) In terms of b̂ operators, the Hamiltonian can be written as
XXp̂2ℏΩn b̂†n b̂n .
(5.5.18)ĤC =+ V (x̂) − p̂ζn (b̂n + b̂†n ) +2mennDetermine the expression of ζn by using O. Using the relation derived in (a), show the following relation(5.5.19)

1−2

1
P

=1+2
m ζ2
e n

X e2 f 2k.
me ℏωkkn ℏΩnCheck that this relation leads to the simplified expression of the mass renormalization in meff of theAD-frame Hamiltonian:Xp̂2+ V (x̂ + Ξ̂) +ℏΩn b̂†n b̂n(5.5.20)
ĤAD = Û † ĤC Û =
2meff
n
where


(5.5.21)X ζn 
iÛ = exp − p̂Ξ̂ , Ξ̂ ≡ib̂†n − b̂n
ℏ
Ωn
k

and

(5.5.22)

meff = me

X e2 f 2k1+2
me ℏωk!
.

k

Exercise 5.4 (Spontaneous emission: 1 point). Consider the isotropic two-level atom coupled tovacuum electromagnetic field. Derive the formula (5.3.52) of the spontaneous emission rate from Fermi’s
golden rule.

CHAPTER 6

Machine learning and quantum science
6.1. Introduction
The field of machine learning, especially artificial neural networks, has witnessed remarkable developments in recent years, and is revolutionizing science and technology. Deep neural networks are now
routinely applied to classify or recognize patterns/images, translate sentences or speech between different languages, predict traffic or market trading, and to attain self-driving cars, device/robot controls, andsuperhuman performance in table/video games. In the context of physical science (especially quantum
science), the idea of machine learning has also led to many interesting developments in these years. In thenext two Chapters, we try to give brief and introductory descriptions about some of key concepts/topics inthis rapidly growing field.
In machine learning, there are mainly three classes of problems, namely, supervised, unsupervised,and reinforcement learning. Supervised learning handles data consisting of a pair of vectors {xi , yi }ni=1with y being often called a label and n being the number of data. The goal of supervised learning is
to estimate a function f that relates these two vectors via y = f (x). We call it as “supervised” since
the correct answer (i.e., label) yi is given for each xi by “teacher”, and machine is trained by “teacher”
according to this dataset. In contrast, unsupervised learning deals with data that only contains some setof vectors {xi }ni=1 that is supposed to be randomly generated by a certain probability distribution p(x).
Typically, unsupervised learning aims to either extract essential features/structures behind the data oreven (approximately) construct the distribution p(x) itself by using some tractable ansatz. Since there isno “teacher” who tells us the label, it is called “unsupervised”.
In these two classes of learning problems, the trained machine can at most reproduce or imitate theways that are taught or predicted by the original dataset; machines thus cannot have sort of creativity, inthe sense that they would neither realize performance beyond the teacher nor predict something new thatis not expected from the dataset.
Roughly speaking, reinforcement learning aims to train machines to acquire that sort of creativity forachieving better performance by repeating a huge number of trials and errors. In reinforcement learning,data is not given in advance in contrast to supervised/unsupervised learning, but machine itself acquires
knowledge about data to find the best solution by interacting with an environment. Each solution associates with some figure of merit called “reward”, and on the basis of its value, the machine tries to search abetter solution from the repetitions of trials and errors. In this sense, there is a feedback loop between themachine (also called agent) and environment, and in fact its theoretical framework fits well with theory ofopen quantum systems (or more specifically, continuous quantum measurement) as we discuss later.
115

6.2. BASIC CONCEPTS

116

Below we begin by introducing some basic concepts of machine learning and black-box optimizations.
Building on these understandings, in the next Chapter, we proceed to learn about (deep) reinforcement
learning.

6.2. Basic concepts
6.2.1. Supervised learning.
6.2.1.1. Function approximation, cost function, and training. As noted above, the goal of supervised
learning is to identify a function approximation f that relates inputs x to outputs y. We do this byhypothesizing a certain functional form fθ parameterized by θ:
y = fθ (x).

(6.2.1)Given a training dataset {xi , yi }ni=1 , we then try to optimize θ in such a way that the resulting fθ imitates
the training dataset best.
To do so, we have to quantify how well the obtained function fθ reproduces the training dataset. For
this purpose, we introduce the cost function that usually decomposes into the sum of each data:n(6.2.2)

1X
λ(xi , yi ; θ).
L(θ) =ni=1

One common way is to use the mean squared error as the cost function:n(6.2.3)

L(θ) =

1X|yi − fθ (xi )|2 ,ni=1

while the specific form of the cost function in general changes depending on type of the problem or achoice of fθ .
After specifying the cost function, we next have to specify how in practice we attempt to find the
optimal θ∗ that would minimize L(θ). A process of finding the optimal values θ∗ is commonly referred
to as training, and the most common way for this is to simply apply the gradient descent method. Recall
that we have already encountered the similar idea in the context of imaginary-time evolution of variational
states in Chapter 4. At each time step, the gradient descent updates the parameters according to(6.2.4)θτ +δτ = θτ − η∇θ L,

where η is some positive and small quantity, often called the learning rate. This means that we change
the parameters along the negative gradient of the cost function. In practice, however, it often takes toomuch time to evaluate L by using all the possible training data. This difficulty can be circumvented by
restricting the ensemble average in L to only a small subset of data called mini batch:(6.2.5)θτ +δτ = θτ − η∇θ Lbatch , Lbatch =

1

nX
batch

nbatchi=1|yi − fθ (xi )|26.2. BASIC CONCEPTS

117

with nbatch < n. This is often referred to as stochastic gradient descent in the sense that we use an approximation of the negative gradient, which brings certain noise compared to the “exact” gradient descentcalculated from all the possible data. As we mentioned in Chapter 4, from a geometrical point of view,the gradient descent may be considered as the discrete-time version of the imaginary time evolution on
the variational manifold spanned by fθ .
It is important to note that the choice of a functional ansatz fθ is crucial in supervised learning, but
its exact form is unknown a priori in most of practical problems. So, on what basis can we specify such
ansatz? A possible guiding principle for this is actually similar to the spirit of variational approach we
discussed in Chapter 4. Specifically, we shall choose a functional ansatz that is (1) flexible, i.e., its representation capacity is large enough such that it can fit well with reasonably different patterns in practicalproblems (2) tractable and efficient, i.e., there is some efficient algorithm to optimize θ within a reasonable amount of time and numerical computing resource, and (3) scalable, i.e., it can easily be scaled up
to handle high-dimensional data. Below we briefly discuss several illustrative examples in supervised
learning.
Example 1: Kernel method, support vector machine, and variational quantum circuits. Suppose that
we have dataset {xi , yi }ni=1 . The kernel method aims to predict output y ∈ R for a given input x ∈ Rm
by using the functional form,

y=(6.2.6)nX

αi k(x, xi ),i=1

where {αi } are parameters of this model and k is a certain fixed (nonlinear) function called kernel. Oneof the most common choice is to use the Gaussian kernel2.
k(x, x′ ) = exp −β x − x′(6.2.7)In this choice, the kernel method tries to fit the data by using a linear superposition of many Gaussianfunctions as in Eq. (6.2.6).
The cost function is typically given by the mean squared error(6.2.8)

2

n

n

i=1

j=1

n

X
X
1X
yi −
αj k(xi , xj ) + σ 2
αi k(xi , xj )αj ,L(α) =nij=1

where the last term is the regularization term that is introduced to avoid the overfitting or divergence ofthe parameters {αi }.
In fact, in the present problem, we can analytically identify the optimal α∗ that minimizes the costfunction (see Exercise):(6.2.9)

αi∗ =

n
X
j=1

−1
K + σ 2 I ij yj ,

6.2. BASIC CONCEPTS

118

where I is the n × n identity matrix and we define the kernel matrix by(K)ij = k(xi , xj ).

(6.2.10)
The resulting trained function is(6.2.11)y=nXαi∗ k(x, xi ) =

i=1

n
X

−1
k(x, xi ) K + σ 2 I ij yj .

ij=1This allows us to estimate output y for a given unknown input x.
It is worthwhile to note that the kernel method also gives a functional ansatz of the so-called support
vector machine. Consider a binary classification problem with output y ∈ {−1, 1}. For a given input
x, the support vector machine predicts which class the input vector belongs to, i.e., it gives a positive(negative) class if the function takes a positive (negative) value:(6.2.12)y = signnX!
αi∗ k(x, xi ) + b∗

∈ {−1, 1} ,i=1where we introduce a bias parameter b. In practice, the optimized vector α∗ may have mostly zero components. One can then simplify the evaluation of the function by restricting the summation over only fordataset that have nonzero αi∗ , rather than over all the possible data.
We also note that, in fact, the kernel method has a close connection with some type of quantum
machine learning algorithms. In the context of quantum machine learning, one typically encodes an input
vector x into a quantum state |ψ(x)⟩. One can then introduce the kernel function by using the innerproduct between two different quantum states by(6.2.13)2

k(x, x′ ) = ⟨ψ(x)|ψ(x′ )⟩ ,P
and use the function ansatz y = ni=1 αi k(x, xi ) in the same manner as above, but at this time the kernel
function is evaluated from the overlap of quantum states.
A specific form of the kernel depends on type of available quantum computer and how one encodes
the information into a quantum state. As a simple example, let us first consider a harmonic oscillator andsuppose that input data is encoded by a displacement operator(6.2.14)†|ψ(x)⟩ = D̂(x)|0⟩ = ex(â −â) |0⟩.

For high-dimensional input data, we consider(6.2.15)|ψ(x)⟩ ≡Y|ψ(xi )⟩.

i

Then, the kernel actually corresponds to the usual Gaussian kernal:(6.2.16)k(x, x′ ) = ⟨ψ(x)|ψ(x′ )⟩

2



2
= exp − x − x′.

6.2. BASIC CONCEPTS

119

Experimentally, this choice would be relevant to photonic quantum computers in which a harmonic oscillator represents each electromagnetic mode.
Let us next consider a two-level system and its encoding:(6.2.17)(6.2.18)



x
x
i
|ψ(x)⟩ = exp − xσ̂ x | ↑z ⟩ = cos| ↑z ⟩ − i sin| ↓z ⟩
2
2
2
Y|ψ(x)⟩ ≡|ψ(xi )⟩.
i

This corresponds to the following kernel function(6.2.19)′′k(x, x ) = ⟨ψ(x)|ψ(x )⟩

2

=

Y
i

cos

2



xi − x′i2.

This situation should be relevant to a quantum computer consisting of, e.g., the superconducting qubits
we explained in Chapter 4. One can then use these kernel functions (which can be evaluated on a quantum
computer) as a functional ansatz to fit real-life data. Very roughly speaking, the idea along this line can beconsidered as one of what has been envisioned in the emerging field of quantum machine learning.
In practice, numerical cost of the kernel method is mainly set by the cost of calculating the inverse
−1matrix K + σ 2 I, which basically scales as O(n3 ) with n being the number of training data. Even if
training has been performed in some way, the function evaluation requires the summation over dataset andits cost scales linear with n. Thus, the kernel method can in general be applied to a problem with rather
modest size of data, such as n ≤ O(103 ). This limitation will become a major difficulty when we haveto deal with a large dataset, namely, “big data”. A possible solution for this is to introduce supervised
learning using deep neural networks, as we briefly discuss next.
Example 2: Neural network and classifying phases of matter. Deep learning basically uses the following multilayer feed-forward neural network as a nonlinear functional ansatz fθ :(6.2.20)fθ (x) = g(wL · · · g(w2 g(w1 x))),where x is the input vector, wi are weight matrices that map input vector at each layer i to another vector
that becomes an input vector to the next layer, and g is some nonlinear function called activation function.
Typical choices for g are(6.2.21)x
1
gsigmoid (x) =
,
g
(x)
=
ReLU
0
1 + e−x

x≥0

,

x<0

where ReLU stands for the Rectified Linear Unit function. One may also introduce the bias vector andconsider the affine transformation wi x + b. Although it is not the purpose of this course to explain about
details of deep learning, we here remark that there is an efficient and simple algorithm (based on thestochastic gradient descent and chain-rule calculations of derivatives) to train this function in a reasonable6.2. BASIC CONCEPTS

120

amount of time even for vastly large dataset. From empirical success of deep learning, we know that thetrained function fθ∗ found by this algorithm would fit well training data set and, surprisingly, somehow
generalizes well (i.e., it can predict well for unknown input). Interestingly, this seems to be the case when
the network is deep and has many layers, for which the function appears to be overparametrized. Currently,
it is a commonly held view that there is still little theoretical understanding behind this unreasonablesuccess of deep learning.
One of the most well-known applications of the multilayer feed-forward neural network (or precisely,
so-called convolutional neural network) is the problem of image classification. It basically gets 2D image
as an input data and predicts which class that image belongs to, say, cat or dog. In the context of physics,
the success of deep learning in this problem inspires physicists to apply it to the problem of classifying
phases of matter. From a close analogy with the variational approach in quantum many-body physics,
deep neural networks also motivate quantum physicists to use them as variational ansatz for quantum
many-body wavefunctions, as we briefly discuss later. Interested students may refer to, for example, [Rev.
Mod. Phys. 91, 045002 (2019)].
6.2.2. Unsupervised learning. In unsupervised learning, we deal with data {xi }ni=1 without labels
and typically aim to extract essential information about the probability distribution that generates training
dataset. In some cases, one attempts to directly construct the probability distribution itself by using acertain functional ansatz and optimizing it to find the best approximations of the distribution. There aremany different ways for doing this, but here let us only mention two of the most well-known examples.
Example 3: Autoencoder. Suppose that we have dataset which should possess certain structure, but
we do not know what structure it specifically represents since the data has no labels; one may imagine,for instance, a set of 2D images of animals for some unknown different species. How can one identify
the possible structure behind the data? The idea of autoencoder is to use neural networks to extract key
features of dataset by compressing the original data into a much smaller vector space, but without loosing
essential information. Specifically, it attempts to reproduce its input vector itself as output vector:(6.2.22)x ≃ fθ (x)by optimizing θ in neural networks whose middle layers have much fewer neurons than those in the input/output layers. In this way, at the middle layers, data is significantly compressed and lives in muchsmaller vector space, while hopefully this compression can be done without loosing too much information
about data, i.e., keeping the most relevant features of the input data. The intuition behind this is that (aslong as the learning is successful) the trained network is supposed to reconstruct the input data only fromthe limited amount of information encoded in the restricted middle layer. The neurons in this middle layer
is often called latent variables, since they represent the relevant features of input data which are a priori
unknown. The philosophy behind autoencoder may fit well with physics, namely, the idea that there must
exist only a few key principles behind data generated from the laws of physics that we want to understand;naturally, autoencoder has already found many applications in physical science. One of the recent illustrative applications is an automated discovery of state variables from observations of experimental data

6.2. BASIC CONCEPTS

121[Nat. Comp. Sci. 2, 433 (2022)].
Example 4: Restricted Boltzmann machine and neural network states. Suppose that dataset {xi }ni=1
is generated by some unknown probability distribution p(x). We want to construct a model that imitates
this distribution by using a flexible, efficient, and scalable variational ansatz pθ (x). In practice, we do
not know the exact form of a true distribution p(x), but can access about the empirical “data” distribution
pData (x) that gives its approximation:n

1X
pData (x) =δx,xi .
n(6.2.23)i=1

The most common way for the optimization, i.e., training of the model pθ (x), is to use the cross entropyn(6.2.24)

1X
L(θ) = −pData (x) ln pθ (x) = −ln pθ (xi ).
n
x
X

i=1

We can rewrite it (aside constant) by(6.2.25)

L(θ) = D [pData (x)||pθ (x)] + const.,

where we define the relative entropy for given probability distributions p, q via

D [p(x)||q(x)] ≡(6.2.26)

X

p(x) ln

x

p(x)
.
q(x)

The relative entropy vanishes D = 0 if and only if the two distributions coincide, i.e., p(x) = q(x) for∀x. Roughly speaking, the relative entropy thus quantifies how much there is difference between two
distributions. The minimization of the cost function above then corresponds to the process that makes themodel pθ similar to the empirical distribution pData as much as possible.
The restricted Boltzmann machine (RBM) is one example of possible variational ansatz pθ , which isdefined by(6.2.27)pθ (v) =Xh(6.2.28)pθ (v, h) ≡X e−Eθ (v,h)
hZ,Eθ (v, h) ≡ −a · v − b · h − vT W h, Z ≡X

e−Eθ (v,h) .

v,hwhere v are actual variables for representing data and called “visible” units, while the “hidden” units h
are summed over before giving the model and do not appear in pθ (v). These units are typically binary
variables vi , hi ∈ {0, 1}. The model parameters θ include real vectors a, b and matrix W ; we emphasizethat there are no direct couplings among hidden variables h and also among v. This particular structure of
the couplings is the reason for calling this model “restricted” since the coupling network of v, h can thenbe represented by the connection between the single hidden layer and the visible layer.

6.3. BLACK-BOX OPTIMIZATION

122

An apparent difficulty in this variational ansatz is that it is computationally intractable to perform thesummation over exponentially large number of configurations for v and h in the partition function Z andthus to evaluate pθ (v). That said, it is noteworthy that one can still efficiently evaluate the conditional
distribution as follows:pθ (v, h) Y ezi hi(6.2.29)pθ (h|v) ≡=, z ≡ b + W v.
pθ (v)
1 + ezi hiiA similar expression can be obtained for pθ (v|h). This fact simplifies the calculation of the gradient of
the loss function ∇θ L to some extent, but the computational difficulty of Z itself is unavoidable anyway.
We mention that there are still several heuristic approaches to tackle this difficulty, such as the method
so-called contrastive divergence.
In the context of quantum many-body physics, the RBM inspires physicists to introduce the following
variational many-body states often called neural network states1 [Science 355, 602 (2017)]:|ΨRBM ⟩ =(6.2.30)X

Ψσ |σ⟩σΨσ ≡(6.2.31)
1 X
exp a · σ + b · h + σ T W h .
Zhnwhere σ ∈ {0, 1} are binary variables representing spin-1/2 states and |σ⟩ corresponds to the basis states.
It is known that any many-body state can then be arbitrarily well approximated by an RBM state if thenumber of hidden units is larger than the number of configurations |σ⟩ satisfying |Ψσ | > 0. Intuitively,
one may say that the correlations of physical variables σ are effectively induced by hidden variables h andthey can be nonlocal if the coupling W is long-ranged2. In practice, to find the optimal RBM variational
state for, e.g., the ground state of some many-body Hamiltonian, one can use the stochastic gradientdescent and standard techniques which are common in variational Monte Carlo method.

6.3. Black-box optimization
To minimize a cost function, one has to choose some optimization algorithms. As introduced above,the most elemental way for this is to use the (stochastic) gradient descent. In Chapter 4, we have also mentioned that one can improve this method and use the so-called natural gradient descent when the geometryof the variational manifold is appropriately taken into account. From a broader perspective, there exist
other types of optimization methods that do not rely on the local gradient as used in the aforementioned
examples, but on some heuristic algorithms, some of which are inspired by ecology. These methods areheuristic in the sense that they usually lack solid theoretical justifications, while are empirically known tobe effective in many challenging optimization problems.
1Precisely speaking, neural network states often refer to a slightly generalized version of the RBM states, in the sense that the

network structure of physical σ and hidden h variables is not necessarily restricted to the bipartite graph with a single hidden
layer as in RBM.
2If couplings W are short-ranged or sparse, one can show that the RBM states have only small entanglement, i.e., they obey thearea law, and thus can be efficiently represented by the matrix product states.

6.3. BLACK-BOX OPTIMIZATION

123

Algorithm 1 Random search
1: Initialize: θ
2: while till convergence of F do
3:
Get: fitness value F (θ)
4:Sample a new parameter set θnew from θ5:
if F (θnew ) > F (θ) then6:
θ ← θnewWe here cover several examples of such heuristic optimization algorithms, also known as black-box
optimizations. In this approach, to optimize the objective (that is not in a solvable, closed form function)with respect to parameters θ, we do not need to calculate its derivative or even require the differentiability,
but use the value of the fitness function F (θ), which gives a certain measure about the optimality of asolution. This fitness function may be the objective function itself, but needs not to be so. While thegradient-based optimization methods tend to be trapped in local minima, black-box optimizations are
gradient-free and can often circumvent this difficulty; this is the reason why black-box optimizations areoften also referred to as global search methods. Below we assume that the optimization problems isformulated in such a way that the goal is to maximize the fitness function (which is consistent with theusual convention in the literature).
Random search. The simplest example of black-box optimization would be the random search.
Firstly, it randomly chooses an initial value θ0 and calculates the fitness F (θ0 ). We then stochasticallygenerate a new set of parameters θ0new starting from θ0 by using some given probability distribution. Calculating the fitness F (θ0new ) for this parameter set, and if the new one is better than the old one, i.e.,
F (θ0new ) > F (θ0 ), we accept the change and set the parameters at the next step as θ1 = θ0new , otherwise
we set θ1 = θ0 . This procedure will be repeated many times until the convergence of the fitness. As aprobability distribution for selecting a new candidate parameter θtnew from θt , we typically use the uniform distribution over the hypersphere with a fixed radius centered at θt . The algorithm is summarized in
Algorithm 1.
Evolutionary strategy. A major difficulty in random search is its low sample efficiency; it literally
searches the whole parameter space randomly and it often fails or takes too much time to finally reach thecorrect optimal solution. In this context, the evolutionary strategies refer to methods that aim to improve
the search efficiency by developing a heuristic algorithm inspired by the evolution process. While there aremany variants of evolutionary strategies, here we consider one concrete example, known as the covariancematrix adaptation evolution strategy (CMA-ES).
Algorithm 2 CMA-ES
1: Initialize: θ, η, σ
2: while till convergence of F do
3:
Sample n random vectors {ϵi }ni=1 , each component sampled from N (0, 1)
4:
Get: fitness values
Fi = F (θ + σϵi ) for i = 1, 2, . . . , nP5:θ ← θ + η n1 ni=1 ϵi Fi

6.3. BLACK-BOX OPTIMIZATION

124

Algorithm 3 Genetic algorithmn1: Initialize: {θi }i=1 , σ, m(≤ n)
2: while till convergence of F do
3:
Get: fitness values Fi = F (θi ) for all i4:
Sort {θi } in the descending order of {Fi }5:θ1′ = θ1 , F1′ = F1
6:
for i = 2, 3, . . . , n do
7:
Sample k from 1, 2, . . . , m. Sample random vector ϵ, each component from N (0, 1)
8:
θi′ = θk + σϵ9:θi ← θi′ for all i

The algorithm of the CMA-ES is summarized in Algorithm 2. The key point is that we use a minibatchPof random vectors {ϵi }ni=1 and use the averaged fitness n1 ni=1 ϵi Fi as a (finite step) differential direction
at the next generation. In this sense, one can say that this algorithm is in the similar spirit of the gradientdescent algorithms, but the advantage here is that one does not need to explicitly calculate the derivative
or even assume the differentiability of the fitness function. While we here discuss the simplest CMA-ES
where only θ is updated, it can in general also update the parameters in the Gaussian sample distribution,i.e., its mean and covariance matrix, at each generation. In this case, under certain conditions, the updateof the parameters of the sample distribution in the CMA-ES can be interpreted as the discretized versionof the natural gradient descent over the space of sample distributions.
Genetic algorithm. Another well-known black-box optimization method is the so-called genetic algorithms. This class of methods generate a population of individual sets of parameters {θi }ni=1 , calculate
the fitness values for a new individual (mutant) obtained by combining some of old individuals (parents),
and pick up the best mutants that are used to create the next generation. While there are again many variants of genetic algorithms, we give the simplest version of the algorithms in Algorithm 4 as an illustration.
Differential evolution. The above algorithms are elementary, representative examples of black box
optimization algorithms, and can already be useful for certain problems as we will see their applications
Algorithm 4 Differential evolution
1: Initialize: θi ∈ Rd for i = 1, 2, . . . , n, f > 0, Cr > 0
2: Get: fitness values Fi = F (θi ) for all i
3: while till convergence of F do
4:for i = 1, 2, . . . , n do
5:
Sample mutually exclusive integers k, l, m ̸= i from 1, 2, . . . , n6:µi = θk + f (θl − θm )7:
Sample β from 1, 2, . . . , d8:for α = 1, 2,(. . . , d do
9:
10:
11:
12:

(θi′ )α =(µi )α(θi )αif α = β or randα,β [0, 1] ≤ Cr
otherwise

Get: fitness value Fi′ = F (θi′ )if Fi′ ≥ Fi thenθi ← θ′ , Fi ← Fi′

6.3. BLACK-BOX OPTIMIZATION125

to deep reinforcement learning in the next Chapter. That said, there have been many proposed ways toimprove the performance of those algorithms and, for the purpose of solving actual problems at your hand,
it may be useful to introduce a more sophisticated algorithm here.
The differential evolution is known to be one of the most competitive black-box optimization algorithms in high-dimensional nonconvex search space. Its basic idea is similar to the above algorithms, but
it includes several heuristic improvements that are empirically known to be able to realize more efficient
search. The algorithm is summarized in Algorithm 4. The key parameter is the crossover factor Cr whichcharacterizes how often the mutation occurs in producing the offspring individuals. A larger Cr leads
to more frequent mutations and thus expedites the exploration while a lower Cr tends to expedite theexploitation of the current information about the search space. To optimize the exploration-exploitation
balance, it may be advantageous to randomly use different sets of parameters at each generation, say(f, Cr ) = (1, 0.1), (1, 0.9), (0.8, 0.2) as proposed in [Wang et al., IEEE Trans. Evol. Comput. 15, 55-66(2011)].

6.3. BLACK-BOX OPTIMIZATION

126

Summary of Chapter 6
Section 6.2 Basic concepts• Supervised learning deals with data having labels, and aims to identify a function approximation, which relates input x to output y, on the basis of hypothesizing a functional form,y = fθ (x). The functional ansatz should be tractable/efficient and possess flexibility andscalability.
• Kernel method provides a functional approximation that consists of a superposition of thekernel functions, and allows one to perform a nonlinear classification known as the support
vector machine. Quantum computer may offer alternative way to define the kernel function
from the overlap between different quantum states.
• Deep learning uses the multilayer feed-forward neural network as a functional ansatz. Thereexist efficient algorithms and many heuristics to optimize this function in a reasonable amountof time even for vastly large dataset.
• Unsupervised learning deals with data without labels, and aims to extract essential information
about the probability distribution that generates the dataset. In the context of deep learning,
one famous example is the autoencoder, which would allow one to compress data into lower
dimensional vector space, but still keep the most relevant features in the data.
• Another example is the restricted Boltzman machine that attempts to approximate the wholeprobability distribution itself by using functional ansatz inspired by the Gibbs distribution.
Section 6.3 Black-box optimization• Gradient-based optimization algorithms, such as the (stochastic) gradient descent, often fail towork due to the presence of local minima. In such a case, a heuristic, gradient-free optimization method called black-box optimization can be useful.
• In black-box optimizations, gradients or even the differentiability of the cost function are
unnecessary, but the parameters are optimized in terms of the fitness function that provides acertain measure about the optimality of a solution.
• The simplest black-box optimization algorithm is the random search, which literally searches
the whole parameter space randomly according to a certain distribution. Its refinement, called
evolutionary strategy, would allow for a more efficient search.
• One of the most competitive approaches, called differential evolution, is similar to the genetic
algorithm, but contains a number of heuristics that can expedite the search over the highdimensional, nonconvex parameter space.

6.4. EXERCISES

127

6.4. Exercises
Exercise 6.1 (Gaussian kernel method: 1 point). Given dataset {xi , yi }ni=1 , consider approximating
function byy=(6.4.1)nX

αi k(x, xi )i=1

with the Gaussian kernel:2k(x, x′ ) = exp −β x − x′.

(6.4.2)

As discussed in the main text, the training is performed by minimizing the following cost function(6.4.3)

n

n

i=1

j=1

2

n

X
1X
σ2 X
yi −
αi k(xi , xj )αj .
L(α) =
αj k(xi , xj ) +
n
n
ij=1

Show that the optimal α∗ is given byαi∗ =(6.4.4)nX

−1
K + σ 2 I ij yj .

j=1

One can now use this α∗ to predict the output ŷ for an unknown input x from

(6.4.5)ŷ =nXαi∗ k(x, xi ) =

i=1

n
X

−1
k(x, xi ) K + σ 2 I ij yj .

ij=1

However, this expression does not tell us about how good this estimation is. Explain how one can determine confidence level of the estimation by showing that the standard deviation of the estimated value ŷ
can be given bysXk(x, xi ) (K + σ 2 I)−1
(6.4.6)δ ŷ = σ 2 + 1 −ij k(xj , x).
ij

Hint: consider Gaussian process regression for this estimation problem and calculate the conditional probability distribution from the Bayesian analysis.

Exercise 6.2 (Global optimization: 1 point). Consider and numerically demonstrate a concrete example of objective functions that is difficult to (globally) minimize by gradient-descent methods, but canbe optimized by black-box optimizations. Discuss why that is the case by plotting the optimization landscape.

CHAPTER 7

Reinforcement learning
7.1. Introduction
As briefly mentioned in Chapter 1, the goal of reinforcement learning is to train machines solely frommany repetitions of trials and errors for achieving a certain task whose performance can be quantified by
the so-called “reward” function. In contrast to supervised/unsupervised learnings, reinforcement learning
does this without given data or knowledge, but rather machine itself acquires information about data tofind the best solution by interacting with an environment. We may say that reinforcement learning canacquire sort of creativity in the sense that a machine can behave in a new and unexpected way (at leastfrom the point of view of human), which makes a sharp contrast to supervised learning where a machinecan, at best, imitate the teacher.
In reinforcement learning (RL), an agent observes the state of the environment and decides how to
act to maximize the reward. This action causes changes in state of the environment and thus there is a
closed feedback loop between the agent and environment. A way of determining the action taken by anagent based on the state of the environment is often called policy. Importantly, there is no teacher who
tells which action should be taken; instead, the agent must discover the optimal action that leads to thebest reward by repeating trials an errors many times. The main goal of Chapter 7 is to help you to gain
intuition/key ideas behind the reinforcement learning algorithms, rather than teaching specific techniques
for numerical implementations, such as how to use existing libraries etc.

7.2. Motivating example
To illustrate these ideas (and another important concepts of RL) in a simple example, consider a single
classical particle on 1D lattice whose position is labelled by x ∈ Z. The particle at position x randomly
hops to the right x + 1 or left x − 1 site at each discrete time t ∈ {0, 1, 2 . . .}. The hopping is asymmetric,
namely, the probabilities of hopping to the right (+) or left (−) directions are
1 ± σδp, σ ∈ {+1, −1} ,2where δp ∈ [0, 1) is a fixed asymmetric parameter. Now suppose that we want to guide this stochastic
particle into the origin of 1D lattice, i.e., x = 0 site. To do so, we consider a situation that, at each time, anagent can change the sign of asymmetric hopping probabilities, σ ∈ {+1, −1} depending on the positionof the particle x. Building on this, each notion of RL framework in the present problem can be formulated
as follows:(7.2.1)

p(±)
σ =

128

7.2. MOTIVATING EXAMPLE

129• Environment: stochastic particle on 1D lattice that randomly hops to right or left sites with(±)
probabilities pσ .
• State: position x of the particle. State at time t is represented by st = xt .
• Agent: external controller that continuously observes the state of the environment and uses this(±)
information to manipulate σ ∈ {+1, −1} in pσ to guide the stochastic particle to the origin
x = 0.
(±)• Action: choice of the sign σ ∈ {+1, −1} in pσ . Action at time t is represented by at = σt .
• Policy: conditional probability distribution πθ (σ|x) of choosing action σ given a state x. The
distribution function is parameterized by a set of parameters θ.
• Reward: the (minus) distance rt = −|xt | from the origin at each time.
• Return: the sum of subsequent reward (with discount factor γ) which is defined by1(7.2.2)

Rt =

∞
X

′

γ t rt+t′ , rt = −|xt |, 0 ≤ γ ≤ 1.

t′ =0

• Quality function: the expectation value of the return for a given state s and action a at time t:(7.2.3)Qπ (s, a) = E[Rt |st = s, at = a].
Note that the ensemble average E is taken over all the possible subsequent stochastic evolutionsof the state conditioned on st = s, at = a. Their sequence is affected by the feedback loop
determined by a given policy π. Thus, the quality function also depends on π as indicated by thesubscript in Qπ (s, a).
• Value: similarly, the expectation value of the return for a given state s at time t is called thevalue:(7.2.4)Vπ (s) = E[Rt |st = s].
Here, the ensemble average E is taken over all the possible subsequent stochastic evolutions
conditioned on st = s.

The ultimate goal of RL then is to find the best policy πθ∗ with the optimized parameters θ∗ , which
maximizes the value V . If we identify the probabilities in πθ themselves as the parameters θ, the optimal
policy in the present example can simply be given by(7.2.5)πθ∗ (σ|x) =


 1+σ

2
 1−σ
2

x<0

.

x>0

In a more complex realistic problem, however, it is almost impossible to analytically find an exact, optimal
policy. We thus need a systematic way to train the parameters θ to maximize the expected reward Vπθ . To
efficiently learn the optimal policy, the agent should tend to perform action that is found to be favorable foracquiring better reward (i.e., the agent needs to exploit the knowledge from experiences), but at the same
1Roughly speaking, the factor γ plays the role of effective cutoff for a number of future rewards that the agent takes into account;it represents the agent’s ability of foreseeing. For instance, the agent with γ = 0 behaves in a greedy manner, i.e., it always tries
to maximize the reward at next step, but does not think about a long-term return.

7.3. FORMALISM: MARKOV DECISION PROCESS

130

time it needs to try new possibilities to find action that leads to better reward in the future (i.e., the agent
also needs to explore to discover a possibly better new solution). The RL algorithms explained below aim
to find the optimal policy by balancing these exploitation and exploration in the search space.
7.3. Formalism: Markov decision process
Let us now introduce a general framework of RL on the basis of Markov decision process. Consider
state space S and action space A. The environment considered by the Markov decision process thensatisfies the property that future dynamics only depends on the current state. In other words, its dynamics
is fully characterized by the transition probability from st ∈ S to st+1 ∈ S given action at ∈ A:p(st+1 |st , at ).

(7.3.1)The Markov decision process also assigns the real number called the reward rt for each pair (st , at ) ofstate and action at time t. In other words, the reward rt is a function rt (st , at ) of the current state-action
pair (st , at )2.
The goal of the agent is to maximize the total reward accumulated over the whole evolution. As
introduced above, we thus have to consider the return, which is the sum of subsequent reward with discountfactor γ:(7.3.2)

Rt =

∞
X

′

γ t rt+t′ , 0 ≤ γ ≤ 1.

t′ =0

We also need to introduce the function, policy, to describe how the agent chooses the next action for agiven current state. Specifically, a policy π is usually defined as a conditional probability distribution
π(a|s) for the agent selecting action a given current state s. To formulate the optimization problem, wethen define the quality function and the value function by(7.3.3)(7.3.4)Qπ (s, a) = E[Rt |st = s, at = a]Vπ (s) = E[Rt |st = s],where the ensemble average E is taken over all the possible subsequent state evolutions generated by thepolicy π with initial conditions st = s, at = a or st = s.
Summarizing, the RL problem on Markov decision process is defined by the optimization problem offinding the following π∗ on a set (S, A, r, p, γ):Vπ∗ (s) ≥ Vπ (s), ∀s, π,(7.3.5)or equivalently,V∗ (s) ≡ maxπ Vπ (s).

(7.3.6)at→ st+1 , however, for the sake of simplicity
t+1 at the next step reached by st −2The reward can in general depend on the state s

we assume that rt only depends on (st , at ) as this situation is most commonly encountered in many practical problems.

7.4. VALUE-BASED SEARCH

131

This is obviously equivalent to the following optimization problems in terms of the quality functions:Q∗ (s, a) = maxπ Qπ (s, a),(7.3.7)

where V∗ (s) = maxa Q∗ (s, a) is satisfied. Note that the optimal policy π∗ achieves both optimal value
and action-value functions.
There are basically two different types of approaches to tackle this optimization problem. The firstone, called a value-based search, attempts to evaluate the quality function Q for possible state and action, and chooses the action at = argmaxa Q(st , a) that gives the maximum quality at each time stepduring the training process. While the policy π does not appear manifestly here, after the quality function
converges to the optimal one Q∗ , this strategy automatically defines the optimal policy π∗ in which theaction is (deterministically) chosen as at = argmaxa Q∗ (st , a). The second strategy, called a policy-based
search, aims to approximate the conditional probability distribution for policy π and directly optimize πby (typically) some iterative methods to maximize the value. Below we briefly review the main ideas andprocedures in each of the two approaches.

7.4. Value-based search
7.4.1. Bellman equation. To formulate an empirical method for a value-based search, we start fromthe following recursive relation satisfied by the value function (often called Bellman equation):"(7.4.1)Vπ (s) =X#

π(a|s) r(s, a) + γ

Xp(s′ |s, a)Vπ (s′ ) .

s′ ∈S

a∈A

This equation can readily be derived from the definition (7.3.4) of the value function. In particular, theoptimal value function (7.3.6) satisfies"#
X(7.4.2)V∗ (s) = max r(s, a) + γp(s′ |s, a)V∗ (s′ ) .
a

s′ ∈S

For a given set of (π, p, r), these equations are ultimately linear problems for a vector Vπ (s) and the S × S
matrix with S being the total number of states. However, in practice, it is almost intractable to exactly
solve this problem because S is typically very large.
However, one may still iteratively solve this equation, called value iteration:"(7.4.3)V (s) ← max r(s, a) + γa#Xp(s′ |s, a)V (s′ ) .

s′ ∈SIn this way, one could find the optimal value function V∗ (s) and thus obtains the optimal policy thatP
chooses the action according to argmaxa r(s, a) + γ s′ ∈S p(s′ |s, a)V∗ (s′ ) . A major difficulty in thisiterative approach is that it contains the summation over all possible states s′ at each iterative step. More
importantly, this method requires the complete knowledge about the exact form of the conditional probability distribution p(s′ |s, a) in the first place, which is usually not available. We thus need to consider
alternative approach.

7.4. VALUE-BASED SEARCH

132

Algorithm 5 Q-learning
1: Initialize: Q(s, a) for ∀s, a
2: while till convergence of Q do
3:
Get: initial state s
4:while till s is terminal do
5:Take(a from s and get next state s′ , based on ϵ-greedy policy of Q:
6:
7:
8:
9:
10:
11:

random action
with probability ϵ′argmaxa′ Q(s, a ) otherwise
if s′ is not terminal thenQ(s, a) ← Q(s, a) + α [r(s, a) + γ maxa′ Q(s′ , a′ ) − Q(s, a)]else
Q(s, a) ← Q(s, a) + α [r(s, a) − Q(s, a)]s ← s′

a=7.4.2. Q learning. One possible way to circumvent these difficulties is to iteratively update the quality function Q(s, a), rather than the value function itself, while using this function to approximate theoptimal policy during the process of training. To this end, we first start from the similar recursive relation
for the optimal quality function:(7.4.4)Q∗ (s, a) = r(s, a) + γ

Xp(s′ |s, a) max
Q∗ (s′ , a′ ),
′
a

s′ ∈S

which can be obtained from Eq. (7.4.2) by noting the relation V∗ (s′ ) = maxa′ Q∗ (s′ , a′ ). In the similarmanner above, this recursive equation can ideally be solved by the iterative method as follows:"#Xp(s′ |s, a) max
Q(s′ , a′ ) − Q(s, a) ,(7.4.5)Q(s, a) ← Q(s, a) + α r(s, a) + γ
′
a

s′ ∈S

where α is a small positive number that acts as the learning rate. This method, however, still includes thePintractable summation s′ ∈S over the whole state space.
The idea of Q-learning is to train the quality function iteratively from a sequence of states, actions,and rewards {st , at , rt }t=0,1,2... experienced by the agent, each of which is called an episode. Then, the exP
pectation value s′ ∈S p(s′ |s, a) maxa′ Q(s′ , a′ ) may be replaced by the estimated value maxa Q(st+1 , a)at the state st+1 arrived from st with action at . Specifically, we arrive at the iterative procedure(7.4.6)hi
Q(st , at ) ← Q(st , at ) + αt r(st , at ) + γ max Q(st+1 , a) − Q(st , at ) ,awhere the residual term r(st , at ) + γ maxa Q(st+1 , a) − Q(st , at ) is often called the temporal difference.
Note that now there is no need to know about the complete form of p(s′ |s, a), which significantly simplifies
the iterative analysis. This approach, coined as Q-learning, can be effective especially when the state andaction space is not very large, and thus one can prepare the whole table of Q function on the variable
space (s, a) ∈ S × A. In particular, under certain conditions, it is theoretically known that the Q-learning
allows the quality function to converge to the optimal one, Q → Q∗ , after a sufficiently large number of
iterations if the learning rate satisfies

7.4. VALUE-BASED SEARCH(7.4.7)∞X

αt → ∞,

t=0

∞
X

133

αt2 < ∞.

t=0

One may wonder how one should choose action at in the middle of training process where the Q
function is still suboptimal. In practice, there are several options for this, but one common approach is theso-called ϵ-greedy algorithm defined by(7.4.8)random action
with probability ϵ,a=′argmax ′ Q(s, a ) otherwiseawhere the action is set to be the one that maximizes the (suboptimal) Q function at the time with probability
1 − ϵ, while otherwise it is chosen in a completely random manner. In this way, one can balance theexploitation and exploration in the RL problem. The usual practice is to start with high ϵ, say ϵ = 1, to
expedite the exploration, while eventually decreases it to smaller value, say ϵ = 0.05 at later stages of the
training. To summarize, the Q-learning proceeds as described in the pseudocode of Algorithm 5.
7.4.3. Deep Q-learning. The dimension of the state-action space is often quite large in many realistic
problems, and in practice it is often intractable to prepare the complete table of the Q function on thewhole state-action space; this makes it challenging to apply the value iteration approach explained above.
To overcome this difficulty, one can use a flexible and efficient functional ansatz to approximate the Q
function. Given recent successes of deep neural networks, it is then natural to use neural network functions
for the purpose of approximating the Q function and apply the RL algorithm to more complex problems
that are otherwise challenging to solve by naive value iterations above.
We recall that the deep neural network is defined by the function(7.4.9)fθ (x) = g(wL · · · g(w2 g(w1 x))),where x is input, wi are weight matrices at each layer i, and g is the nonlinear function which here weshall choose to be the ReLU function:(7.4.10)xg(x) =
0

x≥0

.

x<0

Suppose that we have a single data (x, y) and define the loss function for this by1(y − fθ (x))2 .
2
To approximate the input-output relation y ∼ fθ (x), a standard way was to optimize the neural network
parameters θ by the gradient descent(7.4.11)(7.4.12)L(θ) =θ ← θ − η∇θ L.

7.4. VALUE-BASED SEARCH

134

Algorithm 6 Deep Q-learning with the fixed target network
1: Initialize: network parameters θ in Qθ (s, a) and set Q̃ = Qθ
2: while till convergence of θ do3:
Get: initial state s
4:while till s is terminal do
5:Take(a from s and get next state s′ , based on ϵ-greedy policy of Qθ :
6:
7:
8:
9:
10:
11:
12:

random action
with probability ϵ′argmaxa′ Qθ (s, a ) otherwise′
if s is not terminal thenθ ← θ + η r(s, a) + γ maxa′ Q̃(s′ , a′ ) − Qθ (s, a) ∇θ Qθ (s, a)elseθ ← θ + η (r(s, a) − Qθ (s, a)) ∇θ Qθ (s, a)s ← s′
Every C steps set Q̃ = Qθa=Fixed target network. Building on this, we may now consider applying the deep learning method tothe present RL problem. To do so, we receive the observed state-action pair as input data and use fθ toapproximate the Q function, which is known as the deep Q-network (DQN). The goal is to minimize thetemporal difference and approximate the optimal relation r(s, a) + γ maxa′ Q̃(s′ , a′ ) ∼ Qθ (s, a), whichcorresponds to the target relation y ∼ fθ (x) above. Specifically, we get the correspondence as follows:(7.4.13)x = (s, a), fθ = Qθ ,(7.4.14)y = r(s, a) + γ max
Q̃(s′ , a′ ), s −→ s′ .
′aaHere, note that “data” y is obtained by using the so-called fixed target network Q̃, whose parameters arecopied to those of Qθ only at every C iterations. This use of a separate network to estimate the target
data y is a heuristic method to improve the convergence and stability of training in the RL algorithm. The
resulting loss function is1(yt − fθt (xt ))22
2
1
(7.4.16)=r(st , at ) + γ max Q̃(st+1 , a) − Qθt (st , at ) ,a2
where st+1 is chosen by some policy using Qθ such as the ϵ-greedy method in the similar manner as donein the usual Q-learning. The neural network parameters are then updated as(7.4.15)(7.4.17)(7.4.18)

L(θt ) =θt+1 = θt − η∇θt L(θt )= θt + η r(st , at ) + γ max Q̃(st+1 , a) − Qθt (st , at ) ∇θt Qθt (st , at ).
a

We summarize the algorithm of this deep Q-learning in the pseudocode of Algorithm 6.
Experience replay. Unfortunately, one has to keep in mind that this simple deep RL algorithm still
rarely works as is in many real-life tasks or some physics problems. To overcome this issue, many heuristic
schemes have been proposed in these years; the use of the fixed target network Q̃ above is one such an

7.4. VALUE-BASED SEARCH

135

Algorithm 7 Deep Q-learning with the fixed target network and experience replay
1: Initialize: network parameters θ in Qθ (s, a) and set Q̃ = Qθ
2: Initialize: replay memory D
3: while till convergence of θ do
4:
Sample minibatch of N experiences {ei }N
i=1 randomly from D
5:for every experience ei = (si , ai , ri , s′i ) in minibatch do
6:
if s′ is not terminal then7:yi ← ri + γ maxa′ Q̃(s′i , a′ )
8:
else
9:
yi ← ri
10:11:Pθ ←θ+η N
i=1 (yi − Qθ (si , ai ))∇θ Qθ (si , ai )Every C steps set Q̃ = Qθexample. The progress along this line is very rapid and it is impossible to cover all the heuristics here, but
let us just introduce one commonly used RL technique known as experience replay.
In the deep RL algorithm above, we directly used a sequence of agent’s experience in each episodeto train the network. The difficulty here is that state-action pairs (st , at ) in those sequences are in general
highly correlated as they are generated sequentially in the environment, which often makes the training
inefficient.
The basic idea behind experience replay is to break this correlation between different samples by
shuffling the order of training data used. Specifically, before starting the training, we construct the replay
memory D which is data set of random agent’s experiences et = (st , at , r(st , at ), st+1 ); this can be doneby simply repeating simulations in the environment using, say, the ϵ-greedy method of some initialized Q
function. During the training, we then randomly sample experiences from the memory D, and use themto define the loss function and update the neural network parameters via the stochastic gradient descent.
It is usually advantageous to construct and use minibatch of N experiences from D rather than using thewhole replay memory. Specifically, together with the fixed target network technique above, we calculate
the loss as follows:
2
N 
1 X
′ ′
(7.4.19)
L(θ) =
ri + γ max
Q̃(s
,
a
)
−
Q
(s
,
a
)
,
θ i i
i
a′
2N
i=1

which leads to the stochastic gradient descent

N 
X
′ ′
(7.4.20)θ ←θ+η
ri + γ max
Q̃(si , a ) − Qθ (si , ai ) ∇θ Qθ (si , ai ).
′
i=1

a

The algorithm is summarized in Algorithm 7. For further reading about other heuristic methods to improve
the DQN algorithms, see for example arXiv:1710.02298. Nevertheless, the basic techniques explained
here, including ϵ-greedy method, fixed target network, and experience replay, constitute the key elements
of most of more advanced algorithms; notably, these are the main techniques that have enabled DeepMind
to successfully train DQN to demonstrate the human-level control of 49 Atari video games in [Nature 518,
529 (2015)], despite the common belief at the time.

7.5. POLICY-BASED SEARCH

136

7.4.4. Physics application: quantum control of continuously monitored systems. As one possible application of deep Q-learning to quantum science, here we briefly discuss the measurement-based
feedback control of a quantum particle. Specifically, we assume that the particle feels certain potential V
and is also subject to external force F that is appropriately controlled by feedback loop:p̂2+ V (x̂) − F x̂.
2m
We then consider continuously monitoring the particle position; as we learned before, this measurement
backaction makes the dynamics noisy and nonunitary. The time evolution equation is given by (cf. Eq.
(3.6.50))hi√κ
(7.4.22)d|ψ⟩ = 1 − iĤ − (x̂ − ⟨x̂⟩)2 dt|ψ⟩ + κ(x̂ − ⟨x̂⟩)dW |ψ⟩.
2
To facilitate the deep Q-learning, it is in practice useful to discretize F into ∼ 20 possible values within
the interval [−Fmax , Fmax ].
The reward is chosen depending on the task the feedback controller aims to achieve. For instance,
if the potential satisfies V → ∞ as |x| → ∞ and the goal is to prepare the ground state of the system,
we may choose the reward to be the minus system energy r = −⟨Ĥ⟩. In contrast, if the potential isunbounded V → −∞, |x| → ∞ and the goal is to stabilize (otherwise unstable) state, we may choose thereward in such a way that r = 1 if the wavepacket remains stable at final time t = T while r = 0 if it fails
to do so. The latter problem setting gives the quantum analogue of the common RL problem known as the
CartPole problem.
Summarizing, the present quantum control problem can be formulated as the following RL task:(7.4.21)Ĥ =• State: quantum state |ψ⟩.
• Action: external force F ∈ [−Fmax , Fmax ].
• Environment: the stochastic Schrödinger equation (7.4.22).
• Reward: the minus energy r = −⟨Ĥ⟩ (for the ground-state preparation) or r = 1 only if the
wavepacket remains stable at t = T (for the stabilization of an unstable state).
Note that, in realistic quantum systems, what is available to an observer is the measurement outcome(i.e., realization of a sequence of the Wiener process dW ). Thus, to simulate experimentally relevant
situations, one has to use this information as input to the Q function rather than a quantum state itself
during the training. Nevertheless, if an observer has a good knowledge about the initial state, she/he may
combine it with the measurement outcome to reconstruct the wavefunction by solving the time-evolution
equation, and would use a quantum state itself or its distribution moments as the input. It is reported in[PRL 125, 100401 (2020)] that deep Q-learning can outperform the known control strategies, especiallywhen V contains the nonquadratic terms and thus the problem becomes nonlinear.
7.5. Policy-based search
The DQN algorithms above would be a first choice to solve realistic RL problems if action space isdiscrete and limited in such a way that the search problem in maxa′ Q̃(s′ , a′ ) is still tractable. If not,especially when the action space is continuous, yet another approach called policy-based search is in7.5. POLICY-BASED SEARCH

137

general more favorable. While we do not explain details about the policy-based approaches in this lecture,
let us outline basic ideas below.
We first recall that the policy is defined by the conditional probability distribution of action a for agiven state s:πθ (a|s),(7.5.1)

where θ is a set of parameters that define the policy. The goal of RL is to maximize the total expected
reward Xpθ {(st , at )}Tt=1 R {(st , at )}Tt=1 ,(7.5.2)Eπθ [R] ={(st ,at )}T
t=1

P
where R = Ti=1 γ i−1 ri represents the accumulated total reward along the whole episode {(st , at )}Tt=1
and the summation is taken over all the possible episodes. Due to the Markov property, we can represent
the probability pθ by the product of the conditional probabilities asT Yp(st+1 |at , st )πθ (at |st ).
pθ {(st , at )}Tt=1 =(7.5.3)t=1This allows us to rewrite∇θ log pθ =(7.5.4)TX∇θ log πθ (at |st ).

t=1

The gradient of the expected reward is then given by"(7.5.5)∇θ Eπθ [R] =XRpθ ∇θ log pθ = Eπθ R(7.5.6)θ ← θ − η∇θ Eπθ [R] = θ − ηEπθ R!#∇θ log πθ (at |st ).

t=1{(st ,at )}T
t=1

We may then use this gradient to update the parameters θ
"

T
X

T
X

!#
∇θ log πθ (at |st ).

t=1

In practice, one has to approximate the ensemble average Eπθ by using a certain subset of episodes andrepeat this iterative procedure many times. Similar to the value-based search, it is possible to use deepneural networks to approximate the policy function πθ . Many possible policy-based deep RL algorithms,
such as the actor-critic methods, are proposed; see e.g. arXiv:1810.06339. That said, it would be importantto keep in mind that it has been reported that policy-based RL algorithms tend to be less sample efficientand sensitive to choices of hyperparameters, network architectures, and reward functions, which oftencauses difficulties for reproducing numerical results; see arXiv:1709.06560. Generally speaking, policybased methods would be a better option when (i) action space is continuous and (ii) simulation of theenvironment is fast and numerically cheap.

7.7. SOME TIPS ON IMPLEMENTATION

138

7.6. Black-box optimization in deep RL
Since the performance of many deep RL algorithms tends to be sensitive to choices of specific hyperparameters and it is often this aspect that makes the application of deep RL methods to realistic problems
not straightforward, it would be highly desirable to construct a more stable and versatile approach to deep
RL3. One emerging candidate for solving this issue is to employ the black-box approaches we have reviewed in Chapter 6 to solve the RL problems without relying on specific RL-oriented algorithms, suchas Q learning.
In the black-box approaches, many agents simulate different policies in parallel and, based on the
rewards they obtain, one selects the policy that gives the most reward as the optimal policy. In practice,
the use of the black-box optimization might also be of advantageous for physicists who may only haveusual computational resources such as multicore CPU, since the calculations can be parallelized and no
derivatives of neural networks are necessary (i.e., the advantage of using GPU is marginal).
As we explained in Chapter 6, black-box optimization algorithms prepare a set of individuals eachof which has a set of parameters θ, and then train those individuals solely on the basis of the values ofthe fitness F (θ) at each iteration. In the deep RL problems, the parameters θ correspond to the network
parameters in the deep neural network that represents either the state-action function Qθ (s, a) in the valuebased approaches (for which the policy is simply defined by a = argmaxa′ Qθ (s, a′ )) or the policy itselfπθ (a|s) in the policy-gradient methods. More specifically, the fitness in the value-based methods is givenby the total reward accumulated over each episode {(st , at )}Tt=1 ,(7.6.1)

F (θ) = R(Qθ ) ≡TXr (st , at (θ)) , at (θ) = argmaxa Qθ (st , a),t=1

which depends on the parameters θ through the Q function Qθ . Meanwhile, in the policy-based methods,
the fitness corresponds to the total reward along an episode generated by the policy:(7.6.2)F (θ) = R(πθ ) =TXr (st , at (θ)) , at ∼ πθ (a|st ),t=1

where θ dependence follows from the policy function πθ . Given these identifications, one can directly
apply the black-box optimization algorithms, such as the ones outlined in Algorithms 2,3,4 in Chapter 6,
to the deep RL problems. For further details, see arXiv:1703.03864 or arXiv:1712.06567 for applicationsof evolutionary strategies or genetic algorithms to deep RL methods, respectively.
7.7. Some tips on implementation
While it is not mandatory requirement of this lecture for you to implement the RL algorithms insome simple problem, that would be a very good Exercise if you aim to apply RL methods to actual,
more complex problems at your hand (see, e.g., Exercise 7.1). Here I shall only give a few remarks
for that, which you might find helpful. Obviously, this section is far from complete and we refer theinterested readers to the modern text book for practitioners (Deep Reinforcement Learning Hands-On)
3For further readings about the challenges in deep RL, you may refer to the following blogpost: “Deep Reinforcement Learning

Doesn’t Work Yet”.

7.8. EXERCISE

139

and the sample codes available in GitHub page of this book. As an introduction, PyTorch tutorial and
Chapter 12 of arXiv:2009.05673 would also be useful. You can find some collection of useful sources andsample codes in GitHub page.
To be concrete, I assume PyTorch implementation here. First, it would be useful to install Anaconda
from here, and choose appropriate installer for your OS and just follow the instruction (I assume Mac
from now on, but the similar steps should be followed also in Windows). Once you have completed theinstallation, choose Anaconda-Navigator from the applications. For later use, you may install Jupyter
Notebook in the Home. Then, you can create PyTorch environment by choosing “Environments” in theleft bar and checking the Python package. Launch the environment by clicking the play button → OpenTerminal.
You should now be able to install PyTorch by just doing “conda install pytorch torchvision -c pytorch”
on the terminal (see this page). You can launch the Jupyter Notebook by choosing the “Open with Jupyter
Notebook” in the Pytorch environment and may run a simple Python code to check whether or not the
installation is successful; see Pytorch-tutorial, PyTorchZeroToAll, or Chapter 3 of Deep Reinforcement
Learning Hands-On for further readings about PyTorch introduction.
To implement RL algorithms for solving some prototypical problems, you can use the useful toolkit
called “OpenAI Gym”. To install this, it is usually enough to do “pip install gym” on the terminal you
launched above. To simulate Atari games, which are now known as benchmark problems for modern
deep RL algorithms, you should install ROMS from here and do “python -m atari_py.import_roms <path
to folder>”. In general, to continuously monitor the training process, it is very useful to install TensorBoard; for details, see for instance Chapter 3 of Deep Reinforcement Learning Hands-On. Once you haveinstalled this, you can monitor the training process by doing “tensorboard —logdir <file name>” and typically opening “http://localhost:6006/” in the browser. After these preparations, you should be ready to do
Exercise 7.1.

7.8. Exercise
Exercise 7.1 (Mastering video game “Pong” via Black box optimization: 2 points). In 2015,
DeepMind has demonstrated that the RL algorithm using DQN can achieve human-level control in different types of video games in Atari 2600 [Mnih et al., Nature 518, 529 (2015)]. You may reproduce
this result by following the same approach, namely, Deep Q learning, but (since you have to calculate thederivatives of deep neural networks) it typically requires the GPU acceleration, which may not readily beavailable for physics students. Also, the results are often rather sensitive to choices of hyperparametersand may not be very stable.
Here let us follow another approach we discussed above, namely, black box optimization of DQN (seeSec. 7.6). An advantage here is that the GPU acceleration is no longer essential and you can parallelize
the calculations over multicore CPU on usual computers. The number of hyperparameters is also muchless than those in the other DRL algorithms.
To be concrete, in this Exercise, consider solving the video game named “Pong” among Atari games
by using black-box optimization of DQN based on the genetic algorithm (cf. Algorithm 3). Then, plot

7.8. EXERCISE

140

typical results of the evolutions of the mean and maximum values of the reward during the training process.
Hint: you may built your code on the basis of useful wrapper functions or sample codes available in
GitHub page of Deep Reinforcement Learning Hands-On, especially those in Chpaters 6 and 20. You
should get the results that should look like Fig. 7.8.1 below, where we choose the hyperparameters as• Network structure of DQN: 3 convolutional layers with 32, 64, and 64 channels followed by ahidden layer with 512 units. They use 8 × 8, 4 × 4, and 3 × 3 filters with strides 4, 2, and 1,
respectively. The same as in [Mnih et al., Nature 518, 529 (2015)].
• Population size: n = 800• Parents count: m = 8• Noise standard deviation: σ = 0.004
To obtain the results in this figure, the calculations are distributed over 10 CPU cores on a single machine.
To reach the maximum reward of the elite agent rmax = 20, the learning roughly takes ∼ 1 day. To
achieve the maximum possible reward 21, you have to be a bit more patient; it typically takes another ∼ 1
day to increment rmax from 20 to 21.

F IGURE 7.8.1. Typical learning process of Exercise 7.1 performed on my personal computer with 10 CPU cores.

7.8. EXERCISE

141

Summary of Chapter 7
Sections 7.1-7.4 Reinforcement learning formalism and the value-based search• In reinforcement learning (RL), a machine called agent acquires information about data by
many repetitions of interactions with environment and, on the basis of this experience, it
attempts to find the best solution that maximizes the objective function called reward. A
state of the environment can change according to how the agent act on the environment.
• A way of determining the action a taken by an agent based on the state s of the environment
is given by the conditional probability distribution called policy π(s|a). In the frameworkof Markov decision process, the future dynamics only depends on the current state and thereward rt can be defined as a function of the current state-action pair (st , at ).
• The expectation value of the sum of subsequent reward with initial conditions st = s, at = a
and policy π is called the quality (Q) function and denoted by Qπ (s, a). An iterative method
to find the best policy by optimizing the Q function is known as Q learning.
• Deep Q-learning uses a deep neural network to approximate the Q function and optimizes itby the stochastic gradient descent. There exists a number of heuristic methods to improve thestability and efficiency of the training process.
Section 7.5 Policy-based search• In a policy-based approach, the policy πθ itself is approximated by using some functional
ansatz, such as neural networks, and directly optimized to maximize the total expected reward
along the whole episode.
• Compared to a value-based method, policy-based methods can be advantageous when action
space is continuous and/or simulation of the environment is fast and cheap.
Section 7.6 Black-box optimization in deep RL• Identifying the total reward as the fitness, one can apply black box optimizations to deep RL
algorithms.
• This could be advantageous since the black box approaches typically require much less hyperparameters than the other deep RL methods and can be parallelized on multicore CPU while
GPU acceleration is only marginal.