SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lars Holdijk*
Yuanqi Du*
Ferry Hooft
Computational Chemistry Group
Priyank Jaini
Bernd Ensing
AI4Science Lab
We consider the problem of sampling transition paths between two given metastable states of a molecular system, e.g. a folded and unfolded protein or products and reactants of a chemical reaction. Due to the existence of high energy barriers separating the states, these transition paths are unlikely to be sampled with standard Molecular Dynamics (MD) simulation. Traditional methods to augment MD with a bias potential to increase the probability of the transition rely on a dimensionality reduction step based on Collective Variables (CVs). Unfortunately, selecting appropriate CVs requires chemical intuition and traditional methods are therefore not always applicable to larger systems. Additionally, when incorrect CVs are used, the bias potential might not be minimal and bias the system along dimensions irrelevant to the transition. Showing a formal relation between the problem of sampling molecular transition paths, the Schrödinger bridge problem and stochastic optimal control with neural network policies, we propose a machine learning method for sampling said transitions. Unlike previous non-machine learning approaches our method, named PIPS, does not depend on CVs. We show that our method successful generates low energy transitions for Alanine Dipeptide as well as the larger Polyproline and Chignolin proteins.
Molecular Dynamics (MD) is a central tool in the (bio-)chemistry toolbox. By integrating Newton's equations of motion on a molecular scale, MD can provide insight into chemical processes and systems without requiring expensive lab testing [Frenkel and Smit, 2001, Hollingsworth and Dror, 2018]. However, MD is limited when interested in transitions between two metastable configurations of a system, such as the folding of a protein, general conformational changes, and chemical reactions. These meta-stable states are separated by regions of high energy which are unlikely to be sampled within a reasonable timespan. While machine learning based approximations of the interatomic forces using neural force fields [Unke et al., 2021] have pushed the boundary in terms of system scale, it does not address the problem of sampling molecular transition paths directly [Fu et al., 2022].
To overcome this issue, prior work in computational and physical chemistry has developed several methods for the enhanced sampling of molecular transitions such as transition path sampling [Bolhuis et al., 2002], umbrella sampling[Torrie and Valleau, 1977] and meta-dynamics [Laio and Parrinello, 2002]. Most of these methods speed up the sampling of transition paths by augmenting the MD simulation with a (learned) bias potential that pushes the system to cross the energy barrier separating two states. However, due to the large configuration space of molecular trajectories, finding such a bias potential is in itself a computationally expensive task.
To circumvent this problem, prior methods depend on Collective Variables (CVs). CVs are functions of atomic coordinates that have been identified as playing a role within the transition period. Biasing methods rely on these CVs to reduce the complexity of the bias potential by only biasing the system along them. Limiting the bias potential to act on the CVs is an intuitive approach since the most common reason to sample transition paths, deriving transition dependent quantities such as reaction free-energy and reaction rate, are functions of CVs themselves [Bussi and Branduardi, 2015]. See fig. 1 for an illustration of the free-energy barrier separating two metastable states of the Alanine Dipeptide protein for which the dihedral angles  $\phi$  and  $\psi$  are known to be CVs.
However, while sensible, depending on CVs to reduce the dimensionality of bias potential search space is not always suitable. While some methodological approaches are available [Hooft et al., 2021] for smaller
systems, selecting CVs relies on prior expert knowledge. This limits the applicability of bias potential enhanced sampling to systems for which this information is available. Additionally, when CVs are incorrectly chosen, depending the bias potential on these CVs might result in errors in determining dependent quantities [Bolhuis et al., 2000] and incorrect interpretation of the transition process.
Figure 1: Free-energy surface of Alanine Dipeptide as a function of CV dihedral angles  $\phi$  and  $\psi$  highlighting the high energy barrier separating the two metastable states. White stars indicate saddle points in the high energy barrier where the transition is likely to occur.
For this purpose, we propose PIPS, a Path Integral stochastic optimal control [Kappen, 2005, Kappen and Ruiz, 2016] method for Path Sampling of molecular transitions. PIPS leverages stochastic optimal control theory to train a parameterised bias potential that, unlike previous methods from computational chemistry, operates on the entire geometry of the molecule instead of depending on predetermined CVs. This way, PIPS can be scaled to larger systems.
Contributions and outline Our contributions are organised as follows. First, we introduce the problem of sampling transition paths in section 2. Second, we formally show in section 3 the relationship between the problem of sampling transitions paths, the Schrodinger Bridge Problem (section 3.1) and Stochastic Optimal Control (SOC) (section 3.3). Following this, we use the gained insights regarding SOC in section 4 to propose PIPS; a method based on the PICE algorithm designed for sampling molecular transition paths that does not depend on Collective Variables. Lastly, we demonstrate the efficacy of PIPS on conformational transitions in three molecular systems of varying complexity, namely Alanine Dipeptide, Polyproline, and Chignolin in section 5.
2 Preliminaries, Problem Setup, and Related Work
2.1 Molecular Dynamics
Given the state of a molecular system  $\boldsymbol{x}_t \coloneqq (\boldsymbol{r}_t, \boldsymbol{v}_t)$  consisting of positions  $\boldsymbol{r}_t \in \mathbb{R}^{3n}$  and velocities  $\boldsymbol{v}_t \in \mathbb{R}^{3n}$  at time  $t$  with  $n$  atoms sampled from the Gibbs distribution  $\pi_G(\boldsymbol{x}_t) = \exp(-\frac{1}{k_B T} \mathcal{H}(\boldsymbol{r}_t, \boldsymbol{v}_t))$ , Molecular Dynamics (MD) describe the time evolution of the state over time.  $\mathcal{H}$  is known as the Hamiltonian  $\mathcal{H}(\boldsymbol{r}_t, \boldsymbol{v}_t) = U(\boldsymbol{r}_t) + K(\boldsymbol{v}_t)$ , where  $U(\boldsymbol{r}_t)$  and  $K(\boldsymbol{v}_t) = \frac{1}{2} m \boldsymbol{v}^2$  with mass  $m$ , respectively denote the Potential and Kinetic Energy of the system. The potential energy of a system is defined by a parameterized sum of pairwise empirical potential functions, such as harmonic bonds, angle potentials, inter-molecular electrostatic and Van der Waals potentials.
One common approach of integrating the molecular dynamics is Langevin Dynamics [Bussi and Parrinello, 2007] which couple the deterministic Newtonian equations of motion with a stochastic thermostat that acts like a heat bath. Langevin dynamics obey the following SDEs
$$
\mathrm {d} \boldsymbol {r} = \boldsymbol {v} \cdot \mathrm {d} t \tag {1}
$$
$$
\mathrm {d} \boldsymbol {v} = \frac {- \nabla_ {\boldsymbol {r}} U (\boldsymbol {r})}{\boldsymbol {m}} \cdot \mathrm {d} t - \gamma \boldsymbol {v} \cdot \mathrm {d} t + \sqrt {2 \boldsymbol {m} \gamma k _ {B} T} \mathrm {d} \boldsymbol {W}, \tag {2}
$$
where  $k_{B}$  is the Boltzmann constant,  $T$  the temperature of the heath bath, and  $\mathrm{d}\pmb{W}$  standard Brownian Motion.  $\gamma$ , the friction term, couples the dynamics and the heat bath. Following this SDE samples samples from the Canonical of NVT ensemble with constant temperature.
2.2 Sampling Transition Path Sampling
By sampling an initial configuration  $\pmb{x}_0 = (\pmb{r}_0, \pmb{v}_0) \sim \pi_G$  and following the MD simulation for a fixed amount of time, one can generate trajectories  $\pmb{x}_{0:\tau} = \{\pmb{x}_0, \dots, \pmb{x}_{\tau}\}$ , of length  $\tau$ . These trajectories represent samples from the probability distribution over trajectories given by:
$$
\pi \left(\boldsymbol {x} _ {0: \tau}\right) = \pi_ {G} \left(\boldsymbol {x} _ {0}\right) \cdot \prod_ {t = 1} ^ {\tau} \mathcal {N} \left(\boldsymbol {x} _ {t} \mid \boldsymbol {\mu} _ {t - 1}, \boldsymbol {\Sigma} _ {t - 1}\right), \tag {3}
$$
with  $\pmb{\mu}_t = (\pmb{v}_t \cdot \mathrm{d}t, \frac{-\nabla_r U(\pmb{r}_t)}{\pmb{m}} \cdot \mathrm{d}t - \gamma \pmb{v}_t \cdot \mathrm{d}t)^T$  and  $\pmb{\Sigma} = \mathrm{diag}(0, 2\pmb{m}\gamma k_B T)$ .
However, in the context of sampling transition paths, we are interested in trajectories with a predefined initial and final state. I.e.  $r_0 \in R \subset \mathbb{R}^{3n}$  and  $r_{\tau} \in P \subset \mathbb{R}^{3n}$ . For example,  $R$  can describe the set of reactants and  $P$  the set of products of a chemical reaction. Or,  $R$  can be the set of stable native states of a protein while  $P$  is the set of folded proteins.
We will refer to the distribution over trajectories with restricted initial and target states as the Transition Path (TP) distribution [Dellago et al., 1998].
Definition 1 (Transition Path (TP) distribution). Given a set of initial states  $R$ , target states  $P$ , potential energy  $U$  and a transition length  $\tau$  the Transition Path (TP) distribution is defined as;
$$
\pi^ {*} \left(\boldsymbol {x} _ {0: \tau}\right) = \frac {1}{Z} \mathbf {1} _ {R} \left(\boldsymbol {r} _ {0}\right) \cdot \pi \left(\boldsymbol {x} _ {0: \tau}\right) \cdot \mathbf {1} _ {P} \left(\boldsymbol {r} _ {\tau}\right) \tag {4}
$$
where  $\mathbf{1}_R$  and  $\mathbf{1}_P$  are indicator functions and  $\pi (\pmb{x}_{0:\tau})$  is defined according to eq. (3).
We can naively apply rejection sampling to sample from the TP distribution by sampling a system  $\pmb{x}_0 \sim \mathbf{1}_R(\pmb{r}_0) \cdot \pi_G(\pmb{x}_0)$ , evolving it for  $\tau$  steps according to the MD in eq. (1) and rejecting it when  $\pmb{r}_{\tau} \notin P$ . Unfortunately, when using standard molecular dynamics, it is very unlikely for any trajectory starting in a state  $\pmb{r}_0 \in R$  to terminate with  $\pmb{r}_{\tau} \in P$  due to the two sets of states being separated by a high-energy barrier. I.e. for all  $\pmb{x}_{0:\tau} \sim \pi^*$  some  $\pmb{x}_t$  has  $U(\pmb{r}_t) >> U(\pmb{r}_0)$ . To be able to obtain a representative number of trajectories, one is thus forced to generate a high number of trajectories, making naive sampling from the TP distribution computationally very expensive.
2.3 Bias Potential Enhanced Sampling
To solve the problem caused by high-free energy barriers and to sample from the TP distribution various enhanced sampling approaches are available. These will be further discussed in the related work section. In this work, we will focus on a specific branch of enhanced sampling methods called Bias Potential Enhanced Sampling (BPES). In BPES approaches, the stochastic dynamics are enhanced with a bias potential  $b(\boldsymbol{r}, \boldsymbol{v})$  such that when a system  $\boldsymbol{x}_0 \sim \mathbf{1}_R(\boldsymbol{r}_0) \cdot \pi_G(\boldsymbol{x}_0)$  is transformed according to the biased dynamics
$$
\mathrm {d} \boldsymbol {r} = \boldsymbol {v} \cdot \mathrm {d} t \tag {5}
$$
$$
\mathrm {d} \boldsymbol {v} = \frac {- \nabla_ {\boldsymbol {r}} (U (\boldsymbol {r}) + b (\boldsymbol {r} , \boldsymbol {v}))}{\boldsymbol {m}} \cdot \mathrm {d} t - \gamma \boldsymbol {v} \cdot \mathrm {d} t + \sqrt {2 \boldsymbol {m} \gamma k _ {B} T} \mathrm {d} \boldsymbol {W}, \tag {6}
$$
a trajectory, of length  $\tau$ , always terminates with  $r_{\tau} \in P$ .
Trajectories sampled by following these bias potential enhanced dynamics are sampled according to what we refer to as the Bias Potential enhanced Transition Path (BPTP) distribution
$$
\pi^ {b} \left(\boldsymbol {x} _ {0: \tau}\right) = \mathbf {1} _ {R} \left(\boldsymbol {r} _ {0}\right) \cdot \pi_ {G} \left(\boldsymbol {x} _ {0}\right) \cdot \prod_ {t = 1} ^ {\tau} \mathcal {N} \left(\boldsymbol {x} _ {t} \mid \hat {\boldsymbol {\mu}} _ {t - 1}, \hat {\Sigma} _ {t - 1}\right), \tag {7}
$$
with  $\hat{\pmb{\mu}}_t = (\pmb{v}_t\cdot \mathrm{d}t,\frac{-\nabla_{\pmb{r}}\left(U(\pmb{r}_t) + b(\pmb{r}_t,\pmb{v}_t)\right)}{\pmb{m}}\cdot \mathrm{d}t - \gamma \pmb{v}_t\cdot \mathrm{d}t)^T$  and  $\hat{\Sigma} = \mathrm{diag}(0,2\pmb {m}\gamma k_B T)$ .
Finding the bias potential  $b(\boldsymbol{r}, \boldsymbol{v})$  such that trajectories sampled from the BPTP distribution are distributed according to the TP distribution is referred to as the BPTP problem.
Definition 2 (BPTP problem). Given a set of initial states  $R$ , target states  $P$  and a Potential Energy function  $U$ , the BPTP problem describes the task of finding an optimal bias potential  $b^{*}$  such that trajectories sampled from the BPTP distribution  $\pi^{b^{*}}$  are close to samples sampled to the TP distribution  $\pi^{*}$ , ie.
$$
b ^ {*} = \underset {b} {\arg \min } \mathbb {D} _ {\mathrm {K L}} \left(\pi^ {b} \mid \pi^ {*}\right). \tag {8}
$$
2.3.1 Related Enhanced Sampling Methods
CV dependent Enhanced Sampling Most closely related to our work are the metadynamics [Laio and Parrinello, 2002, Bussi and Branduardi, 2015, Barducci et al., 2008] and the Adaptive Biasing Force (ABF) methods [Darve and Pohorille, 2001, Comer et al., 2015]. In metadynamics, the bias potential is iteratively built as a sum of Gaussians centered at conformational states previously visited during the MD simulation. This consecutively pushes the system outwards to regions of higher energy not previously visited. Contrarily to metadynamics, ABF does not aim to learn the bias potential  $b(\boldsymbol{r}, \boldsymbol{v})$ , but instead aims to control the system through the bias force  $\boldsymbol{b}(\boldsymbol{r}, \boldsymbol{v}) = \nabla_{\boldsymbol{r}} b(\boldsymbol{r}, \boldsymbol{v}) \in \mathbb{R}^{3n}$ . The intuition behind ABF is to learn a bias force that cancels out the deterministic force from the molecular potential. As a result, the only remaining driving force is the stochastic Langevin thermostat which is not affected by the high energy barriers. Other approaches to sampling transition paths using a bias potential include umbrella sampling Torrie and Valleau [1977], hyper-MD [Voter, 1997], the Wang-Landau method [Wang and Landau, 2001] and various less commonly applied others [Sprik and Ciccotti, 1998, Grubmüller, 1995, Huber et al., 1994, Carter et al., 1989]. All these methods depend on dimensionality reduction steps using CVs while our proposed method does not.
CV free Enhanced Sampling In addition to the CV dependent methods a different family of MCMC based approaches for direct sampling from the TP distributions is available. These methods, such as Transition Path Sampling [Dellago et al., 1998, Bolhuis et al., 2002] and Nudge Elastic Band sampling [Henkelman et al., 2000], generally do not use a bias potential or CVs.
Recently, several machine learning solutions for the BPTP and related problems have been proposed. For example, Das et al. [2021] use Reinforcement Learning to sample from the TP distribution under Brownian dynamics, Schneider et al. [2017] consider the modelling of the free-energy surface using neural networks, and both Sultan et al. [2018] and Sun et al. [2022] use neural networks find CVs.
3 Sampling Transition Paths using Stochastic Optimal Control theory
In this section we will formally discuss the relationship between the BPTP problem and two topics from the machine learning literature; the Schrodinger Bridge problem and Stochastic Optimal Control.
3.1 The BPTP problem is a Schrodinger Bridge Problem
First introduced by Schrodinger [Schrodinger, 1931, 1932], the Schrodinger Bridge (SB) problem studies the transition between two distributions over time under some fixed drift and diffusion components. Formally, the SP problem is defined as
Definition 3 (Schrodinger Bridge (SB) problem). Given a reference distribution  $\pi\big(x_{0:\tau}\big)$  over trajectories with predefined marginals  $\pi_0$  and  $\pi_{\tau}$ , the Schrodinger Bridge (SB) Problem aims to find an alternative distribution  $\hat{\pi}\big(x_{0:\tau}\big)$  such that
$$
\hat {\pi} ^ {*} \left(\boldsymbol {x} _ {0: \tau}\right) := \underset {\hat {\pi} \left(\boldsymbol {x} _ {0: \tau}\right) \in \mathcal {D} \left(\pi_ {0}, \pi_ {\tau}\right)} {\arg \min } \mathbb {D} _ {\mathrm {K L}} \left(\hat {\pi} \left(\boldsymbol {x} _ {0: \tau}\right) \| \pi \left(\boldsymbol {x} _ {0: \tau}\right)\right) \tag {9}
$$
where  $\mathcal{D}(\pi_0,\pi_\tau)$  is the space of path measures with marginals  $\pi_0$  and  $\pi_{\tau}$ .
Recently, machine learning approaches for parameterizing this alternative distribution  $\hat{\pi}$  to approximate the reference distribution  $\pi$  have received attention [Vargas et al., 2021, De Bortoli et al., 2021]. In the following theorem, we show that these approaches also provide a solution to the BPTP problem when the correct marginal distributions are specified.
Theorem 3.1 (BPTP problem is a SB problem). Let  $b$  be the set of functions such that  $\pi_0 = \pi_G(\pmb{x}_0) \cdot \mathbf{1}_R(\pmb{r}_0)$  and  $\pi_\tau = \pi_G(\pmb{x}_\tau) \cdot \mathbf{1}_R(\pmb{r}_\tau)$ , we have that a solution to the SB problem with reference distribution  $\pi^*$  is also a solution to the BPTP problem, i.e.
$$
\arg \min  _ {b} \mathbb {D} _ {\mathrm {K L}} \left(\pi^ {b} \mid \pi^ {*}\right) = \underset {\pi^ {b} \in \mathcal {D} \left(\pi_ {0}, \pi_ {\tau}\right)} {\arg \min } \mathbb {D} _ {\mathrm {K L}} \left(\pi^ {b} \| \pi^ {*}\right) \tag {10}
$$
Proof. This follows from the definition of the BPTP and SB problems.
Following this theorem, we can use proposed solutions for solving the SBP to solve the BPTP problem using a bias potential. In this work, we will specifically focus on Stochastic Optimal Control theory, which has been shown to solve the SBP in [Chen et al., 2016].
3.2 Background: Stochastic Optimal Control
Before formally introducing the relation between Stochastic Optimal Control (SOC) and the BPTP problem we first review some of the basic concepts of SOC and, more specifically, the Path Integral Stochastic Optimal Control (PISOC) branch of SOC theory.
Given an arbitrarily controlled dynamical system
$$
\mathrm {d} \boldsymbol {x} _ {t} = \boldsymbol {f} (\boldsymbol {x} _ {t}) \mathrm {d} t + \boldsymbol {G} (\boldsymbol {x} _ {t}) \cdot (\boldsymbol {u} (\boldsymbol {x} _ {t}) \mathrm {d} t + \mathrm {d} \boldsymbol {W}), \quad \boldsymbol {x} _ {0} \sim \pi_ {0}, \tag {11}
$$
where  $\pmb{f}:\mathbb{R}^d\times \mathbb{R}^+ \to \mathbb{R}^d$  and  $\pmb {G}:\mathbb{R}^{d}\times \mathbb{R}^{+}\to \mathbb{R}^{d\times d}$  are deterministic functions representing the drift and volatility of the system and  $\mathrm{d}\pmb{W}$  is Brownian Motion with variance  $\nu$ , Stochastic Optimal Control theory aims to find a policy  $\pmb{u}(\pmb{x}_t)$  that minimizes some expected cost  $C$  over the trajectories:
$$
\boldsymbol {u} ^ {*} = \underset {\boldsymbol {u}} {\arg \min } \mathbb {E} _ {\boldsymbol {x} _ {0: \tau} \sim \pi_ {u}} \left[ C \left(x _ {0: \tau}\right) \right] \tag {12}
$$
Here  $\pi_u$  represents the distribution over trajectories similar to eq. (7) with  $\pmb{\mu}_t = \pmb{x}_t + \pmb{f}(\pmb{x}_t, t) \mathrm{d}t + \pmb{G}(\pmb{x}_t)(\pmb{u}(\pmb{x}_t) \mathrm{d}t)$  and  $\Sigma_t = \pmb{G}(\pmb{x}_t)^T \nu \pmb{G}(\pmb{x}_t)$ .
In this work we will specifically rely on a branch of SOC called Path Integral Control (PISOC), first introduced by Kappen [2007]. In PISOC the cost of a trajectory is defined as
$$
C \left(\boldsymbol {x} _ {0: \tau}\right) = \frac {1}{\lambda} \left(\varphi \left(\boldsymbol {x} _ {\tau}\right) + \sum_ {t = 0} ^ {\tau - 1} \frac {1}{2} \boldsymbol {u} \left(\boldsymbol {x} _ {t}\right) ^ {T} \boldsymbol {R} \boldsymbol {u} \left(\boldsymbol {x} _ {t}\right) + \boldsymbol {u} \left(\boldsymbol {x} _ {t}\right) ^ {T} \boldsymbol {R} \boldsymbol {\varepsilon} _ {t}\right) \tag {13}
$$
where  $\varepsilon_{t} = G^{-1}(\pmb{x}_{t})(\mathrm{d}\pmb{x} - \pmb{f}(\pmb{x}_{t})\mathrm{d}t) - \pmb{u}(\pmb{x}_{t}), \varphi$  denotes the terminal cost,  $\lambda$  is a constant and  $\pmb{R}$  is the cost of taking action  $\pmb{u}$  in the current state and is given as a weight matrix for a quadratic control cost. To clarify,  $\varepsilon_{t} \sim \mathrm{d}\pmb{W}$  is the noise introduced into the trajectories by the Langevin thermostat.
The last term in the cost function in eq. (13) relating the Brownian motion and the control is unusual and devoid of a clear intuition. However, this term plays an important role when relating the cost to a KL-divergence which we will establish next. Additionally, as discussed by Thijssen and Kappen [2015], the additional cost vanishes under expectation  $(\mathbb{E}_{\boldsymbol{x}_{0:\tau} \sim \pi_u}[\boldsymbol{u}(\boldsymbol{x}_t)^T \boldsymbol{R} \boldsymbol{\varepsilon}_t] = 0)$  and thus, does not influence the optimal control  $\boldsymbol{u}^*$  given by eq. (12).
3.3 Stochastic Optimal Control solves the BPTP problem
We can see that SOC dynamical system (eq. (11)) is similar to the dynamics of the BPTP distribution (eq. (5)). In fact, as we will see next, with a properly defined  $\varphi$ , minimizing the trajectory cost (eq. (13)) results in finding a control  $u$  that solves the BPTP problem.
Theorem 3.2 (SOC solves the BPTP problem). Given  $\pmb{x}_t = (\pmb{r}_t, \pmb{v}_t)^T$ ,  $\pmb{f}(\pmb{x}_t) = (\pmb{v}_t, \frac{-\nabla_{\pmb{r}_t} U(\pmb{r}_t)}{\pmb{m}} - \gamma \pmb{v}_t)^T$ ,  $\pmb{G}(\pmb{x}_t) = (\mathbf{0}_{3n}, \mathbb{I}_{3n})^T$ ,  $\pmb{u}(\pmb{x}_t) = \frac{-\nabla_{\pmb{r}_t} b(\pmb{r}_t, \pmb{v}_t)}{\pmb{m}}$ ,  $\nu = 2m\gamma k_B T$ , and  $\pi_0 = \pi_G$ , such that the SOC dynamics (eq. (11)) describe the dynamics of the BPTP distribution  $\pi^b$  (eq. (5)).
If we define  $\varphi (\pmb {x}_\tau) = -\lambda \log (\pmb {1}_P(\pmb {r}_\tau))$ $\pmb {R} = \lambda \nu^{-1} = \lambda (2\pmb {m}\gamma k_B T)^{-1}$  and assume  $\pmb {r}_0\in R$  , we have
$$
\underset {b} {\arg \min } \mathbb {E} _ {\boldsymbol {x} _ {0: \tau} \sim \pi^ {b}} \left[ C (\boldsymbol {x} _ {0: \tau}) \right] = \underset {b} {\arg \min } \mathbb {D} _ {\mathrm {K L}} \left(\pi^ {b} | \pi^ {*}\right), \tag {14}
$$
where  $\pi^{*}$  is the TP distribution.
Proof. See appendix A. The proof relates  $\pi^b$  and  $\pi^0$  using Girsanov's theorem to rewrite the expectation over cost  $C$  as the summation of the terminal cost and a KL divergence.
Using the established connection we now thus have a tool to solve the BPTP problem by learning a parameterized control  $\boldsymbol{u}_{\theta}(\boldsymbol{x}_t) = \frac{-\nabla_{\boldsymbol{r}_t}b_{\theta}(\boldsymbol{r}_t,\boldsymbol{v}_t)}{\boldsymbol{m}}$  and consequently the parameterized bias potential  $b_{\theta}$  using SOC theory. In the following section we will look at one specific approach for doing so.
4 PIPS: Path Integral SOC for Path Sampling
Following the formal construction of the relationship between SOC and the BPTP problem, we now introduce our proposed method to find the parameterized bias potential  $b_{\theta}$  based on this connection. We refer to this method as PIPS: Path Integral Path Sampling. PIPS is an adaptation of the Path Integral Cross Entropy (PICE) [Kappen and Ruiz, 2016] method to the setting of sampling molecular transition paths where we have a single initial  $R = \{r_0^*\}$  and target  $P = \{r_{\tau}^{*}\}$  system.
Background: Path Integral Cross Entropy Kappen and Ruiz [2016] introduced the Path Integral Cross Entropy (PICE) method for solving Equation (12). The PICE method derives an explicit expression for the distribution  $\pi^{\pmb{u}^{*}}$  under optimal control  $\pmb{u}^{*}$  when  $\lambda = \nu R$  given by:
$$
\pi^ {\boldsymbol {u} ^ {*}} = \frac {1}{\eta (\boldsymbol {x} , t)} \pi^ {\boldsymbol {u}} \left(\boldsymbol {x} _ {0: \tau}\right) \exp (- C \left(\boldsymbol {x} _ {0: \tau}\right)) \tag {15}
$$