SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Although one could have used the negative of cost as a reward, this is inconsistent with our Bayesian net model since we require positive rewards to define the reward augmented trajectory model. The following result addresses this issue by transforming a problem with costs to one with a suitable reward specification.
Proposition B.1. Given costs  $c_{ij} \geq 0$  the following two problems are equivalent:
$$
\underset {\boldsymbol {\theta}} {\arg \min } \frac {1}{N} \sum_ {i = 1} ^ {N} \sum_ {j = 0} ^ {H} c _ {i j} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) \equiv \underset {\boldsymbol {\theta}} {\arg \max } \frac {1}{N} \sum_ {i = 1} ^ {N} \sum_ {j = 0} ^ {H} \underbrace {\left(\sum_ {n = 0} ^ {H} c _ {i n}\right) \left(1 - \frac {c _ {i j}}{\sum_ {n = 0} ^ {H} c _ {i n}}\right)} _ {r _ {i j} ^ {\prime}} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j})
$$
Proof. Starting with the cost minimization problem, we may write:
$$
\underset {\boldsymbol {\theta}} {\arg \min } \frac {1}{N} \sum_ {i = 1} ^ {N} \sum_ {j = 0} ^ {H} c _ {i j} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) = \underset {\boldsymbol {\theta}} {\arg \min } \frac {1}{N} \sum_ {i = 1} ^ {N} c _ {i} \sum_ {j = 0} ^ {H} \tilde {c} _ {i j} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) \tag {42}
$$
where  $c_{i} \coloneqq \sum_{j=0}^{H} c_{ij}$  and  $\tilde{c}_{ij} = \frac{c_{ij}}{c_i}$ . Since  $\sum_{j=0}^{H} \psi_j^{\pmb{\theta}}(\mathbf{s}_{ij}) = 1$ , we may subtract the constant  $\frac{1}{N} \sum_{i=1}^{N} c_i \sum_{j=0}^{H} \psi_j^{\pmb{\theta}}(\mathbf{s}_{ij})$  from the objective yielding:
$$
\begin{array}{l} \underset {\boldsymbol {\theta}} {\arg \min } \frac {1}{N} \sum_ {i = 1} ^ {N} \sum_ {j = 0} ^ {H} c _ {i j} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) = \underset {\boldsymbol {\theta}} {\arg \min } \frac {1}{N} \sum_ {i = 1} ^ {N} c _ {i} \sum_ {j = 0} ^ {H} (\tilde {c} _ {i j} - 1) \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) \\ = \arg \max _ {\boldsymbol {\theta}} \frac {1}{N} \sum_ {i = 1} ^ {N} c _ {i} \sum_ {j = 0} ^ {H} (1 - \tilde {c} _ {i j}) \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) \\ = \underset {\boldsymbol {\theta}} {\arg \max} \frac {1}{N} \sum_ {i = 1} ^ {N} \sum_ {j = 0} ^ {H} \underbrace {\left(\sum_ {n = 0} ^ {H} c _ {i n}\right) \left(1 - \frac {c _ {i j}}{\sum_ {n = 0} ^ {H} c _ {i n}}\right)} _ {r _ {i j} ^ {\prime}} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {i j}) \\ \end{array}
$$
where we have expanded  $c_{i}$  and  $\tilde{c}_{ij}$ . This completes the proof.
C Neural Fitted Q-iteration approaches: DNN-FQI and RNN-FQI
Fitted Q-iteration methods [39, 40] use the Wald-Bellman equation (WBE) as follows: First, given parameterized function approximations  $K_{j}^{\theta}(S_{j})$ , a single Wald-Bellman step is bootstrapped, for a batch of trajectories yielding  $\hat{V}_{j}(s_{ij}) = \max \{r_{ij},K_{j}^{\theta}(s_{ij})\} ,\forall j < H,1\leq i\leq N$ . Next the parameters of the continuation function are fit:  $\theta^{*} = \arg \min \sum_{i}\sum_{j}(K_{j}^{\theta}(s_{ij}) - \hat{V}_{j + 1}(s_{ij + 1}))^{2}$  and the process is iterated. To provide competent DNN/RNN baselines for fitted Q-iteration (FQI) methods that are missing in the literature, we introduce a temporal FQI loss (Algorithm 2).
Algorithm 2 Pseudo-code for mini-batch computation of the FQI loss
Input:  $\mathbf{R} := [r_{ij}]$ ,  $\mathbf{K} := \left[K_j^\theta (\mathbf{s}_{ij})\right]$ $\{1\leq i\leq N_b,0\leq j\leq H,N_b$  is batch size}
$\mathbf{V}_{:,0:H-1} = \text{stop-gradient}(\max\{\mathbf{R}_{:,0:H-1}, \mathbf{K}_{:,0:H-1}\})$  {bootstrap WBE to get value targets}
$\mathbf{V}_{\therefore H} = \mathbf{R}_{\therefore H}$  {final target is reward for last step}
$\mathbf{J} = \mathbf{MSE}(\mathbf{V}_{:,1:H},\mathbf{K}_{0,0:H - 1})$  {next step value is target for current continuation function}
D Further details of the experiments
All experiments were performed on a shared server configured with  $2 \times$  Intel Xeon Silver 12-core,  $2.10\mathrm{GHz}$  CPUs with 256GB RAM and equipped with 6 NVIDIA 2080Ti GPUs. However, experiments were run on a single GPU at a time and no computation was distributed across GPUs.
D.1 Model hyper-parameter settings
Table 2 shows general hyper-parameter settings used for all experiments. We apply Batch normalization to the input and outputs of layer activation functions at all hidden layers. Due to the dense correlation structure between assets at each time-step of the American option pricing experiment, we choose the hidden units to be greater than the input dimension  $d$ .
As with RL policy gradients, we may subtract a baseline value[29] to reduce variance. The OSPG algorithm uses baseline  $b$  that does not depend on time-index  $j$ , sufficient to guarantee an unbiased OSPG estimator (see Appendix E). In our experiments, we use:
$$
b = \frac {1}{N H} \sum_ {i = 1} ^ {N} \sum_ {j = 0} ^ {H} r _ {i j} \tag {43}
$$
Table 2: model hyper-parameter settings
method	hyper-parameter	tuned range	value
DOS, DNN-FQI, DNN-OSPG	num hidden layers	n/a	2
RRLSM, RNN-FQI, RNN-OSPG	num hidden layers	n/a	1
All models	hidden layer units	n/a	20
All models (Am. Option pricing)	hidden layer units	n/a	20 + d
All models	batch size	n/a	64
All models (Am. Option pricing)	batch size	n/a	128
All models	learning rate	{0.01, 0.001, 0.0001}	0.001
All models	epochs	early stopping	100
All models	batches/epoch	n/a	200
All models	optimizer	n/a	Adam
RRLSM	Kernel noise std	n/a	0.0001
RRLSM	Recurrent noise std	n/a	0.3
D.2 Pricing Bermudan options
Table 3 shows model sizes (in trainable parameters), training, and inference times (per time-step). Model sizes grow with input dimension except for RRLSM, which uses an RNN with random, non-trainable weights to extract features from the input. The parameter size of DOS is about an order of magnitude higher than DNN-FQI and DNN-OSPG since parameters in backward induction methods like DOS grow linearly with the number of time-steps (parameters are not shareable across time-steps). Training and inference times are also high since individual models must be fit and inferred at each time-step.
Table 3: model sizes and compute times for Bermudan max-call experiment
method	assets	model-size (params)	mean training time (seconds)	mean time/prediction (μ-seconds)
DOS	20	9,225	271	3
DNN-FQI	20	1,047	23	5
DNN-OSPG	20	1,047	29	7
RRLSM	20	198	2	14
RNN-FQI	20	2,807	32	8
RNN-OSPG	20	2,807	37	8
DOS	50	15,165	289	3
DNN-FQI	50	1,707	28	6
DNN-OSPG	50	1,707	29	7
RRLSM	50	198	2	14
RNN-FQI	50	4,667	32	8
RNN-OSPG	50	4,667	32	8
DOS	100	25,065	326	3
DNN-FQI	100	2,807	30	6
DNN-OSPG	100	2,807	28	6
RRLSM	100	198	2	14
RNN-FQI	100	7,767	33	8
RNN-OSPG	100	7,767	30	8
DOS	200	44,865	317	3
DNN-FQI	200	5,007	30	6
DNN-OSPG	200	5,007	27	6
RRLSM	200	198	2	15
RNN-FQI	200	13,967	35	8
RNN-OSPG	200	13,967	30	8
Table 4: American geometric-call option pricing: Results
d	s0	p*	average return (error %)
LS [23]	PDE-DGM [37]	PDE-BSDE [7]	DNN-FQI [34]	DNN-OSPG
7	90	5.9021	5.8440 (0.98%)	NA	5.8822 (0.34%)	5.7977 (1.77%)	5.8704 (0.54%)
7	100	10.2591	10.1736 (0.83%)	NA	10.2286 (0.30%)	10.1022 (1.53%)	10.2518 (0.07%)
7	110	15.9878	15.8991 (0.55%)	NA	15.9738 (0.09%)	15.0487 (5.87%)	15.9699 (0.11%)
13	90	5.7684	5.5962 (3.00%)	NA	5.7719 (0.06%)	5.7411 (0.47%)	5.7436 (0.43%)
13	100	10.0984	9.9336 (1.60%)	NA	10.1148 (0.16%)	9.9673 (1.30%)	10.0691 (0.29%)
13	110	15.8200	15.6070 (1.40%)	NA	15.8259 (0.04%)	14.7759 (6.60%)	15.8107 (0.06%)
20	90	5.7137	5.2023 (9.00%)	NA	5.7105 (0.06%)	5.6607 (0.93%)	5.6983 (0.27%)
20	100	10.0326	9.5964 (4.40%)	10.0296 (0.03%)	10.0180 (0.15%)	9.6372 (3.94%)	10.0100 (0.23%)
20	110	15.7513	15.2622 (3.10%)	NA	15.7425 (0.06%)	14.9345 (5.19%)	15.7553 (0.03%)
100	90	5.6322	OOM	NA	5.6154 (0.30%)	5.3858 (4.38%)	5.6211 (0.20%)
100	100	9.9345	OOM	9.9236 (0.11%)	9.9187 (0.16%)	9.3954 (5.43%)	9.8954 (0.40%)
100	110	15.6491	OOM	NA	15.6219 (0.17%)	14.6335 (6.49%)	15.6301 (0.12%)
200	100	9.9222	OOM	9.9004 (0.22%)	9.9088 (0.14%)	9.3772 (5.49%)	9.8991 (0.23%)
D.3 Pricing American options
The scope of the paper is solving discrete-time, finite-horizon, model-free optimal stopping problems. Bermudan options that have discrete exercise opportunities are one example application. American options, which are more popular, are based on continuous-time asset price evolution and have a continuum of possible exercise times. One way to convert this to our discrete-time setting is to solve related Bermudan options. These options limit exercise opportunities to a fine discrete time grid.
State-of-the-art algorithms for pricing American options are based on Partial differential equation (PDE) methods. These methods are model-based since they start with a PDE (such as the multidimensional Black-Scholes Model) defining process evolution. For example, PDE methods often assume Markovian Black-Scholes Dynamics, and the PDEs to be solved require the Black-Scholes model parameters, such as covariance of the Brownian motion, volatility, risk-free interest rate, and dividend yield. In contrast, model-free methods, such as FQI and OSPG algorithms, do not use prior information on the evolution dynamics of the underlying stochastic process.
Nevertheless, we compare our model-free OSPG method against state-of-the-art PDE methods such as the Deep Galerkin Method (DGM) [37] and Backward Stochastic Differential Equations method (BSDE)[7] by suitable discretization of the original continuous time-problem. Note that the PDE methods also require discretization of the original PDE (ex, using the Euler-Maruyama scheme) or random sampling (as used in DGM) but do not end up directly solving a Bermudan option.
We consider multi-dimensional continuous-time American geometric-average call options with Black-Scholes dynamics considered in [37] and [7]. The payoff of these options depends on the price of  $d$  underlying assets with multi-dimensional Black-Scholes dynamics and the strike price  $K$ . The dynamics are Markovian, with payoff (reward) given by:
$$
S _ {t} ^ {m} = s _ {0} ^ {m} \exp \left(\left[ r - \delta_ {m} - \sigma_ {m} ^ {2} / 2 \right] t + \sigma W _ {t} ^ {m}, \quad R _ {t} = \left(\left[ \prod_ {m = 1} ^ {d} S _ {t} ^ {m} \right] ^ {\frac {1}{d}} - K\right) ^ {+}\right. \tag {44}
$$
for  $m = 1,2,\dots d$ .  $r \in \mathbb{R}$  is the risk-free rate,  $s_0^m \in (0,\infty)$  represents the initial price,  $\delta_m \in [0,\infty)$  is the dividend yield,  $\sigma_m \in (0,\infty)$  the volatility and  $W$  is a  $d$ -dimensional Brownian motion, with instantaneous correlation between its  $d$  components given by  $\mathbb{E}[W_t^i W_t^j] = \rho_{ij}t$ . The reward for exercise at time  $t$  is given by  $R_{t}$ . We discretize  $t$  into  $H + 1 = 100$  possible exercise opportunities, using times  $t_j = jT / H$  for  $j = 0,1,\dots H$ .  $T$  is the option expiration duration in years. This yields the stopping problem:  $\sup_{0\leq \tau \leq H}\mathbb{E}[R_{\tau}]$ .
The specific option we consider is characterized by the following parameters:  $K = 100$ ,  $r = 0.0$ ,  $\sigma_{m} = 0.25$ ,  $\rho_{ij} = 0.75 \forall i \neq j$ ,  $\delta_{m} = 0.02$ ,  $T = 2$ . The exact price of this option,  $p^{*}$ , can be determined semi-analytically for comparison [7]. We generate 10,000 batches (with a batch size of 128) for training and compute option prices on a 3,000-batch test sample. We compare vs. published results from state-of-the-art model-based PDE baselines, including the Deep Galerkin Method (PDE-DGM) [37], the Backward Stochastic Differential Equation (PDE-BSDE) method [7]. We also include published results [7] from the industry standard Longstaff-Schwartz (LS) option
Table 5: Stopping a fractional Brownian motion: Results
h	average return (standard deviation)
DOS	DNN-FQI	DNN-OSPG	DOS-ES	RRLSM	RNN-FQI	RNN-OSPG
0.05	0.70 (0.01)	0.87 (0.06)	1.14 (0.00)	1.18 (0.01)	1.16 (0.00)	1.24 (0.04)	1.28 (0.01)
0.10	0.54 (0.01)	0.68 (0.04)	0.92 (0.01)	0.93 (0.01)	0.94 (0.00)	0.99 (0.03)	1.03 (0.02)
0.20	0.34 (0.01)	0.42 (0.08)	0.58 (0.00)	0.55 (0.01)	0.57 (0.00)	0.59 (0.04)	0.64 (0.01)
0.30	0.19 (0.00)	0.14 (0.07)	0.29 (0.10)	0.28 (0.01)	0.29 (0.00)	0.27 (0.08)	0.36 (0.01)
0.40	0.10 (0.01)	0.02 (0.03)	0.13 (0.00)	0.09 (0.01)	0.10 (0.00)	0.04 (0.04)	0.15 (0.00)
0.50	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)
0.60	0.08 (0.01)	0.03 (0.03)	0.10 (0.00)	0.08 (0.01)	0.06 (0.00)	0.03 (0.03)	0.10 (0.03)
0.70	0.16 (0.01)	0.08 (0.08)	0.17 (0.02)	0.18 (0.01)	0.19 (0.00)	0.10 (0.07)	0.20 (0.00)
0.80	0.23 (0.01)	0.03 (0.05)	0.25 (0.00)	0.26 (0.01)	0.27 (0.00)	0.18 (0.09)	0.27 (0.01)
0.90	0.31 (0.01)	0.14 (0.12)	0.28 (0.10)	0.32 (0.01)	0.33 (0.00)	0.21 (0.12)	0.33 (0.00)
0.95	0.35 (0.01)	0.01 (0.05)	0.34 (0.01)	0.36 (0.00)	0.36 (0.00)	0.25 (0.08)	0.36 (0.00)
pricing algorithm [23], which uses linear approximation with multi-dimensional basis functions, suitable for low dimensional settings.
Table 4 summarizes the experiment's results. Published results [7] for LS, DGM, and BSDE are included. NA denotes results not reported in the literature, while OOM indicates out of memory. Our model-free DNN-OSPG algorithm compares favorably with state-of-the-art model-based PDE-based option pricing methods that have prior knowledge of process evolution. Specifically, unlike LS, whose accuracy degrades with the number of correlated assets, and DNN-FQI, whose accuracy degrades up to  $6.5\%$ , DNN-OSPG retains excellent performance ( $<0.55\%$  error) closely approaching the model-based PDE methods ( $<0.35\%$  error). Of course, a key advantage of DNN-OSPG is that it can be used to price options for which no known PDE is available to describe process evolution.
D.4 Stopping a fractional Brownian motion
Table 5 provides numerical values corresponding to Figure 2. RNN-OSPG dominates competing methods across all values the Hurst parameter in this non-Markovian setting.
Table 6: Stopping a fractional Brownian motion: Model sizes and compute times
method	model-size (params)	mean training time (seconds)	mean time/prediction (μ-seconds)
DOS	62,900	2250	26.9
DNN-FQI	651	21	0.5
DNN-OSPG	651	29	0.5
DOS-ES	276,300	2228	25.9
RRLSM	2,200	9	5.0
RNN-FQI	1,691	14	1.1
RNN-OSPG	1,691	79	1.1
D.5 Early stopping a sequential multi-class classifier
We start with the 34 datasets, and corresponding values of  $\alpha$  used in the [1] and remove binary classification datasets. This leaves 17 datasets. However, many of these have very few series or have a large number of classes relative to series size, which might make them unsuitable for training RNN classifiers. Nevertheless, we report results on all 17 datasets. Table 7 reports the experiment's mean and standard deviation (over ten random splits) results over 17 UCR time-series datasets. Figure 2 provides a graphical visualization of the same results.  $N$  denotes the number of trajectories,  $H$  denotes the length of the series, and  $K$  denotes the number of classes.  $\alpha$  trades-off classification accuracy and earliness. RNN-OSPG achieves the best performance on 15 of the 17 datasets.
Table 7: Early stopping a sequential multi-class classifier: Results
Dataset	N/H/K/α	average cost (standard deviation)
RRLSM	RNN-FQI	RNN-OSPG
CBF	930/128/3/0.8	0.192 (0.017)	0.308 (0.138)	0.186 (0.011)
ChlorineConcentration	4,307/166/3/0.4	0.468 (0.011)	0.523 (0.073)	0.455 (0.003)
Crop	24,000/46/24/0.06	0.428 (0.005)	0.407 (0.010)	0.389 (0.008)
ECG5000	5000/140/5/0.5	0.110 (0.005)	0.217 (0.059)	0.099 (0.006)
ElectricDevices	16,637/96/7/0.1	0.257 (0.015)	0.260 (0.012)	0.228 (0.009)
FaceAll	2,250/131/14/0.01	0.108 (0.025)	0.100 (0.010)	0.092 (0.017)
FaceUCR	2,250/131/14/0.5	0.358 (0.035)	0.465 (0.049)	0.328 (0.028)
FiftyWords	905/270/50/0.5	0.667 (0.027)	0.812 (0.041)	0.634 (0.029)
InsectWingbeatSound	2,200/256/11/1	0.666 (0.064)	0.897 (0.110)	0.646 (0.080)
MedicalImages	1,141/99/10/0.07	0.296 (0.033)	0.309 (0.031)	0.263 (0.038)
MelbournePedestrian	3,633/24/10/0.8	0.591 (0.065)	0.597 (0.068)	0.487 (0.044)
MixedShapesRegularTrain	2,925/1024/5/0.1	0.194 (0.024)	0.194 (0.008)	0.191 (0.037)
NonInvasiveFetalECGThorax2	3,765/750/42/0.04	0.418 (0.094)	0.343 (0.008)	0.295 (0.064)
StarLightCurves	9,236/1024/3/0.3	0.120 (0.043)	0.315 (0.067)	0.165 (0.035)
Symbols	1,020/398/6/0.2	0.103 (0.013)	0.208 (0.030)	0.239 (0.070)
UWaveGestureLibraryX	4,478/315/8/0.5	0.523 (0.023)	0.658 (0.026)	0.498 (0.040)
WordSynonyms	905/270/25/0.6	0.760 (0.024)	0.913 (0.043)	0.727 (0.041)
E Baseline subtraction for variance reduction
As with RL policy gradients, we may subtract a baseline value[29] to reduce variance. However, unlike the general RL case, due to the stopping-time-based formulation of the OSPG, the OSPG baseline should not be time-varying.
Proposition E.1 (baseline subtraction for OSPG). The optimal stopping policy gradient (OSPG) of Theorem 4.1 is invariant to the subtraction of a constant (w.r.t. trajectory) baseline from every reward in the trajectory. Thus:
$$
\begin{array}{l} \nabla_ {\pmb {\theta}} J _ {O S} (\pmb {\theta}) = \mathbb {E} _ {\mathbf {s} _ {H} \sim \mathbb {P} (\mathbf {s} _ {H})} \left[ \sum_ {j = 0} ^ {H} r _ {j} \psi_ {j} ^ {\pmb {\theta}} (\mathbf {s} _ {j}) \nabla_ {\pmb {\theta}} \log \psi_ {j} ^ {\pmb {\theta}} (\mathbf {s} _ {j}) \right] \\ { = } { \mathbb { E } _ { \mathbf { s } _ { H } \sim \mathbb { P } ( \mathbf { s } _ { H } ) } \left[ \sum _ { j = 0 } ^ { H } ( r _ { j } - b ) \psi _ { j } ^ { \boldsymbol { \theta } } ( \mathbf { s } _ { j } ) \nabla _ { \boldsymbol { \theta } } \log \psi _ { j } ^ { \boldsymbol { \theta } } ( \mathbf { s } _ { j } ) \right] } \\ \end{array}
$$
Proof. It suffices to show  $\mathbb{E}_{\mathbf{s}_H\sim \mathbb{P}(\mathbf{s}_H)}\left[\sum_{j = 0}^H b\psi_j^\theta (\mathbf{s}_j)\nabla_\theta \log \psi_j^\theta (\mathbf{s}_j)\right] = 0$ . We proceed as follows:
$$
\begin{array}{l} E _ {\mathbf {s} _ {H} \sim \mathbb {P} (\mathbf {s} _ {H})} \left[ \sum_ {j = 0} ^ {H} b \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {j}) \nabla_ {\boldsymbol {\theta}} \log \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {j}) \right] = E _ {\mathbf {s} _ {H} \sim \mathbb {P} (\mathbf {s} _ {H})} \left[ \sum_ {j = 0} ^ {H} b \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {j}) \frac {\nabla_ {\boldsymbol {\theta}} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {j})}{\psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {j})} \right] \\ { = } { E _ { \mathbf { s } _ { H } \sim \mathbb { P } ( \mathbf { s } _ { H } ) } \left[ \nabla _ { \boldsymbol { \theta } } \sum _ { j = 0 } ^ { H } b \psi _ { j } ^ { \boldsymbol { \theta } } ( \mathbf { s } _ { j } ) \right] } \\ = E _ {\mathbf {s} _ {H} \sim \mathbb {P} (\mathbf {s} _ {H})} \left[ \nabla_ {\boldsymbol {\theta}} b \underbrace {\sum_ {j = 0} ^ {H} \psi_ {j} ^ {\boldsymbol {\theta}} (\mathbf {s} _ {j})} _ {= 1} \right] \\ = 0 \tag {45} \\ \end{array}
$$