## 2 Theoretical Considerations of the Digital Filter
### Theoretical Considerations of the PF Algorithm  
The theoretical considerations of the PF model start by considering a hidden Markov model (HMM) that observes the outputs \(x_{i}\) indirectly using state \(y_{i}\), and we can specify a simple model as follows  
\[x_{1}\sim\mu(x_{1}) \tag{38}\]  
\[f(x_{k}\left|x_{k-1}\right.) \tag{39}\]  
where in state equation (transition) for \(k>1\), and  
\[h(y_{k}\left|x_{k}\right.)\text{ in measured state (observation)} \tag{40}\]  
where all states are homogeneous, and the probabilities of transitions and observations are independent of time, see Figure 4.  
Figure 4: HMM showing the transition and observation states.  
Figure 3: System block diagram of Extended Kalman Filter (EKF).  
The goal is to estimate \(x_{k}\), which is equivalent to \(x_{i}\) the original datasets, given all observations up to point (\(y_{1:n}\)). Alternatively, we need to find the posterior distribution of \(p(x_{1:n}\left|y_{1:n}\right.).\) Using Bayes, we end up with two steps as follows:  
1. Update step: \[p(x_{k}\left|y_{1:k}\right.)=\frac{h(y_{k}\left|x_{k}\right.)p(x_{k}\left|y_{1: k-1}\right.)}{p(y_{k}\left|y_{1:k-1}\right.)}\] (41)
2. Prediction step: \[p(x_{k}\left|y_{1:k-1}\right.)=\int f(x_{k}\left|x_{k-1}\right.)p(x_{k-1}\left| y_{1:k-1}\right.)dx_{k-1}\] (42)  
We often find that these distributions were intractable, especially the nonlinear and Gaussian model in closed form. The solution is to approximate the distributions using a large number of samples (particles). To address the problems in Equations (41) and (42), it might not be too difficult to appropriate the intractable integrals appearing in those equations directly, where the alternative is to use important sampling or sequential importance sampling (SIS). The advantage of SIS is that it does not guarantee to fail as \(t\) increases and it becomes more and more skewed, especially when sampling high-dimensional spaces [26].  
Thus, we propose a bootstrap PF, which is an iterative method of Bayesian inference for a dynamic state space. The algorithm of the bootstrap PF model is described as follows:  
1. Assume \(p\left(x_{k-1}\left|y_{k-1}\right.\right)\) is the posterior probability distribution at \(k-1\) where the transition state (state equation) is \(S_{k-1}=\left\{\left(x_{k-1}^{i},w_{k-1}^{i}\right);i=1,2,...,N\right\}\)
2. Resample \(p\left(x_{k}\left|x_{k-1}\right.\right),\) which is the prior probability distribution at \(k-1\) using the bootstrap algorithm
3. Find a weight by Monte Carlo integration using \(p\left(x_{k-1}\left|y_{k-1}\right.\right)\) and \(p\left(x_{k}\left|x_{k-1}\right.\right)\) to obtain, \(\int p\left(x_{k}\left|x_{k-1}\right.\right)\cdot p\left(x_{k-1}\left|y_{k-1} \right.\right)\mathrm{d}x_{k-1}\approx\sum\limits_{i}w_{k-1}^{i}p\left(x_{k} \left|x_{k-1}^{i}\right.\right)\)
4. From (3), use \(\sum\limits_{i}w_{k-1}^{i}p\left(x_{k}\left|x_{k-1}^{i}\right.\right)\) to estimate the particle at \(k-1\) and obtain \(\hat{S}_{k-1}=\left\{\left(\hat{x}_{k-1}^{i},\hat{w}_{k-1}^{i}\right);i=1,2,...,N\right\}.\)
5. Update the likelihood function \(p\left(y_{k}\left|x_{k}\right.\right)\) with \(\hat{x}_{k-1}^{i}\) and \(\hat{w}_{k-1}^{i}.\)
6. From (5) use the new updated likelihood to update \(p\left(x_{k}\left|y_{k}\right.\right),\) which is the posterior probability distribution at \(k\) using a normalised weight from time to time. The weight can either be the averaging of all the weights or the last weight only.
7. Finally, we obtain \(p\left(x_{k}\left|y_{k}\right.\right)\) the posterior probability distribution at \(k.\)