# Optical Flow-Based Extraction of Breathing Signal from Cone Beam CT Projections
## 3 Materials and Methods
### Feature Tracking Using Optical Flow  
Optical flow is the distribution of 2 D velocities in an image caused by the motion of luminance patterns also known as pixel intensities [28]. The dense optical-flow method with enhanced classical Horn-Schunck method [28] provided by [PERSON], [PERSON], and [PERSON] [29] is implemented in this work. For two subsequent frames acquired at time \(t\) and \(t+1\) in a sequence of images, pixel \((x,y)\) in frame 1 changes position by \((x+\delta x,y+\delta y)\) in frame 2. The motion of pixel intensities in an image sequence can thus be described by Equation (1).  
\[I(x+\delta x,y+\delta y,t+1)=I(x,y,t) \tag{1}\]  
The apparent motion of pixel intensities between subsequent projection images while estimating respiratory motion can be given as Equation (2).  
\[\frac{\partial I}{\partial x}\frac{dx}{dt}+\frac{\partial I}{\partial y}\frac{ dy}{dt}+\frac{\partial I}{\partial t}=0 \tag{2}\]  
Let \(U=\frac{dx}{dt}\) and \(V=\frac{dy}{dt}\) be the horizontal and vertical optical-flow components of \(I(x,y,t)\).  
Then,  
\[I_{x}U+I_{y}V+I_{t}=0 \tag{3}\]  
where \(I_{x},~{}I_{y},~{}I_{t}\) represent the partial derivative of image intensity in respective directions. Motion fields were estimated through a coarse-to-fine approach. For every adjacent projection image, an image pyramid is built by sub-sampling. The base level of the pyramid consists of the original frame of size \((M\times M)\), and adjacent layers are occupied as \((1\times 1,~{}2\times 2,~{}4\times 4,\dots,~{}2^{a}\times 2^{a})\) so that \(M=2^{a}\). The pyramidal approach involves the computation of optical flow initially at the layer with images at minimum resolution, to the layer with the original frame.  
In this work, the optical flow is applied to all image pixels of every projection. Thus, a data matrix is created for each dataset to store the output of the optical-flow vectors. The total number of columns in the matrix is equal to (2 \(\times\) the dimension of the projection), as each pixel is described by two components: the horizontal \(\Delta u_{(prj~{}1,~{}pxl~{}1)}\) and the vertical one \(\Delta v_{(prj~{}1,~{}pxl~{}1)}\). On the other hand, the number of rows in the matrix is equal to the total number of projections. The matrix is filled row by row, such that all pixel displacement components for the first CBCT projection are filled in the first row of the matrix, and this is continued till the final CBCT projection.  
The data matrix \(D\) can be described by Equation (4).  
\[D=\begin{bmatrix}\Delta u_{(prj~{}1,~{}pxl~{}1)}~{}\Delta v_{(prj~{}1,~{}pxl~{}1 )}~{}\cdots~{}\Delta u_{(prj~{}1,~{}pxl~{}m)}~{}\Delta v_{(prj~{}1,~{}pxl~{}m)} \\ \vdots~{}\vdots~{}\vdots~{}\vdots~{}\vdots~{}\vdots\\ \Delta u_{(prj~{}n,~{}pxl~{}1)}~{}\Delta v_{(prj~{}n,~{}pxl~{}1)}~{}\cdots~{} \Delta u_{(prj~{}n,~{}pxl~{}m)}~{}\Delta v_{(prj~{}n,~{}pxl~{}m)}\end{bmatrix} \tag{4}\]  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Projection Size** & **Number of** & **Imaging System** & **Ground-Truth** \\ \hline Patient 1 & 512 \(\times\) 512 & 701 & XVI 3.5 (Elekta) & Diaphragm motion \\ \hline Patient 2 & 768 \(\times\) 1024 & 1220 & OBI (Varian) & Internal marker trace \\ \hline Patient 3 & 768 \(\times\) 1024 & 2396 & OBI (Varian) & Internal marker trace \\ \hline \hline \end{tabular}
\end{table}
Table 1: Description of the patient datasets.  
where _prj_ and _pxl_ represent projection number and pixels, respectively. The final projection within a set of data is represented as \(n\), and \(m\) represents the last pixel in the projection being processed. \(\Delta u\) and \(\Delta v\) signify the horizontal and vertical component of the optical-flow vectors.