## 2 Materials and Methods
### Tracking Filter  
To combine the output of the classical hue thresholding approach (which had proven quite reliable in close quarters) and the neural network approach to image processing, an extended Kalman filter (EKF) was introduced into the system. The node containing the filter operates in the image coordinate space, and fuses together aMussel position measurements. It also acts as a tracking filter, using previously acquired information about the speed of the aMussel's movement in the image to ensure more accurate state predictions. The filter operates at a fixed rate of 10 Hz, so a time step of \(\Delta t=0.1\) s is used in all internal calculations. The state vector of the filter is given with  
\[\mathbf{x}= \begin{bmatrix}\hat{e}_{x}&\hat{e}_{y}&v_{x}&v_{y}\end{bmatrix} ^{T} \tag{5}\]  
where \(v_{x}\) and \(v_{y}\) are velocities derived from the difference of the last two estimated states, and \(\hat{e}_{x}\) and \(\hat{e}_{y}\) are position estimates based on measurements fused from the two sources. The state model used can be expressed with \(\mathbf{\hat{e}}= \begin{bmatrix}v_{x}&v_{y}&0&0\end{bmatrix} ^{T}\). The measurement vector of the filter is given with  
\[\mathbf{y}= \begin{bmatrix}e_{x}&e_{y}\end{bmatrix} ^{T}+ \begin{bmatrix}r_{x}&r_ {y}\end{bmatrix} ^{T} \tag{6}\]  
where \(e_{x}\) and \(e_{y}\) represent \((x,y)\) image coordinate pairs calculated using measurement fusion and scaled from pixel values to a \([-1,1]\) range. Results received from the two measurement sources are represented with \(\mathbf{e_{nn}}= \begin{bmatrix}e_{nn,x}&e_{nn,y}\end{bmatrix} ^{T}\) containing the results of neural net processing and \(\mathbf{e_{hue}}= \begin{bmatrix}e_{hue,x}&e_{hue,y}\end{bmatrix} ^{T}\) containing the results of hue-based image thresholding. Meanwhile, \(r_{x}\) and \(r_{y}\) are the respective measurement variances used to take into account measurement quality and reliability along each axis. These are defined as:  
\[r_{x} \sim\mathcal{N}(0,\zeta_{x}) \tag{7}\] \[r_{y} \sim\mathcal{N}(0,\zeta_{y}) \tag{8}\]  
The values of \(\zeta_{x}\) and \(\zeta_{y}\) are determined using values for each measurement source, \(\mathbf{n_{nn}}= \begin{bmatrix}\zeta_{nn,x}&\zeta_{nn,y}\end{bmatrix} ^{T}\) for neural net processing and \(\mathbf{n_{hue}}= \begin{bmatrix}\zeta_{hue,x}&\zeta_{hue,y}\end{bmatrix} ^{T}\) for hue-based image thresholding. For neural network data, the values of \(\zeta_{nn}\) are set to an empirically determined constant:  
\[\zeta_{nn,x}=\zeta_{nn,y}=0.075 \tag{9}\]  
However, for the hue thresholded data, the \(\zeta_{hue}\) values were defined as depending on the vertical position of the detected pixel blob in the image. The further away on the y-axis the detected blob is, the higher the variance of the hue-based detector, i.e., hue-based detection is less trusted. It is set with:  
\[\zeta_{hue,x}=\zeta_{hue,y}=\frac{1-e_{hue,y}+0.2}{2} \tag{10}\]  
Considering the image y-coordinate ranges from \(-1\) at the bottom of the image, and \(1\) at the top, the variance values used in the filter achieve a range of [0.1, 1.1], which is always higher than any concurrently available neural network detector value. Thus, using methods explained in [37; 38], the fusion of measurements from two sources is done with:\[e_{x}=\frac{\zeta_{hue,x}e_{nn,x}+\zeta_{nn,x}e_{hue,x}}{\zeta_{hue,x}+\zeta_{nn,x}} \tag{11}\]  
\[e_{y}=\frac{\zeta_{hue,y}e_{nn,y}+\zeta_{nn,y}e_{hue,y}}{\zeta_{hue,y}+\zeta_{nn,y}} \tag{12}\]  
\[\zeta_{x}=\frac{\zeta_{hue,x}\zeta_{mm,x}}{\zeta_{hue,x}+\zeta_{nn,x}} \tag{13}\]  
\[\zeta_{y}=\frac{\zeta_{hue,y}\zeta_{nn,y}}{\zeta_{hue,y}+\zeta_{nn,y}} \tag{14}\]  
Implementation-wise, three distinct cases exist. If hue-based detection has produced a result, but no detection data has been received from the neural net, \(\zeta_{nn}\) values will both become zero, while \(\zeta_{hue}\) will be calculated using (10).