# Application of machine learning for detecting and tracking turbulent structures in plasma fusion devices using ultra fast imaging

Sarah Chouchene, F. Brochard, Mikael Desecures, Nicolas Lemoine, Jordan Cavalier

► To cite this version:

Sarah Chouchene, F. Brochard, Mikael Desecures, Nicolas Lemoine, Jordan Cavalier. Application of machine learning for detecting and tracking turbulent structures in plasma fusion devices using ultra fast imaging. Scientific Reports, 2024, 14 (1), pp.27965. 10.1038/s41598-024-79251-z. hal-04784770

HAL Id: hal-04784770

https://hal.science/hal-04784770v1

Submitted on 15 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Application of machine learning for detecting and tracking turbulent structures in plasma fusion devices using ultra fast imaging

Sarah Chouchene<fnref n="1" />,<fnref n="2" />*, Frédéric Brochard<fnref n="1" />, Mikael Desecures<fnref n="2" />, Nicolas Lemoine<fnref n="1" />, and Jordan Cavalier<fnref n="3" />

## ABSTRACT

This study explores the application of machine learning techniques for detecting and tracking plasma filaments around the boundary of magnetically confined tokamak plasmas. Plasma filaments, also called blobs, are responsible for enhanced turbulent transport across magnetic field lines, and their accurate characterization is crucial for optimizing the performance of magnetic fusion devices. We present a novel approach that combines machine learning methods applied to data obtained from ultra-fast cameras, including YOLO (You Only Look Once) for object detection, semantic segmentation, and specific tracking methods. This approach enables fast and accurate detection and tracking of filaments while overcoming the limitations of conventional methods, which are time-consuming and prone to human subjectivity. A significant advance in our study lies in the development of a method for automatically labeling a large batch of data, which greatly facilitates the training of supervised machine learning algorithms. Using these techniques, we obtained promising results demonstrating a significant improvement over conventional tracking methods, achieving a detection accuracy of up to 98.8 %, while reducing the inference time per frame by 15% to 31% compared to conventional Kalman filter tracking. These results open up new perspectives for investigating turbulent phenomena in tokamaks, and could have important implications for the development of controlled nuclear fusion.

## Introduction

Magnetized plasmas host numerous instabilities that can lead to turbulent transport of particles and heat across magnetic field lines. In nuclear fusion plasmas, a portion of this transport manifests as elongated plasma filaments along magnetic flux surfaces, commonly referred to as "blobs"<fnref n="1" />. By their nature, these blobs tend to reduce the confinement time of plasma energy, thus opposing the maintenance of optimal fusion conditions. Understanding their formation and movement is crucial for optimizing reactor performance by controlling these heat and particle fluxes<fnref n="2" />. More broadly, understanding the mechanisms of blob generation and propagation can provide insights into turbulence processes in magnetized plasmas in general, and is therefore of interest in other fields of plasma physics and engineering, such as astrophysical plasmas<fnref n="3" /> or plasma thrusters for space propulsion⁴.

Since the early 2000s, increasingly powerful high-speed cameras have made it possible to visualise, with steadily improving temporal resolution, the filaments in the volume located at the periphery of the last closed magnetic surface (LCFS), where the presence of neutral or non-fully ionised particles allows visible light to be emitted. In order to localise the filaments in 3D and improve the spatial resolution of the measurements, gas puff imaging (GPI) techniques based on localised gas injection⁵ or tomographic inversions⁶ using for instance the Tomotok python package⁷ have been developed, resulting in large volumes of data to be exploited. Data presented in this study were obtained from recordings made on the COMPASS tokamak⁸,⁹ before its final shutdown in 2021. The aim of the work presented in this article is to facilitate the use of such data to better characterize the radial and poloidal dynamics of plasma filaments.

Previously, we developed a Machine Learning (ML) method to study filament dynamics and interactions in the poloidal direction using fast imaging data¹⁰. In this previous work, the main objective was to detect and count the phenomena of coalescence, splitting or rapid alterations in trajectories resulting from mutual interactions between

<footnote n="1">¹Université de Lorraine, Institut Jean Lamour, CNRS, Nancy, 54000, France</footnote>

<footnote n="2">²APREX Solutions, Pulligny, 54160, France</footnote>

<footnote n="3">³Institute of Plasma Physics of the Czech Academy of Sciences, U Slovanky 2525/1a, 182 00 Prague 8, Czech Republic</footnote>

<footnote>*sarah.chouchene@univ-lorraine.fr</footnote>
filaments. To achieve this, we focused on analysing the dynamics only in the main direction of movement of the
filaments, along the magnetic flux surfaces. The different types of interactions and the filament speeds were deduced
thanks to a pattern recognition AI method applied to kymographs. Contrary to this 1-D approach, this new study aims
at testing whether Artificial Intelligence-based detection and tracking models could improve 2D filament tracking.
Knowing the radial and poloidal velocities of the turbulent structures in a tokamak is crucial for several important
reasons linked to the performance and understanding of fusion plasmas. Firstly, the radial velocities of the turbulent
structures are directly linked to the transport of particles and energy across magnetic field surfaces, contributing to
faster energy losses and increasing plasma-wall interactions. Secondly, both radial and poloidal motions influence
plasma stability and the formation of transport barriers, required to achieve high-confinement modes and to maintain the
conditions necessary for fusion in the core plasma. This work was partly motivated by the desire to overcome certain
interpretive biases attributable to human operators in conventional analysis methods, where some parameters are more
or less arbitrarily set: threshold level for segmentation detection, Kalman filter initialization for tracking, etc...

To achieve this, we combined specialized Convolutional Neural Networks (CNNs) and focused on two fundamental
perception problems, namely, i) object detection and semantic segmentation based on the state of the art YOLO models¹¹
for detection and 2) object tracking with CNNs specialized in tracking such as DeepSort¹², StrongSort¹³, BotSort¹⁴
and ByteTrack¹⁵. The 2D radial and poloidal velocities maps obtained with these CNNs were then compared with
those obtained with conventional detection and tracking methods. It should be noted that while neural networks are
increasingly being used in tracking applications, such as recently for honey bee segmentation tracking¹⁶, their use in
turbulent systems has so far remained very limited. Recently, a work based on similar, albeit different, approaches has
already demonstrated the possibility of using neural networks for filament tracking on the TCV tokamak¹⁷. Supervised
ML needs large sets of annotated data to be effective and to give consistent results. This generally has a twofold cost, in
terms of manpower devoted to training and also in terms of the subjectivity of the people carrying out the labelling.
In order to overcome these issues, the solution tested on TCV for blob detection was based on the use of optical flow
vectors (RAFT, GMA, Flow Walk) or masks (Mask R-CNN), trained on synthetic data prior to the application to
true experimental data. In our case, both training and analysis were executed on true experimental data, thanks to an
automatic data labeling method developed on purpose. Another difference between both studies is that on TCV, the
data used favoured very high temporal resolution (0.5 µs), made possible by the use of GPI, over spatial resolution,
resulting in 10x12 pixel images. In our study, carried out with and without gas injection, the temporal resolution is
lower (1-3.7 µs), although still sufficient to resolve the dynamics of the structures correctly, but the spatial resolution is
much better, with 128x40 pixels images at minimum also offering a larger field of view. Finally, as artificial intelligence
is a rapidly developing field, we were also able to test state-of-the-art neural network models, more recent than those
presented in the TCV study.

This work enhances the study of turbulence dynamics in hot and magnetized plasmas through the following
contributions:

1. Use of state-of-the-art neural networks for the detection and tracking of filaments, trained on experimental fast imaging data from both GPI and passive imaging,

2. Development of an automatic data labeling method, enabling supervised learning without wasting time doing it by human hands,

3. Production of radial and poloidal velocity maps of plasma filaments of similar or better quality and faster than with conventional methods.

The article is structured as follows. Firstly, we explain how experimental data were obtained from recordings made
on the COMPASS tokamak. We then present the methods used to detect filaments using different versions of the YOLO
model and compare the results with those of a conventional thresholding method. In the following section, we intro-
duce different CNNs specialized in tracking and compare the results obtained on filament tracking with those obtained
with a Kalman filter. Finally, we discuss the results as a whole and their application prospects for studying blob dynamics.

**Methods**

**Experimental setup and datasets collection**

Massive sets of fast imaging data have been recorded until COMPASS shutdown in 2021, with the aim to improve the
understanding of turbulent transport in tokamaks plasmas. For that purpose, we used modern high-speed visible cameras
capable of extremely fast frame rates (up to 2.1 Mfps) and high spatial resolution. After tomographic inversion, the
collected data allow to evidence individual plasma blobs in the vicinity of the LCFS and to investigate their shapes, trajectories, speeds, and mutual interactions¹⁰. In this study we focus on the shot #20846 recorded at 1 Mfps with reduced GPI and shot #15487 recorded at a lower frame rate (270 kfps) without GPI. Table 1 shows the main camera settings and parameters of the discharges.

<table><caption>**Table 1.** Main parameters of the shots investigated in our study</caption><thead><tr><th>Shots</th><th>Frame rate (kfps)</th><th>Exposure time (µs)</th><th>Toroidal magnetic field B<sub>T</sub> (Tesla)</th><th>Plasma current I<sub>p</sub> (kA)</th><th>Flat-top plasma density n<sub>e</sub> (10<sup>19</sup> m<sup>-3</sup>)</th><th>Camera window Width x Height (pixels)</th></tr></thead><tbody><tr><td>20846</td><td>1008</td><td>0.68</td><td>-1.15</td><td>182</td><td>4</td><td>128 x 40</td></tr><tr><td>15487</td><td>270</td><td>2.08</td><td>-1.15</td><td>185.14</td><td>3</td><td>128 x 144</td></tr></tbody></table>

**Camera and tools.** Fig 1a represents a sketch of the experimental apparatus, which is very similar to the one detailed in *Cavalier et al.*<sup>6</sup>, except for the two relay lenses which are of focal 75 mm instead of 45 mm, and for the camera, since a Photron SA-Z monochrome camera was used in shot #20846 instead of a Photron SA-X2 in previous measurements campaigns. The spectral sensitivity of the camera ranges from 400 nm to about 1000 nm, with a maximum around 680 nm. No interferential filter was used and the light captured by the camera is assumed to come mostly from the $D_α$ emission line at 656.3 nm, which largely dominates the visible spectrum during the experiments, and for which the quantum efficiency of the camera is close to its maximum of 46 %. The camera is located at a distance of approximately 1 m from the vessel. The 16 mm focal length lens is directed toward an aluminium mirror mounted inside the vacuum vessel and facing the LCFS. The mirror is tilted so as to have lines of sights (LOS) locally tangential to the magnetic field lines, in order to improve the quality of the tomographic reconstruction. The field of view of the camera thus obtained in a poloidal cross-section is depicted by the red square in Fig. 1b.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** a) Overview of the optical setup<sup>6</sup> combined with top view of the COMPASS tokamak showing typical toroidal magnetic field orientation $B_{tor}$ and viewing angle of the fast camera. The north direction is also shown for orientation purposes<sup>9</sup>. b) Poloidal cross-section of a field of view of the camera in the focal plane. [color online]</figcaption></figure>

**Auto labelling method for filaments' supervised learning.**

The main advantage of supervised machine learning over conventional methods (traditional segmentation) lies in its ability to manage greater variability (variation in brightness, noise, low contrast) and to considerably improve detection and clipping performance. Eliminating the need to manually adjust thresholds on a case-by-case basis to perform contouring structures is crucial. However, for training the model, it is always necessary first to build up a database containing a large number of labelled examples representative of the expected variability. If labelling is carried out
manually, image by image, structure by structure, the time cost may be too high for this approach to be viable. For
this reason, an automatic labelling method using classical segmentation has been developed. This automatic labelling
method is based on classical segmentation techniques, requiring the initial definition of intensity thresholds to isolate
turbulent structures in a subset of a few images selected from the several tens of thousands to be auto-labelled. These
images were chosen with the aim of being representative of the variability observed in the videos studied, particularly in
terms of brightness and contrast. For each sample, intensity thresholds were determined manually, enabling the contours
of structures to be segmented while minimising tomographic reconstruction noise and artefacts. Once set up, the auto
labelling tool can label a large volume of images with limited variability, without the need for frequent threshold
adjustments, which saves considerable time. However, when contouring proves inaccurate, particularly in poor contrast
conditions, manual adjustments are required to refine detection and correct segmentation errors. The virtual elimination
of manual annotation means that a subset of images can be quickly checked using tools such as Labelme¹⁸, ensuring
the accuracy of the defined contours (bounding boxes or polygon points surrounding the structure). This approach has
considerably speeded up data processing, while ensuring optimum quality when spot adjustments were required, making
the large-scale application of supervised learning techniques viable for the characterization of turbulent structures. This
autolabelling method is illustrated in Fig. 3(b).

DETECTION

Turbulents structures detection with CNNs.

**Object detection and semantic segmentation.** In the field of computer vision, two fundamental tasks that play a
crucial role in understanding visual data are semantic segmentation¹⁹ and object detection²⁰. Although both processes
involve analyzing images and identifying specific elements within them, they do it differently. Semantic segmentation
involves dividing an image into semantically meaningful regions and assigning class labels to each pixel. Essentially,
it provides detailed information about object boundaries and regions of interest. On the other hand, object detection
focuses on locating and classifying specific objects in an image, providing bounding boxes around them. By combining
these approaches, segmentation identifies filaments, object detection identifies the localization of the filaments inside
bounding boxes, and classification assigns labels and categories.

**YOLO.** YOLO series are a family of single real-time object detectors. The first YOLO model, which stands for "You Only Look Once", was introduced in 2016¹¹. Since then, it has proliferated the industry with speed and accuracy. For more details, the original proposal was to predict bounding boxes and class probabilities from an image, in a single evaluation using a single neural network. Several versions of Yolo have been developed since 2016, YoloV9²¹, and YoloV10²² are probably not the last ones. Our filament database has been trained on several versions since the beginning of this project, such as YoloV3, V4, V5, V6, V7, V8 and VX, but we are focusing on two versions that have shown the most promising results, i.e. YoloV7 and YoloV8.

Customized YOLOv7-seg, YOLOv8-seg model network.

In this study, filaments detection was achieved by utilizing the powerful YoloV7²³ and YoloV8²⁴ base models, which are
state-of-the-art supervised machine learning models. More specifically the ‘small’ variant marked ‘s’ such as YoloV8s
have been used, which is a good compromise for applications requiring a balance between speed and precision.
YoloV7, introduced in 2022, represents a significant advance with respect to its predecessors in the YOLO series. It
features increased efficiency and speed, thanks to specific optimisation techniques and architectural improvements such
as advanced feature aggregation blocks. These improvements enable better fusion of spatial information and features
at different levels, increasing the accuracy of object detection. YoloV7 also uses new training techniques, including
regularisation methods and advanced weight updating schemes, which improve model convergence and performance
compared to YoloV6.

To date YoloV8 is the penultimate version of the YOLO family, bringing even more innovations. It introduces more
advanced backbone architectures, such as CSPDarknet53, which better capture object characteristics and improve
detection in complex scenes. YoloV8 uses more sophisticated data augmentation techniques, such as mixup and mosaic,
increasing the robustness and generalisability of the model. In addition, optimisations to the detection head allow better
management of objects of different sizes and aspect ratios, while advanced post-processing techniques, such as duplicate
removal and clustering, improve the accuracy of final detections. YoloV8 also offers significant gains in performance
and scalability, running efficiently on a wider range of hardware, from high-performance GPUs to embedded devices. In
summary, YoloV8 generally outperforms YoloV7 with significant improvements in accuracy, efficiency, robustness and
scalability, making it a leading choice for real-time object detection applications. However, instead of training these
models on the large-scale COCO datasets²⁵ typically used for all the versions of YOLO, we trained them using our own
filaments dataset where both models have shown great results. Additionally, we combined the YoloV7 model with a
pixel-level characterization model, known as semantic segmentation, using Detectron2²⁶ as the mask branch of YoloV7.
This combined model is referred to as YoloV7-seg model. YoloV8 has an integrated semantic segmentation function. We applied transfer learning through fine-tuning to adapt the YoloV7-seg and YoloV8-seg models for classifying and detecting turbulent structures. This involved removing the fully connected nodes at the end of the network, which made class label predictions and initialized new nodes tailored to the "filament" category we wanted to detect. The earlier convolutional layers in the network were frozen to preserve the robust features learned by CNN. We then trained the model only on the fully connected layers, achieving successful results as described in the results section.

## TRACKING

Usual object tracking methods generally rely on the Kalman Filter (KF) algorithm<sup>27,28</sup>, which is also the basis of predictions included in tracking algorithms designed with CNNs and described in this article. Another real-time object tracking method is the optical flow method, used for instance by *Han et al.*,<sup>17</sup> for detecting blobs in TCV tokamak. Despite its good accuracy for capturing small movements and its robustness to variations in lighting, the optical flow method suffers from limitations in case of occlusion, large deformations, small gradients and sudden changes<sup>29</sup>. Indeed, sudden deformations occur frequently when dealing with turbulent structures.

The KF is a powerful recursive algorithm used for estimating the state of a dynamic system from a series of noisy measurements. In the context of 2D object tracking, it is employed to predict the object's future positions and adjust these predictions based on new incoming measurements. The principle of the KF algorithm is described in detail in Fig. 2. The process begins with the initialization of the state vector, which includes the initial position and velocity of the structure $S_i (x_{k-1}, y_{k-1})$ and $(v_{x,k-1}, v_{y,k-1})$, expressed as $\hat{x}_{k-1}$ at time step (image) $k-1$ in Fig. 2, and the initial covariance matrix $P_{k-1}$ that quantifies the uncertainty of these initial estimates. The prediction phase follows, where the state transition matrix $A_k$ at time step $k$ is applied to forecast the object's next position and velocity. This model assumes a constant velocity, updating the position based on the current velocity and the elapsed time. Simultaneously, the covariance matrix is updated to account for the process noise $Q_{p,k}$, reflecting the growing uncertainty over time. Upon receiving a new measurement by the detection algorithm, the filter proceeds to the measurement update phase. Here, the measurement model representing the detected position of the structure $S_i$ links the measurement matrix $H_k$ to the predicted state, allowing the calculation of the Kalman Gain $K_k$. The Kalman Gain determines the extent to which the prediction should be adjusted based on the new measurement, balancing the prediction uncertainty against the measurement noise expressed by the measurement noise covariance matrix $R_{p,k}$. The state estimate is then corrected by comparing the predicted state with the measurement $y_k$, weighted by the Kalman Gain, improving the precision. Additionally, the covariance matrix $P_k$ is updated to reflect the enhanced certainty in the state estimate after incorporating the new measurement. The accuracy of Kalman filter predictions generally improves with time and the accumulation of data. Initial predictions may be approximate, but as the filter processes more measurements and adjusts its estimates, it becomes increasingly accurate. This cycle of prediction and measurement update is iteratively repeated at each time step, allowing the KF to continuously refine its estimates of the object's position and velocity. By effectively combining the predictions derived from the motion model with the corrections provided by the measurements, the KF achieves robust and accurate tracking of the object's trajectory in 2D space, even in the presence of noise and other uncertainties. This makes the KF an essential tool in various applications, from navigation systems to computer vision and robotics. In this paper, we compare conventional tracking using a standard Kalman filter and other algorithms using CNNs. On the one hand, for the conventional method, we have manually set the pixel tolerance of the KF which refers to the allowable margin of error in predicting or detecting the position of the filament to be tracked. It is used to determine the maximum distance between the position predicted by the KF and the actual position of the filament to consider the prediction valid. Pixel tolerance is built into the KF primarily via the measurement noise covariance matrix $R$. By increasing the values in $R$ to reflect a larger tolerance, the filter automatically adjusts the confidence in the measurements relative to the predictions, making monitoring more robust to uncertainties and variations in detections. On the other hand, tolerance or accuracy in the tracking algorithms is mainly controlled by other parameters such as decision thresholds for associating detections between frames, thresholds for removing false detections, and parameters of the deep association model used for estimating the similarity between detected objects. These parameters indirectly affect tracking accuracy by influencing object association and prediction decisions.

## Turbulents structures tracking with CNNs.

The objective of multi-object tracking (MOT) involves both detecting and tracking all the turbulent structures within a scene, ensuring each filament maintains a distinct identifier throughout the process. Four tracking models have been tested. These models use a deep neural network for feature extraction such as bounding boxes coordinates and high-dimensional appearance (color, texture and shape), and a traditional tracking algorithm (usually the Kalman filter or Hungarian algorithm) to associate object detections across images. DeepSORT<sup>12</sup> and StrongSORT<sup>13</sup> focus more
<figure><img src="image_3.png" /><figcaption>**Figure 2.** Diagram of the Kalman filter prediction and correction steps</figcaption></figure>

on advanced appearance descriptors to distinguish objects, while BotSORT¹⁴ concentrates on optimizing bounding boxes and motion patterns. ByteTrack¹⁵ uses a combination of appearance and position and is unique in its explicit management of weak detections, which enhances its robustness. All are designed to be fast and efficient and adapted to real-time tracking (considering standard cameras up to few tens of frames per second).

## Global scheme of Deep learning

### Principle of data collection and preparation for machine learning. Fig.3.1

On the one hand, the data pre-processing principle is described in Fig. 3.a. Firstly, data are collected using an ultra-fast camera that observes the poloidal plane of the COMPASS tokamak. To perform tomographic inversion, we must know precisely from where the light collected by a pixel of the camera comes from. Each pixel is associated with a line of sight that passes through the pupil of the camera. A bright image of the tokamak with the camera is superposed to a CAD model of the chamber thanks to a software called Calcam³⁰ that calculates the equation of each line of sight in the frame of the machine. This operation is called a calibration. Calibration is essential to correct distortions and imperfections in the images, then tomographic inversion is applied⁶. This technique reconstructs the spatial distribution of plasma properties from camera data. In this context, it is used to obtain inverted images of turbulent structures or filaments present in the plasma of the COMPASS tokamak. These inverted images are crucial for subsequent analysis and detection of filaments.

On the other hand, after obtaining inverted images of plasma turbulent structures from the pre-processed data in Fig.3.(a), the process of data labeling presented in Fig. 3.b begins, for which more than 36,000 inverted images are labeled using the auto-labeling method developed specifically for this project. This method assigns annotations to each image, describing the position and spatial characteristics of turbulent filaments. Two techniques are used: mask labeling, which extracts polygon points to define filament contours, or bounding box labeling, which extracts positional information from top, left, width and height points to delimit bounding boxes around filaments. These labeled annotations are then
transformed into a format compatible with machine learning models, in particular YOLO models. Spatial information extracted from filament bounding box or filament contours is converted into text files in YOLO format (.txt extension), containing bounding box coordinates and the corresponding categories of detected objects which represent filaments.

<figure><img src="image_4.png" /><figcaption>**Figure 3.** This diagram illustrates the data processing and collection process used to study filaments in tokamak plasmas. a) Use of an ultra-fast camera to capture images of the Compass tokamak poloidal plane, enabling visualisation of turbulent plasma structures, followed by image calibration with Calcam to correct distortions and imperfections, before application of tomographic inversion to obtain inverted images of turbulent structures in the chosen poloidal plane. b) Auto-labeling method for assigning spatial annotations to images, describing the contours of turbulent filaments and the use of mask labeling and bounding box labeling to extract filament position in a Yolo format as a supervised learning data preparation for ML algorithm. [color online]</figcaption></figure>

**Machine learning procedure. Fig.4.2**

Fig. 4 summarizes the process of detecting and tracking turbulent filaments in tokamak plasmas, divided into two distinct parts. In the first part (Fig. 4.a) video frames, accompanied by bounding box or polygon annotations for semantic segmentation, are used as input for filament detection. These data are processed by a convolutional neural network (CNN) in the YOLO backbone. To efficiently process objects of different sizes in the image, YOLO often uses a Feature Pyramid Network (FPN) in the neck of its network. The FPN is an architecture that combines information from different
spatial scales of the feature map to form a multi-scale feature pyramid. This enables the network to detect objects at different spatial scales. After generating the feature map from the input image using the network backbone and neck, regions of interest (ROIs) are identified using bounding boxes proposed by the model. These bounding boxes delimit potential regions where filaments could be present in the image. ROI pooling is then applied to each ROI to extract the relevant information. After that, the network uses on one side a fully connected layer in the network head for the prediction of bounding boxes and object classes (bbox regression and classification), and on the other side a mask branch for semantic segmentation contouring the filament. Finally, Non-Maximum Suppression (NMS) is used to obtain the final filament detections, either in a bounding box or mask. NMS is an essential step in object detection algorithms. Its main role is to remove redundant predictions in order to retain only the most reliable detections. After the model has made predictions on an image, several bounding boxes may be detected for each object, often with significant overlaps between them. Each predicted bounding box is associated with a confidence score, representing the probability that this bounding box contains a real object. The NMS intervenes to compare confidence scores and remove bounding boxes that have a high overlap with another bounding box with a higher confidence score. It uses a predefined overlap threshold (generally around 0.5 or 0.6) to determine which bounding boxes to delete. If the overlap between two bounding boxes is greater than this threshold, the bounding box with the lowest confidence score is deleted. By eliminating redundant detections, NMS improves the overall accuracy of the object detection model and helps deliver more reliable results. During this phase, the model learns how to recognize and locate turbulent filaments in the images. Optimization and hyperparameter tuning techniques are used to improve model performance. A detailed structures of both YoloV7 and yoloV8 can be found in *MMyolo contributions*<sup>31, 32</sup> respectively, with a comprehensive summary in *Terven et al.*<sup>33</sup> The second part of the diagram, Fig. 4.b, focuses on filament tracking. For feature extraction, the images are first cropped to select only the filaments and divided into a train set and a validation set. The training set consists in 30 randomly selected images grouped into 3 sub-folders, each one containing a specific image processing. For example, the first folder includes cropped filaments with bilateral processing, the second folder has a binary filter cropped images, and the last one contains the raw images without processing. The same is done for the validation set but with fewer images. Then, a data augmentation is applied to enlarge the number of training images with horizontal or vertical rotation techniques, size change, etc., These data are then passed through a Resnet18 CNN model<sup>34</sup> to extract the features required for tracking, where all the layers have been frozen but the last one. These features are then used as input for the trackers. The Kalman filter is used as the basis procedure for tracking filaments, it includes the prediction and the correction part as illustrated in Fig. 2. Four other trackers (DeepSORT, ByteTrack, BotSORT, and StrongSORT) are also used to track the filaments and a comparison between all these trackers has been made and described in the Result section. As output, these trackers provide bounding boxes or contour structures with identifiers for tracking and velocity calculation.

## Evaluation metrics.

**Precision and recall.** Precision and recall are two important metrics used to evaluate the performance of a machine learning model. Precision measures the proportion of true positive (TP) predictions among all the positive predictions made by the model. In other words, precision measures how well the model avoids false positive (FP) predictions. Precision is calculated as:

$$ Precision = \frac{TP}{TP + FP} \qquad (1) $$

Recall, on the other hand, measures the proportion of true positive predictions among all the actual positive cases in the dataset. In other words, recall measures how well the model avoids false negative (FN) predictions. Recall is calculated as:

$$ Recall = \frac{TP}{TP + FN} \qquad (2) $$

**mean Average Precision (mAP).** mAP is a metric used to evaluate the accuracy of algorithms that rank items based on their relevance to a class: mAP is defined as the average of the average precision (AP) scores for each class in a test set. As we have only one category "Filament", the average precision for a single class is calculated by measuring the area under the precision-recall curve. The precision-recall curve is a graph that shows the trade-off between precision and recall at different ranking thresholds, a higher mAP score indicates better performance, with a maximum possible value
<figure><img src="image_5.png" /><figcaption>**Figure 4.** Diagram illustrating (a) blob detection using YOLO architecture and (b) tracking using The Kalman filter as in conventional tracking or several other trackers based on features extracted by CNNs. [color online]</figcaption></figure>

of 1.0.

**Intersection over Union (IoU).** IoU is used to measure the overlap between a predicted bounding box and a ground truth bounding box. In image segmentation, IoU is used to evaluate the accuracy of a segmentation algorithm by comparing the predicted segmentation mask to the ground truth mask. The IoU is calculated by finding the intersection area between the two masks and dividing it by the union area of the masks and it is expressed as a value between 0 and 1, where a value of 0 indicates no overlap between the predicted and ground truth bounding boxes, while a value of 1 indicates perfect overlap. In our study, a threshold value of 0.5 is used to determine if a detection is considered a true positive or false positive.

$$IoU = \frac{\text{Area of overlap}}{\text{Area of union}} \qquad (3)$$
## Computational tools and hyperparameters settings.

On the one hand, the conventional analysis of tomographically reconstructed data was carried out with the AX R&D software³⁵, by using conventional methods for image filtering, simple intensity thresholding for filament detection, and predictive Kalman filter tracking. On the other hand, for machine learning analysis, the training data set was constituted by more than 36000 images divided into an 80% training set and a 20% validation set. The automatic labelling of filaments was also done with the AX R&D software, which includes the developed automatic labeling method presented in Fig. 3.b. It resulted in 46,730 polygons labels for the "filament" class, whose validity was then partly verified with the online image annotation application Labelme.

ML methods require a rather significant amount of memory space to train a model effectively on large datasets. The training process was performed on a Microsoft operating system, utilizing a XEON processor and Nvidia RTX 4000 GPU with 8GB of memory. The training was conducted using the CUDA 11.1 library toolkit and cuDNN 8.1, with Python version 3.8 and the PyTorch 1.11.0 deep learning framework.

The original image size in the dataset was 64 x 64 pixels, but during the training stage, the model resized the images to 640 x 640 pixels. The batch size for training was set between 4 to 8, and the model was trained for 300 epochs. The stochastic Gradient Descent (SGD) algorithm was used as the optimizer to update gradients, with a maximum learning rate of 0.1 and a momentum of 0.937. Data augmentation techniques such as horizontal and vertical translation, as well as adjustments to light intensity and colors (contrast, Hue, etc.), were applied.

## Results

### Filaments detection.

For filament detection, we have tested YoloV3, V4, V5, V6, V7, V8, and YoloX. Fig. 5 depicts the Average Precision (AP) achieved with YoloV7, very close to what is obtained with YoloV8. As a comparison, the AP of YoloV4 is only 56% and YoloX is 36%, as summarized in Table 2 at 50 % of IoU.

<table><caption>**Table 2.** Average Precision of the different versions of Yolo applied for the filaments detection</caption><thead><tr><th>Yolo versions</th><th>YoloV4</th><th>YoloV5</th><th>YoloV6</th><th>YoloV7</th><th>YoloV8</th><th>YoloX</th></tr></thead><tbody><tr><th>Average Precision (AP)</th><td>0.56</td><td>0.984</td><td>0.877</td><td>0.988</td><td>0.987</td><td>0.36</td></tr></tbody></table>

The inference time is also a crucial criterion for evaluating the effectiveness of detection and tracking models and algorithms. When comparing YoloV7-seg and YoloV8-seg, YoloV8-seg performed better with a reduced inference time of 13.5 ms per frame compared to YoloV7-seg as summarized in Table 3. This improvement is attributable to the optimisations and architectural advances incorporated into YOLOv8-seg, which enable faster processing while maintaining high detection accuracy.

<table><caption>**Table 3.** Inference speed of both of YoloV7-seg and YoloV8-seg detection process, the inference speed of the four tracking models tested on filaments detection and tracking.</caption><thead><tr><th colspan="5">Inference Speed [ms/frame]</th></tr><tr><th>Detection models</th><th>YoloV7-seg</th><th>YoloV8-seg</th><th colspan="2">Conventional thresholding</th></tr></thead><tbody><tr><td></td><td>43.7</td><td>13.5</td><td colspan="2">13.1</td></tr><tr><th>Tracking models</th><td>DeepSORT</td><td>StrongSORT</td><td>BotSORT</td><td>ByteTrack</td></tr><tr><td></td><td>52.8</td><td>46.8</td><td>42.8</td><td>52.7</td></tr><tr><td></td><td></td><td></td><td></td><td>Kalman filter</td></tr></tbody></table>

Fig. 6 shows a visual comparison between the actual labels and the predictions made by a semantic segmentation and bounding box detection model trained on YOLOv8. At the top of Fig. 6a and Fig. 6b, the semantic segmentations of the objects in the image are displayed, with each filament coloured with a hue. Object bounding boxes are also drawn around each object, with class labels placed above or next to the boxes. The bottom of Fig. 6a and Fig. 6b show the semantic segmentations predicted by the YoloV8 model, where each object is coloured similarly to the actual labels. The predictions include a confidence percentage displayed on each segment, indicating the probability that the model attributes to the presence of that specific object. The predicted bounding boxes are superimposed on the detected objects, accompanied by labels indicating the class of the object, which for us is 'filament', and the percentage of confidence in the prediction. Confidence percentages are displayed alongside the class labels, providing an indication of the model's certainty for each prediction. High values (close to 100%) indicate high confidence, while lower values suggest greater
<figure><img src="image_6.png" /><figcaption>**Figure 5.** The area under the curve representing the average precision of YoloV7 model</figcaption></figure>

uncertainty. The juxtaposition of actual labels and predictions makes it easy to visualise differences and similarities, highlighting segmentation and detection errors, such as missing or misclassified objects, providing insights for model improvement.

For filament detection, data from several discharges were trained in order to enrich the database. To compare the ML and traditional methods, we chose to test both approaches on 40.000 frames from shot 15487, as shown in Fig. 7. Fig. 7a depicts the position of the center of mass of the filaments obtained using YOLOv7-seg. The colours indicate the density of the structures detected, with more intense hues representing a larger concentration of filaments. As a comparison, Fig. 7b shows the location of the filaments detected by conventional thresholding methods. It can be noted that in both cases, the filaments are predominantly detected in the Scrape-Off Layer, and exhibit a distribution parallel to the LCFS, which is physically consistent with the capabilities offered by passive visible imaging diagnostics. Although rather similar, the YoloV7 seg map reveals a narrower distribution of filaments than the one obtained with the traditional methods where the detection area is only slightly larger (44.16 cm² VS 53.56 cm²), but the greater density is attributed to YoloV7 seg. The total number of filaments within this area after trajectory filtering, considering a minimum of 2 frames per trajectory, is substantially larger in the first case: 35344 VS 29808. In our opinion, the physically relevant localization of these additional filaments convincingly demonstrates the superiority of the method based on Machine Learning compared to conventional thresholding for analyzing the turbulence dynamics in tokamak plasmas accurately. However, there is a price to pay in terms of processing time, with an inference time of 43.7 ms/frame with YOLOV7-seg, compared to an average analysis time of 13.1 ms/frame with conventional thresholding, while YoloV8-seg has made great strides, achieving a detection inference time of 13.5 ms per frame. (See Table 3)

The 2D poloidal maps reconstructed with the other ML models are very similar to the one obtained with YoloV7-seg.
<figure><img src="image_7.png" /><figcaption>**Figure 6.** Labeled and predicted filaments with the model level of certainty normalized to 1 a) inside Bbox detected by YoloV8 and b) Semantic segmentation detected by YoloV8seg. [color online]</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 7.** Comparison between the number of structures represented in a poloidal plane a) detected by YoloV7-seg and b) detected by conventional methods in COMPASS shot #15487. The colorbar indicates the number of filaments detected on each pixel. The highest density of turbulent structures is seen in the SOL, parallel to the LCFS, depicted by the solid red line. [color online]</figcaption></figure>

Another more accurate means for evaluating the efficiency of the various models is to compare their Average Precision (AP) as Table 3 illustrates.
Filaments tracking.

With regard to the tracking algorithms, a comparison of the inference times between several trackers was carried out and summarized in Table 3: BoTSORT, StrongSORT, ByteTrack, DeepSORT, and the conventional method using the Kalman filter. The results show that BoTSORT has the shortest inference time, followed by StrongSORT. ByteTrack came next despite its first position when compared in the literature with the COCO dataset, with a slightly slower but still efficient performance. DeepSORT, despite being older, ranks almost similar to ByteTrack in terms of speed. Finally, the conventional method using the Kalman filter, although effective for simple scenarios, is the slowest, particularly in environments with a high density of objects and complex movements: from this point of view, the complex dynamics of blobs experiencing mutual interactions such as merging, splitting and fast reversal of the velocity¹⁰ is particularly challenging. To evaluate the quality of a model, the quality of the tracking is, of course, a more important result than the inference time. Performing tracking requires assigning each detected object an unique identifier and maintaining the continuity of this identifier in successive images. Fig. 8 is an example of two filaments tracked by DeepSORT in a frame, showing their IDs followed by the class "filament" surrounding the mask of the detected filament.

<figure><img src="image_9.png" /><figcaption>**Figure 8.** Filaments tracking using YoloV7 DeepSort, where the number before the class name represents the ID of the filament. [color online]</figcaption></figure>

In order to compare the different tracking models used, we tested them on the analysis of the same dataset by considering shot 20846, which allows for a good resolution of the filament dynamics thanks to an acquisition rate of 1 Mfps. The time-average 2D maps of the poloidal velocities of filaments obtained with all the tested models are first depicted in Fig. 9. The LCFS separator, marked by a solid red line, delimits the boundary between the confined plasma region and the Scrape-Off-Layer. Fig. 9a, obtained using a conventional tracking method, shows a visible poloidal shear zone parallel to the LCFS and a velocity distribution radially homogeneous, although limited in coverage. The location of this shear zone, verified by cross-comparisons with measurements from reciprocal probes, and its general appearance enable the validity of the tomographic reconstruction and tracking to be established. The different coverage compared to Fig. 7 is due to i) different camera settings and ii) the slight gas puff injection in this shot, which enables to detect structures beyond the LCFS, in the confined plasma. Fig. 9b, using DeepSORT, shows results similar to the conventional method, with a well-defined shear zone, but lower poloidal velocities in the confined region. Fig. 9c, using StrongSORT, reveals holes in the velocities at the heart of the plasma, at radial locations where the detection is mostly intermittent due to weak visible light emission. The shear zone is clearly visible and the magnitude of the velocities show larger extrema than the results obtained with all the other models. Fig. 9d, using BotSORT, also shows holes in the velocities, particularly in the central part of the plasma. Fig. 9e, using ByteTrack, is characterised by a more homogeneous and extensive coverage, without any hole and with a clearly visible shear zone. A common characteristic of all the ML models is a better coverage of the velocity map in area corresponding to the Scrape-Off-Layer, with the better global coverage obtained with ByteTrack.
<figure><img src="image_10.png" /><figcaption>**Figure 9.** 2D maps of poloidal velocities of turbulent structures (Shot #20846) represented in a poloidal plane obtained by (a) conventional tracking (Kalman filter) (b) DeepSORT, (c) StrongSORT, (d) BotSORT and (e) ByteTrack algorithms. Positive poloidal velocities, depicted in warm colors, are directed upwards, while cold colors represent negative poloidal velocities, directed downwards. [color online]</figcaption></figure>

The spatial coverage is identical when considering the 2D maps of radial velocities, as illustrated in Fig. 10a for the map obtained with conventional Kalman filter and in Fig. 10b for the one obtained with ByteTrack. The velocity map obtained with conventional tracking does not show any comprehensive feature, such as a well-defined pattern in the poloidal shear zone. In contrast, the radial velocity map produced with ByteTrack exhibits a similar pattern as the one observed on the poloidal velocity map, with a shear zone clearly visible, highlighted by a continuous short line parallel to the LCFS. This distinction demonstrates the advantage of machine learning techniques, such as ByteTrack, in detecting more complex structures and subtle dynamics in the plasma, demonstrating greater efficiency than traditional methods.

In order to determine how these different models perform under less favorable conditions, we also analyzed a downsampled version of the same shot 20846. For this analysis, we kept only every other frame, simulating a video acquired at a rate of 500 kfps. The results, for the maps of poloidal velocities, are depicted in Fig. 11. Not surprisingly, the resulting maps show a significantly reduced coverage, but this mainly concerns the confined zone, where the blobs are detected much more intermittently. The conventional tracking method (Fig. 11a) remains effective, showing a visible shear zone and a velocity distribution radially homogeneous in the reconstructed area. On the other hand, StrongSORT (Fig. 11b) and BotSORT (Fig. 11c) show inferior performance, with visible holes in the speeds and less homogeneous coverage, as well as a noisier poloidal velocity distribution in the shear zone regarding StrongSORT. ByteTrack (Fig. 11d), on the other hand, continues to stand out as the best candidate, offering a well-defined shear zone with positive velocities outside the separator and negative velocities inside, and a wider coverage compared to the two other ML models.
<figure><img src="image_11.png" /><figcaption>**Figure 10.** Comparison between radial velocities represented in a poloidal plane (a) Conventional tracking and (b) ByteTrack tracking of shot#20846. Yellow colors represent radially outwards blob displacements, and blue colors radially inwards displacements. [color online]</figcaption></figure>

These comparisons illustrate the importance of choosing the right tracking algorithm based on the specific requirements of the application, balancing accuracy and time efficiency. These results show that ByteTrack performs best, especially at a reduced sampling rate, offering a better representation of poloidal velocities and a clear definition of the shear zone.

## Discussion and conclusion

A major challenge in the search for practical fusion energy lies in understanding and controlling turbulent transport in plasma. Turbulence in the fusion plasma gives rise to the formation of coherent structures called “blobs” or “filaments” characterized by complex dynamics. The application of machine learning techniques to images from the boundary plasma in the COMPASS tokamak, captured by ultra-fast cameras and processed by tomographic inversion on a poloidal section centered on the separatrix (LCFS), has shown promising results in the detection and tracking of turbulent filaments. These filaments, which play a crucial role in transporting particles and heat toward the walls of the tokamak, can seriously deteriorate the confinement of the hot plasma.

The integration of ML algorithms enabled us to accurately identify the turbulent filaments and to analyse their dynamics with high spatial and temporal resolutions. Since a good detection model is crucial for a good tracking, a rigorous evaluation of the results based on the evaluation metrics has been made. More precisely, this study demonstrates the effectiveness of the application of the YOLO algorithm for the detection of turbulent filaments in tokamak plasmas, compared to conventional thresholding techniques. This result is all the more remarkable because these successes were achieved through an automated labeling method based on conventional thresholding techniques. It is therefore an example of learning where one can say that “the student surpasses the teacher”. The use of this automated labeling method also eliminates human subjectivity, presents a considerable time savings, and can be applied to experimental data of the same nature as the data to be analyzed. The automatically labeled data was used to train various Yolo models to detect filaments, using different strategies such as object detection and semantic segmentation. The most accurate results were obtained with YoloV7-seg, followed by YoloV8-seg and YoloV5. However, YoloV8-seg proved to be superior in its ability to quickly analyze images, with an inference time more than three times shorter than YoloV7. Regarding tracking, we relied on the most accurate detection method, obtained with YoloV7-seg, and well-sampled temporal data at 1 Mfps. Four recent CNN-based tracker models were tested—DeepSORT, StrongSORT, BotSORT, and ByteTrack—and compared with conventional tracking using a Kalman filter. The most convincing results were obtained with ByteTrack, which surpasses conventional tracking methods both in accuracy and in terms of processing time. The analysis of temporally downsampled data shows less significant gains, but again ByteTrack seems more accurate than the other ML models tested. However, this gain in accuracy comes at the cost of an inference time up to 20% longer
<figure><img src="image_12.png" /><figcaption>**Figure 11.** 2D maps of poloidal velocities of turbulent structures (Shot #20846 downsampling 500k fps) represented in a poloidal plane obtained by (a) Conventional tracking, (b) StrongSORT, (c) BotSORT and (d) ByteTrack algorithms [color online]</figcaption></figure>

than other ML models tested, notably BotSORT, contrary to what is observed on generalist databases not including turbulent dynamics.

All in all, the integration of ML algorithms enabled us to accurately identify the turbulent filaments and to analyse their dynamics with high spatial and temporal resolutions. As an example, the efficient construction of 2D maps of radial and poloidal velocities consistently evidenced the shear zone located in the Scrape-Off-Layer. The inference times of the models used are found to be competitive with the analysis times of conventional methods and are progressing rapidly. Although it is not yet possible to conduct this type of analysis in real-time, it is conceivable that in the future, pre-trained ML models could enable certain imaging analyses to be performed live. By enabling the counting and localization of filaments, and even the localization of shear zones, these models could contribute to reactor control and improve their stability.

We feel it is important to point out that, in order to be useful, the application of the AI techniques presented in this article to tokamak data still requires the data to be pre-processed by fusion physicists: the selection of imaging data as a function of the plasma parameters, the calibration of the tomographic inversion of the data, and then the manual
thresholding of the images required for the Auto Labelling procedure all require expertise specific to the operation of
tokamaks. It should also be kept in mind that if it is preferable to validate the obtained results by comparing them with
other experimental or modelling data. To do this, we recommend checking the accuracy of the location of the shear zone,
which can be deduced from other plasma diagnostics such as reciprocating probes, used on COMPASS, or Doppler
reflectometry measurements on other tokamaks.

Finally, beyond the scope of fusion plasmas, we have demonstrated that ML models can be more effective than
conventional methods for studying the dynamics of coherent structures in turbulent environments, which offers prospects
for applications in the study and control of fluid environments in general.

Data Availability

The datasets used and/or analyzed during the present study are available from the corresponding author upon reasonable
request. The data can be found in "Chouchene, S., Brochard, F., Lemoine, N., and Cavalier, J. (2024). Detection
and tracking of turbulent structures in the edge of tokamak plasmas using machine learning [Data set]. Zenodo.
https://doi.org/10.5281/zenodo.12608068 "
References

1. D'Ippolito, D. A., Myra, J. R. & Zweben, S. J. Convective transport by intermittent blob-filaments: Comparison of theory and experiment. *Phys. Plasmas* **18**, 060501 (2011).

2. Federici, G. et al. Overview of the demo staged design approach in europe. *Nucl. Fusion* **59** (2019).

3. El-Taibany, W. et al. Comparison study of the energy and instability of ion-acoustic solitary waves in magnetized electron-positron-ion quantum plasma. *Sci. Reports* **12**, 19078 (2022).

4. Takahashi, K., Charles, C., Boswell, R. W. & Ando, A. Demonstrating a new technology for space debris removal using a bi-directional plasma thruster. *Sci. reports* **8**, 14417 (2018).

5. Zweben, S. J., Terry, J. L., Stotler, D. P. & Maqueda, R. J. Invited review article: Gas puff imaging diagnostics of edge plasma turbulence in magnetic fusion devices. *The Rev. scientific instruments* **88** 4, 041101 (2017).

6. Cavalier, J. et al. Tomographic reconstruction of tokamak edge turbulence from single visible camera data and automatic turbulence structure tracking. *Nucl. Fusion* **59** (2019).

7. Svoboda, J. et al. Tomotok: python package for tomography of tokamak plasma radiation. *J. Instrumentation* **16** (2021).

8. M. Hron et al, Overview of the compass results. *Nucl. Fusion* **62** (2021).

9. Vlainic, M. *Studies of Runaway Electrons in COMPASS Tokamak*. Ph.D. thesis, Ghent University (2017).

10. Chouchene, S. et al. Mutual interactions between plasma filaments in a tokamak evidenced by fast imaging and machine learning. *Phys. Rev. E* **109**, 045201, DOI: 10.1103/PhysRevE.109.045201 (2024).

11. Redmon, J., Divvala, S. K., Girshick, R. B. & Farhadi, A. You only look once: Unified, real-time object detection. *2016 IEEE Conf. on Comput. Vis. Pattern Recognit.* (CVPR) 779–788 (2015).

12. Wojke, N., Bewley, A. & Paulus, D. Simple online and realtime tracking with a deep association metric. In *2017 IEEE international conference on image processing (ICIP)*, 3645–3649 (IEEE, 2017).

13. Du, Y. et al. Strongsort: Make deepsort great again. *IEEE Transactions on Multimed*. (2023).

14. Aharon, N., Orfaig, R. & Bobrovsky, B.-Z. Bot-sort: Robust associations multi-pedestrian tracking. *arXiv preprint arXiv:2206.14651* (2022).

15. Zhang, Y. et al. Bytetrack: Multi-object tracking by associating every detection box. In *European conference on computer vision*, 1–21 (Springer, 2022).

16. Kongsilp, P., Taetragool, U. & Duangphakdee, O. Individual honey bee tracking in a beehive environment using deep learning and kalman filter. *Sci. Reports* **14**, 1061 (2024).

17. Han, W. et al. Tracking blobs in the turbulent edge plasma of a tokamak fusion device. *Sci. Reports* **12**, 18142 (2022).

18. Torralba, A., Russell, B. C. & Yuen, J. Labelme: Online image annotation and applications. *Proc. IEEE* **98**, 1467–1484 (2010).

19. Hao, S., Zhou, Y. & Guo, Y. A brief survey on semantic segmentation with deep learning. *Neurocomputing* **406**, 302–321 (2020).

20. Zou, Z., Chen, K., Shi, Z., Guo, Y. & Ye, J. Object detection in 20 years: A survey. *Proc. IEEE* **111**, 257–276 (2023).

21. Wang, C.-Y., Yeh, I.-H. & Liao, H.-Y. M. Yolov9: Learning what you want to learn using programmable gradient information. *arXiv preprint arXiv:2402.13616* (2024).

22. Wang, A. et al. Yolov10: Real-time end-to-end object detection. *arXiv preprint arXiv:2405.14458* (2024).

23. Wang, C.-Y., Bochkovskiy, A. & Liao, H.-Y. M. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. *2023 IEEE/CVF Conf. on Comput. Vis. Pattern Recognit.* (CVPR) 7464–7475 (2022).

24. Reis, D., Kupec, J., Hong, J. & Daoudi, A. Real-time flying object detection with yolov8. *arXiv preprint arXiv:2305.09972* (2023).

25. Lin, T.-Y. et al. Microsoft coco: Common objects in context. In *Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V* **13**, 740–755 (Springer, 2014).
26. Wu, K. A. M. F. L. W. G. R., Y. Detectron2. https://github.com/facebookresearch/detectron2 (2019). Accessed: April 2023.

27. Welch, G., Bishop, G. et al. An introduction to the kalman filter. (1995).

28. Kim, Y., Bang, H. et al. Introduction to kalman filter and its applications. *Introd. Implementations Kalman Filter 1*, 1–16 (2018).

29. Sun, D., Roth, S. & Black, M. J. Secrets of optical flow estimation and their principles. In *2010 IEEE computer society conference on computer vision and pattern recognition*, 2432–2439 (IEEE, 2010).

30. Silburn, S. et al. Calcam, DOI: 10.5281/zenodo.10116099 (2023).

31. Contributors, M. MMYOLO: OpenMMLab YOLO yolov7 benchmark. https://github.com/open-mmlab/mmyolo/tree/main/configs/yolov7 (2023).

32. Contributors, M. MMYOLO: OpenMMLab YOLO yolov8 benchmark. https://github.com/open-mmlab/mmyolo/tree/main/configs/yolov8 (2023).

33. Terven, J., Córdova-Esparza, D.-M. & Romero-González, J.-A. A comprehensive review of yolo architectures in computer vision: From yolov1 to yolov8 and yolo-nas. *Mach. Learn. Knowl. Extr.* **5**, 1680–1716 (2023).

34. Targ, S., Almeida, D. & Lyman, K. Resnet in resnet: Generalizing residual architectures. *arXiv preprint arXiv:1603.08029* (2016).

35. APREX Solutions website. https://aprex-solutions.com/en/. Accessed: May 2024.

## Acknowledgements

This work was supported by the «PLUS» project co-funded by FEDER-FSE Lorraine et Massif des Vosges 2014-2020, a European Union Program, and by the french national research agency (Agence Nationale de la Recherche) under project PLATUN ANR-21-CE30-0063, and co-funded by the MEYS project No. LM2023045. This work has also been carried out within the framework of the EUROfusion Consortium, funded by the European Union via the Euratom Research and Training Programme (Grant Agreement No 101052200 — EUROfusion). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the European Commission can be held responsible for them. S.C. would like to thank APREX Solutions and the Region Grand-Est for the funding of her Ph.D. work and all the COMPASS team for carrying out the experiments.

## Author contributions statement

S.C: Conceptualization, software, data cleaning, formal analysis, figures, writing, revision. F.B: interpretation, writing, editing, revision, ressources. M.D: software and editing. N.L: tomographic inversion, writing and editing. J.C: data acquisition, tomographic inversion, writing and editing.