title: Integrating Machine Learning and Computer Vision into PiSCAT: An Open Source Python Package for Interferometric Scattering Microscopy
title: Integration von maschinellem Lernen und Computer Vision in PiSCAT: Eine Open-Source-Python-Bibliothek für interferometrische Streuungsmikroskopie"
title: AI Integration in Interferometric Scattering Microscopy
titleType: AlternativeTitle
Creators: name: Dastjerdi, Houman Mirzaalian
Publisher: name: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Year: 2024
Subjects: subject: PiSCAT, iSCAT microscopy, Computer Vision (CV), Machine Learning (ML), Open-source Python package

description: PiSCAT attempts to address researchers' analysis challenges while deploying iSCAT microscopy for high spatiotemporal resolution imaging applications. PiSCAT tackles heavy quantitative analysis with performance-optimized, physically accurate, verified Computer Vision (CV) and Machine Learning (ML) algorithms. The topic of PiSCAT, an open-source Python package for interferometric scattering microscopy, was initially introduced and sought after in 2019. PiSCAT has been developed with a user-friendly Graphical User Interface (GUI), allowing users, regardless of their programming skills, to easily analyze microscopy data, particularly iSCAT data. This approach makes the framework accessible to a broad and diverse audience. PiSCAT has also been deployed with installation instructions, tutorials, unit tests, deployment tests, and Application Programming Interface (API) support available to all users as an open-source package on GitHub. The current version of PiSCAT includes specific detection and mass estimation of unlabeled protein detection, three-dimensional (3D) tracking of nanoparticles, and the physical vectorial model of the iSCAT Point Spread Function (iPSF) model. However, iSCAT algorithms are modular and may be utilized for other microscopic applications. These analyses help to detect nanoparticles with a very weak Signal-to-Noise Ratio (SNR) using cutting-edge CV and ML background correction techniques, such as Differential Rolling Average (DRA), Fixed Pattern Noise correction (FPNc), Sparse in Time Affine Registration (SITAR), and Dictionary Learning (DL), as well as a range of localization and tracking algorithms. The PiSCAT website offers interactive Jupyter tutorials to help users become familiar with the various PiSCAT APIs and functionalities. PiSCAT offers several interactive visualization displays to achieve this goal, assisting users in selecting the optimum analysis parameters. Additionally, it provides a variety of statistical evaluations and the ability to visualize and interact with the temporal/spatial traces of the detected nanoparticles in large videos. In the new version of the analysis, PiSCAT uses the ability of a deep neural network to combine spatial-temporal features to track low SNR particles against noisy background with noise structures similar to real iPSFs. The End-to-End Unsupervised Anomaly Identification (UAI) deep neural networks are robust, fully automated algorithms which manipulate features of iPSFs and background speckles into a new feature domain which can separate the footprint of the iPSF from background speckles. Since this method is unsupervised, it can be used for different applications with only fine-tuning of the hyperparameters. This method produces remarkable results in pushing the sensitivity limit in label-free detection of single proteins below 10 kDa. According to Google's most recent data study, there are 2,242 users worldwide. Among these users, approximately 42% are from Europe, 34% from Asia, 21% from the Americas, and 3% from other continents. The current PiSCAT version can execute analyses in parallel on CPUs and GPUs with various levels of computational power.