# Demystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools
## 5 Tooling in MLOps
### Requirements on Tools in MLOps
#### 5.1.1 Data Preprocessing Phase (DPR)  
ML models are only as good as their input data. The data collected from different sources undergo different preprocessing steps through which they are prepared to be used by an ML algorithm. There are various requirements for implementing the preprocessing steps in ML projects. DPR-1: Data Source Handling  
Integrating different data sources in a project's pipeline can be accomplished by using connectors or interfaces to load data. As every data science project depends on data sources, handling various sources (databases, files, network sockets, complete services, and many more) is vital to the pipeline. DPR-2: Data Preprocessing and Transformations  
As input data are frequently formatted in incompatible ways, the preprocessing and transformation of data must be supported. Often, the domain-specific input data are of a structured nature like text or numbers. As the ML project is dependent on cleaned data, other unstructured formats like audio, images, or videos must be taken into consideration, too. DPR-3: Data Quality Measurement Support through Metrics  
Automated data validation is a critical component of MLOps. In order to ensure data quality for images, one can utilize reference-based Image Quality Assessment (IQA) methods. These methods require a high-quality reference image to which other images are compared. Some of these methods are listed in Table 1. Other aspects of data quality measurements are auditioning of data (unit and integration testing for data). \begin{table}
\begin{tabular}{p{113.8 pt} p{284.5 pt}} \hline \hline
**Metric** & **Brief Metric Description** \\ \hline SSIM [46] & Predict image quality by measuring the structural similarity between two images. \\ \hline MS-SSIM [47] & SSIM based on modeling of image luminance, contrast and structure at multiple scales. \\ \hline MC-SSIM [47] & Motion-compensated SSIM by comparing input to previous images/frames in a video. \\ \hline IW-SSIM [48] & Determines the Information Content of an image and performs Weighting SSIM. \\ \hline
3-SSIM [49] & Three-component SSIM determines edges, textures, and smooth regions, and associates different weights for each region. \\ \hline G-SSIM [47] & Gradient SSIM compares edge information between distorted and original images. \\ \hline NQM [50] & The Noise Quality Measure models distorted images by using a linear frequency distortion with additive noise injection. \\ \hline VIF [51] & Visual information fidelity is calculated by relation of Distorted Image Information to the Reference Image Information. \\ \hline FSIM [52] & Feature Similarity Index maps features and compares them to the original image. \\ \hline GMSM [53] & Gradient Magnitude Similarity Mean makes use of average pooling on GMS maps. \\ \hline GMSD [53] & Gradient Magnitude Similarity Deviation using pixel-wise GMS captures local image information. The standard deviation of the GMS-map is the final image quality index. \\ \hline UQI [54] & Compute an ’Universal Quality Index’ by loss of correlation, luminance distortion. \\ \hline MAD [55] & The most apparent distortion metric utilizes contrast sensitivity, local luminance, and contrast masking for high-resolution images. For low-resolution images, changes in local statistics between the originals image sub-bands are compared. \\ \hline DSCSI [56] & Directional Statistics based Color Similarity Index can handle chromatic and achromatic distortions. \\ \hline VSI [57] & Visual Saliency-Induced Index uses ’saliency map’ as a quality feature and weighting function to characterize the importance of local regions. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Full-reference-based deterministic metrics for automated image quality assessment. DPR-4: Data Management  
Managing data sets in a centralized cockpit is an obvious requirement of MIOps.