## 3 Methodology: A CRISP-DM Approach to a Real Case
### Data Preparation  
Following the exploratory assessment of the data landscape, the data preparation phase focused on the systematic transformation, cleaning, and integration of heterogeneous data sources to enable the development of machine learning models. In line with the CRISP-DM framework, this phase served to operationalize the findings from the previous stage by implementing data engineering strategies capable of addressing sparsity, inconsistency, and semantic heterogeneity across the datasets. The data corpus was initially fragmented across multiple files and formats, including structured spreadsheets and delimited text files sourced from independent subsystems such as PatOnline and PATS. These sources contained overlapping, complementary, and in some cases, redundant information related to equipment specifications, supplier attributes, user interactions, and importation diagnostics. As a first step, a robust data merging pipeline was developed to consolidate these inputs into a cohesive dataset. This involved schema alignment through the detection of shared column names, resolution of conflicting data types, and elimination of deprecated or contextually irrelevant sheets (e.g., legacy contact directories or inactive administrative entities). The merging logic preserved table-level granularity when necessary to retain meaningful relational context. Data cleaning routines were subsequently applied to improve data quality and prepare features for downstream tasks. Missing value treatment was performed based on field-level sparsity thresholds. Columns with more than 50% null values were excluded from further analysis, while others were retained with appropriate imputation or flagging. Summary statistics and missingness matrices were employed to guide these decisions. This process \begin{table}
\begin{tabular}{l l} \hline \hline
**Attribute** & **Description** \\ \hline Complaint ID & Unique identifier for each informatics complaint \\ \hline Timestamp & Date and time when the complaint was logged \\ \hline User ID & Identifier of the user who logged the complaint \\ \hline Interaction Log & Sequence of actions or events from user interaction logs \\ \hline Software/System & Name of the software or system involved \\ \hline Complaint Text & Unstructured natural language description of the issue \\ \hline Category & Initial categorization or classification of the complaint \\ \hline Severity & Assessed impact or urgency level of the issue \\ \hline Status & Current resolution status (e.g., open, closed) \\ \hline Resolution & Outcome or actions taken to resolve the complaint \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Overview of attributes in the informatics complaint dataset. led to a significant reduction in dimensionality while preserving the features most relevant to the predictive modeling and semantic interpretation objectives. To address the substantial lexical variability observed in free-text fields, particularly those containing equipment names, part numbers, and error messages, text normalization techniques were introduced. These included case standardization, removal of extraneous characters, and the application of fuzzy string-matching algorithms to harmonize semantically equivalent but syntactically divergent entries. For example, variants such as \"HP ProBook 430 GS\" and \"HP Probook 430G5\" were reconciled using custom modules based on token-level similarity and domain-specific regular expressions. This step was essential to ensure consistency in entity recognition tasks and improve the reliability of subsequent machine learning models. Particular attention was given to the preprocessing of the MensagemErroImportacao field, which served as a critical input for both classification and clustering models. This unstructured field was transformed into numerical representations using sentence-level embeddings.