## 2 Materials and Methods
### Patient-Level Decision  
This part relates to different ways to achieve classification at the patient-level based on the same two categories: \"malignant\", as the positive class, and \"benign\". With this assumption, a patient should be considered \"malignant\" if at least one image is considered to be \"malignant\". Additionally, this part needs to consider the varying number of samples per patient (as a reminder, the number of instances per patient can vary between 2 and 833 images). In order to achieve this, the best combination of the feature extraction method and the classification model from Section 2.3 was used. The classification model provided two types of information for each image: the score was based on prediction probabilities and the decision (i.e., the class that achieved the best probability). In both cases, due to the varying number of images per patient, the information needs to be transformed into constant-size matrices to make a decision for existing and for new patients. At the score level, the structure was composed of patients, images, and scores of classes and transformed into a new matrix of size P \(\times\) C, where P is the number of patients and C is the number of classes. A dynamic threshold was then used to adjust the positive class that maximizes the chosen metric. Multiple strategies can be used to achieve this:  
* Mean--Allows the contribution of each instance on the patient to be retained
* Maximum--Retention of the best confidence prediction as to the trusted one. At the decision level, the structure was composed of structure, combining patients, images, and scores of classes and transformed into a new matrix of size P*C, where P is the number of patients and C is the number of classes. Also, C refers to the probability vector of each decision between 0 and 1. * At Least One--At least one positive decision to consider the input as positive (initial assumption)
* Dynamic--Find a dynamic threshold that minimizes false-positive decisions. A global overview of the processing scheme of this method is presented in Figure 4. In a second stage, a number of multiple instance learning (MIL) concepts are implemented in this part, as they fit the issue at hand: a patient is constitutive of several instances (consider it as a bag) and a positive instance assumes that the patient should be positive. Furthermore, only the patient label is known, and the annotation step of individual images is time-consuming. Such a problem can be set as a pair \(\{X|y\}\), in which \(X=\{X^{1},X^{2},\dots,X^{b}\}\) is a bag containing \(b\) instances and each \(X^{b}\) formulated as follows: \(X^{b}=\{x^{b}_{1},x^{b}_{2},\dots,x^{b}_{n}\}\), in which \(n\) is the number of features and \(y\) is the patient label [38]. Two ideas are developed regarding this context in the paragraphs that follow. In a first stage, \begin{table}
\begin{tabular}{c c l} \hline \hline
**Name** & **Parameter** & **Values** \\ \hline \multirow{2}{*}{CART/RF/ERT} & Maximum depth & [3, \(\infty\)] \\ \cline{2-3}  & Criterion & [Gini, Entropy] \\ \hline \multirow{2}{*}{GB} & Maximum depth & [3, \(\infty\)] \\ \cline{2-3}  & Criterion & [Mean squared error, Mean absolute error] \\ \hline SVM - Linear & C & [0.01, 0.1, 1, 10, 100, 1000] \\ \hline \multirow{2}{*}{SVM - RBF} & C & [0.01, 0.1, 1, 10, 100, 1000] \\ \cline{2-3}  & Gamma & [0.01, 0.1, 1, 10, 100, 1000] \\ \hline \hline \end{tabular}
\end{table} 
Table 3: List of all of the classification models performed in this study and their referring evaluated hyper-parameters. a Single-Instance Learning (SIL) classification is used in which a bag is considered as negative if all of the instances are considered to be negative, and positive if at least one of the instances has a positive label that fits our initial formulation of the patient label. In a second stage, the MI-SVM is an extension of Support Vector Machine (SVM) upon MIL theory and is employed in these due to the results of Section 2.3.