## 1 Introduction  
In domains where the decisions made by a machine learning model are critical, the interpretability of the model becomes a fundamental requirement, since in general, we will want to ensure that the abstraction learned by the model is free of errors [1]. From the medical point of view, the interpretability of machine learning models is especially relevant for regulatory aspects and also to gain confidence in the system by the medical specialist, as it allows us to know on what criteria the model has been based to arrive at a certain prediction. In some cases, it can even allow the medical specialist to find relationships in the data that were previously hidden from the naked eye or not given enough attention or importance [2]. In this context, Shapley values have achieved a high preponderance among the methods for explaining the decisions of a machine learning system. In the review carried out by [PERSON] et al. [3] on the application of explainability techniques in artificial intelligence in the healthcare sector during the last decade, it is worth noting that almost half of the works reviewed used them for this purpose. The Shapley values refer to a method that comes from the theory of cooperative games [4]. This method provides us with a way to distribute the benefits obtained in a cooperative game between the players of that game [5]. Moving this method to the field of machine learning, a prediction from a model could be explained assuming that each feature for a given instance is a \"player\" in a game in which the prediction is the payoff that we can obtain. The Shapley values tell us how to \"fairly\" distribute that payoff among the features, depending on the contribution of each one [1]. In a previous work [6], we explored the use of saliency maps to interpret convolutional neural networks for glaucoma diagnosis, highlighting both their potential and their limitations. In contrast, Shapley values offer a fundamentally different approach to assigning a degree of importance to each feature in a decision. They have the advantage of being based on a solid mathematical foundation derived from game theory, ensuring a series of desirable properties: efficiency, symmetry, null (or dummy) player, and linearity (or additivity) [5; 7]. However, their main disadvantage is that they require evaluating all possible combinations of features in order to calculate these values accurately. When applied in a machine learning context where the number of features is high, this would produce a combinatorial explosion that could be unfeasible in many cases since it would require performing on the order of \(2^{n}\) predictions of our model, being \(n\) the total number of features. In addition, in general, a machine learning model is not prepared to make predictions for any combination of input features, such as a smaller subset of them, so re-training may be required. For this reason, some methods arise that calculate the Shapley values in an approximate way, such as the sampling method of [8], the different methods based on SHAP proposed by [9] or Deep SHAP, specifically intended for Deep Learning systems [9; 10]. The main objective of this work is the use of Shapley values to explain the decisions of convolutional neural networks trained to predict glaucoma. Glaucoma is one of the leading causes of irreversible blindness in the world [11], and it is a pathology that is difficult to address because of the lack of clear and unequivocal biomarkers that warn of its existence, especially in the most incipient stages [12]. Thus, only highly specialized experts can deal with this issue, usually following a holistic approach to diagnosis, using both functional and structural techniques. In this category of structural techniques are those based on images, and more specifically retinal or fundus images, which are the ones used in this research. Figure 1 shows an example of this type of image with the most relevant anatomical structures.