# Explainable Artificial Intelligence: Advancements and Limitations
## 2 Black-Box Nature of CNNs  
First proposed in 1998 for handwritten character recognition [44], CNNs have shown a remarkable performance, especially in vision tasks. Thanks to the ImageNet dataset [14] and efficient GPUs that enabled thousands of computations in parallel, AlexNet by [PERSON] et al. achieved state-of-the-art performance in ImageNet LSVRC-2012 competition [45]. Following AlexNet, several even more performant architectures have been proposed including but not limited to VGG [46], GoogleNet [47], ResNet [48], DenseNet [49], MobileNet [50], EfficientNet [51], and ConvNeXt [52, 53]. With these CNN-based architectures, human-level performance has already been reached and arguably outperformed in various tasks. However, CNNs' black-box nature has been seen as the main obstacle to their further deployment, especially in fields where human life is at stake.  
The so-called black-box nature of CNNs arises from their end-to-end learning process, which contrasts with traditional ML methods such as decision trees [54] and support vector machines (SVMs) [55] where features are manually designed by experts--for example, edge and texture detectors or colour histograms--tailored to specific tasks [56, 57]. As the decision process of traditional ML techniques is built around these predefined, human-understood features, it is relatively easy to follow the reasoning behind their predictions. In contrast, CNNs automatically learn features directly from data through multiple layers of abstraction without any human intervention. This automatic feature extraction helps CNNs achieve state-of-the-art performance, but it also complicates our understanding of what the model is learning at each stage. The learned representations are highly complex and the decision process is non-linear, which leads to an opaque decision-making that obscures the internal workings of the model [58].  
Several methodologies were proposed to tackle these opaque predictions and make deep networks more transparent. In this direction, efforts to enhance the interpretability of CNNs have focused on two primary strategies: visualising model behaviour and aligning learnt high-level features with human-understandable concepts.  
Visualisation techniques aim to illuminate how CNNs build complex representations hierarchically from simpler ones, as early studies revealed that initial layers learn basic features like edges and lines, while later layers capture textures and patterns, culminating in the penultimate layer focusing on object parts--insights contributing to what in [59] is described as _Algorithmic Transparency_[60, 61, 62]. Additionally, pixel attribution methods highlight specific regions of input images that most strongly influence model predictions, providing a clearer understanding of what drives specific decisions [63, 64].  
In contrast, feature-concept alignment methodologies seek to map high-level features learnt by CNNs to human-interpretable concepts [65, 66, 67]. These approaches reconnect abstract representations of deep networks with predefined, semantically meaningful concepts, akin to the handcrafted features used in traditional models. By doing so, they aim to bridge the gap between the opaque, high-dimensional workings of CNNs and human intuition, making the decision-making process more transparent and understandable.  
An ideal XAI methodology would seamlessly address both directions detailed above: it would not only match high-level features automatically extracted during training to human-understandable concepts but also localise these concepts spatially within an examined input sample. Such methodology would provide a more holistic explanation, enabling users to understand not only what the model has learned but also where these concepts are represented in the input. This dual capability would allow for more intuitive and interpretable model explanations, improving trust and transparency in AI systems across various domains.