# Embrace the opaque: A sociology of the xAI community

Nicolas Berkouk, Mehdi Arfaoui, Romain Pialat, Pierre Francois, Laurence Barry

<figure><img src="image_1.png" /></figure>

## To cite this version:

Nicolas Berkouk, Mehdi Arfaoui, Romain Pialat, Pierre Francois, Laurence Barry. Embrace the opaque: A sociology of the xAI community. 2025. hal-05320301

HAL Id: hal-05320301

https://hal.science/hal-05320301v1

Preprint submitted on 17 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Working paper

# Embrace the opaque

## A sociology of the xAI community

October 2025

linc.cnil.fr

Nicolas Berkouk<fnref n="1" />, CNIL, France

Mehdi Arfaoui<fnref n="1" />, CNIL/EHESS, France

Romain Pialat<fnref n="1" />, CNIL, France

Pierre François, Sciences Po./CNRS, France

Laurence Barry, CREST, France

<footnote n="1">¹ This research was conducted as part of the authors' institution research and experimentation missions. Consequently, the point of views expressed in this article solely are the authors' one, and do not engage CNIL's opinion on the matter.</footnote>
# Summary

<table><tr><td>ABSTRACT</td><td>3</td></tr><tr><td>KEYWORDS</td><td>3</td></tr><tr><td>INTRODUCTION</td><td>4</td></tr><tr><td>PREVIOUS WORK</td><td>4</td></tr><tr><td>XAI, A PRE-PARADIGMATIC FIELD?</td><td>5</td></tr><tr><td>METHODOLOGY</td><td>6</td></tr><tr><td>QUANTITATIVE ANALYSIS OF THE XAI LITERATURE</td><td>6</td></tr><tr><td>QUALITATIVE ANALYSIS OF THE XAI LITERATURE</td><td>7</td></tr><tr><td>DEEP LEARNING, OR THE RENEWAL OF THE EXPLAINABILITY PROBLEM</td><td>8</td></tr><tr><td>A PROLIFIC, YET UNFORMALIZED FIELD OF RESEARCH</td><td>10</td></tr><tr><td>A PROLIFERATION OF HETEROGENEOUS TECHNIQUES</td><td>11</td></tr><tr><td>...WITH NO COMMON CATEGORIES</td><td>12</td></tr><tr><td>...NOR STRUCTURED SCIENTIFIC DEBATE</td><td>14</td></tr><tr><td>THE SOCIAL FUNCTION OF XAI</td><td>16</td></tr><tr><td>ANSWERING A GROWING LIST OF STAKEHOLDERS AND DESIDERATA</td><td>16</td></tr><tr><td>A SUBORDINATE RELATIONSHIP</td><td>19</td></tr><tr><td>CONCLUSION: THE XAI COMMUNITY AS THE FRONTIER OF AI</td><td>20</td></tr><tr><td>REFERENCES</td><td>22</td></tr></table>
**Abstract**

Ten years after DARPA's initial call to foster the field of Explainable AI (xAI), its research productions proliferate but are commonly accepted to lack formalism and robustness, leading to both internal and external criticisms. This situation is becoming a political problem as AI regulation is expanding and entering into force.

In order to understand why and how the xAI field faces these limitations, the core of this article will focus on characterizing its productions and explaining its specific development. We propose to adopt an empirical approach using a combination of quantitative and qualitative methods based on a corpus of over 16,000 xAI publications.

We defend the hypothesis that the singular properties of xAI are due to the position occupied by this field in relation to connectionist AI. We argue that understanding the function and organization of xAI requires considering its relationship with the field of deep learning and the "connectionist" AI agenda in general. We posit that xAI constitutes a boundary in a topographical sense - a space that is contiguous with AI but does not merge with it - and in a functional sense - xAI protects deep learning-based AI by responding to stakeholders' criticism and desiderata. To us, the epistemological characteristics and limitations of xAI derive from this subordinate position, which requires it to respond to external demands, preventing it from engaging in the traditional processes of self-regulation proper to scientific disciplines - they are, in other words, the consequence of xAI's heteronomy.

**Keywords**

Explainable AI, Sociology of AI, Heteronomy, Boundary Space, Sociotechnical Analysis, AI regulation.
# Introduction

The rapid ascent of deep learning has marked a pivotal moment in the history of artificial intelligence, allowing unprecedented capabilities across numerous domains. Yet, this progress comes with a significant shortcoming: the “black box” nature of the very algorithms driving this revolution. As these “connectionist systems” become increasingly integrated into high-stakes decision-making processes, their inherent opacity poses a critical challenge. Consequently, a dedicated field of research, now widely known as *Explainable AI* (xAI), has emerged and expanded dramatically over the last decade, aiming to render these complex systems intelligible to their developers, users, and regulators.

This paper offers an analysis of the xAI field of study. First, we will argue that while the issue of algorithm explainability is not new, the rise of deep learning presents a fundamental renewal of this issue. Unlike earlier “symbolic AI” systems, where the logic was explicitly programmed, connectionist systems develop operational principles that are not designed or fully understood even by their creators, thus renewing both the nature and the urgency of the explainability challenge.

Second, we will examine the scientific field that has organized itself to meet this challenge. Our analysis of over 16,000 research papers reveals a field characterized by a prolific explosion of proposed techniques but a striking lack of scientific consensus. We observe a proliferation of heterogeneous and often juxtaposed methods without a stable conceptual framework, shared evaluation standards, or robust debate over foundational limitations. This has resulted in a body of research that produces an important amount of solutions but struggles to formalize and structure itself as a coherent scientific field of study.

This observation leads us to adopt a sociological perspective on the field. We argue that these scientific limitations are not merely signs of a young, immature field of research but are linked to the social function that xAI research has come to occupy. We demonstrate that xAI primarily serves as a protective boundary for the core development of deep learning, absorbing and responding to external criticism and regulatory pressure. This subordinate role, we argue, shapes the very nature of its scientific output, prioritizing “reactive” justification over the development of a self-critical research paradigm.

## Previous work

Previous literature in social science has also explored the limits of this field of research. As Explainable AI has grown in prominence, it has increasingly attracted critical attention in sociology, HCI, philosophy or medias studies. This scholarship can be broadly divided into two streams. The first engages in a form of constructive critique, intervening directly in computer science debates to argue that xAI research often fails to account for the rich social contexts of its application (Ehsan et al., 2021). These scholars aim to improve xAI by proposing new conceptualizations (Aizenberg and Van Den Hoven, 2020; Cao and Yousefzadeh, 2023; Felzmann et al., 2019), mobilizing novel methods to explore user needs (Munk et al., 2022),
or advocating for explanations grounded in practical use cases rather than abstract technical
properties (Miller, 2019).

The second stream, more aligned with our own approach, does not seek to produce better
explanations but rather to study the social world that produces them. This “sociology of ML
explainability”, as articulated by Borch and Hee Min (2022), involves “examining all aspects of
the practical work that goes into explaining the actionable predictions of ML systems” (p. 3).
Much of this valuable work has focused on the implementation and consequences of xAI in
specific domains, such as finance (Borch and Hee Min, 2022) and policing (Hälterlein, 2021),
its effect on user trust and priorities (Kieslich et al., 2022), or its role in the broader political
economy of artificial intelligence. A particularly salient theme is the critique that xAI can serve
to depoliticize fundamental questions about AI's social impact by framing them as narrow
technical problems to be solved (Freiesleben and König, 2023; Knowles, 2020).

xAI, a pre-paradigmatic field?

While our work builds upon this literature, it diverges in its analytical focus. Rather than
examining a particular sector, the uses or consequences of explainability, we analyze the
scientific structure of the xAI research community itself. Few studies have approached the
field's internal limitations (its heterogeneous methods and lack of formal consensus) as the
central object of inquiry. Our contribution is to connect these observable scientific
characteristics to their social context of production.

To analyze this context and understand xAI structural limitations, we're inspired by Kuhnian description of a "normal science" as a field of research grounded in a "paradigm": a shared framework of theories, principles, instruments, and exemplary problems that structures the work of the field. It is this paradigm that allows for the definition of relevant questions, the validation of methods, and the coherent accumulation of knowledge.

While we do not adopt the entirety of Kuhn's thesis, his concept of a paradigm provides a powerful heuristic for interpreting the specific condition of xAI. We suggest that xAI exemplifies a research domain that, despite its prolific output, so far struggles to reach a paradigmatic stage. In the remainder of this paper, we will see that this absence of a shared paradigm nonetheless has direct and observable consequences but can be understood by looking at the fields' internal logics and external dependencies. We argue that the "pre-paradigmatic" state of xAI is a direct consequence of its fundamental relationship of dependence on the field of connectionist AI as well as the social function it performs. The sociological perspective therefore is essential to understand not only what xAI produces, but why it produces in the way that it does.
# Methodology

## Quantitative analysis of the xAI literature

### SemanticScholar corpus

In order to study the research community and publication on xAI, we established a database constituted of 16000+ research papers related to this topic. For that we used the *SemanticScholar* API<fnref n="2" /> that allows regular expression search over titles and abstract of their own database. Papers have a unique id, and comes with several well-structured fields, such as: authors, citation list, year of publication or field of study, etc.

The *SemanticScholar* API matches regular expressions with the text contained in the title and abstract of papers in their database. Nevertheless, this matching process is blind to stop words and link words such as “or”, “of”, “and”, which led to getting a significant amount of outlier papers. Therefore, our querying process was twofold:

1. Query the Semantic Scholar API with a given regular expression query to obtain a corpus;

2. Apply a second time the *query* to the title and abstract of papers in corpus, but this time in *Python* using the *re* package. This is done in order to get rid of the aforementioned outlier papers.

We proceeded iteratively in order to design the final query. We started with an initial short query containing relevant words that were present in most famous xAI papers. In order to complete it, we then used ChatGPT in order to obtain a list of additional keywords. This is relevant since auto-regressive Large Language Models (LLM) such as ChatGPT are designed specifically in order to infer correlations between semantically similar expressions. We then iteratively added some of these keywords, by qualitatively cross-checking the top 100 most cited papers that we would obtain. Merging the best of those two key-words lists, we finally got the following query:

```
query = "Interpretable Model" | "Explainable Model" | "Model Interpretability" | "Models Interpretability" | "Model Explanations" | "Models Explanations" | "Explanations of Models" | "Explaining Models" | "Explainable Artificial Intelligence" | "xAI" | "Explainable AI" | "Interpretable AI" | "Interpretable Artificial Intelligence" | "Explanations from Deep Networks"
```

It is to be noted that our vocabulary both include terms relating to "explanations" and "interpretations". We made this methodological choice because it appeared to us that, in the literature, while some authors give different meanings to these terms, some others use them interchangeably, as exemplified by the title of this widely cited survey (Linardatos et al., 2020).

<figure><img src="image_3.png" /></figure>

<footnote n="2">² *SemanticScholar* is an AI powered research tool for scientific literature, owned by the Allen Institute for AI, a non-profit research institute, gathering several sources of scientific publications. The documentation regarding the API is accessible here: https://api.semanticscholar.org/api-docs/.</footnote>
Studying precisely the way that each author uses these terms is outside of the scope of this
article, but will be an interesting line of future research.

We have started our temporal scope to the year 1970, since no relevant papers were present
in the database before this date, and obtained finally 16 413 papers, on the 30th of December
2024.

Quantitative analysis.

This database allowed us to perform a statistical description of the xAI publication and
community. In particular, we used *SemanticScholar* machine learned attribution of the field
of study, because it had the advantage of being computed in the same way across all the
corpus.

We then used the citations and references data for each paper in order to build a co-citation
network. We mostly used it in order to navigate within the corpus and formulate hypothesis.

Qualitative analysis of the xAI literature

Qualitative coding of the 100 most cited papers in our database

We carried out a qualitative analysis of the 100 most cited publications in our database, to
categorize the type of publication (review, user or case study, technical proposal, conceptual
framework) and to define the authors' motivation for producing this publication or for
defending the production of work in xAI in general. The authors' motivations are most often
expressed at the start of the publication - in the sections that set the context for the
publication and justify the need for research in this field - and at the end of the publication -
in the sections generally intended to open up perspectives and place the publication's
contributions in relation with concrete cases. In the publications most explicitly designed to
meet these desiderata, the motivations are clearly stated in the abstract or even the title of
the publication. We thus coded each paper by indicating one or more motivations/purposes
(trust, performance, transparency, etc.) associated with one or more citations, as well as the
stakeholders considered in each paper to be the recipients or beneficiaries of current
publications in xAI (users, regulators, developers, deployers, affected parties). To do so, we
based ourselves on the typology proposed in a review by Langer et al. (2021).

Snowballing approach to build critical publication review

To identify related works, we attempted to systematically analyze critiques of xAI. To do this,
we used our coding of the 100 most cited publications in our database (see above) to identify
an initial set of critiques. We then delved into the bibliographies of this first batch to identify
other critique publications cited within them. We then completed our list using academic
search engines (SemanticScholar, GoogleScholar, OpenAlex, etc.) to identify publications using
the same critical lexicon as the first two batches. Finally, we shared this list with colleagues
during academic meetings to obtain any papers that might still be missing. This resulted in
under forty critical academic publications that we deemed relevant and representative for our
study.
*Most cited authors' bibliographical analysis.*

We also wanted to learn more about the academic trajectory of authors publishing in xAI, and in particular their position relatively to the field of AI (deep learning, machine learning). We determined a sample of 100 authors by selecting the last author in the order listed on the publication team – being often the most senior author of the writing team– of each of the 100 most cited publications in our database. We then used *GoogleScholar* to explore their bibliography, notably to find out whether the authors had published papers in deep learning or machine learning before entering the field of xAI. We then categorized each author as having been “previously in deep learning” (Yes/No), and “previously in machine learning” (Yes/No). Since 5 of the 100 authors did not have a page on *GoogleScholar*, the data processing was carried out on 95 authors.

At last, when necessary, we also conducted a mathematical analysis of the content of the papers, in order for instance to understand the content of the description of certain limits of xAI techniques (such as (Bilodeau et al., 2024)).

Deep learning, or the renewal of the Explainability problem

For decades, most of AI systems deployed for applications relied on symbolist techniques, whose logic consisted of implementing in the computation of the algorithm a theory on the environment one tries to act on (Cardon et al., 2018). For example, in order to distinguish images of cats from images of giraffes, one could implement the theory ‘giraffes have longer necks than cats’ by analysing the relative neck size of the animal pictured in each image and deciding that it is a giraffe above a certain threshold and a cat below. Symbolic AI tools are therefore, at least potentially, perfectly transparent, since the ‘intelligence’ of the machine is merely that of its programmer. The shift that occurred in the mid-2010s saw this symbolist logic replaced by a connectionist approach, which quickly proved its effectiveness. Connectionism attempts to metaphorically mimic brain cognition in order to incorporate ‘intelligence’ into a machine through a learning process. In this paradigm, ‘intelligence’ emerges from trial and error, by adapting the parameters of the algorithm to best fit the examples provided in the training set: for instance, one would train a neural network based on a huge set of pictures already labelled “cat” or ‘giraffe’, and one would algorithmically adjust the network’s parameters when it outputs the wrong label.

While making sense of symbolist machines boils down to tracing the intention of its programmers during its development (Merigoux et al., 2021, 2024), it is by construction of connectionist systems that the way they operate to solve a task does not originate from a theory of their developers on it. Consequently, their computations do not make sense a priori, and any attempt to construct a meaning out of their inner functioning is necessarily exterior and ex post to their development. In particular, the learning process of neural networks has, so far, no mathematical description, even from the fields of statistics or optimisation. When such tentative work exists, it often makes unrealistic assumptions, such as a network with an infinitely wide single hidden layer, or no non-linearities (Chizat et al., 2024; Chizat and Bach,
2020). In other words, none of the actors in connectionist AI, neither its users nor (more surprisingly) its most knowledgeable designers, have a clear understanding of the operations performed by neural networks.

Under these circumstances, the opacity of ML models quickly became a problem – to the point that over the last ten years, a series of proposals and practices grouped under the term ‘Explainable AI’ (or xAI) has developed, aiming to explain how connectionist tools come to make predictions, propose classifications, or make decisions. The growth of this field of research has been spectacular, going from less than 100 papers per year in 2015 to about 5,500 papers in 2024. Before 2015, research papers tend to use mostly the lexical field related to “interpretation” rather than the one related to “explanation” (see Figure 1), whereas the vocabulary related to the concept of ‘explanation’ has become clearly predominant since 2018.

<figure><img src="image_4.png" /></figure>

Number of papers of our corpus containing words from our query related to “Explainability”, “Interpretability”, or both, per year of publication.

<figure><img src="image_5.png" /></figure>

Number of papers of our corpus per principal field of study (as detected by SemanticScholar), per year of publication.

<caption>Figure 1 - The xAI corpus across time</caption>

This change of vocabulary as well as this spectacular growth in the xAI reference responds to the injunctions addressed by institutions to AI players from the mid-2010s, when AI began to be applied in many fields of application. Among these injunctions, those addressed by DARPA<fnref n="3" /> played a fundamental role, - as shown by the stabilization of the lexicon around xAI, a term chosen by DARPA and ignored, at least initially, by the EU regulation (GDPR). The DARPA consequently launched in 2015 its Explainable AI program (Gunning et al., 2021), that aimed “to create a suite of machine learning techniques that: (i) Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and (ii) Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners”. It appears clear from the program’s cover, which depicts a neural network (see Figure 2), that machine learning refers here to deep learning. xAI was thus substantially encouraged by a call sent by DARPA’s engineers to provide technical

<footnote n="3">³ The Defense Advanced Research Projects Agency (DARPA) is a research and development agency of the United States Department of Defense responsible for the development of emerging technologies for use by the military. It is one of the main research funding agencies of the USA.</footnote>
means to understand *why deep learning* based systems work. As Gunning et al. put it in the
retrospective paper regarding this program (Gunning et al., 2021): “*the year 2015 was an
inflection point in the need for xAI. Data analytics and machine learning had just experienced
a decade of rapid progress. The deep learning revolution had just begun, following the
breakthrough ImageNet demonstration in 2012.*” Since then, the problem itself has become
more of a political issue as AI systems are becoming widely publicly deployed, and, as the
legal scope applied to AI-based systems is expanding (from the GDPR to the EU AI Act),
regulators will have to interpret and put into practice the debated ‘right to an explanation’
developed in these texts (Edwards and Veale, 2017).

<figure><img src="image_6.png" /><figcaption>Figure 2 - DARPA Explainable AI program</figcaption></figure>

A prolific, yet unformalized field of research

On the surface, xAI appears to be a dynamic and thriving field, unified by the common goal of
making those opaque systems intelligible. Despite this decade of efforts and prolific output, a
closer examination reveals a research landscape that lacks scientific consensus and a stable
theoretical framework. Behind the sheer volume of publications lies a significant degree of
fragmentation, where the rapid accumulation of methods has not translated into the
cumulative, structured knowledge characteristic of a mature scientific field of research.

To demonstrate this, our analysis in this section will proceed from three key observations: the striking heterogeneity of the techniques proposed, their mere juxtaposition without a shared formalization, and the notable lack of structured scientific debate surrounding their limitations (Deck et al., 2023). Drawing inspiration from Thomas Kuhn's work on scientific development, we will argue that these characteristics are symptomatic of a "preparadigmatic" state. These three key observations will encourage us to interrogate the social context in which xAI evolves in the last part of the paper.
A proliferation of heterogeneous techniques

The first observation draws on Kuhn's old idea (Kuhn, 1962; Wray, 2007) that a scientific field
is based on a more or less stable list of relevant questions and conceptual and methodological
tools for answering them. If we look at the content of the xAI proposals, it appears that a
significant proportion of the core publications propose algorithmic techniques for computing
explanations of the outputs of deep learning models. Our qualitative analysis of the 100 most
cited publications in our database shows that 30.5% consist of technical proposals (reviews,
which we explore in the next section, account for 45.7% of the most cited publications). These
technical proposals come in a very heterogeneous fashion. In particular, some dimensions of
this heterogeneity remain invisible when analysing xAI methods only with a technical lens.

Let us illustrate this stance on two examples in image classification. The first and most famous
technique in this setting is GradCAM (Selvaraju et al., 2020). For each image, it points out the
pixels whose smallest variation implies the highest variation on the network's prediction, see
Figure 3. Importantly, it does not propose a general criterion for all images of what the model
has learned to recognize as a "cat" or a "dog". Instead, it provides a criterion that is applied
on an image-by-image basis. In addition, GradCAM provides a measure of the importance, in
a correlative dimension, of an image area for class assignment. This technique does not
propose a causal scheme between explanatory variables for the assignment of the "cat" class
to an image. We have therefore identified two constituent characteristics of GradCAM results:
their *locality* and their *correlative* dimension.

Another family of xAI techniques are *post hoc* examples. Broadly speaking, this consists, given
an input x, in finding a few elements in the training set that activate the network in a similar
way to x. This is illustrated in Figure 4, where a convolutional neural network has been trained
to detect the digit present in an image. On the left, we see an image of digit "7" and digit "6"
that are correctly classified by the model, and for each image, three examples that activate
the network similarly. Same thing on the right-hand side, but this time with images that trick
the algorithm. We can see, for example, that the "8" image, classified as a "3", activates the
network in the same way as "3" images whose bottom part closes. Like GradCAM, this method
offers a local explanation, in that it is specific to each input. On the other hand, it does not
correlate the network's prediction with a set of variables specific to the input data. It searches
across known data for which there exists a notion of 'ground truth', for those activating the
network most similarly to x. It is this similarity that constitutes the explanation, which could
be described as operating by *analogy*.
<figure><img src="image_7.png" /><figcaption>Figure 3 - GradCAM explanation for the class "cat" (Selvaraju et al., 2020).</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 4 - Post hoc example explanation (Kenny et al., 2021).</figcaption></figure>

These two simple cases exemplify that techniques from xAI come in a diverse and heterogeneous manner. Characterizing precisely this heterogeneity is out of the scope of this paper and will be part of future work. We can at least see that some dimensions of this heterogeneity, such as the local versus global nature of explanations, are well captured through a technical lens and identified in the literature, see for instance Figure 6. This exclusively technical perspective characterizing the field of xAI prevents researchers from addressing the ontology of its explanations, i.e., what actually *makes* an explanation. As a matter of fact, the distinction between analogy-based techniques and correlation-based techniques is not formally brought out in the literature, illustrating the distance taken from fundamental considerations on explanations.

## ...with no common categories

Facing this proliferation of recipes, a certain kind of articles aim at organizing this disorganized and illegible production: the literature review. We show below that these reviews are developing in unusual proportions and, moreover, that they are not organized according to consensual criteria.

The reviews constitute 13 % of the entire corpus, and span 45,7% of the top 100 most cited xAI papers that we qualitatively analyzed. In order to understand if this situation is common for such a field, or whether it is a characteristic aspect of the research in xAI, we performed a comparison with other fields within computer science, or at the intersection of computer sciences and applications. To do so, we did not use the *Semantic Scholar API*, because we would have had to find a well-defined query for a list of other fields, which is time and computation wise very costly. Instead, we used another scholar database called *OpenAlex* (Priem et al., 2022), that had performed a machine learned “topic” detection based on the title, abstract, citations and journal of a paper (Research)<fnref n="4" />.

<figure><img src="image_9.png" /></figure>

<footnote n="4">⁴ We shall take the preliminary methodological precautions here. The corpus associate to *OpenAlex* topic "Explainable AI (xAI)" differs slightly from the one we have constituted from querying *SemanticScholar*. Nevertheless, we qualitatively observed that it had about the same size, and that the 100 most cited papers were</footnote>
<figure><img src="image_10.png" /><figcaption>Figure 5 - Percentage of reviews in our sample</figcaption></figure>

In Figure 5, we observe that the topic “Explainable AI” has at least five time more reviews in its literature than other topics of computer sciences that are linked to machine learning and/or applications. This proliferation of reviews is an answer by the xAI community to the heterogeneity of xAI techniques, and to the absence of scientific consensus on what makes a good explanation. Indeed, the most cited review papers describe the contribution of their work as to propose a taxonomy of xAI techniques along technically framed categories: “The main contribution of this study is a taxonomy of the existing machine learning interpretability methods that allows for a multi-perspective comparison among them” (Linardatos et al., 2020). One key aspect of this organization work, is that it does not question the heterogeneity of xAI techniques, but take it as a constitutive property of the domain, as exemplified by the following final sentence of the conclusion of the review (Gilpin et al., 2018), which recognized that taxonomies are not overcoming the heterogeneity aspect of xAI techniques, but leave it for future work : “We also highlight the lack of standardized evaluation metrics, and propose research crossing areas of the taxonomy as future research directions”.

sharing great similarity in both corpuses. Secondly, the way each of these two tools classify the type of papers differs, as shown in (Haupka et al., 2024). Therefore, the estimation obtained in *OpenAlex* for the percentage of reviews amongst papers whose topic is “Explainable AI” is lower than the one we obtained thanks to Semantic Scholar and our query. But it is still methodologically meaningful to compare this percentage with the one from other topics of *OpenAlex*.
<figure><img src="image_11.png" /><figcaption>Figure 6 - Two examples of taxonomies from xAI review papers.</figcaption></figure>

One last and important aspect of xAI review papers, is that the technical categories into which they propose to classify every xAI techniques, are themselves unstable across such review papers. The xAI authors indeed produce maps without any consensus on the metric system, on the definition of the cardinal points or on the basic principles of geodesy used to identify each other's positions (see Figure 6). To highlight the qualitative difference between the taxonomies proposed in (Arrieta et al., 2020) and (Linardatos et al., 2020), which are both highly cited review papers, let us for instance observe that the category “Post hoc explanations” is a final node of the taxonomy in (Arrieta et al., 2020), while being the very first level of distinction of (Linardatos et al., 2020). Therefore, this concept plays a role of high importance to organize xAI techniques for (Linardatos et al., 2020), while it is only pertinent after several ramifications for (Arrieta et al., 2020).

## ...nor structured scientific debate

The lack of stabilisation in the body of concepts or methods does not necessarily mean that xAI is not in its way to become a paradigmatic field of study: it may be a symptom of its youth and pre-paradigmatic state. In the meantime, within new fields of knowledge, one of the roles of controversy is precisely to gradually build consensus and bring out dividing lines and contrasting options (Jasanoff, 2019). The field of xAI is precisely characterised by its lack of critical engagement with the field's formal contradictions.

This lack of critical engagement can be illustrated with an analogy between xAI and fairness in machine learning. It has been shown at a mathematical level (Bilodeau et al., 2024), that an explanation technique satisfying commonly agreed formalized properties of what an
explanation should be and which are satisfied for instance by some of the most used xAI
techniques such as SHAP (Lundberg and Lee, 2017) and Integrated Gradients (Sundararajan et
al., 2017), can provably fail to improve on random guessing for inferring model behavior. This
result is in some sense, very similar to the now famous Impossibility Theorem of Fairness,
which states that a risk attribution system cannot satisfy in general simultaneously the three
most common mathematical formalization of “being fair”. Despite the existence of theoretical
limitations, such as the impossibility results presented in (Bilodeau et al., 2024), the xAI
community exhibits limited engagement with these findings. This contrasts sharply with the
algorithmic fairness field, where the influential impossibility theorem by (Kleinberg et al.,
2016) “has shaped ongoing discussion on the proper conditions for fairness in prediction”
(Sahlgren, 2024). On the contrary, the limited impact of (Bilodeau et al., 2024) (cited only 85
times according to SemanticScholar as of 26/08/2025, two years after publication on the
arXiv), compared to the widespread citation of SHAP’s paper (over 23k citations) (Lundberg
and Lee, 2017) suggests a lack of critical engagement with fundamental challenges within the
xAI field.

Despite the existence of a wide range of heterogeneous methods and proven impossibility
theorems, the xAI research community appears largely immune to significant controversies.
Weaknesses, sometimes very significant ones, are sometimes clearly identified. But they do
not prevent proposals from proliferating, which no metrics or formalism organises. Actually,
the very authors of this literature underline the field's current lack of formalism. As set in
(Guidotti et al., 2018): "It is evident that the research activity in this field is not providing yet a
sufficient level of importance in the study of a general and common formalism for defining an
explanation". The field would miss clear definitions, research questions and goals (Linardatos
et al., 2020; Lipton, 2016), making methodological rigor difficult to achieve (Haufe et al., 2024).
This ambiguity would also hamper the development of reliable self-assessment tools (Adadi
and Berrada, 2018), a crucial step towards establishing xAI as a truly scientific field, according
to the authors (Freiesleben and König, 2023; Weber et al., 2024).

In other words, because they are rarely subjected to academic dispute and scientific
controversy, many xAI techniques are designed with a critical lack of formalism. This
unconstrained development results in limited robustness, making those techniques
susceptible to manipulation and technical flaws. (Bove et al., 2024; Dombrowski et al., 2019;
Slack et al., 2020). For example, in the setting of local explanation in image classification, small
perturbations in the input image can lead to a drastic change in the explanation computed
with xAI techniques. These perturbations can be made adversarial, by adding a small
perturbation to an input image, to obtain almost any explanation (see an illustration of our
re-implementation Figure 7). Furthermore, in (Slack et al., 2020), it is shown that
manipulations of explanations can easily be used to hide racist biases in classifiers. It is
therefore not the rigour shared by the various proposals that explains the absence of
controversy. If we want to understand how they can coexist without polemics or evaluation
of their respective qualities, we must identify the role they play in relation to the techniques
they are intended to clarify.
<figure><img src="image_12.png" /><figcaption>Figure 7 - Example of adversarial perturbation</figcaption></figure>

# The social function of xAI

The conclusion that Explainable AI allegedly remains in a pre-paradigmatic state, characterized by heterogeneous methods and a lack of formalism, begs a trivial question: why? A conventional explanation might suggest that xAI is simply a young field still in its formative stages, or that the “black box” problem is so intrinsically difficult that scientific coherence is not yet possible. While these factors may play a role, they are insufficient to explain the persistent lack of structured debate and the field's seeming immunity to foundational critiques.

To find a more comprehensive answer, we contend that it is necessary to shift our analytical lens from a purely internal critique of its scientific output to a sociological examination of the field itself. Understanding why xAI is maintained in this state requires us to investigate its internal workings, the motivations that drive its community, and its complex interactions with the outside world. More specifically, we must scrutinize its relationship with the dominant field it is *de facto* related to: deep learning.

This section will therefore argue that the pre-paradigmatic condition of xAI is not an accidental flaw, but rather a direct consequence of its primary social function. We will posit that the field operates as a crucial interface between the technically-focused world of connectionist AI development and the escalating demands for accountability from society, regulators, and users. By analyzing the xAI community's composition, its stated objectives, and its subordinate position relative to core connectionist AI research, we will demonstrate how this function shapes its scientific production, favoring a continuous output of solutions over the establishment of a stable, self-critical field of research.

## Answering a growing list of stakeholders and desiderata

However powerful DARPA's desiderata and fundings may be, they alone cannot explain why researchers have continued to work for ten years to explain the operations at work in deep learning-based tools. To understand the motivations of these researchers, we set out to analyze their motivations as expressed directly through their publications. Despite the technical objectives at the very origin of xAI, since 2016, we note that xAI publications have also focused on nontechnical desiderata likely to justify the existence of this field. We show that since 2016, the list of these desiderata has continued to grow, as has that of the

<figure><img src="image_13.png" /></figure>
stakeholders who embody them. Far from being a field of study strictly aimed at AI professionals, xAI strives to embody a space for responding to the criticisms and demands for accountability addressed to deep learning-based AI.

While the birth of xAI was strongly determined by a technical motivation, from the very beginnings of this field, the motivations expressed by researchers were never strictly technical. The 2015 DARPA program itself suggests the need to provide a technical response to the expectations of operators likely to be making important decisions, by considering the fact that users must “trust” the system beyond the simple technical understanding of tools by the developer. Users’ trust appears as a fundamental criterion for the successful development of connectionist AI, making xAI a sine qua non condition of AI’s development: “A research field holds substantial promise for improving trust and transparency of AI-based systems. It is recognized as the sine qua non for AI to continue making steady progress without disruption” (Adadi and Berrada, 2018). Our coding of the most cited publications in xAI indeed highlights “users” as an almost systematically considered stakeholder by the authors (97 publications out of 100 mention the “user” as a stakeholder), while “trust” is the most frequently mentioned desiderata to be answered (46 publications out of 100), often associated with “trustworthiness” (15 publications out of 100). Answering the issue of trust and trustworthiness is also often related to the objective of increasing the “acceptance” of AI systems by the users (14 publications out of 100). As shown in Figure 8, the yearly proportion of xAI publications containing the term “trust” in the abstract then increases from less than 2% in 2015 to over 16% in 2024.

<figure><img src="image_14.png" /><figcaption>Figure 8 - Proportion of xAI publications in explainable AI containing the term “trust” in our corpus</figcaption></figure>

In the years following DARPA's program launch, the xAI literature demonstrates a diversification of the desiderata and stakeholders to which xAI authors aim to respond. This broadening scope of xAI's objectives aligns with the escalating public scrutiny of these new AI systems. At the same period, the European GDPR was being discussed - yet the discussions and the text did not refer explicitly to the concept of “explanation” in its core: its Article 15.1.h mentions, regarding to automatic decision-making, the right for data subjects to access “meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject”. Later on (see for example (Edwards and Veale, 2017)), substantial debates developed among scholars in order to know whether
this article should be interpreted as a right to an explanation. This question has just been clarified in the prejudicial decision n° C-203/22 of February 2025 by the Court of Justice of the European Union, but as early as 2017, papers explicitly mention European law as a "huge motivation for our approach are rising legal and privacy aspects. The new European General Data Protection Regulation (GDPR and ISO/IEC 27001) entering into force on May 25th 2018, will make black-box approaches difficult to use in business" (Holzinger et al., 2017). The recent adoption of EU's AI Act, as well as the opinion of the European Data Protection Board about the application of GDPR to AI models (Board 2024), may continue to make explainability a growing central legal concern. In the meantime, media coverage of incidents and hazards associated kept increasing - the OECD notes that AI incidents and hazards as reported by reputable international media have increased between 2016 and 2024 (and dramatically since mid-2023), going from dozens reports every three months to more than 600 in the beginning of 2024 (Observatory, 2024).

This heightened public attention has fueled a strong demand for transparency from AI systems, significantly influencing the evolving goals of the xAI research field, contributing to a multiplication of expectations and interlocutors to whom AI professionals are accountable. The incorporation by xAI researchers of desiderata coming from the outside of the field is finely documented and categorized in a review by Langer et al. and entitled "What do we want from xAI?" (2021). We used this categorization to code the first 100 most cited papers in our database, showing in an essentially stakeholder-focused (or heteronomous) literature (see Figure 9).

<table><caption>Figure 9 - Stakeholders and desiderata mentions in the first 100 most cited papers of our database</caption><thead><tr><th></th><th>Times expressed</th></tr></thead><tbody><tr><td>Stakeholders</td><td></td></tr><tr><td>User</td><td>97</td></tr><tr><td>Developer</td><td>44</td></tr><tr><td>Deployer</td><td>11</td></tr><tr><td>Regulator</td><td>37</td></tr><tr><td>Affected</td><td>14</td></tr><tr><td>Desiderata (most expressed)</td><td>Times expressed</td></tr><tr><td>Trust</td><td>46</td></tr><tr><td>Usability</td><td>34</td></tr><tr><td>Transparency</td><td>33</td></tr><tr><td>Safety</td><td>27</td></tr><tr><td>Accountability</td><td>27</td></tr><tr><td>Legal compliance</td><td>22</td></tr><tr><td>Performance</td><td>20</td></tr><tr><td>Accuracy</td><td>20</td></tr><tr><td>Fairness</td><td>18</td></tr><tr><td>Trustworthiness</td><td>15</td></tr><tr><td>Ethics</td><td>14</td></tr><tr><td>Acceptance</td><td>14</td></tr></tbody></table>

Our coding shows that, as the xAI field expands, the list of stakeholders expands and diversifies. AI system "developers" appear as the second most cited stakeholder group (considered in 44 publications out of 100), ahead of regulators (37 publications out of 100), affected parties (14) and system deployers (11). The presence of all these stakeholders in the
most cited literature shows a concern for a multitude of desiderata: transparency (33 publications out of 100), safety (27), accountability (27) and legal compliance (22), as well as fairness (18) and ethics (14). It is important to note that reviews are often the place where stakeholders and their desiderata are recalled and added up by xAI authors (reviews consider in average 5,2 different desiderata, and 2,7 stakeholders, while technical papers consider in average 2,5 different desiderata, and 1,3 stakeholders). Trivially speaking, we argue that reviews have the *de facto* function of maintaining the bond between xAI and the set of imperatives and actors it is supposed to address.

<figure><img src="image_15.png" /><figcaption>Figure 10 – Illustration from (Arrieta et al., 2020).</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Figure 11 – Illustration from (Yang et al., 2021).</figcaption></figure>

With a more schematic view, (Arrieta et al., 2020) proposes five categories of actors and purposes of xAI, showing that the field as a whole seeks to meet exogenous needs (see Figure 10). Similarly, (Yang et al., 2021) summarizes the many challenges xAI aught to answer in order to achieve a “trustable AI” and a GDPR compliant AI (see Figure 11).

As connectionist AI thrives and its opacity raises questions, it is not surprising that the population of those to whom it must justify the nature of its operations is growing and becoming more heterogeneous. But answering these questions is also a way of enlisting them. In this sense, whatever the heterogeneity and validity of the techniques produced by these authors, xAI plays an essential socio-political role, sustaining the development and deployment of connectionist AI by being the main arena for demonstrating that stakeholders' expectations are being taken into account. Broadening the spectrum of stakeholders that xAI must consider is not a way of weakening it, but on the contrary the path that will enable it to consolidate the hold of AI, which remains its objective function. Analyzing the explicit motivations of xAI authors allows us to re-inscribe their production in a logic of dependence of xAI towards connectionist AI, and the heteronomy of xAI's objectives. This heteronomy has consequences for the properties of the xAI players and the nature of their productions, which we will now present.

## A subordinate relationship

We suggest the very special way in which xAI operates relatively to the traditional mechanisms and properties of scientific production is due to its subordinate position in the field of connectionist AI. This position can be identified by looking at the trajectory of xAI scholars. Two observations can be made, both of which bear witness to this subordinated position: the most cited authors in xAI have published little or no major publication in deep learning as such; and the most cited authors in deep learning have little or no involvement in the field of xAI.
We conducted a comprehensive bibliographical analysis focusing on the authors of the 100 most cited papers within our xAI dataset, in order to understand how their involvement in xAI research fits within their broader academic careers, particularly examining their prior engagement with the field of deep learning. Almost two-thirds (59%) of the last authors of the most cited xAI authors had no prior publication record in deep learning before entering the xAI domain: in other words, a significant proportion of the leading voices in xAI research originate from other academic backgrounds, including Medicine.

The subordination of the xAI field is also clear from the fact that the main authors of deep learning are noticeably marginally interested in xAI. To assess the level of engagement of leading deep learning researchers in xAI, we initially identified the 100 most cited authors in the topic "Neural Networks and Application" based on the OpenAlex database (Research). Notably, only 10 of these prominent researchers were found to have authored any of the 16,413 xAI papers analyzed in our database. These include Yoshua Bengio, Bernhard Schölkopf, and Andrew Y. Ng, among others. Furthermore, when examining the 100 most cited publications within the xAI domain, only one of these 10 leading deep learning researchers was represented. While these prominent figures may have occasionally contributed to xAI research, their involvement appears to be primarily through less influential publications.

At the individual level, xAI has been identified as an opportunity for researchers who initially come from outside the field of deep learning to position themselves in the draft of a particularly dynamic area of research, much like cyclists benefit from the draft of a speeding peloton. This has led to a division of labor: AI researchers work on the innovative core of deep learning, while xAI researchers defend and help the industry find justifications. At the collective level, this division of labor places xAI in a subordinate position to the field of AI, acting as a boundary dedicated to ensuring its socio-political defense.

xAI's subordinate position and its function as a protective boundary has a decisive consequence regarding its status. xAI is not organised around a stabilized conceptual and methodological corpus, and its specific role actively impedes this stabilization. By functioning as a 'shield' that absorbs and responds to external demands directed at connectionist AI, xAI prioritizes reactive and demonstrative explanations over the internal processes of consensus-building around core scientific problems, methods and objects.

## Conclusion: The xAI community as the frontier of AI

This paper began by examining a central paradox in contemporary artificial intelligence: the growth of Explainable AI (xAI) as a research field, contrasted with its persistent absence of a stable scientific consensus. Our analysis of the xAI literature has revealed a field characterized by a proliferation of heterogeneous techniques, a multitude of non-converging taxonomies, and a notable absence of the structured controversies that typically forge a mature field of research.

The central argument of this paper is that this state is not a sign of scientific immaturity, but rather a direct consequence of the particular social function xAI performs. We propose that xAI should be understood as a boundary field, operating in two distinct but related senses.
Topographically, it constitutes a space contiguous to, but separate from, the core enterprise of deep learning, often populated by researchers from outside this core. Functionally, and more importantly, it acts as a protective interface, a shield that absorbs and responds to the escalating demands for trust, transparency, and accountability placed upon AI by a growing list of stakeholders, including regulators and the public.

This subordinate and heteronomous role explains the scientific characteristics we observed. Governed by external pressures rather than an internal research agenda, xAI is incentivized to produce a continuous stream of demonstrative and "reactive" solutions. This prioritizes breadth over depth, leading to the catalogue of juxtaposed techniques that lack a unifying formal structure or hierarchy. The goal becomes demonstrating that the problem of opacity is being addressed, rather than building the internal consensus required to rigorously solve it. The field's limitations are therefore not a bug, but a feature of its place in the division of scientific labor.

This conclusion carries significant implications. For policymakers and practitioners, it suggests that xAI should not be viewed as a purely technical toolkit that can objectively "solve" the black box problem. Instead, its outputs should be understood as part of a socio-political negotiation over the legitimacy and acceptance of AI systems. For the sociology of science, the case of xAI provides an interesting model for how new fields of knowledge are shaped when they emerge in a subordinate relationship to a dominant technological driver, a dynamic likely present in other areas like "AI Ethics" or "AI Fairness" for example. Ultimately, this study demonstrates how a field's production, or even more, a field's epistemology, is profoundly shaped by its socio-political function with a clear case of scientific heteronomy, where an agenda driven by external demands from regulators, industry, and users prioritizes solutionism over scientific methods and robustness. This functional imperative, in turn, explains the absence of structured controversy within the field: rather than a sign of consensus, the tendency to ignore or marginalize foundational critiques, such as impossibility theorems, is a necessary condition for xAI to perform its role as a protective "frontier". Engaging potentially destabilizing internal debate would undermine this role, illustrating how the "epistemic culture" of the field is a direct consequence of its utility to the broader AI enterprise.

Future research could build on this foundation, further exploring the specific mechanisms of this dependency and examining whether similar fields are forming around other contested technologies.
References

Adadi A and Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). *IEEE access* 6. IEEE: 52138–52160.

Aizenberg E and Van Den Hoven J (2020) Designing for human rights in AI. *Big Data & Society* 7(2). 2. SAGE Publications Sage UK: London, England: 2053951720949566.

Arrieta AB, Díaz-Rodríguez N, Del Ser J, et al. (2020) Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. *Information fusion* 58. Elsevier: 82–115.

Bilodeau B, Jaques N, Koh PW, et al. (2024) Impossibility theorems for feature attribution. *Proceedings of the National Academy of Sciences* 121(2). 2. National Acad Sciences: e2304406120.

Borch C and Hee Min B (2022) Toward a sociology of machine learning explainability: Human–machine interaction in deep neural network-based automated trading. *Big Data & Society* 9(2). 2. SAGE Publications Sage UK: London, England: 20539517221111360.

Bove C, Laugel T, Lesot M-J, et al. (2024) Why do explanations fail? A typology and discussion on failures in XAI. *arXiv preprint arXiv:2405.13474*. Epub ahead of print 2024.

Cao X and Yousefzadeh R (2023) Extrapolation and AI transparency: Why machine learning models should reveal when they make decisions beyond their training. *Big Data & Society* 10(1). 1. SAGE Publications Sage UK: London, England: 20539517231169732.

Cardon D, Cointet J-P and Mazières A (2018) La revanche des neurones: L'invention des machines inductives et la controverse de l'intelligence artificielle. *Réseaux* 211(5). La Découverte: 173–220.

Chizat L and Bach F (2020) Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss. In: *Proceedings of Thirty Third Conference on Learning Theory* (eds J Abernethy and S Agarwal), 9 July 2020, pp. 1305–1338. Proceedings of Machine Learning Research. PMLR. Available at: https://proceedings.mlr.press/v125/chizat20a.html.

Chizat L, Colombo M, Fernández-Real X, et al. (2024) Infinite-width limit of deep linear neural networks. *Communications on Pure and Applied Mathematics* 77(10). 10. Wiley Online Library: 3958–4007.

Deck L, Schoeffer J, De-Arteaga M, et al. (2023) A critical survey on fairness benefits of XAI. *XAI in Action: Past, Present, and Future Applications*. Epub ahead of print 2023.

Dombrowski A-K, Alber M, Anders C, et al. (2019) Explanations can be manipulated and geometry is to blame. *Advances in neural information processing systems* 32.

Edwards L and Veale M (2017) Slave to the algorithm? Why a'right to an explanation'is probably not the remedy you are looking for. *Duke L. & Tech. Rev.* 16. HeinOnline: 18.
Ehsan U, Liao QV, Muller M, et al. (2021) Expanding Explainability: Towards Social Transparency in AI systems. In: *Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems*, New York, NY, USA, 7 May 2021, pp. 1–19. CHI ’21. Association for Computing Machinery. Available at: https://doi.org/10.1145/3411764.3445188 (accessed 3 October 2025).

Felzmann H, Villaronga EF, Lutz C, et al. (2019) Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns. *Big Data & Society* 6(1). 1. SAGE Publications Sage UK: London, England: 2053951719860542.

Freiesleben T and König G (2023) Dear XAI community, we need to talk! Fundamental misconceptions in current XAI research. In: *World Conference on Explainable Artificial Intelligence*, 2023, pp. 48–65. Springer.

Gilpin LH, Bau D, Yuan BZ, et al. (2018) Explaining explanations: An overview of interpretability of machine learning. In: *2018 IEEE 5th International Conference on data science and advanced analytics (DSAA)*, 2018, pp. 80–89. IEEE.

Guidotti R, Monreale A, Ruggieri S, et al. (2018) A survey of methods for explaining black box models. *ACM computing surveys (CSUR)* 51(5). 5. ACM New York, NY, USA: 1–42.

Gunning D, Vorm E, Wang Y, et al. (2021) DARPA’s explainable AI (XAI) program: A retrospective. *Authorea Preprints*. Authorea. Epub ahead of print 2021.

Hälterlein J (2021) Epistemologies of predictive policing: Mathematical social science, social physics and machine learning. *Big data & society* 8(1). 1. SAGE Publications Sage UK: London, England: 20539517211003120.

Haufe S, Wilming R, Clark B, et al. (2024) Explainable AI needs formal notions of explanation correctness. *arXiv preprint arXiv:2409.14590*. Epub ahead of print 2024.

Haupka N, Culbert JH, Schniedermann A, et al. (2024) Analysis of the publication and document types in OpenAlex, Web of Science, Scopus, PubMed and Semantic Scholar. *arXiv preprint arXiv:2406.15154*. Epub ahead of print 2024.

Holzinger A, Biemann C, Pattichis CS, et al. (2017) What do we need to build explainable AI systems for the medical domain? *ArXiv abs/1712.09923*.

Jasanoff S (2019) Controversy studies. *The Blackwell Encyclopedia of Sociology*. John Wiley & Sons, Ltd.: 1–5.

Kenny EM, Ford C, Quinn M, et al. (2021) Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in XAI user studies. *Artificial Intelligence* 294. Elsevier: 103459.

Kieslich K, Keller B and Starke C (2022) Artificial intelligence ethics by design. Evaluating public perception on the importance of ethical design principles of artificial intelligence. *Big Data & Society* 9(1). 1. SAGE Publications Sage UK: London, England: 20539517221092956.

Kleinberg J, Mullainathan S and Raghavan M (2016) Inherent trade-offs in the fair determination of risk scores. *arXiv preprint arXiv:1609.05807*. Epub ahead of print 2016.

Knowles B (2020) Explainable AI: Another Successful Failure? *Conference Proceedings*. Epub ahead of print 2020.

<figure><img src="image_17.png" /></figure>
Kuhn TS (1962) *The Structure of Scientific Revolutions*. University of Chicago Press.

Langer M, Oster D, Speith T, et al. (2021) What do we want from Explainable Artificial Intelligence (XAI)?–A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. *Artificial Intelligence* 296. Elsevier: 103473.

Linardatos P, Papastefanopoulos V and Kotsiantis S (2020) Explainable ai: A review of machine learning interpretability methods. *Entropy* 23(1). 1. MDPI: 18.

Lipton ZC (2016) The mythos of model interpretability. *Communications of the ACM* 61: 36–43.

Lundberg SM and Lee S-I (2017) A Unified Approach to Interpreting Model Predictions. In: *Advances in Neural Information Processing Systems* (eds I Guyon, UV Luxburg, S Bengio, et al.), 2017. Curran Associates, Inc. Available at: https://proceedings.neurips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf.

Merigoux D, Chataing N and Protzenko J (2021) Catala: a programming language for the law. *Proceedings of the ACM on Programming Languages* 5(ICFP). ICFP. ACM New York, NY, USA: 1–29.

Merigoux D, Alauzen M, Banuls J, et al. (2024) De la transparence à l’explicabilité automatisée des algorithmes : comprendre les obstacles informatiques, juridiques et organisationnels. RR-9535, January. INRIA Paris. Available at: https://inria.hal.science/hal-04391612.

Miller T (2019) Explanation in artificial intelligence: Insights from the social sciences. *Artificial intelligence* 267. Elsevier: 1–38.

Munk AK, Olesen AG and Jacomy M (2022) The thick machine: Anthropological AI between explanation and explication. *Big Data & Society* 9(1). 1. SAGE Publications Sage UK: London, England: 20539517211069892.

Observatory OAP (2024) Artificial intelligence. Available at: https://www.oecd.org/en/topics/artificial-intelligence.html.

Priem J, Piwowar H and Orr R (2022) OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. *arXiv preprint arXiv:2205.01833*. Epub ahead of print 2022.

Sahlgren O (2024) What’s Impossible about Algorithmic Fairness? *Philosophy & Technology* 37(4). 4. Springer: 1–23.

Selvaraju RR, Cogswell M, Das A, et al. (2020) Grad-CAM: visual explanations from deep networks via gradient-based localization. *International journal of computer vision* 128. Springer: 336–359.

Slack D, Hilgard S, Jia E, et al. (2020) Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In: *Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society*, 2020, pp. 180–186.

Sundararajan M, Taly A and Yan Q (2017) Axiomatic attribution for deep networks. In: *International conference on machine learning*, 2017, pp. 3319–3328. PMLR.

Weber RO, Johs AJ, Goel P, et al. (2024) XAI is in trouble. *AI Magazine* 45(3). 3. Wiley Online Library: 300–316.

<figure><img src="image_18.png" /></figure>
Wray KB (2007) Kuhn, Thomas and Scientific Paradigms. *The Blackwell Encyclopedia of Sociology*. Wiley Online Library: 1–3.

Yang G, Ye Q and Xia J (2021) Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond. *An International Journal on Information Fusion* 77: 29–52.