# A Rubric for Implementing Explainable AI in Production Logistics

Amita Singh, Erik Flores Flores-García, Yongkuk Jeong, Magnus Wiktorsson

► To cite this version:

Amita Singh, Erik Flores Flores-García, Yongkuk Jeong, Magnus Wiktorsson. A Rubric for Implementing Explainable AI in Production Logistics. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2022, Gyeongju, South Korea. pp.190-197, 10.1007/978-3-031-16407-1_23. hal-05386059

HAL Id: hal-05386059

https://inria.hal.science/hal-05386059v1

Submitted on 27 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# A Rubric for Implementing Explainable AI in Production Logistics

Amita Singh<sup>[1]</sup>, Erik Flores Garcia<sup>[2]</sup>, Yongkuk Jeong<sup>[3]</sup>, and Magnus Wiktorsson<sup>[4]</sup>

KTH Royal Institute of Technology, Södertälje 151 36, Stockholm, Sweden
{amitas,efso1,yongkuk,magwik}@kth.se

**Abstract.** With the advent of Industry 4.0, the world is witnessing increasing use of data and data-driven services. This phenomenon has penetrated through different sectors of production including logistics. The purpose of this study is to explore the use of Artificial Intelligence (AI) and Machine Learning (ML) in production logistics. This paper is the first step in the direction of understanding the complexity of AI and ML algorithms and thus explaining the black-box-like characteristics of these algorithms. This is coupled with the definition of eXplainable AI (XAI) in the domain. The paper furthers describes the needs for XAI and consequently presents a rubric for implementing XAI in the domain of production logistics and discusses it in detail.

**Keywords:** Production logistics · Explainable artificial intelligence · Explainability · Interpretability

## 1 Introduction

For many years artificial intelligence (AI) has mostly focussed on theory without looking at the applications of the real world [27]. However, this is changing with the help of improved learning algorithms, more computational power at hand, and easier access to data enabled advances in the field leading to wider industrial adoption of AI [20]. AI and ML are seeping into our everyday lives in different domains, ranging from medicine and healthcare [2, 13, 14, 36] to banking sectors [21, 26] and the domain of production logistics is no different [23, 38, 42]. The AI and ML techniques have demonstrated remarkable potential in these varied fields [5]. These techniques can benefit from the fact that they can learn automatically based on the historical data and that they can learn from experience while predicting results for new data.

However, with the increasing capacity of these models to be able to predict accurately, the complexity of these AI/ML models are increasing. Many AI algorithms are considered black box when they deal with a large number of data points for training. This black-box-like characteristic is a major roadblock
when it comes to implementing these results [39]. Additionally, since users lack
the ability to explain the results obtained through the implementation of these
AI algorithms it becomes difficult to trust the results. Wang et al. [41] describe
this as preventing “responsible exploitation” of their decisions by users. This
phenomenon thus gives rise to the need to motivate these decisions through
explanations resulting in more interest in eXplainable AI (XAI) [4].

Researchers and practitioners have started using XAI extensively in the field of medicine and healthcare [2, 31, 33] as it involves decisions of utmost importance. However, this phenomenon has evolved in different aspects of business domains. Production logistics is one such business domain where this change is taking place. Through this work the authors want to define XAI in terms of what it means in the domain of production logistics and why it is needed. The paper then describes a rubric for implementing XAI in production logistics.

The remainder of the paper is structured as follows: Sect. 2 describes the
related work in the field production logistics and AI/ML followed by the reasons
why XAI is an emerging and urgent need for production logistics. In Sect. 4, we
discuss a set of rules for implementing XAI in the domain and the paper ends
with a conclusion.

## 2 XAI in Production Logistics

In literature, production logistics is described as a system that deals with planning, coordinating and service functions to carry out manufacturing activities [40]. Production logistics refers to all activities involving the transportation of materials and information in a factory. This definition has been reinforced in literature by several other works [30,40]. In time dimension, the scope of production logistics is spread over the endpoints from where demand of a product is determined to the point where it is fulfilled which includes flow of product, service or information crossing entrepreneurial and state boundaries [43]. Coordination of these complex systems poses a challenge to both academicians and practitioners.

Nowadays, AI and ML are creeping into the solution space of almost all
domains as we discussed earlier and so is the case of production logistic [22].
Next, we describe, in short, artificial intelligence and machine learning followed
by the definition of explainable AI (XAI) in the domain of production logistics.

With the advent of Industry 4.0 machine learning and artificial intelligence
has come to the forefront of problem-solving in production domains. With the
growing complexity of AI and ML models, it is becoming increasingly impor-
tant to understand and decode the complexity in order to take informed and
understandable decisions.

However, so far there is no concrete definition of XAI in the domain for
production and the research team of this work believes that as a first step it is
important to understand and define XAI in this domain.

It is important to note that we in this paper adopt the definitions of explainability and interpretability and hence the difference between the two as per the literature [1,29].
We understand XAI in production logistics as the capability to assist users in understanding the AI models with clear reasoning behind them that enables users to use, trust, verify and validate the outputs produced that eventually helps in building rigorous business models and facilitates business needs.

# 3 Reasons for XAI in Production Logistics

In this section, we discuss what are the core needs for explanation of AI and where do they stem from. The research team based on existing literature and based on the discussions found these needs or reasons for XAI in the domain of production logistics.

The first purpose of XAI in production logistics comprehends justifying the results of a model. AI propelled research and the results are reported to have yielded controversial results [1, 10, 19] which demands more and more justification for the results presented through AI. XAI should thus be able to provide explanation for a decision-making specifically for unexpected results [1]. This would give provable and auditable results which further defend that algorithms are fair and logical thus building trust in AI. According to the authors, these characteristics of being provable and auditable would play an important role in the implementing XAI and encouraging AI in production logistics. Furthermore, these explanations provide compliance with legislation in the domain of production logistics.

The second reason to employ XAI in produciton logistics is to be able to control. According to Adadi and Berrada [1], explainability can help prevent untoward events by giving more insights into a model which, in turn, provides visibility over vulnerabilities and flaws. This can prove to be an important aspect in supply chain and production systems where flaws in AI models can lead to disruption on an assembly line leading to, for example, shortage of material or bottleneck in the assembly.

Explainability to evaluate AI models in production logistics for us mean that both domain experts, logistics planners and software developers, how to evaluate and compare different models. This would require understanding the models in depth to check how the results are yielded and how models perform with changing input criteria.

If the model is explained well, it can then be improved as well. This forms a part of continuous improvement which forms the basis for lean management in production logistics. If planners responsible for logistics in companies understand why a certain output is produced, it will make it possible for the planners to improve it. From the developers’ perspective, the increased visibility over vulnerabilities in the model help understand and debug the problem thus giving enhanced performance of the AI model [16].

In industrial setup, it is important to convince the management to use AI algorithms and equally important for the management to have an understanding of the results and vulnerabilities of a model. As pointed out in the literature. [8, 29, 34], it is explainability to manage has an overarching theme that encompasses justification, control, improvement and evaluation of models.
# 4 Rubric for Implementation of XAI

## 4.1 Low Effort

As production logistics concerns with short throughput times and high planning reliability together with fulfilling customer demands all of which has cost attached to it [32]. Therefore, it is important to do a cost benefit analysis [25] of the model to be explained before implementing XAI methods to it. Accordingly, the cost to explain the model should not exceed, and in most cases should be far below than the profit incurred from a particular model. Ideally the fewer resources are used in explaining the model the better it is [29].

The cost to explain a model could also be a one-time cost if the explanation is *model agnostic*. Model agnostic explanation refers to the explanation that can be used to explain any model and not just a specific model.

## 4.2 High Fairness

Binns [9] on their work on fairness in XAI discuss how *fair* should be defined: equal probability of all explainees benefiting from the explanation or minimizing risk for the least advantaged stakeholder [7, 18]. Among many definitions of formalisation of fairness, Dieterich et al. proposed ‘equality of opportunity’ which is a well-suited definition in case of many stakeholders. Following this multi intra and interfactories stakeholders, this presents itself as the most suited explanation of fairness as shown in Fig. 1.

Production logistics inherently is a multiple stakeholder scenario where different stakeholders

are engaged in a supply chain. These stakeholders could be personnel responsible for inbound and outbound logistics together with external suppliers.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Characteristics proposed for XAI in production logistics.</figcaption></figure>

## 4.3 High Comprehensibility

Comprehensibility refers to the aid that explanation provides in human understanding of an AI/ML model [29]. This has also been discussed as *plausibility* in the literature [15], but the research team for this work believes that plausibility which refers to as a measure regarding the acceptance of the explanation. Comprehensibility, thus, encompasses the plausibility aspect of the model.

*Interpretability* is discussed in the XAI literature as an extent to which the explanation is understood by humans [17]. The research team upon discussion decided that in production logistics interpretability could be seen as a part of comprehensibility and does not need to be differentiated (Table 1).
<table><caption>**Table 1.** Relevant literature corresponding to the XAI characteristics</caption><thead><tr><td>XAI characteristic</td><td>References</td></tr></thead><tbody><tr><td>Low Effort</td><td>Nyhuis et al. [32], Layard et al. [25], Meske et al. [29]</td></tr><tr><td>High Fairness</td><td>Binns [9], Barocas et al. [7], Haian et al. [18], Dietrich et al. [12]</td></tr><tr><td>High Comprehensibility</td><td>Meske et al. [29], Furnkranz et al. [15], Guidotti et al. [17]</td></tr><tr><td>High Fidelity</td><td>Arbatli et al. [6], Andrews et al. [3], Maskowska-Kaczmar et al. [28], Zilke et al. [44], Ribeiro et al. [37]</td></tr><tr><td>High Generalizability</td><td>Meske et al. [29], Ras et al. [35], Ribeiro et al. [37]</td></tr><tr><td>Legal</td><td>Montjoye et al. [11]</td></tr></tbody></table>

## 4.4 High Fidelity

Fidelity in the context of XAI refers to the degree to which the method applied agrees to the input-output mapping of the particular AI technique. The property is considered the most important of all characteristics for explaining an AI model [3, 6, 28, 37, 44]. Ribeiro et al. [37] call this property local fidelity as it is not possible for an explanation to be true for the complete model other than when it is indeed explanation of the complete model itself which brings in the word *locally*. This means how a model behaves in the vicinity of the prediction made. Thus, there exists a trade-off between explainability or interpretability and fidelity. This has been discussed in the literature. However, in context of production logistics local fidelity would be sufficient as global fidelity of complex models is difficult to achieve.

Depending upon the management to which the explanation is to be presented from the factory level to shop floor level, the trade-off between explainability and fidelity is made.

## 4.5 High Generalizability

According to Meske et al. and Ras et al. [29,35] refers to the range of methods which the XAI technique can be used to explain or interpret. Higher the generalizability more the usefulness of the XAI would be. Literature also discusses *explanatory power* [35,37] as an important characteristic for XAI which refers to how generic an explanation is. For example, an explanation that is able to extend to general models has more explanatory power than one that is specific to a model. In other words, model agnostic explanations have more explanatory power than model specific explanations. The research team views this property as a subset of comprehensibility and generalizability.

## 4.6 Legal

*Privacy* is an important aspect mentioned the existing literature in the personalized explanation space which includes the risk of meta-data of personnel involved in the process or draw conclusions about behaviour of personnel [11]. However,
the research team believes that in the context of production systems and specifically logistics, it is crucial to secure the data and metadata of upstream and downstream suppliers which may include performance data, cost-related figures, etc. Data sharing and explainability in context of the model developed is therefore has legal concerns which encompasses the abovementioned criteria.

Therefore, we used the term *legal* instead of only privacy concerns for the production domain.

# 5 Conclusion

Recently, XAI has been used in the result critical fields of medicine and healthcare. However, current challenges in production logistics make the use of XAI increasingly relevant. Accordingly, this study explored the use of XAI for production logistics and identified a rubric for its implementation including six characteristics of XAI in production logistics. Additionally, the study identified the reasons for applying XAI in production logistics. Finally, the authors present a rubric for implementing explainable AI in the domain. In future work, it would be interesting to see how a real-world production logistics problem is solved using the rubric presented. Validation of the presented rubric is also an important future work.

# References

1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access (2018)

2. Ahmad, M.A., Eckert, C., Teredesai, A.: Interpretable machine learning in healthcare. In: Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (2018)

3. Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl. Based Syst. (1995)

4. Anjomshoae, S., Najjar, A., Calvaresi, D., Främling, K.: Explainable agents and robots: results from a systematic literature review. In: 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, Canada, 13–17 May 2019. International Foundation for Autonomous Agents and Multiagent Systems (2019)

5. Antoniadi, A.M., et al.: Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: a systematic review. Appl. Sci. **11**(11) (2021)

6. Arbatli, A.D., Akin, H.L.: Rule extraction from trained neural networks using genetic algorithms. Nonlinear Anal. Theory Methods Appl. (1997)

7. Barocas, S., Selbst, A.D.: Big data’s disparate impact. Calif. Law Rev. **104**, 671 (2016)

8. Berente, N., Gu, B., Recker, J., Santhanam, R.: Managing Ai. MIS Q. (2019). Call for Papers

9. Binns, R.: Fairness in machine learning: lessons from political philosophy. In: Conference on Fairness, Accountability and Transparency. PMLR (2018)
10. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N.: Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015)

11. De Montjoye, Y.A., Radaelli, L., Singh, V.K., Pentland, A.S.: Unique in the shopping mall: on the reidentifiability of credit card metadata. Science (2015)

12. Dieterich, W., Mendoza, C., Brennan, T.: Demonstrating accuracy equity and predictive parity performance of the compas risk scales in broward county (2016)

13. Dua, S., Acharya, U.R., Dua, P.: Machine Learning in Healthcare Informatics. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-40017-9

14. Esteva, A., et al.: A guide to deep learning in healthcare. Nat. Med. (2019)

15. Fürnkranz, J., Kliegr, T., Paulheim, H.: On cognitive preferences and the plausibility of rule-based models. Mach. Learn. 109(4) (2020)

16. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA). IEEE (2018)

17. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. (CSUR) (2018)

18. Hajian, S., Domingo-Ferrer, J.: A methodology for direct and indirect discrimination prevention in data mining. IEEE Trans. Knowl. Data Eng. (2012)

19. Howard, A., Zhang, C., Horvitz, E.: Addressing bias in machine learning algorithms: a pilot study on emotion recognition for intelligent systems. In: 2017 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO). IEEE (2017)

20. Jordan, M.I., Mitchell, T.M.: Machine learning: trends, perspectives, and prospects. Science (2015)

21. Khandani, A.E., Kim, A.J., Lo, A.W.: Consumer credit-risk models via machine-learning algorithms. J. Bank. Financ. (2010)

22. Klumpp, M., Hesenius, M., Meyer, O., Ruiner, C., Gruhn, V.: Production logistics and human-computer interaction-state-of-the-art, challenges and requirements for the future. Int. J. Adv. Manuf. Technol. 105(9) (2019)

23. Knoll, D., Prüglmeier, M., Reinhart, G.: Predicting future inbound logistics processes using machine learning. Procedia CIRP (2016)

24. Kusner, M.J., Loftus, J., Russell, C., Silva, R.: Counterfactual fairness. Adv. Neural Inf. Process. Syst. (2017)

25. Layard, P.R.G., et al.: Cost-Benefit Analysis. Cambridge University Press, Cambridge (1994)

26. Le, H.H., Viviani, J.L.: Predicting bank failure: an improvement by implementing a machine-learning approach to classical financial ratios. Res. Int. Bus. Financ. (2018)

27. Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable ai: a review of machine learning interpretability methods. Entropy 23, 18 (2020)

28. Markowska-Kaczmar, U., Wnuk-Lipiński, P.: Rule extraction from neural network by genetic algorithm with pareto optimization. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 450–455. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24844-6_66

29. Meske, C., Bunde, E., Schneider, J., Gersch, M.: Explainable artificial intelligence: objectives, stakeholders, and future research opportunities. Inf. Syst. Manag. (2022)
30. Nagy, G., Illés, B., Bányai, Á.: Impact of industry 4.0 on production logistics. In: IOP Conference Series: Materials Science and Engineering, vol. 448. IOP Publishing (2018)

31. Nazar, M., Alam, M.M., Yafi, E., Mazliham, M.: A systematic review of human-computer interaction and explainable artificial intelligence in healthcare with artificial intelligence techniques. IEEE Access (2021)

32. Nyhuis, P., Wiendahl, H.P.: Fundamentals of Production Logistics: Theory, Tools and Applications. Springer Science & Business Media, Heidelberg (2008). https://doi.org/10.1007/978-3-540-34211-3

33. Panigutti, C., Perotti, A., Pedreschi, D.: Doctor XAI: an ontology-based approach to black-box sequential data classification explanations. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (2020)

34. Rai, A., Constantinides, P., Sarker, S.: Next generation digital platforms: toward human-ai hybrids. Mis Q. (2019)

35. Ras, G., van Gerven, M., Haselager, P.: Explanation methods in deep learning: users, values, concerns and challenges. In: Escalante, H.J., et al. (eds.) Explainable and Interpretable Models in Computer Vision and Machine Learning. TSSCML, pp. 19–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98131-4_2

36. Ravì, D., et al.: Deep learning for health informatics. IEEE J. Biomed. Health Inform. (2016)

37. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)

38. Singh, A., Wiktorsson, M., Hauge, J.B.: Trends in machine learning to solve problems in logistics. Procedia CIRP (2021)

39. Stepin, I., Alonso, J.M., Catala, A., Pereira-Fariña, M.: A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence. IEEE Access 9 (2021)

40. Strandhagen, J.W., Alfnes, E., Strandhagen, J.O., Vallandingham, L.R.: The fit of industry 4.0 applications in manufacturing logistics: a multiple case study. Adv. Manuf. 5(4) (2017)

41. Wang, N., Pynadath, D.V., Hill, S.G.: Trust calibration within a human-robot team: comparing automatically generated explanations. In: 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE (2016)

42. Woschank, M., Rauch, E., Zsifkovits, H.: A review of further directions for artificial intelligence, machine learning, and deep learning in smart logistics. Sustainability (2020)

43. Wu, S.D., Roundy, R.O., Storer, R.H., Martin-Vega, L.A.: Manufacturing logistics research: taxonomy and directions. Technical report, Cornell University Operations Research and Industrial Engineering (1999)

44. Zilke, J.R., Loza Mencía, E., Janssen, F.: DeepRED-rule extraction from deep neural networks. In: Calders, T., Ceci, M., Malerba, D. (eds.) DS 2016. LNCS (LNAI), vol. 9956, pp. 457–473. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46307-0_29