# Towards Cognitive Intelligence-Enabled Manufacturing

Reuben Agbozo, Pai Zheng, Tao Peng, Renzhong Tang

► To cite this version:

Reuben Agbozo, Pai Zheng, Tao Peng, Renzhong Tang. Towards Cognitive Intelligence-Enabled Manufacturing. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2022, Gyeongju, South Korea. pp.434-441, 10.1007/978-3-031-16411-8_50. hal-04768976

HAL Id: hal-04768976

https://inria.hal.science/hal-04768976v1

Submitted on 6 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Towards cognitive intelligence-enabled manufacturing

Reuben Seyram Komla Agbozo¹, Pai Zheng²[0000-0002-2329-8634], Tao Peng¹[0000-0003-1646-6166]*, Renzhong Tang¹[0000-0002-9866-7609]

¹ Institute of Industrial Engineering, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China

² Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, HKSAR, China

*tao_peng@zju.edu.cn

**Abstract.** Cognitive intelligence-enabled manufacturing (CoIM) uses machines to utilize technologies that mimic human cognitive abilities to solve complex problems in manufacturing. With the support of a cognitive intelligence-enabled manufacturing system (CoIMS) architecture, information flow is organized and coordinated appropriately, starting from the machine sensory system, central system to the motor system. Machine perceptive abilities monitor, sense and capture equipment performance, aggregate data, and help gain valuable insights into the production process. It uses the industrial internet of things, data analytics, artificial intelligence and related techniques and cognitive computing and related technologies to address production issues in an autonomous manner. As such, CoIMS solves complex production problems. It also transforms manufacturing by improving product quality, productivity, and safety, reducing costs and downtimes, identifying knowledge gaps, and enhancing customer experience. Even so, a CoIMS is not responsible for making the final decision. Instead, it supplements information on the fly for engineers to take necessary actions.

**Keywords:** Cognitive Intelligence, Manufacturing, Self-X cognition, Smart Decision Making, Artificial Intelligence

## 1 Introduction

Smart factories are automated production facilities that use sensors, cyber-physical systems (CPS), industrial internet of things (IIoTs), artificial intelligence (AI), robotics, and other modern technologies to improve efficiency and reduce costs [1]. These technologies also aid in monitoring, diagnostics, and prognostics. However, cognitive intelligence can strengthen the learning mechanism in manufacturing systems. For instance, the 5C architecture for implementing CPS inherently supports the reasoning in making manufacturing cognitive intelligent [2]. The CPS manages the interconnected systems between the physical assets and computational capabilities while leveraging the interconnectivity of machines to become cognitive intelligent [3]. Nevertheless, the CPS coupled with IIoTs adds a layer of knowledgeability to any system through data collection and monitoring. This is established through the Data-Information-Knowledge-Wisdom (DIKW) hierarchy, which facilitates the functional relationship
between data, information, knowledge, and wisdom to better understand a subject [26], a trait of human cognition. Strube (2001) defines cognition as “*a class of advanced control mechanisms that allow for sophisticated adaption to changing needs (e.g., learning and planning) through computations operating on mental representations*” [4]. These mental actions form the foundation for cognitive science. Cognitive science studies human thought processes, including perception, memory, language, reasoning, problem-solving, decision making, planning, and learning [5]. It has been used across many fields. For instance, Tesla electric vehicles utilize cognitive intelligence features in their vehicle maintenance program to monitor their health condition continuously, and users are signalled in case of any aberrations while proactively and independently ordering the replacement ahead of servicing schedules [6].

The human mind is a complex system otherwise described as an information pro-
cessing machine that receives, stores, retrieves, transforms, and transmits information
through computational processes [5, 7]. With the help of the sensory receptors, humans
extract meaning from a sensation due to contact with external stimuli through percep-
tion [8]. Fig. 1 illustrates the stages of information processing in humans. Humans use
their senses to gather information acquired through environmental stimulation. It is then
followed by perception, cognition, or recognition within the information processing re-
gion. For example, humans recognize objects by their shape, colour, size, or speech.
Consequently, the human cognitive system makes decisions by identifying and com-
prehending these patterns. In this paper, the authors use the information processing
mechanism in Fig. 1 and the general view of human cognitive architecture in Fig.2 as
the foundation and inspiration for proposing a CoIMS, where Fig.2 details the stages of
information transfer.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Cognition, perception, and infor-
mation processing</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** A general view of human cognitive ar-
chitecture</figcaption></figure>

Manufacturing is evolutional, and CoIM is a phase in this advancement. This progres-
sion toward human-machine collaboration and Self-X cognitive systems is the inspira-
tion behind CoIM and the proposed architecture of CoIMS. In addition, this paper high-
lights the enabling technologies of CoIMS and ongoing applications across selected
industries using cognitive intelligence-enabled technologies.
## 2 Fundamentals of CoIMS

The CoIMS imitates ordinary brain-related skills and acts as the information pro-
cessing region, including memory, attention, concentration, problem-solving, creativ-
ity, and critical thinking. A CoIMS applies these imitated skills to executing regular
manufacturing tasks. The information processing capacity of humans, plus the system's
mechanism, is described as cognitive architecture (see Fig. 4) [5]. Like brain-based
skills, the CoIMS has different compartments with specific skills for performing cate-
gorical functions, as shown in Fig. 3.

<figure><img src="image_5.png" /><figcaption>Fig. 3. CoIMS Composition</figcaption></figure>

Engineers have, over the years, made tremendous progress in automating the replication of human senses. These human-like sensory systems are used in smart manufacturing for system diagnostics and prognostics. For instance, machine vision [9] technologies are used in visual inspection, fault diagnostics, and product defect detection. Other machine perceptions include machine audition [10], machine olfaction [11], machine touch or tactile perception [12], and machine gustatory (the process of mimicking the perception of taste and feel experiences food at a machine level), a term coined purposely for illustration, has seen very little progress. Machine perceptions mimic human senses to gather data on the manufacturing operations and their surroundings. Data collected is forwarded for analysis and interpretation. Then, the machine motor systems coordinate certain machine motor functions as per the directives of the CoIMS to solve production issues. Based on this logic of mimicking human cognition, perception, and information processing to solve problems as described in this paper, we define *cognitive intelligence-enabled manufacturing (CoIM) as using machines to utilize technologies that mimic human cognitive abilities to solve complex problems in manufacturing*.

Generally, human cognition and learning are based on three thematic theories: "the nature of knowledge, learning and transfer, and the nature of motivation and engagement" [13]. A CoIMS is structured on the same pattern of cognition and learning. Unlike humans, machines acquire knowledge through data supply (historical and real-time). Subsequently, they learn new information from the acquired data using knowledge graphs [14] to map data points using relationships. Identical to human
knowledge transfer and learning is the machine learning technique called transfer learn-
ing [17]. Transfer learning is used in transferring knowledge in different but related
source domains to improve the performance of target learners on a target domain [15-
16]. As humans learn and transfer knowledge, they are motivated extrinsically or in-
trinsically to become better at their tasks. Likewise, machines use reinforcement learn-
ing (RL) [21]. Applications of RL are seen in robotic control, end-to-end control, rec-
ommendation systems, and natural language dialogue systems [18-19]. However, a crit-
ical cognitive skill, Metacognition, is barely used in smart systems. Metacognitive pro-
cesses refer to the ability to reflect on one's thinking processes and evaluate them for
improvements [20]. Metacognitive skills such as self-monitoring and evaluation, help
the system to reflect on its performance, thus enabling it to be self-motivating, engag-
ing, and enthusiastic about finding problems. Such abilities enable machines to identify
problems early and take corrective action.

3 CoIMS characteristics and enabling technologies

CoIMS simulates human cognition to perform production tasks and solve problems. It learns from experience and adapts its behaviour based on system feedback. However, the CoIMS needs the support of enabling technologies, to learn from data and perform tasks efficiently without explicit programming.

**3.1 Characteristics of CoIMS**

A CoIMS is knowledgeable, flexible, and attentive to be adaptive. It imitates human
cognitive architecture using flexible programmable systems and technology for reason-
ing, planning, and solving problems. A CoIMS independently performs diagnostics and
prognostics. It utilizes metacognitive abilities, cognitive reasoning, learning, and trans-
ferring knowledge, knowledge mapping, and graphing.

## **3.2 Enabling technologies in CoIMS**

CoIM is dependent on enabling technologies which are integrated and work as collab-
orative systems. They include cognitive computing and informatics, machine learning,
deep learning, big data and analytics, robotics, CPS, IIoTs, machine vision, cloud com-
puting, knowledge graphs, natural language processing (NLP), and reinforcement
learning. These techniques and technologies serve as tools for mimicking human cog-
nitive abilities and applying them to manufacturing processes. As a result, the system
performs diagnostics and suggests preventive and corrective actions. For example, to
take proactive measures in CoIM, the machine learning (ML) and deep learning (DL)
models are trained on the data harvested within the machine sensory system. The
trained models acquire knowledge to identify patterns within a defined data group and
present detailed information on the right approach to solve a production issue. The
CoIMS supplements relevant information to engineers for analysis and implementation,
thereby enhancing an interactive change and innovation drive.
## 4 The proposed system architecture of CoIM

A CoIMS functions effectively and efficiently with defined architecture, named CoIMS architecture in this paper (see Fig. 4). The compositions of the CoIMS architecture in Fig.4 are depicted in Fig.3, which are modelled using Figs.1 and 2. Inspired by the functionalities of Figs.1 and 2, machine sensory systems perform the role of machine perceptions in a CoIMS. The machine central systems and motor systems in a CoIMS mimic human central thinking systems and motor systems shown in Fig 2. The machine central system is the engine of the CoIMS, where it analyzes data from the machine sensory system (to identify inefficiencies and potential production issues). The machine motor system is also responsible for coordinating real-time and historical data in the machine central system for decision making. As a result, the CoIMS interacts with cognitive machines, processors, devices, and cloud platforms to identify issues and communicate them to engineers.

The CoIMS architecture is the underlying principle behind a cognitive intelligence-enabled factory. As depicted in Fig.5, a cognitive intelligence-enabled factory comprises enabling technologies, a Self-X cognitive manufacturing network [21], machine-to-machine cognitive-mutual collaboration, a real-time data centre, a CoIMS system, and CoIM digital twin. Robots and other manufacturing equipment in this factory adopt self-X cognitive abilities in their operations. In this factory, machines can self-learn and unlearn. Enabling technologies and techniques enhance machine-to-machine interaction while optimizing operations and making the system adaptive and flexible to understand changes in the information process flow. It identifies problems by pulling historical data if the data in the problem is incomplete by using cloud computing methodologies. Thereafter, predictive actions are forwarded to engineers for decision-making and implementation based on data analytics. Simply, the CoIMS can be described as a consultant (an expert using key enabling technologies to function), and the human is the overseer. For instance, cognitive chatbots mimic human thinking processes using NLP to engage in causal analysis and advisory interactions with engineers on ways to improve total system performance. Besides, this factory adopts a cognitive digital twin (CDT) technology, a real-time virtual cognitive system [25]. The CDT aids in analyzing real-time performance, adaptability, and cognitive capabilities. Thence, results in real-world system optimization, problem-solving, and taking proactive measures.

Solving previously unknown issues birth new information to effectively leverage manufacturing operations. For instance, if a new issue of material quality defect arises during the production process, CoIMS will refer to historical data, draw inferences, and make diagnostics or prognostics where necessary. Solving this new problem adds to the existing knowledge database. As such, the CoIMS can be described as a system equipped with the ability to gather new information in the manufacturing plant and use the acquired information to improve product quality, productivity, and safety. The same approach applies to cost and downtime reduction, identifying knowledge gaps, and enhancing customer experience.
Besides Tesla, which comprehensively applies cognitive intelligence to their sys-
tems at a product level, IBM Watson IoT perfectly illustrated how cognitive intelli-
gence-enabled systems could dramatically enhance efficiency and maximize perfor-
mance on the shop floor within the manufacturing plant.

<figure><img src="image_6.png" /><figcaption>Fig. 4. A cognitive intelligence-enabled manufacturing system (CoIMS) architecture</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. A cognitive intelligence-enabled factory</figcaption></figure>

For instance, in their illustrative video case study, an NLP-enabled cognitive chatbot relies on an enormous amount of manufacturing data, similar to the operation of the machine sensory systems in Fig. 3. The chatbot used a visual inspection system to identify defective products and uses data acquired from the production process and historical data, it recommended probable servicing actions to the maintenance team [24]. After that, the new solution was fed into the system for future proactive decision-making.

Furthermore, Zheng et al. (2021) highlighted the Self-X cognitive manufacturing network and focused on an industrial knowledge graph (IKG)-based multi-agent reinforcement learning approach in manufacturing networks with a higher level of automation. They used a simulated multi-robot based on their proposed IKG-based MARL-enabled approach, comprising the Self-X cognitive attributes [22]. The success of the Self-X cognitive manufacturing network lays a strong foundation for a cognitive intel-
ligence-enabled factory in Fig. 5. Also, Zheng et al. (2022) introduced the visual rea-
soning-based mutual-cognitive human-robot collaborative (HRC) system. The pro-
posed system will enable industry robots to develop visual cognitive reasoning and per-
ception during task executions, which is relevant to activities on a manufacturing shop
floor. With a holistic scene analysis, robots can logically and cognitively understand
activities around them, and then predict and support human actions in that environment
[23].

Lastly, works on cognitive digital twins (CDT) for manufacturing are explored for
future implementations where it uses learning, reasoning, and automatic adjustment for
better decision-making using real-time IoT data [25, 27]. A CDT is relevant as it re-
quires semantic modelling, systems engineering, and product lifecycle to achieve
higher levels of automation and intelligence [27]. Nevertheless, it is a critical technol-
ogy for the establishment of a cognitive intelligence-enabled factory.

5 Conclusion and future work

This paper described the inspiration behind CoIM. Fundamentals and enabling technol-
ogies in CoIMS coupled with illustrative examples were also discussed. In addition, we
echo the Self-X cognitive manufacturing network with multi-robot collaboration and
envision a cognitive intelligence-enabled factory that generates a massive amount of
data to enable manufacturers to gain deeper insight into the workings of CoIMS. After
that, the cognitive system can gather new information in the manufacturing plant and
use the acquired information to subsequently improve product quality, productivity,
and safety. Others include cost and downtime reduction, identifying knowledge gaps,
and enhancing customer experience. In the near future, the authors shall explore the
fluidity in collaboration between varying robots (having distinct functions) with Self-
X cognitive capabilities in a cognitive intelligence-enabled factory.

References

1. Li, J., Tao, F., Cheng, Y., Zhao, L.: Big data in product lifecycle management. The International Journal of Advanced Manufacturing Technology 81(1), 667-684 (2015).

2. Lee, J., Bagheri, B., Kao, H. A.: A cyber-physical systems architecture for industry 4.0-based manufacturing systems. Manufacturing letters 3, 18-23 (2015).

3. Baheti, R., Gill, H.: Cyber-physical systems. The impact of control technology 12(1), 161-166 (2011).

4. Strube, G.: Cognitive Science: Overview. In: Smelser, N.J., Baltes, P.B. (eds.) International Encyclopedia of the Social and Behavioral Sciences, pp. 2158-2166. Elsevier, Amsterdam (2001).

5. Stillings, N. A., Chase, C. H., Weisler, S. E., Feinstein, M. H., Garfield, J. L., Rissland, E. L.: Cognitive science: An introduction. 2nd edn. MIT Press, Massachusetts (1995).

6. Tesla, https://twitter.com/Tesla/status/1125465424529887232, last accessed 2022/02/09.

7. Simon, H. A.: The human mind: The symbolic level. In: Proceedings of the American Philosophical Society, 137(4), pp. 638-647. JSTOR (1993).
8. Levitin, D. J.: Foundations of cognitive psychology: core readings. MIT Press, London (2002).

9. Klette, R.: Concise computer vision. Springer, London (2014).

10. Wang, W.: Machine Audition: Principles, Algorithms, and Systems: Principles, Algorithms, and Systems. IGI Global. Hershey (2010).

11. Haddad, R., Medhanie, A., Roth, Y., Harel, D., Sobel, N.: Predicting odor pleasantness with an electronic nose. PLoS Computational Biology 6(4), (2010).

12. Fleer, S., Moringen, A., Klatzky, R. L., Ritter, H.: Learning efficient haptic shape exploration with a rigid tactile sensor array. PloS one 15(1), (2020).

13. Greeno, J. G., Collins, A. M., Resnick, L. B.: Cognition and learning. In: Berliner, D., Calfee, R.(eds.), Handbook of Educational Psychology, pp. 15-46. MacMillan, New York (1996).

14. Hogan, A., Blomqvist, E., Cochez, M., d'Amato, C., Melo, G., Gutierrez, C., Kirrane, S., Gayo, J., Navigli, R., Neumaier, S., Ngomo, A., Polleres, A., Rashid, S., Rula, A., Schmelzeisen, L., Sequeda, J., Staab, S., Zimmermann, A.: Knowledge Graphs. Synthesis Lectures on Data, Semantics, and Knowledge. 12, 1-257 (2021).

15. Torrey, L., Shavlik, J.: In Handbook of research on machine learning applications and trends: algorithms, methods, and techniques. pp. 242-264. IGI Global, 2010.

16. Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., He, Q.: A comprehensive survey on transfer learning. Proceedings of the IEEE 109(1), 43-76 (2020).

17. Pan, S. J., Yang, Q.: A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359 (2009).

18. Wang, H. N., Liu, N., Zhang, Y. Y., Feng, D. W., Huang, F., Li, D. S., Zhang, Y. M.: Deep reinforcement learning: a survey. Frontiers of Information Technology & Electronic Engineering 21(12), 1726-1744 (2020).

19. Li, Y.: Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274. (2017).

20. Livingston, J.A.: Metacognition: An overview. U.S. Department of Education, 1-9 (2003).

21. Wiering, M. A., Van Otterlo, M.: Reinforcement learning. Adaptation, learning, and optimization, 12(3), 729 (2012).

22. Zheng, P., Xia, L., Li, C., Li, X., Liu, B.: Towards Self-X cognitive manufacturing network: An industrial knowledge graph-based multi-agent reinforcement learning approach. Journal of Manufacturing Systems (61), 16-26 (2021).

23. Zheng, P., Li, S., Xia, L., Wang, L., Nassehi, A.: A visual reasoning-based approach for mutual-cognitive human-robot collaboration. Cirp Annals Manufacturing Technology, (2022).

24. Woolfe, T.: Cognitive Manufacturing in Action - IBM Watson IoT, https://www.youtube.com/watch?v=f3WB2e3vXWQ&t=438s&ab_channel=TobyWoolfe, last accessed 2022/03/03.

25. Al Faruque, M. A., Muthirayan, D., Yu, S. Y., Khargonekar, P. P.: Cognitive Digital Twin for Manufacturing Systems. In: 2021 Design, Automation & Test in Europe Conference & Exhibition, pp. 440-445. IEEE, (2021).

26. Mourtzis, D.: Towards the 5th industrial revolution: a literature review and a framework for process optimization based on big data analytics and semantics. Journal of Machine Engineering 21(3), 5-39 (2021).

27. Zheng, X., Lu, J., Kiritsis, D.: The emergence of cognitive digital twin: vision, challenges and opportunities. International Journal of Production Research, 1-23 (2021).