# Self-learning Production Control Using Algorithms of Artificial Intelligence

Ben Luetkehoff, Matthias Blum, Moritz Schroeter

► To cite this version:

Ben Luetkehoff, Matthias Blum, Moritz Schroeter. Self-learning Production Control Using Algorithms of Artificial Intelligence. 18th Working Conference on Virtual Enterprises (PROVE), Sep 2017, Vincenza, Italy. pp.299-306, 10.1007/978-3-319-65151-4_28. hal-01674863

HAL Id: hal-01674863

https://inria.hal.science/hal-01674863

Submitted on 3 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Self-Learning Production Control using Algorithms of Artificial Intelligence

Ben Luetkehoff, Matthias Blum and Moritz Schroeter

FIR at RWTH Aachen University, Production Management, Campus-Boulevard 55, 52074
Aachen, Germany
{lh, bl, sch}@fir.rwth-aachen.de

**Abstract.** Manufacturing companies are facing an increasingly turbulent market – a market defined by products growing in complexity and shrinking product life cycles. This leads to a boost in planning complexity accompanied by higher error sensitivity. In practice, IT systems and sensors integrated into the shop floor in the context of Industry 4.0 are used to deal with these challenges. However, while existing research provides solutions in the field of pattern recognition or recommended actions, a combination of the two approaches is neglected. This leads to an overwhelming amount of data without contributing to an improvement of processes. To address this problem, this study presents a new platform-based concept to collect and analyze the high-resolution data with the use of self-learning algorithms. Herby, patterns can be identified and reproduced, allowing an exact prediction of the future system behavior. Artificial intelligence maximizes the automation of the reduction and compensation of disruptive factors.

**Keywords:** production control, self-learning algorithms, data analytics

## 1 Introduction

Driven by the Internet of Things and countless initiatives in the context of Industry 4.0, the shop floor level is steeped by IT. This poses the question as to which extent these approaches can be used to reduce and control the increasing production complexity. The collection of real-time data and its interpretation with data analytics are supposed to enable improved planning decisions. The research project “Intelligent Production Control” (iProd) takes exactly this point into account: technologically and theoretically the progress towards completion, machine condition and the current flow of material can be captured by installing sensor systems and identification technology. The difficulty lies in, on the one hand, the connection of heterogeneous data to form an overall picture to generate recommended actions [1, 2]. On the other hand, the reaction speed of implemented planning and control solutions is limited by the detection and evaluation of deviations by the employee [3]. Moreover, current algorithms are based on inflexible heuristic optimizations and static analyses. Even though, simulation can be based on this kind of information, the reality is still far
away from a self-learning production control that, to a certain degree, compensates
deviations and disturbances independently [4, 5].

Therefore, the goal of this project is the application of existing technologies, architectures and algorithms from the field of artificial intelligence to the problems of production, so that the open loop production control is expanded to an advanced closed loop production control.

Before the project is described in detail, the motivation of this paper is outlined and a short outline of the necessary basics and state of the art is given. Additionally, the requirements for an intelligent closed loop production control are presented. Afterwards, the concept itself is introduced and discussed before a conclusion is given and necessary further research is outlined

## 2 Motivation

Data analytics is a scientific process of mathematical-logical transformation of data to improve decision making [6]. Four stages of the decision-making process exist: descriptive, diagnostic, predictive and prescriptive analytics. The decision making process assisted by data analytics can be described as a four-stage model. The first stage is called descriptive analytics and deals with the question “What happened?” This stage aims at analyzing large amounts of data in order to find out what happened in the past. The next stage, called diagnostic analytics, deals with the question “Why did it happen?” by analyzing interactions in the first stage. The third and fourth stage also allow proactive optimization. Predictive analytics focusses on the question “What will happen?” and predicts future behavior by applying methods of pattern recognition as well as the use of statistics. Prescriptive analytics focusses on the question “What should I do?”. The assistance of data analytics either serves as a support mechanism for the decision process or implements decisions autonomously. [7]

## 3 State of the Art

In this chapter, the basic principles deviation management, information modelling, Artificial Intelligence, machine learning and their application in production planning and control are discussed.

A multitude of software solutions has been developed in the past in order to manage the dynamics of production planning and control with the use of IT. In context of production control, especially Enterprise Resource Planning and Manufacturing Execution Systems have been used for detailed production planning and control [8]. Approaches that focus on closed loop production, i.e. an additional feedback loop to regular open loop control, can be found in several dissertations [4, 9, 10]. However, these are not based on artificially intelligent methods but instead on procedures of discrete-event simulation. These require exact models of influencing variables, which come along with manual effort. Even the automatic generation of models has only been theoretically discussed within this field [11].
Intelligent, self-learning systems are aware of their current performance. They are therefore able to evaluate it in comparison to the ideal performance, enabling them to develop strategies to improve the solution. Machine learning (ML) methods enable intelligent systems to complete tasks in unexpected situations [12]. Learning the behavior strategies of a technical system with the use of ML has already been used for many purely virtual scenarios and for a few scenarios in robotics. The best-known examples for this come from the technology company Google, e.g. the latest application example for learning gripping operations [13]. Recently, data-driven ML has become progressively more important in the context of Industry 4.0. However, significant research results have not yet been presented.

Research for the application of Artificial Intelligence in the context of Production Planning and Control is nothing new. In the 90ies, the application of artificial neuronal networks in business has already been researched by Schneider, who identified different adequate network types and parameters [14]. Huang analyzed the suitability and weaknesses of existing approaches for the application of neuronal networks in production systems [15]. In the following years, approaches to build groups for flexible manufacturing systems were developed [16] for capacity planning and sequencing. Reddy et al. used artificial neuronal networks for pattern recognition on the production process level [17]. Scholz-Reiter et al. introduced a system with cascaded control loops for a neuronal closed loop production control. Implicit knowledge is integrated into the network with the help of case based reasoning. The training and selection of the suitable artificial neuronal network remains a task for experts [18]. Hamann developed a control concept for production systems that is based on "Computational Intelligence". Neuronal networks control the stock of work systems by determining the optimal target stock heuristically. Next, the quality of the model is shown with theoretical observations and practical examples. The quality can be increased continuously by training the neuronal networks with new examples [4]. This approach is suited mainly for bottlenecks due to the high implementation complexity and does not represent a universal concept for the entire production control. Di Orio et al. discuss a paradigm called Self-Learning Production System (SLPS) to ensure the evolvability of industrial systems over time. This is achieved with context-awareness and data mining techniques in order to ensure the adaption of a system to changing processes. The discussed approach focusses on the systemization of technical and theoretical developments to provide a baseline for the SLPS solution [19].

The State of the Art shows that existing approaches for the rapid elimination of disturbances and deviations often focus solely on reproducing interdependencies in simulation models or do not offer the feedback of real-time data. The implementation and adaption of these models is despite extensive research effort in the context of automated model generation linked to manual input [11, 20]. In the framework of simulation models, forecasts can be calculated for different scenarios, the exact reproduction of real interdependencies in the form of these scenarios is linked to high manual effort. Methods of pattern recognition allow the formalization of knowledge. Because of increasing computing capacities, promising approaches can be found in the field of artificial intelligence – especially in ML. The high requirements for reliable and robust processes as well as an efficient use of resources complicate the
application of complex trial and error methods parallel to the ongoing production. Extended data driven solutions are needed that can be applied purposefully and allow efficient learning of new tasks. Here, a combination of ML and model driven approaches seems imaginable in order to achieve a broad field of application [21].

# 4 Requirements

The research project has five requirements in order to achieve an intelligent closed loop production control: 1) a real-time capability, 2) a pattern recognition, 3) an analysis of deviations, 4) a derivation of response strategies and 5) a feedback.

Real-time representation of data is required and the essential element for handling with deviations. Decisions are based on data and therefore the provision is one of the main challenges. The earlier data are available and presented in a useful way the better the response time of the system. Furthermore, the real-time representation serves to a better overview of the production processes. With the help of data, the production is presented in a transparent manner, which leads to a better process understanding and can help to stabilize the production processes.

Another condition is the pattern recognition. Machine learning methods evaluate a database to detect anomalies. A basic prerequisite for machine learning methods is the reliability and real time availability of data. Through this, machine-learning methods are able to discover patterns early and just in time. Pattern recognition support the analysis of error causes and error sequences. This draws on specifically designed algorithms. If there is a deviation in the production process, the deviation needs to be captured completely.

Afterwards the deviation has to be analyzed, which is the next main requirement of the system. It is very important to understand the causes and effects of the deviation. The analysis is the fundament of the classification process, in which deviations allocated to clusters.

The next step is to create a link between a type of deviation and a response strategy. For this allocation, different response strategies have to be defined. According to this, response strategies are another requirement for a working system. A distinction is made between automated, semi-automated and manual response strategies. One aspect of a response strategy is the development of a morphology with characteristic attributes and properties captured by sensors and IT. In order to prioritize the reaction strategies, the aspect of time needs to be regarded as well.

# 5 iProd – A Concept for Data Driven Closed-Loop Production Control

The research project iProd intends to develop a robust and intelligent closed loop production control that continuously reacts to deviations within the production system by target-performance comparison. It configures the parameters of the production order accordingly, based on model-based forecast scenarios. With a basis of real-time, high-resolution data from the direct production environment, the optimization model
applies algorithms of artificial intelligence in order to deduce effective recommended actions. These recommendations are supposed to contain (partly) autonomous decisions, made by a pattern recognition system. The system analyses the observations, assesses them with reaction strategies and links them to a course of action. A feedback into the control loop rates the result of the measures. The research project reproduces the application of a closed loop production control very precisely and describes one of the relevant problems of manufacturing companies – the simple control and laying-up of the production with regard to a turbulent environment. All of the addressed processes are collected on a data platform. The concept is visualized in the following Fig. 1.

<figure><img src="image_2.png" /><figcaption>Fig. 1: The concept of the research project</figcaption></figure>

The result is an increased capacity of the personnel that allows the search, analysis and removal of causes. This leads to a sustainable stabilization of the entire production and relieves employees by reducing the amount of short time, highly critical decisions. A positive impact on cycle times and work in progress affect delivery dates positively. The degree of capacity utilization – for employees and machines – is harmonized and sudden surge in workload can be averted. Thus, the capacity can be planned better without risking the flexibility of the company.

The proposed concept is based on four elements: A) the collection and aggregation of real-time data, B) the ability to recognize patterns with self-learning algorithms, C) the deduction of response strategies and D) the implementation of a data platform for production control.

A) Real-time feedback data: The basis for the concept is the identification and virtual representation of real-time data along the material flow. The analysis and evaluation of structured and unstructured real-time data enables an optimization of complex production systems. The digital shadow of the production is the basis for a databased predictability of future situations within a real system and for the realization of potentials for innovation and optimization [22].

The overarching goal of iProd covers the development and validation of an Industry 4.0 suitable solution concept, which exploits technical possibilities in digitalization. Therefore, the digital image of a production forms the basis for a holistic analysis an evaluation of the automatized captured data.
B) Pattern recognition: The collected data in form of a digital shadow allows recognizing patterns that would otherwise stay hidden. This is achieved by descriptive, predictive or prescriptive analysis of the data. Different methods can be applied for data analytics. For example, there classical data mining and machine learning tools that can be used on structured data and other tools that can be used for unstructured data [23]. In this case, there is only structured data from the production environment. Classical tools for the analysis of structured data are, for example, Online Analytical Processing (OLAP), querying and reporting [24] and will mainly be used in this project. Based on patterns from the production system, the project enables the accurate forecast of the future system behavior in consideration of influencing disruptive factors. This is the basis for the implementation of an intelligent closed loop production control.

C) Response strategies: The intelligence of the production control results from the ability to implement response measures in a systematical and partly autonomous way. The goal is a systematic allocation of response strategies, which can be either automatic, semi-automatic or manual, in order to react to deviations in production. This way, the achievement of the logistical targets can be ensured through the focused feedback of measured values and self-learning control units, in spite of dynamic influencing factors. Transferred to industry partners participating in the project, this means that manufacturing companies from different kinds of industries become capable to design their production reactively and efficiently with the implementation of machine learning methods.

D) Data platform: The production control is the fundamental condition for capturing and comparing the relevant control variables with the planning guidelines (reference variables) continuously and in real-time. For the approximation of the control variable, the action alternatives are rated in the data platform of iProd. For the determination of the regulating variables, for example work- and machine usage plans, the optimal configuration can be defined. This way, adequate measures for the parameterization of the order-generation, the order-authorization, the sequencing and the capacity-coordination of the production can be derived.

In summary, iProd intends the substitution of classical control logic, in which for example set-up times, process times and transition times are based on average values or even on subjective evaluation. Instead of this, iProd allows the forecast of future disruptive factors and resulting deviations besides the identification of current deviations, to fix the control difference in a proactive way by an optimized parameter configuration.

# 6 Conclusion and Further Research

Manufacturing companies are facing an increasingly turbulent market defined by products growing in complexity and shrinking product life cycles. This leads to an increase in planning complexity and higher error sensitivity. In practice, IT systems and sensors integrated into the shop floor in the context of Industry 4.0 are used to deal with such challenges. However, while existing research provides solutions in the field of pattern recognition or recommended actions, a combination of the two
approaches is neglected. This leads to an overwhelming amount of data without contributing to an improvement of processes. Therefore, this paper presents a new platform-based concept that enables an increased logistical capability. This is achieved by collecting and analyzing high-resolution data with the use of self-learning algorithms. Thereby, patterns can be identified and reproduced, allowing an exact prediction of the future system behavior influenced by interfering factors. The use of artificial intelligence maximizes the automation of the reduction and compensation of the identified disruptive factors.

However, the presented paper also has a few shortcomings. First, further research is needed to enable and improve the proposed concept. It is necessary to elaborate the discussed approach to solve the discussed problem of production control. Moreover, the concept still only focusses on the manufacturing industry while other industries are neglected. Furthermore, use cases need to be found for a possible application and validation of the proposed concept. In addition, it is important to improve the man-machine interaction, e.g. visualization, to enable a broad dissemination of the concept.

## References

1. Kropp, S.K.: Entwicklung eines Ereignismodells als Grundlage der Produktionsregelung, 1st edn. Schriftenreihe Rationalisierung, vol. 137. Apprimus, Aachen (2016)

2. Schuh, G., Blum, M.: Design of a data structure for the order processing as a basis for data analytics methods. In: Portland International Conference on Management of Engineering and Technology (PICMET), pp. 2164–2169, Honolulu (2016)

3. Meier, C.: Echtzeitfähige Produktionsplanung und -regelung in der Auftragsabwicklung des Maschinen- und Anlagenbaus, 1st edn. Edition Wissenschaft, vol. 117. Apprimus, Aachen (2013)

4. Hamann, T.: Lernfähige intelligente Produktionsregelung. Informationstechnische Systeme und Organisation von Produktion und Logistik, vol. 7. GITO, Heidelberg (2008)

5. Hauptvogel, A.: Bewertung und Gestaltung von cyber-physischer Feinplanung, 1st edn. Produktionssystematik, vol. 6. Apprimus Verlag, Aachen (2015)

6. Knabke, T., Olbrich, S., Lehner, W., Piller, G. (eds.): Towards agile BI: applying in-memory technology to data warehouse architectures (2011)

7. Stich, V., Hering, N.: Daten und Software als entscheidender Wettbewerbsfaktor. Industrie 4.0 Magazin : Zeitschrift für integrierte Produktionsprozesse, 8–13 (2015)

8. Kletti, J., Schumacher, J.: Die perfekte Produktion. Manufacturing Excellence durch Short Interval Technology (SIT), 2nd edn. Springer, Heidelberg (2014)

9. Simon, D.: Fertigungsregelung durch zielgrößenorientierte Planung und logistisches Störungsmanagement. iwb Forschungsberichte, vol. 85. Springer, Heidelberg (1995)
10. Zetlmayer, H.: Verfahren zur simulationsgestützten Produktionsregelung in der Einzel- und Kleinserienproduktion. iwb Forschungsberichte, Bd. 74. Springer, Berlin, New York (1994)

11. Selke, C.: Entwicklung von Methoden zur automatischen Simulationsmodellgenerierung, vol. 193. Utz, München (2005)

12. Giese, H., Burmester, S., Klein, F., Schilling, D., Tichy, M.: Multi-Agent System Design for Safety-Critical Self-Optimizing Mechatronic Systems with UML. In: Crocker, R. (ed.) Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, pp. 21–23. n.n. (2003)

13. Levine, S., Pastor, P., Krizhevsky, A., Quillen, D.: Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection. http://arxiv.org/pdf/1603.02199 (2016)

14. Schneider, B.: Neuronale Netze für betriebliche Anwendungen: Anwendungspotentiale und existierende Systeme. Universität Münster (1993)

15. Huang, S.H., Hong-Chao Zhang: Artificial neural networks in manufacturing. Concepts, applications, and perspectives. IEEE Transactions on Components, Packaging, and Manufacturing Technology: Part A, 212–228 (1994)

16. Kulkarni, U.R., Kiang, M.Y.: Dynamic grouping of parts in flexible manufacturing systems — a self-organizing neural networks approach. European Journal of Operational Research, 192–212 (1995)

17. Reddy, D.C., Ghosh, K., Vardhan, V.A.: Identification and interpretation of manufacturing process patterns through neural networks. Mathematical and Computer Modelling, 15–36 (1998)

18. Scholz-Reiter, B., Hamann, T., Gronau, N., Bogen, J.: Fallbasierte neuronale Produktionsregelung : Nutzung des Case-Based Reasoning zur Produktionsregelung mit neuronalen Netzen. wt - Werkstattstechnik online, 293–298 (2005)

19. Di Orio, G., Cândido, G., Barata, J.: Self-Learning Production Systems: A New Production Paradigm. Sustainable Design and Manufacturing 2014 Part 2, 887–898

20. Bergmann, S., Straßburger, S., Schulze, T.: Automatische Generierung adaptiver Modelle zur Simulation von Produktionssystemen. TU Ilmenau Universitätsbibliothek, Ilmenau (2014)

21. Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A.: Learning Movement Primitives. In: Dario, P., Chatila, R. (eds.) Robotics Research. The Eleventh International Symposium., vol. 15. Springer Tracts in Advanced Robotics, pp. 561–572. Springer Heidelberg (2005)

22. Schuh, G., Blum, M., Reschke, J., Birkmeier, M.: Der Digitale Schatten in der Auftragsabwicklung. ZWF, 48–51 (2016)

23. Gröger, C., Kassner, L., Hoos, E., Königsberger, J., Kiefer, C., Silcher, S., Mitschang, B.: The Data-driven Factory. ICEIS 2016, 40–54 (2016)

24. Kassner, L., Gröger, C., Mitschang, B., Westkämper, E.: Product Life Cycle Analytics – Next Generation Data Analytics on Structured and Unstructured Data. Procedia CIRP, 35–40 (2015)