## 3 Concept
### Embedding in Manufacturing IT Landscape  
To enable the ACSN to support manufacturing control effectively it must be seamlessly integrated into the common manufacturing IT landscape of manufacturing companies (cf. requirement 7). The proposed modular approach (cf. Figure 7) was inspired by a more general event processing architecture described in the literature [66]. It has been adapted for the processing of events from the supply network as well as extended to provide a suitable basis for a RL-based controller. The building blocks of the architecture are oriented to the automation pyramid with manufacturing level, manufacturing control level and enterprise control level. A data layer and a digital twin supplement these. Regarding stakeholders, both external companies and internal users are considered. Consequently, interfaces could be either human machine interfaces or application programming interfaces.  
Figure 7: Architecture for event processing of supply network events.  
The manufacturing level consists of the equipment and work centers, their sensors, actuators and IT interfaces. Since newer approaches to manufacturing control see dispatching as a decentralized task assigned to the work centers, this is taken into account accordingly in the architecture, in contrast to the classic automation pyramid.  
Although the manufacturing level mainly consists of physical equipment, the manufacturing control level and enterprise control level are IT driven and, therefore, responsible for data processing. The manufacturing control level consists of operating data acquisition and machine data acquisition to establish a data-based link to the manufacturing level. This link needs to be modular, flexible, and well scalable, which could be achieved by techniques such as a manufacturing service bus. Furthermore, an integration layer such as a manufacturing service bus allows for an integration with Internet of Things (IoT) devices and Cyber-physical Systems (CPS) within a smart factory [67]. Additionally, the Manufacturing Execution System (MES) is included as the central system of manufacturing control. The ACSN controller supplements the manufacturing control level and integrates the ACSN (cf. Section 3.2) with its RL-based controlling element (cf. Section 3.3) into the architecture.  
On the enterprise control level, the systems to control the company (enterprise resource planning) and supply network (supply chain management) are found. Consequently, this also represents the main interface to external companies.  
The ACSN's decision on how to respond to events is made at the manufacturing control level. However, data from the enterprise control level (events) and the manufacturing level (manufacturing situation) are explicitly taken into account and included in the decision. The decision-making process is further supported by the data layer and the digital twin. Going beyond the architecture of classical event processing systems, a learning system in the data layer--besides events--also needs the history of production plans, manufacturing states and its model parameters. These data are fed by the systems of the three levels mentioned above. The digital twin addresses the need for an RL agent for a safe environment to perform learning while at the same time use data and system behavior as close as possible to the real environment. Therefore, semantics and algorithms are used to preprocess data of the data layer for simulation models representing physical and functional characteristics of the manufacturing system.