# Digital Twin and Reinforcement Learning-Based Resilient Production Control for Micro Smart Factory
## 1 Introduction  
Personalized production has become the core paradigm in manufacturing research owing to the need for highly diversified products [1, 2]. Customized products with affordable quality, cost, and delivery can be manufactured via this production process to meet customer requirements [1, 2, 3, 4, 5]. To realize this personalized production, the following three limitations need to be addressed: access, cost, and performance hurdles [2, 3, 6, 7, 8]. Among these hurdles, cost and performance are closely correlated. The access hurdle pertains to the difficulty in accurately judging customer needs through customer interaction; cost hurdle includes increase in cost due to more complex manufacturing systems; and performance hurdle involves performance degradation caused by the complexity of the production process, dynamic situation, and increased preparation time [2, 3, 6, 7, 8, 9, 10]. Additionally, personalized production needs to employ make to order (MTO) entirely or partly. Because the MTO production environment cannot handle inventory, which allows managing fluctuations within certain margins, it is necessary to address these limitations [7, 10, 11]. Modular manufacturing systems (MMSs) enable the management cost hurdles, and the concept of resilience helps overcome the performance hurdles [12, 10, 13, 14]. The realization of MMS is expected to restructure the manufacturing system rapidly and easily and enable personalized production of highly diversified products [14]. In addition, theMMS has an advantage in terms of the cost of product change in production and is suitable for frequent changes related to production [14,15]. Moreover, the elements in the physical work center are managed by a module with independent functional units [14]. Resilience is a characteristic that corresponds to robustness and prevents the degradation of performance indicators. Impermissible events that can cause bullwhip and ripple effects are managed under the resilient production control [10,12,15]. Resilience needs to satisfy five core functional requirements for handling the events: (1) action selection, (2) key performance indicator (KPI) measurement, (3) monitoring, (4) fluctuation notification, and (5) adjustment [12]. In the production control perspective, requirements 1, 2, and 5 correspond to achieving resilience [10]. By configuring the MMS for personalized production, the micro smart factory (MSF) produces personalized products [2,5]. The MSF acts as the work center for production of personalized products that are requested by factory as a service (FaaS) platform [2,5,16]. Additionally, the MSF operates with cyber-physical production system (CPPS) to achieve resilience [7]. CPPS establishes and revises the production plan and schedule, provides time-machine monitoring, and extracts off-line programming (OLP) codes to the physical MSF [5,7]. These technical functionalities satisfy the five core requirements for achieving resilience and support the production control [7]. Although the MSF overcomes the cost hurdle and CPPS solves the performance hurdle in the personalized production perspective, there are still limitations that need to be solved. MMS often reconstructs its configuration to accommodate product change and the control-related characteristics [14,17,18]. To enable efficient production operation of MMS, these limitations must be overcome [17,18]. In addition, three functional requirements, as mentioned above, are necessary to achieve resilience [10]. Thus, the production control of MMS needs to select an action with restructured configuration, measure the KPI, and adjust the rule for selecting action with revised plans and schedules. In particular, adjusting the rule to select action needs to be executed when the impermissible events are detected, and the reactive plan and schedule are established for handling such events. To consider the abovementioned roles, the dispatching rule for production control is also changeable.