## 5 Analysis of Simulation Experiments
### Performance Analysis  
Figure 5 compares the performance of the implemented RL-based frameworks. Observing this figure, one can indicate that the RL/ad hoc control mechanisms demonstrate better performance in terms of profitability. Specifically, the two-stage manufacturing system is considerably profitable when controlled by RL(EK-CBM) and RL(BS-CBM). It is evident that the coupling of EK, BS, and CBM enhances the decision-making process of the two-agents by including awareness regarding customer demand, waste produced by low-quality items and potential maintenance activities. Another remark can be made concerning the similar performance demonstrated by the implemented versions of the proposed reinforcement learning mechanism. This could be attributed to the exploited parametric manufacturing control policies. For example, let us examine the performance of RL(EK-PM) and RL(BS-PM). Utilizing the same maintenance policy, that is, PM, they incorporate different manufacturing policies, namely EK and BS. The exploited manufacturing policies, as described in Section 4.3, implement a similar functionality, as EK is formed according to BS and KANBAN. In this regard, both policies authorize production operations taking into consideration the incoming orders. As a result, their performance is not substantially dissimilar. An analogous performance is displayed by frameworks, which exploit diverse parametric control policies. For instance, RL(\((s,s)\), CM) and RL(KANBAN-CBM) obtain nearly identical revenues. Clearly, the amalgamation of these policies considerably improves the efficiency of the manufacturing system in terms of manufacturing and maintenance operations. As KANBAN and \((s,S)\) control manufacturing process on the basis of inventory levels, CM and CBM maintain the system according to the observed failures and a predefined threshold, respectively. It is likely that the maintenance operations are regularly authorized, as the equipment is frequently deteriorated due to the manufacturing operations. As a result, the system scarcely finds itself in downtime and the product stock is in good condition. Finally, RL(R-Learning) does not display the same awareness as that demonstrated by its counterparts. This is behavior is attributed to its decision-making agents, which are merely dependable on the state of their environment. In that case, the system would generate a stock of products that would not be remanufactured, or sold to customers. Moreover, it is assumable that maintenance activities are recurring in the system \begin{table}
\begin{tabular}{c c c} \hline \hline
**Mechanisms** & **Ad Hoc Production** & **Ad Hoc Maintenance** \\ \hline RL(EK-CBM) & EK & CBM \\ \hline RL(EK-CM) & EK & CM \\ \hline RL(EK-PM) & EK & PM \\ \hline RL(BS-PM) & BS & PM \\ \hline RL(BS-CM) & BS & CM \\ \hline RL(BS-CBM) & BS & CBM \\ \hline RL(\((s,s)\)-CBM) & \((s,S)\) & CBM \\ \hline RL(\((s,S)\)-PM) & \((s,S)\) & PM \\ \hline RL(\((s,S)\)-CM) & \((s,S)\) & CM \\ \hline RL(KANBAN-CBM) & KANBAN & CBM \\ \hline RL(KANBAN-PM) & KANBAN & PM \\ \hline RL(KANBAN-CM) & KANBAN & CM \\ \hline RL(CBM) & N/A & CBM \\ \hline RL(PM) & N/A & PM \\ \hline RL(CM) & N/A & CM \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Employed policies in the RL-based mechanisms. improving the system's condition. Although, they are significantly costly when they are conducted in later degradation stages. Figure 6 illustrates the profits attained by best-performing versions of the proposed mechanism compared to those obtained by ones integrating only maintenance policies. A clear indication is that the inclusion of production policies, such as EK, improves the sustainability of the manufacturing system, when the customers recurrently place orders and the manufacturing activities are not scarce. In that case, the stock is often replenished with a high variety of new products. Thus, the system satisfies the customers, whether the products are standard, remanufactured, or refurbished.