## 3 Proposed Method
### Experimental Results  
The concepts adopted in this study can be summarized as follows. First, a partial environmental method was established to reduce the size of the state and action compared to that of the overall environment. Second, an inverse reward, rather than a negative reward, was used. Third, regarding the timing of a reward, instead of giving a reward at the end of an actual event, a reward was given in advance by predicting the end of an event as soon as it occurs. To prove the above, the experiments were conducted in the following order. First, an experiment using the entire environmental space was conducted, and a method that involves giving inverse rewards and rewards at the time of completion of an action was used. The algorithm was compared to both the DQN and PPO algorithms. Based on the results, the PPO method had much better indicators than the DQN method; thus, the former was used in all subsequent experiments. To prove the first concept, the improvement in performance achieved using a partial environmental space compared to the performance achieved when the entire environmental space is employed was determined. Moreover, inverse rewards and rewards at the time of completion of the actions were used. To prove the second concept, the effectiveness of an inverse reward compared to that of a negative reward was examined. Similar to the above experiment, a partial environmental space was used, and rewards were given after the end of an event. An experiment was also conducted using a negative reward instead of an inverse reward. Finally, a partial environmental space and an inverse reward were used to prove the concept, and a method of predicting and giving rewards in advance at the start of the event, instead of awarding them at the end of an event, was adopted. Figure 14 shows the learning results based on the first experimental environment. Subsequently, the total processing time required was measured and compared with that in the next experiment to verify the scheduling performance of each method. Three existing heuristic algorithms (proximity first, first-in-first-out (FIFO), and process first) and the reinforcement learning algorithm were used in the experiments on the entire and partial environments with inverse and negative rewards and method of reward timing. The total processing completion time of the reinforcement learning model was measured and used as an indicator. The proximity-priority scheduling method is a method of delivering materials to idle machines closest to the RTU in real time. The FIFO method delivers materials to each process machine in the order of materials currently contained in the buffer. Finally, the process-first scheduling method focuses on moving a material to the next process. Tables 4 and 5 show the results of measurements on scheduling performance by method in the two experimental environments. Figure 14: Results of learning by each method. (**a**) Trend of rewards per learning time step; (**b**) trend of cumulative rewards per learning time step; (**c**) trend of total machining completion time per learning time step. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multirow{2}{*}{**Space**} & \multirow{2}{*}{**Reward**} & \multicolumn{1}{c}{**Learning Time Step**} & \multicolumn{1}{c}{**} & \multicolumn{1}{c}{**} & \multicolumn{1}{c}{**Learning Time Step**} & \multic