## 3 Proposed Method
### Reward Definition  
A reward is defined based on the processing time of a machine and the travel time to a destination using the current position of the RTU supplying the material. The objective of this study is to schedule each task such that the total time is minimized. As shown  
Figure 8: Reinforcement learning action.  
Figure 7: Defining final state of reinforcement learning.  
in Figure 9, long processing and material transfer times imply a small reward. Hence, a reward can be expressed as an inverse or negative value with respect to time.  
As shown in Figure 10, a sum of negative numbers is calculated as work progresses when a reward is a negative number. If no action is taken and no separate reward is set for waiting, a zero reward is obtained, which is greater than a negative value. In this case, the agent may possibly learn that it can wait without any action to receive a higher reward. Conversely, using a reverse reward is easier than additionally considering an appropriate reward when waiting for work. Therefore, in this study, a reward is defined in the reverse order of time.  
Figure 10: Cumulative reward difference according to the type of reward.  
Figure 9: Relationship between event time and reward.  
Another important consideration when designing rewards is whether a reward should be penalized. If a reward is given as an inverse of the event time for all actions, learning is compromised, because rewards for irrelevant actions that do not contribute to achieving the objective accumulate. The primary action in this study is to move the RTU with a material buffer to specific machines for processing. Therefore, an instance in which the RTU is moved to a specific machine, but the material cannot be processed, is regarded as a penalty.  
Figure 11 shows a case in which materials are to be input into a machine that is currently processing. As mentioned in Section 1, because a machine can only process one material at a time, in this case, material processing cannot proceed. A case related to the buffer state is shown in Figure 12. Figure 12 depicts the case in which the action of moving to a machine for processing has occurred. This requires the presence of the material in the previous process completion buffer; however, in the current state, processing cannot proceed because the buffer is empty. Subsequently, it is necessary to retrieve a processed material from a machine and move it to the relevant process completion buffer. However, as shown in the case in Figure 12, the processed material cannot be retrieved because the buffer is full. In this buffer state, instructing an action to move the RTU will not lead to material processing. If the material can be processed, it is given a reward because it is the right choice. Conversely, if all the machines in the group are in operation and the material cannot be allocated, this is a wrong choice, and it penalizes them because processing cannot proceed.  
Figure 11: Reward penalty according to machine state.  
Figure 12: Reward penalty for machine material input/recovery: (**a**) Penalty for inputting machine materials; (**b**) penalty for recovering machine materials.