## 4 System Implementation and Experimental Results
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Type** & **Function** & **Detailed Function** & **Configurations** \\ \hline \multirow{4}{*}{Panel} & Application of & - & Iteration/parameters’ setting. & - & Define an iteration \\  & competitive network & - & Defense/attack range. & - & By defining the defense \\  & (e.g., Soccer Game). & - & Button generation. & - & range and attack range, \\  & & & Button generation. & - & specify the range of the \\ \hline \multirow{4}{*}{Panel} & \multirow{4}{*}{Network topology.} & - & Monitoring players’ & - & Representation of a \\  & & & network topology. & - & coexisting network topology \\  & & - & Monitoring system’s reward. & - & only with current status of \\  & & & & & both players’ group. \\ \hline \multirow{4}{*}{Panel} & Players’ statue network topology distribution & - & Monitoring both players’ (e.g., Blue and Read team) & - & Representation of a \\  & (2-dimension). & & (e.g., Blue and Read team) & - & coexisting network topology \\  & & (2-dimension). & network topology. & - & with current status and the \\ \hline \multirow{4}{*}{Panel} & Players’ statue network topology distribution & - & Representing 3-dimensional & - & Representation of probability \\  & (3-dimension). & & network topology. & - & distributions depicting the \\ \hline \multirow{4}{*}{Window} & \multirow{4}{*}{Adaptive human evaluation.} & \multirow{4}{*}{-} & \multirow{4}{*}{
\begin{tabular}{} \end{tabular} } & - & The network topology \\  & & & & learned and evaluated by \\  & & - & Input window for human & - & humans in each iteration \\  & & - & reward evaluation. & - & It is updated by the adaptive \\  & & - & Calculation of adaptive & human evaluation strategy \\  & & - & human evaluation. & - & between the estimated \\  & & & & values and the current \\  & & & & evaluated values in \\  & & - & & the iteration. \\ \hline \multirow{4}{*}{Window} & \multirow{4}{*}{Reward calculation and updates.} & - & Analyzing & - & The rewards obtained for \\  & & - & integrated reward. & - & each learning are calculated \\ \cline{1-1}  & & - & Measurement of & - & The average value is \\ \cline{1-1}  & & - & system performance. & - & updated, and system \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Implemented Cooperative Human–Machine Evaluation Software Program. Table 4 shows how the proposed cooperative human-machine evaluation framework differs in learning performance according to the degree of human intervention. The results are summarized  
Figure 6: Comparisons of three evaluation methods, (**a**) the proposed cooperative human–machine. Evaluation framework, (**b**) human–machine evaluation with binary human evaluation, and (**c**) traditional Markov decision process with human evaluation. according to the parameter of the degree of human intervention rate \(\delta\). The problem addressed in this experiment is that two networks each have 10 nodes and coexist in the learning environment simultaneously. The problem is solved in three cases by applying the proposed cooperative human-machine evaluation framework. The first case solved the problem only with the system reward of the network topology existing inside the environment without any human intervention. As the training iterations were repeated, the reward value increased, and the system learned to converge to the maximum; however, the performance was poor and needed enough iterations. On the other hand, in the second case, if the policy for updating reward is updated by applying the reward calculated through the system's internal reward and the proposed cooperative human-machine evaluation framework at a 50% rate, convergence tended to occur quickly to the maximum reward value. In the last case, where a 90% rate was applied, the highest Q-function value was obtained at the same number of learning while converging to the maximum reward value as the target. This is shown by comparison in Figure 7.  
These results may vary depending on the assumed conditions and parameters.