# Learning Ad Hoc Cooperation Policies from Limited Priors via Meta-Reinforcement Learning
## 4 Method
### Teammate Identification
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{**Average**} & **Affiliated** \\  & **Win Rate (\%)** & **Caught Prey ID** & **Number of Steps** & **Known Repository** \\ \hline Type 01 & 92.00 & 0 & \(58.35\pm 43.46\) & \(N=2\), \(3\), \(4\) \\ Type 02 & 100.00 & 3 & \(37.67\pm 20.76\) & \(N=2\), \(3\), \(4\) \\ Type 03 & 81.00 & 2 & \(13.67\pm 2.91\) & \(N=3\), \(4\) \\ Type 04 & 57.00 & 1 & \(93.53\pm 63.53\) & \(N=4\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance metrics for the known teammate types. The success rates of completing tasks, the IDs of prey captured, and the average number of steps required to complete tasks are not completely identical across different types. The final column specifies the category of the teammate type repository to which each teammate type is affiliated. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{**(a)** Quantity of types} \\ \hline  & \(\Theta\) & \(\Theta^{\prime}\) & \(\Theta_{T}\) \\ \hline \(N=2\) & 2 & 37 & 67 \\ \(N=3\) & 3 & 38 & 67 \\ \(N=4\) & 4 & 39 & 67 \\ \hline \hline \multicolumn{3}{c}{**(b)** Composition of type repository} \\ \hline  & \(\mathbf{In}\Theta\) & \(\mathbf{In}\Theta^{\prime}\) & \(\mathbf{NFIn}\Theta^{\prime}\) \\ \hline FEAT & \(\Theta^{\prime},\Theta\) & \(\Theta^{\prime},\Theta^{\prime}\) & \(\Theta^{\prime},\Theta_{T}\) \\ PLASTIC-\(\Theta\) & \(\Theta,\Theta\) & \(\Theta,\Theta^{\prime}\) & \(\Theta,\Theta_{T}\) \\ PLASTIC-\(\Theta^{\prime}\) & \(\Theta^{\prime},\Theta\) & \(\Theta^{\prime},\Theta^{\prime}\) & \(\Theta^{\prime},\Theta_{T}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Explanation of type repositories varies in different experimental settings. Table (**a**) delineates the number of types provided and the respective quantities of types contained within three distinct type repositories. Table (**b**) depicts the combinations of testing type repositories provided to the ad hoc agent and potential teammate types during the online testing phase for three algorithms across various teammate configurations. \begin{table}
\begin{tabular}{c c c c c c c c c}  & \multicolumn{2}{c}{\(N=2\)} & \multicolumn{2}{c}{\(N=3\)} & \multicolumn{2}{c}{\(N=4\)} & \multicolumn{2}{c}{\(N=5\)} \\ \cline{2-9}  & **InoD** & **InoD** & **NFInoD** & **InoD** & **InoD** & **NFInoD** & **InoD** & **NFInoD** \\ \hline FEAT & \(52.05\pm 27.23\) & \(28.27\pm 3.22\) & \(26.89\pm 4.10\) & \(49.57\pm 30.55\) & \(27.94\pm 3.29\) & \(26.08\pm 2.00\) & \(46.48\pm 15.58\) & \(31.08\pm 5.14\) & \(27.97\pm 2.89\) \\ PAISTIC-O & \(72.6\pm 49.96\) & \(51.66\pm 5.86\) & \(42.30\pm 5.08\) & \(60.88\pm 49.49\) & \(47.21\pm 8.13\) & \(41.90\pm 4.00\) & \(48.32\pm 7.80\) & \(46.16\pm 9.15\) & \(43.36\pm 7.46\) \\ PAISTIC-O & \(68.08\pm 35.99\) & \(36.59\pm 5.91\) & \(28.34\pm 3.25\) & \(50.38\pm 2.402\) & \(33.36\pm 3.93\) & \(30.31\pm 2.57\) & \(60.69\pm 2.86\) & \(36.16\pm 6.70\) & \(29.95\pm 4.11\) \\ ESORON & \(13.00\pm 34.76\) & \(103.33\pm 14.43\) & \(167.01\pm 11.73\) & \(27.68\pm 35.92\) & \(146.36\pm 15.77\) & \(17.34\pm 15.77\) & \(153.58\pm 3.48\) & \(107.81\pm 5.99\) & \(105.03\pm 10.73\) \\ Random & \(151.30\pm 14.73\) & \(171.73\pm 14.73\) & \(167.01\pm 11.73\) & \(146.75\pm 35.60\) & \(144.65\pm 17.57\) & \(170.104\pm 11.73\) & \(115.35\pm 35.54\) & \(107.015\pm 2.209\) & \(107.10\pm 11.73\) \\ \end{tabular}
\end{table} 
Table 4: The average step lengths and associated standard deviations for successful task completions by various algorithms.