## 8 Numerical Examples
For chatbot 1, in the first round, the first row of matrix \(M_{1}\) (i.e., the first row of Table 3) was filled in with the coefficients \(Q_{i,1}\), \(i=1\), \(\ldots\), \(N\). Then, all the locations were (unsuccessfully) scanned once, after which the \(Q_{i^{\star}(1),1}=max_{i}Q_{i,1}=Q_{4,1}=0.5112\) and the corresponding location \(i^{\star}(1)=4\) were found. Next, during this round, chatbot 1 scanned location 4 once more; we assumed that in this example, it did not discover the relevant target information there. Then, the number of unsuccessful scans of this location increased to 2, and the corresponding parameter \(Q_{4,2}\) was calculated according to (4), (equals 0.0204) and placed in the cell (2,4) of matrix \(M_{2}\). All other elements of the second row of this matrix were left unchanged--the same as in the previous row of \(M_{1}\). The first round of the algorithm was completed, and then the second round started. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{1}{c}{**Round**Location} & **1** & **2** & **3** & **4** & **5** & **6** & \(\mathbf{\pi^{2}}\) & \(F_{2}(\mathbf{\pi_{2}})\) \\ \hline
1 & 0.3001 & 0.2032 & 0.2928 & **1.1980** & 0.2744 & 0.3507 & 4 & 0.1140 \\ \hline
2 & 0.3001 & 0.2032 & 0.2928 & 0.1198 & 0.2744 & **0.3507** & 6 & 0.2389 \\ \hline
3 & **0.3001** & 0.2032 & 0.2928 & 0.1198 & 0.2744 & 0.1227 & 1 & 0.2816 \\ \hline
4 & 0.0900 & 0.2032 & **0.2928** & 0.1198 & 0.2744 & 0.1227 & 3 & 0.3208 \\ \hline
5 & 0.0900 & 0.2032 & 0.1171 & 0.1198 & **0.2744** & 0.1227 & 5 & 0.3402 \\ \hline
6 & 0.0900 & **0.2032** & 0.1171 & 0.1198 & 0.0549 & 0.1227 & 2 & 0.3476 \\ \hline
7 & 0.0900 & 0.0203 & 0.1171 & 0.1198 & 0.0549 & **0.1227** & 6 & 0.3562 \\ \hline
8 & 0.0900 & 0.0203 & 0.1171 & **0.1198** & 0.0549 & 0.0430 & 4 & 0.3574 \\ \hline
9 & 0.0900 & 0.0203 & **0.1171** & 0.0120 & 0.0549 & 0.0430 & 3 & 0.3591 \\ \hline
10 & 0.0900 & 0.0203 & 0.0187 & 0.0120 & **0.0549** & 0.0430 & 5 & 0.35931 \\ \hline
11 & **0.0900** & 0.0203 & 0.0187 & 0.0120 & 0.0110 & 0.0430 & 1 & 0.359696 \\ \hline
12 & 0.0027 & 0.0203 & 0.0187 & 0.0120 & 0.0110 & **0.0430** & 6 & 0.360275 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Calculations of the optimal policy for chatbot 2. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{1}{c}{**Round**Location} & **1** & **2** & **3** & **4** & **5** & **6** & \(\mathbf{\pi^{1}}\) & \(F_{1}(\mathbf{\pi_{1}})\) \\ \hline
1 & 0.2287 & 0.1073 & 0.1952 & **0.5112** & 0.1543 & 0.3237 & 4 & 0.0486 \\ \hline
2 & 0.2287 & 0.1073 & 0.1952 & 0.0204 & 0.1543 & **0.3237** & 6 & 0. In round 2, the next location to be scanned was selected by computing \(Q_{i^{\star}(2),2}=\max_{i}Q_{i,2}=Q_{6,2}=0.3237\), and hence, the location \(i^{\star}(2)=6\) was chosen. \(Q_{i^{\star}(2),2}\) was placed in cell (2,6) of the second row of matrix \(M_{2}\), and this item was highlighted in bold. Next, during this round, chatbot 1 scanned location 6 once more; we assumed that the human user was not satisfied by the target information extracted there. Then, the number of unsuccessful scans of this location increased to 2; the corresponding parameter \(Q_{6,3}\) was calculated according to (4) (equals 0.0204) and placed in cell (3,6) of the next matrix \(M_{3}\). All other elements of the third row of this matrix were left unchanged--the same as in the previous row of \(M_{2}\). The second round was completed. During the subsequent rounds, the algorithm sequentially filled in the next rows 3, 4,..., 12 of Table 3, following the instructions of the algorithm, as in round 2. We assumed that the human user was satisfied by the target information retrieved in the 12 th round. The results obtained for each chatbot over 12 rounds were as follows: (1) The optimal policies for chatbots 1 and 2 were equal to \(\boldsymbol{\pi}^{\star(1)}\) = (4, 6, 1, 3, 5, 2, 6, 1, 3, 4, 6, 5) and \(\boldsymbol{\pi}^{\star(2)}\) = (4, 6, 1, 3, 5, 2, 6, 4, 3, 5, 1, 6), respectively.