SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Assuming gradients have a bounded norm  $\| \nabla l_{j}(\theta)\| \leq \sigma_{l}$  and  $\| \nabla f_j(\theta)\| \leq \sigma_f$ . Thus, from reverse triangle inequality, we can write
$$
\mathbb {E} \left[ \| g _ {S} ^ {t} \| ^ {2} \right] = \mathbb {E} \left[ \| \sum_ {j \in S} \gamma_ {j} \nabla l _ {j} \left(\theta^ {t}\right) \| ^ {2} \right] \leq \sigma_ {l} ^ {2}. \tag {25}
$$
From Lemma 1, we can assume that the subset  $S$  and corresponding per-element weights  $\gamma_{j}$  can approximate the full gradient with an expected error at most  $\epsilon >0$ , i.e.,  $\mathbb{E}_{y\sim p_{\theta^t}}\left[\| \sum_{i\in Q}\nabla l_i(\theta) - \sum_{i\in S}\gamma_j\nabla l_j(\theta)\|\right]\leq \epsilon$ . Thus, from reverse triangle inequality  $\mathbb{E}[\| g_S^*\| ]\leq \mathbb{E}[\| g^*\| ] + \epsilon$  and  $\mathbb{E}[\| g^{*}\| ]$  can be further derived as follows:
$$
\begin{array}{l} \mathbb {E} [ \| g ^ {*} \| ] = \frac {1}{| Q |} \mathbb {E} \left[ \| \sum_ {i \in Q} \nabla l _ {i} (\theta^ {*}) \| \right] \leq \frac {1}{| Q |} \sum_ {i \in Q} \mathbb {E} \left[ \| \nabla l _ {i} (\theta^ {*}) \| \right] \\ = \frac {1}{| Q |} \sum_ {i \in Q} \mathbb {E} _ {y _ {i} \sim p _ {\theta t}} \left[ \| \nabla_ {\theta = \theta^ {*}} (- y _ {i} \ln f _ {i} (\theta) - (1 - y) \ln (1 - f _ {i} (\theta))) \| \right] \\ = \frac {1}{| Q |} \sum_ {i \in Q} \mathbb {E} _ {y _ {i} \sim p _ {\theta t}} \left[ \| - \frac {y _ {i}}{f _ {i} \left(\theta^ {*}\right)} \nabla f _ {i} \left(\theta^ {*}\right) + \frac {1 - y _ {i}}{1 - f _ {i} \left(\theta^ {*}\right)} \nabla f _ {i} \left(\theta^ {*}\right) \| \right] \\ = \frac {1}{| Q |} \sum_ {i \in Q} \| \nabla f _ {i} \left(\theta^ {*}\right) \| \mathbb {E} _ {y _ {i}} \left| \frac {y _ {i}}{f _ {i} \left(\theta^ {*}\right)} - \frac {1 - y _ {i}}{1 - f _ {i} \left(\theta^ {*}\right)} \right| \\ = \frac {1}{| Q |} \sum_ {i \in Q} \| \nabla f _ {i} \left(\theta^ {*}\right) \| \sum_ {y \in \{0, 1 \}} \frac {p _ {\theta^ {*}} \left(q _ {i} , y _ {i} = y\right)}{p _ {\theta^ {*}} \left(q _ {i} , y _ {i} = y\right)} \\ = \frac {1}{| Q |} \sum_ {i \in Q} \| \nabla f _ {i} \left(\theta^ {*}\right) \| \mathbb {E} _ {\left(q _ {i}, y _ {i}\right) \sim p _ {\theta^ {t}}} \left[ \frac {1}{p _ {\theta^ {*}} \left(q _ {i} , y _ {i}\right)} \right] \\ \leq \sigma_ {f} \mathbb {E} _ {(q, y) \sim p _ {\theta^ {t}}} \left[ \frac {1}{p _ {\theta^ {*}} (q , y)} \right], \tag {26} \\ \end{array}
$$
where  $p_{\theta}(q,y)$  is the response distribution of the student with the ability  $\theta$ , and  $f_{i}(\theta) = p_{\theta}(q_{i},y_{i} = 1)$  is the output of IRT. Also assuming that  $\| \theta - \theta^{*} \| \leq D$ , we have,
$$
\begin{array}{l} \mathbb {E} \left[ \| g _ {S} ^ {*} \| \| \theta^ {t} - \theta^ {*} \| \right] \leq D \mathbb {E} [ \| g ^ {*} \| ] + \epsilon D \\ \leq \sigma_ {f} D \mathbb {E} _ {(q, y) \sim p _ {\theta t}} \left[ \frac {1}{p _ {\theta *} (q , y)} \right] + \epsilon D. \tag {27} \\ \end{array}
$$
Combining the Equation (24) to (26) and let  $\eta = \frac{1}{\alpha}$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \| \theta^ {t + 1} - \theta^ {*} \| ^ {2} \right] \leq \frac {2 \epsilon D \alpha + \sigma_ {l} ^ {2}}{\alpha^ {2}} + \frac {2 \sigma_ {f} D}{\alpha} \mathbb {E} _ {(q, y) \sim p _ {\theta^ {t}}} \left[ \frac {1}{p _ {\theta^ {*}} (q , y)} \right] \\ = \frac {2 \epsilon D \alpha + \sigma_ {l} ^ {2} + 2 \sigma_ {f} D \alpha H _ {p} \left(\theta^ {t} , \theta^ {*}\right)}{\alpha^ {2}}, \tag {28} \\ \end{array}
$$
where  $H_{p}(\theta^{t},\theta^{*}) = \mathbb{E}_{(q,y)\sim p_{\theta^{t}}}\left[\frac{1}{p_{\theta^{*}}(q,y)}\right]$ . So we complete the proof.
C Proofs of Theorem 2
Theorem 2. Assume that  $\theta^t$ , estimated by the cross-entropy loss, can minimize the empirical risk i.e.,  $\sum_{i\in S_t}l_i(\theta^t) = 0$ . Then  $H_{p}(\theta ,\theta^{*})$  can take its minimum when  $\theta = \theta^t$ , that is
$$
H _ {p} \left(\theta^ {t}, \theta^ {*}\right) \leq H _ {p} \left(\theta , \theta^ {*}\right), \quad \forall \theta \in \Theta \tag {29}
$$
Proof. When we use the binary cross-entropy (BCE) loss to estimate the ability  $\theta$  and minimize the empirical risk at step  $t$ , we have
$$
\begin{array}{l} \theta^ {t} = \arg \min _ {\theta} \sum_ {i \in S _ {t}} l _ {i} (\theta) \\ = \arg \max  _ {\theta} \sum_ {i \in S _ {t}} y _ {i} \log f _ {i} (\theta) + (1 - y _ {i}) \log (1 - f _ {i} (\theta)) \\ = \arg \max  _ {\theta} \sum_ {i \in S _ {t}} \log p _ {\theta} \left(q _ {i}, y _ {i}\right) \\ \approx \arg \max  _ {\theta} \mathbb {E} _ {(q, y) \sim p _ {\theta_ {0}}} \log p _ {\theta} (q, y), \tag {30} \\ \end{array}
$$
where  $f_{i}(\theta) = p_{\theta}(q_{i},y_{i} = 1)$  is the output of the IRT. We argue that the student's response to the question  $q$  is determined by the true ability  $\theta_0$  in the CAT process, i.e.,  $y_{i} = \arg \max_{y}p_{\theta_{0}}(q_{i},y)$ . Therefore, when the above empirical risk achieves its minimum, i.e.,  $\sum_{i\in S_t}l_i(\theta^t) = 0$ ,  $\mathbb{E}_{(q,y)\sim p_{\theta_0}}\log p_\theta (q_i,y_i)$  can reach the maximum 0, and the real and predicted responses are the same:
$$
y _ {\max } = \arg \max  _ {y} p _ {\theta^ {t}} (q, y) = \arg \max  _ {y} p _ {\theta_ {0}} (q, y), \forall q \tag {31}
$$
and the predicted probability should approach 1:  $p_{\theta^t}(q, y_{max}) \to 1$ .
Next, we discuss the minimization of the statistical distance  $H_{p}(\theta^{t},\theta^{*}) = \mathbb{E}_{(q,y)\sim p_{\theta^{t}}}[1 / p_{\theta^{*}}(q,y)]$  term in the upper bound in Theorem 1. In general, make now the simplifying assumption that distribution  $p_{\theta^*}(q,y)$  is smooth and has a global maximum probability  $p_{max}$ , attained at a point  $(q,y_{max})$  for question  $q$ , so that  $1 / p_{\theta^{*}}(q,y)$  has a global minimum at the same  $y_{max}$ . Obviously, we can choose:
$$
p _ {\theta^ {t}} (q, y) = \delta (y - y _ {\max },) \tag {32}
$$
where  $\delta$  is the delta function [59], making  $\mathbb{E}_{(q,y)\sim p_{\theta^t}}[1 / p_{\theta^*}(q,y)]$  to its minimum. Based on the above findings, in the CAT situation (binary classification), the optimal distribution  $p_{\theta^t}$  (minimizing the  $H_{p}$  in the upper bound) needs to: have the same classification result as  $p_{\theta^*}$ , i.e.,  $y_{max} = \arg \max_y p_{\theta^t}(q,y) = \arg \max_y p_{\theta^*}(q,y) \approx \arg \max_y p_{\theta_0}(q,y)$ , and its corresponding probability is as large as possible, i.e.,  $p_{\theta^t}(q,y_{max}) \to 1$ . In this case,  $H_{p}(\theta^{t},\theta^{*})$  can take its minimum. All of these findings are consistent with the conclusion that minimizing BCE loss for ability estimation in Eq.(30) and (31). So we complete the proof.
Figure 4: The illustration of the distance measurements in similarity function  $\widetilde{w}(i,j)$ : replace the entire ability space  $\Theta$  with the student's possible ability estimates  $\Theta_{t} = \{\theta_{i}^{t}\}_{i=1}^{m}$  to reduce search space and speed up the selection.
D Implementation Details of BECAT
To improve its complexity, we provide two implementation tricks:
(1) Lazy Evaluations. By exploiting the submodularity, we use the lazy evaluations approach presented in [30, 31] to speed up the selection process and make the running time faster in practice. At step  $t$ , the greedy selection algorithm must identify the question  $q$  with maximum marginal gain  $\Delta(q|S_{t-1})$ . Instead, this lazy method involves using a max-heap  $O(1)$  lookup and  $O(\log(n))$  insertion) to keep an upper bound on the gain of each question that comes from submodularity, i.e., the marginal benefits of any question  $q \in Q$  are monotonically nonincreasing during the selection:
$$
\Delta (q \mid S _ {t - 1}) \geq \Delta (q \mid S _ {t}) \quad \forall q \in Q. \tag {33}
$$
Instead of recomputing  $\Delta(q|S_{t-1})$  at each step for each element  $q \in Q$  (requiring  $O(|Q|)$  computations), the accelerated lazy algorithm maintains a list of upper bounds  $\Delta'(q)$  (initialized to  $\infty$ ) on the marginal gains sorted in decreasing order (max-heap order). Specifically, at each step, the algorithm first selects the maximal from this ordered list, i.e., the top of the heap. It then updates this bound  $\Delta'(q) \gets \Delta(q|S_{t-1})$  in the heap. As soon as the  $\Delta'(q)$  is still at the top of the heap, then submodularity Eq.(33) guarantees that  $\Delta(q|S_{t-1}) \geq \Delta(q'|S_{t-1})$  for all  $q' \neq q$ , and therefore we do not need to evaluate any more items. If it does not satisfy this condition, we just insert it with  $\Delta'(q)$  as the new upper bound and repeat the above procedure until the qualified question is selected. While the worst case is the same, in practice this method has enormous speedups over the standard greedy algorithm [60].
(2) Reducing the Ability Space  $\Theta$  In our method, note that the similarity function  $\widetilde{w}(i,j) = d - \max_{\theta \in \Theta} \mathbb{E}_{y \sim p_{\theta t}}[\|\nabla l_i(\theta) - \nabla l_j(\theta)\|]$  requires finding the worst case over the entire ability parameter space  $\Theta$ , which is too expensive for CAT systems. In fact, it only needs to be calculated in each student's own possible ability space. In other words, we can find an ability subspace  $\Theta_t \subseteq \Theta$  specialized to each student. We utilize a novel MLE-based estimation approach [32], which models student's multifaceted nature and sequentially generates a set of possible abilities  $\Theta_t = \{\theta_i^t\}_{i=1}^m$  at each step  $t$ :
$$
\theta_ {i} ^ {t} = \arg \min  _ {\theta_ {i}} \sum_ {i \in S _ {t}} l _ {i} \left(\theta_ {i}\right) - \frac {\lambda}{2} \left\| \theta_ {i} - \bar {\theta} _ {i} \right\| ^ {2} \quad f o r i = 1, \dots , m, \tag {34}
$$
where  $\bar{\theta}_i$  is the average of previous  $i - 1$  estimates and the term  $\left\| \theta_i - \bar{\theta}_i\right\|^2$  ensures the diversity of abilities in  $\{\theta_i^t\}_{i = 1}^m$ . We refer the reader to [32] for more details about this estimation method. As shown in Figure 4 we replace the entire parameter space in the optimization problem with student's  $m$  potential estimates  $\Theta_t = \{\theta_i^t\}_{i = 1}^m$ . The complexity can be reduced to  $O(|Q|^2 m)$  and  $m \ll |\Theta|$ .
E Details of Experiment
The impact of test length. We use simulation experiments to verify the choice of max length  $T$ : Due to the unknown of the true ability  $\theta_0$ , we artificially generate it and conduct the Simulation
Figure 5: Simulation experiments of ability estimation using MSE:  $\mathbb{E}[\| \theta^t -\theta_0\| ^2 ]$
of Ability Estimation experiment on the EXAM dataset using the mean square error  $\mathbb{E}[\| \theta^t -\theta_0\| ^2 ]$  between the ability estimate  $\theta^t$  at each step and the true ability  $\theta_0$  (Figure 5): The classic Fisher method can reduce the evaluation error quickly and 20 is sufficient for the length of a typical adaptive test. Thus, we fix the max length  $T = 20$
Experimental Implementation Details. As 20 is sufficient for the length of a typical test [13], we also fix the max length  $T = 20$ . We implement all the methods with PyTorch. We set batch size to 64 and the learning rate to 0.001, and optimize all the parameters using the Adam algorithm [61] on a Tesla V100-SXM2-32GB GPU.
E.1 Statistics of the datasets
Table 2: Statistics of the datasets
Dataset	ASSIST	NIPS-EDU	EXAM
#Students	20,704	220,274	9,214
#Questions	15,071	27,613	1,650
#Response logs	1,768,253	19,181,192	133,398
#Response logs per student	85.41	87.08	14.48
#Response logs per question	117.33	694.64	80.85
E.2 Detailed Evaluation Method
The goal of CAT is to estimate the student's ability accurately with the fewest steps. However, since the true ability cannot be obtained as the ground truth, there are usually two tasks to verify the performance of different CAT methods following previous works [9, 12]: 1) Student Score Prediction and 2) Simulation of Ability Estimation:
1) Student Score Prediction. To evaluate the ability estimate output by the CAT system, this estimate can be used for predicting the student's scores (correct or wrong) on the questions he/she has answered in the held-out response data. This is an indirect evaluation method. Following the common strategy [12], we use  $70\% - 20\% - 10\%$  students for training, validation, and testing respectively, and the students in validation/testing set won't appear in training. The training set is used for initializing some question's parameters in IRT (e.g., difficulty parameter in IRT), and the data-driven selection algorithm baselines (e.g., BOBCAT). In the validation/testing, the responses of each student  $i$  are further divided into the candidate  $(Q_{i})$  and meta  $(M_{i})$  question sets to simulate CAT procedure, following [9, 12]. Specifically, at each step, (1) different selection algorithms first select a question from  $Q_{i}$ ; (2) IRT then updates the ability estimate with his/her response to it; (3) evaluate this estimate's accuracy by predicting binary-valued responses (correct or wrong) on the held-out meta set  $M_{i}$ . This task assumes that the more accurate the score prediction is, the more accurate the ability
Figure 6: Comparison of the time consumed by different methods at the selection step. The implementation tricks provide  $2 \times$  to  $3 \times$  speedup.
estimate is. This task considers that the accuracy of score prediction can reflect the accuracy this ability estimates. Thus, from this binary classification perspective, we use Prediction Accuracy (ACC) [46] and Area Under ROC Curve (AUC) [62] for the evaluation of different selection algorithms.
2) Simulation of Ability Estimation. This is CAT's traditional evaluation method [2]. Since the ground truth of student ability  $\theta_0$  is not available, we artificially generate their  $\theta_0$  and further simulate student-question interaction process within CAT systems. For the rationality of the generated  $\theta_0$ , we use all the students' responses in the dataset to estimate their ability  $\{\theta_0^1, \theta_0^2, \dots, \theta_0^N\}$  as the ground truth [9, 63]. Also, the dataset is used to learn questions' parameters and fix them. Different from the first task, such settings can simulate students with  $\theta_0$  responding to any question in  $Q$ , thus the candidate question in selection is the entire bank  $Q$ . Specifically, (1) different selection algorithms first select a question from the entire bank  $Q$ ; (2) IRT then updates the estimate with the response to it; (3) evaluate this estimate's accuracy by computing the difference between the estimated and the true ability. In this way, we can use Mean Square Error (MSE) to evaluate the accuracy of estimation.
E.3 Implementation Tricks for Speedups
To solve the BECAT optimization problem, we need to calculate the gradients of all items in question bank, leading to high computation requirements for large datasets/examinations. To this end, we consider two speed-up tricks: lazy evaluations and multifaceted estimation. Lazy evaluations take advantage of submodularity to avoid calculating the conditional gain  $\Delta(q|S_{t-1})$  of all the candidate items. The multifaceted estimation method [32] can effectively reduce the ability space when calculating the similarity between two items, thus reducing the time to calculate  $\Delta(q|S_{t-1})$  of each item. In Figure 6, we compare the time (second) spent on question selection by different methods<sup>3</sup>, and find that the proposed implement tricks give the average speedup of  $3\times$ , and achieve  $7\times$  to recent MAAT. And it is almost the same time as the traditional informativeness-based method KLI. This demonstrates that our greedy selection algorithm in BECAT is fast in practice.
E.4 BECAT Analysis
In this section, we will further analyze its selection time/latency, the effectiveness of Theorem 1 and 2, and the characteristics of questions selected by BECAT, respectively.
Upper Bound Analysis of Estimation In Theorem 1, the most important component is  $H_{p}(\theta^{t},\theta^{*})$ , which determines the upper bound on estimation error and the convergence behavior of the proposed BECAT method. Therefore, to verify the effectiveness and theoretical guarantees of our explicit selection algorithm, i.e., the estimation error bound conclusion in Theorem 1, we compute  $\bar{H}_{p}$  for each step on the strongly convex loss function: the cross-entropy loss of the L2-regularized IRT. This experiment is still based on the simulation setting in Appendix E.2 and the results are shown in
3There is no comparison here of reinforcement learning methods, due to the extra training overhead they require.
(b)
Figure 7: (a) The value of  $H_{p}(\theta^{t},\theta^{*})$  in the upper-bound of estimation. (b) Normed difference between the full gradient on  $|\dot{Q}|$  and the gradient of the question subset found by different methods.
Figure 7(a): In the first few steps, the  $H_{p}(\theta^{t},\theta^{*})$  can rapidly decrease to the minimum. As the test progresses, the ability estimate tends to be accurate (Figure 7(a)), and the upper-bound  $H_{p}$  remains near the minimum, which reflects the good convergence behavior of BECAT. This demonstrates that our expected gradient difference approximation is reliable in practice.
Gradient Approximation. In Proposition 1 we show that  $\theta^{*}$  is an approximation of student true ability. To approach the new target  $\theta^{*}$ , an approximation to the full gradient of the bank  $Q$  is required. Figure 7(b) demonstrates the norm of the difference between the weighted gradient of the questions subset found by BECAT and the full gradient. This figure also compares the normed gradient difference of other subsets found by other methods where each response is weighted by  $|Q| / |S|$ . The gradient difference is calculated by sampling the full gradient at various points in the parameter space. Note that the gradient difference obtained by BECAT decreases significantly with the increase of  $t$  and it is much smaller than that of other methods, which proves that the expected gradient difference approximation method is accurate. Combining the experimental results on the two tasks in Experiments, the better the prediction/estimation performance of the method, the smaller the gradient difference. This demonstrates that the closeness to  $\theta^{*}$  reflects the closeness to the true ability  $\theta_{0}$ , which further proves the rationality of using  $\theta^{*}$  as a new target.
Table 3: The variance results of different methods on ACC and AUC metrics.
(a) Variance results on ASSIST
CDM	IRT	NeuralCDM
Metric@Step	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20
Random	0.0378/0.0462	0.0090/0.0226	0.0052/0.0142	0.0064/0.0072	0.0013/0.0042	0.0012/0.0038
FSI	0.0446/0.0257	0.0147/0.0076	0.0067/0.0024	-	-	-
KLI	0.0163/0.0058	0.0042/0.0037	0.0023/0.0019	-	-	-
MAAT	0.0121/0.0202	0.0082/0.0150	0.0162/0.0286	0.0123/0.0073	0.0083/0.0276	0.0253/0.0242
BOBCAT	0.0120/0.0152	0.0052/0.0057	0.0165/0.0148	0.0054/0.0066	0.0047/0.0024	0.0126/0.0023
NCAT	0.0063/0.0041	0.0065/0.0055	0.0012/0.0007	0.0032/0.0031	0.0037/0.0029	0.0035/0.0020
BECAT	0.0100/0.0136	0.0062/0.0040	0.0055/0.0023	0.0022/0.0022	0.0019/0.0011	0.0012/0.0010
(b) Variance results on NIPS-EDU
CDM	IRT	NeuralCDM
Metric@Step	ACC / AUC@5	ACC / AUC@10	ACC / AUC@20	ACC / AUC@5	ACC / AUC@10	ACC / AUC@20
Random	0.0185 / 0.0406	0.0183 / 0.0431	0.0182 / 0.0478	0.0185 / 0.0406	0.0183 / 0.0431	0.0182 / 0.0478
FSI	0.0270 / 0.0350	0.0269 / 0.0357	0.0274 / 0.0401	-	-	-
KLI	0.0231 / 0.0315	0.0218 / 0.0278	0.0196 / 0.0247	-	-	-
MAAT	0.0207 / 0.0347	0.0232 / 0.0377	0.0256 / 0.0412	0.0192 / 0.0298	0.0216 / 0.0306	0.0253 / 0.0363
BOBCAT	0.0203 / 0.0311	0.0185 / 0.0267	0.0169 / 0.0247	0.0197 / 0.0315	0.0190 / 0.0289	0.0200 / 0.0310
NCAT	0.0178 / 0.0246	0.0159 / 0.0214	0.0142 / 0.0198	0.0176 / 0.0258	0.0163 / 0.0232	0.0169 / 0.0246
BECAT	0.0216 / 0.0294	0.0185 / 0.0248	0.0169 / 0.0225	0.0204 / 0.0304	0.0167 / 0.0251	0.0165 / 0.0225
(c) Variance results on EXAM
CDM	IRT	NeuralCDM
Metric@Step	ACC / AUC@5	ACC / AUC@10	ACC / AUC@20	ACC / AUC@5	ACC / AUC@10	ACC / AUC@20
Random	0.0171 / 0.0189	0.0114 / 0.0136	0.0084 / 0.0086	0.0025 / 0.0107	0.0071 / 0.0102	0.0070 / 0.0090
FSI	0.0195 / 0.0112	0.0097 / 0.0099	0.0093 / 0.0069	-	-	-
KLI	0.0195 / 0.0112	0.0097 / 0.0099	0.0076 / 0.0098	-	-	-
MAAT	0.0193 / 0.0115	0.0104 / 0.0094	0.0082 / 0.0065	0.0191 / 0.0112	0.0182 / 0.0099	0.0053 / 0.0062
BOBCAT	0.0036 / 0.0133	0.0070 / 0.0114	0.0086 / 0.0092	0.0142 / 0.0087	0.0095 / 0.0091	0.0012 / 0.0021
NCAT	0.0033 / 0.0146	0.0005 / 0.0004	0.0004 / 0.0004	0.0160 / 0.0120	0.0148 / 0.0090	0.0003 / 0.0003
BECAT	0.0196 / 0.0121	0.0022 / 0.0006	0.0020 / 0.0007	0.0172 / 0.0134	0.0063 / 0.0012	0.0090 / 0.0029