SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yan Zhuang $^{1,2}$ , Qi Liu $^{1,2}$ ; GuanHao Zhao $^{1,2}$ , Zhenya Huang $^{1,2}$ , Weizhe Huang $^{1,2}$ , Zachary A. Pardos $^{3}$ , Enhong Chen $^{1,2}$ , Jinze Wu $^{2,4}$ , Xin Li $^{2,4}$
1: Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China
2: State Key Laboratory of Cognitive Intelligence
3: University of California, Berkeley
4: iFLYTEK Co., Ltd
{zykb, ghzhao0223, hwz871982879, hxwjz}  $@$  mail.ustc.edu.cn,
{qiliuql,huangzhy,cheneh,leexin}@ustc.edu.cn, pardos@berkeley.edu
Computerized adaptive testing (CAT), as a tool that can efficiently measure student's ability, has been widely used in various standardized tests (e.g., GMAT and GRE). The adaptivity of CAT refers to the selection of the most informative questions for each student, reducing test length. Existing CAT methods do not explicitly target ability estimation accuracy since there is no student's true ability as ground truth; therefore, these methods cannot be guaranteed to make the estimate converge to the true with such limited responses. In this paper, we analyze the statistical properties of estimation and find a theoretical approximation of the true ability: the ability estimated by full responses to question bank. Based on this, a Bounded Ability Estimation framework for CAT (BECAT) is proposed in a data-summary manner, which selects a question subset that closely matches the gradient of the full responses. Thus, we develop an expected gradient difference approximation to design a simple greedy selection algorithm, and show the rigorous theoretical and error upper-bound guarantees of its ability estimate. Experiments on both real-world and synthetic datasets, show that it can reach the same estimation accuracy using  $15\%$  less questions on average, significantly reducing test length.
As the landscape of education is changing rapidly, especially after COVID-19, many schools and institutions move from in-class to online platforms, providing individualized education, such as educational measurement and recommendation. They are looking to "right-size" the learning experience of students according to their ability level [1, 2]. To this end, Computerized Adaptive Testing (CAT) [3] becomes an indispensable tool to efficiently measure student's ability in the areas of standardized testing, computer tutoring, and online courses, through automatically selecting best-suited questions for individual students. Compared with the time-consuming and burdensome paper-and-pencil tests, CAT has been proven to require fewer questions to reach the same measurement accuracy [4, 2].
A typical CAT system is shown in Figure 1: At test step  $t$ , the Cognitive Diagnosis Model, e.g., Item Response Theory (IRT), as the user model based on psychology, first uses student's previous  $t$  item answer responses to estimate his/her current ability  $\theta^t$ . IRT family has been used for ability estimation in several state assessments, such as OECD/PISA Project [5, 6]. Next, the selection algorithm selects the next item from the entire question bank according to some criteria [7, 8, 9]. Most of them are
*Corresponding Author.
Figure 1: An illustration of the CAT system: At test step  $t \in [1, \dots, T]$ , the selection algorithm uses the current ability estimate  $\theta^t$  to select the next question  $q_{t+1}$  from the question bank. When the test stops, the  $\theta^T$  (i.e., the final estimate of his/her true ability  $\theta_0$ ) will be output.
informativeness metrics such as selecting the question with difficulty closest to his/her current ability estimate  $\theta^t$ , i.e., the student's probability of answering it correctly is closest to  $50\%$  [7]. Obviously, the selection algorithm is the core component to realize CAT's adaptivity and seeks to answer the following question about accuracy and efficiency: Can we estimate student's true ability by asking him/her as few questions as possible, with negligible estimation error?
From the perspective of machine learning, CAT can be viewed as a parameter estimation problem with the least cost: it is essentially to select the fewest data samples (questions to be answered) sequentially from the whole unlabeled data (question bank), so that after obtaining their labels (correct/wrong responses), model's hidden parameters (student true ability  $\theta_0$ ) can be accurately estimated. Unfortunately, the exact true ability of student is unknown even to the students themselves, thus it is impossible to find such ground truth in datasets to design/train selection algorithms. As a result, most selection algorithms are not designed explicitly with the goal of accurate and efficient estimation. Existing approaches either select representative/diverse items solely from question feature space [9] (deviating from the goal of ability estimation), or require additional training overhead (e.g., Reinforcement Learning-based methods [10, 11, 12, 13]). Although these implicit methods achieve good results in experiments, the theoretical guarantee on approximating student's true ability is also critical for reliable CAT systems especially in standardized tests.
Obviously, the biggest challenge of designing reliable explicit methods is: student's true ability  $\theta_0$  is unknown. Therefore, in this work, we propose a general (upper-)Bounded Ability Estimation CAT framework (BECAT), which explicitly targets the accuracy and efficiency of ability estimation. Due to the unknown  $\theta_0$ , we first find its theoretical approximation  $\theta^*$  as the alternative: the ability estimated by his/her full responses on the entire question bank. Hence, our key idea is to select questions such that the estimate can best approximate the ability estimated by full responses. Specifically, we propose an expected gradient difference approximation method based on recent data efficiency/summary technique [14, 15, 16], and design a practical greedy selection algorithm in submodular function, which essentially finds representative items to approximate the gradient of full responses. We further provide the theoretical analysis about its upper-bound of ability estimation error.
To validate BECAT's effectiveness, we conduct experiments on three real-world datasets from different educational platforms. Empirical results show that this simple greedy selection achieves state-of-the-art performance compared with other implicit methods. The main contributions are:
- To better estimate the unknown  $\theta_0$ , we find its theoretical approximation as the new target for designing an explicit selection algorithm. Based on this, we formally redefine and transform CAT into an adaptive subset selection problem in data summary manner for the first time.
- An effective expected gradient-based selection algorithm is proposed to select appropriate items, which exactly minimizes the estimation error term, therefore admitting theoretical guarantees on ability estimation in CAT systems.
- We show the generality of BECAT — it can be applied to any gradient-based method, including IRT and neural network methods. We observe that BECAT outperforms existing CAT methods at reducing test length, requiring  $10\% - 20\%$  less questions to reach the same estimation accuracy.
2 Problem Definitions of CAT
For accurate and efficient assessment, CAT needs to sequentially select best-fitting questions for each student from the question bank  $Q$ ; then uses the corresponding responses for ability estimation.
When the test stops, the final estimate is output as the result/score of this test. The goal of CAT is to accurately estimate examinee's true ability  $\theta_0$ , while minimizing the number of questions asked [17].
2.1 Preliminaries
Specifically, at test step  $t \in [1,2,\dots,T]$ , given the student's previous  $t$  responses  $S_{t} = \{(q_{1},y_{1}),\ldots ,(q_{t},y_{t})\}$ , where  $\{q_i\}_{i = 1}^t \subseteq Q$  are selected sequentially by the selection algorithm and  $y$  is the binary outcomes of correct or incorrect; student's current ability can be estimated by minimizing the empirical risk (e.g., binary cross-entropy) from the whole ability space  $\Theta$ :
$$
\theta^ {t} = \underset {\theta \in \Theta} {\arg \min } \sum_ {i \in S _ {t}} l _ {i} (\theta) = \underset {\theta \in \Theta} {\arg \min } \sum_ {i \in S _ {t}} - \log p _ {\theta} \left(q _ {i}, y _ {i}\right), \tag {1}
$$
where  $p_{\theta}(q_i, y_i)$  represents the probability of the response  $(q_i, y_i)$  towards a student with  $\theta$ , and the specific form of  $p_{\theta}$  is determined by IRT. Since the size of  $S_t$  is small, Standard Gradient Descent [18, 19] is sufficient to minimize Eq.(1), and requires the computations of  $\sum_{i \in S_t} \nabla l_i(\theta) - \sum_{i \in S_t} \nabla l_i(\theta)$  — sum of the gradients over the previous  $t$  response data. It takes repeated steps in the opposite direction of the gradient, thus leading to a minimum of the empirical risk in Eq.(1).
Next, the selection algorithm selects the next question  $q_{t + 1}$  from bank  $Q$  according to various criteria [7, 8, 12, 13]. The above process will be repeated for  $T$  times², i.e.,  $|S| = T$  ( $T \leq 20$  in most tests [13]), ensuring the final step estimate  $\theta^T$  close to the true  $\theta_0$ , i.e.,
Definition 1 (Traditional Definition of CAT). At each step  $t$ , it will select the most suitable/informative question, according to student's current ability  $\theta^t$ . When the test ends ( $t = T$ ), the final ability estimate  $\theta^T = \arg \min_{\theta \in \Theta} \sum_{i \in S} l_i(\theta)$  can approximate the true ability:
$$
\min  _ {| S | = T} \left\| \theta^ {T} - \theta_ {0} \right\|. \tag {2}
$$
Unfortunately, directly solving the above optimization problem is infeasible. Because the ground truth ability  $\theta_0$  cannot be obtained and, even students themselves cannot know the exact value. As a result, traditional informativeness-based methods [7, 8] use asymptotic statistical properties of Maximum Likelihood Estimation to reduce estimation uncertainty, e.g., selecting the one whose difficulty is closest to student's current ability  $\theta^t$ ; but they are all IRT-specific, i.e., they can not be applied into recent neural networks methods. Although recent active learning-based [9] and reinforcement learning-based [12, 13] methods achieve good experimental results, there is no evidence that they can theoretically guarantee that estimate can efficiently approach  $\theta_0$ , which is unacceptable for CAT systems applied in standardized tests. Testing reliability requires not only satisfactory experimental results, but also good theoretical guarantees [3].
2.2 New Definition of CAT
Given that there is no such ground truth  $\theta_0$  in the dataset, thus, for designing explicit selection algorithms, we find its approximation as the new target.
Proposition 1. The student's one ability estimate  $\theta^{*}$ , estimated by his/her full responses to the entire question bank  $Q$ , is an approximation of his/her true ability  $\theta_{0}$ , that is,
$$
\theta^ {*} \approx \theta_ {0} \tag {3}
$$
Proof. When we use consistent estimation approaches, such as Maximum Likelihood Estimation (cross-entropy loss) in Eq.(1), we have  $\lim_{t\to \infty}p(|\theta^t -\theta_0|\geq \epsilon) = 0$ , where  $t$  is the number of responses (steps) for ability estimation. The size of CAT's question bank is finite (i.e.,  $t\in [0,|Q|]$ ) and  $\theta^{*} = \lim_{t\to |Q|}\theta^{t}$ , thus  $\theta^{*}\approx \lim_{t\to \infty}\theta^{t}\approx \theta_{0}$ , i.e.,  $\theta^{*}$  can be regarded as an approximation of  $\theta_0$ .
Since this proposition exploits estimator's asymptotic property and may make the approximation not perfect. For example, both the bank size  $|Q|$  and various perturbations in student's response
2In this paper, we only consider the most common fixed-length tests  $(T = 20)$ , and the effect of test length can be found in Appendix E
Figure 2: (a) Simulation experiments about Proposition 1 using MSE:  $\mathbb{E}[\| \theta^{*} - \theta_{0}\|^{2}]$ . In addition to the normal situation (blue), we also show the MSE under different perturbations, for example: Slip  $5\%$  means that the label has a  $5\%$  probability of changing from 1 to 0; Guess  $25\%$  means that the label changes from 0 to 1 with  $25\%$ . (b) The illustration of the optimization problem: selecting subset  $S$  to cover the whole response data on  $Q$ . The rectangles represent a student's full responses to the bank  $Q$ , and  $w(i,j)$  measures the similarity of response pair  $(i,j)$ .
can impact this proposition. Thus, we also conduct simulation experiments to further verify it: we randomly sample  $100\theta_0$  from  $\Theta$  as groundtruth, using the smallest EXAM dataset  $(|Q| = 1650)$  in Section 4; then use IRT with these  $\theta_0$  to simulate the response behavior (correct/wrong) of 100 students. Figure 2(a) shows that when the bank size exceeds  $300\left(\approx |Q| / 5\right)$ , the estimated  $\theta^{*}\approx \theta_{0}$  (blue). Even if some extreme perturbations (e.g., guess and slip factors [13]) are added, their MSE will not exceed 0.1. Therefore, it is reasonable to replace  $\theta_0$  with  $\theta^{*}$  as the ground truth in optimization.
In this way, the selection algorithm can aim to approach  $\theta^{*}$  instead of the unknown  $\theta_0$ . We can design explicit selection algorithms: Select a subset of questions  $S$  from the bank  $Q$ , so that the student's ability is estimated only on the subset  $S$  while still (approximately) converging to the optimal solution  $\theta^{*}$  (i.e., the estimate that would be obtained if optimizing on the full responses to  $Q$ ). As mentioned in Preliminaries, the ability estimation usually adopts cross-entropy loss with gradient computations, and denote the full gradient of the loss w.r.t. ability parameters by  $\sum_{i\in Q}\nabla l_i(\theta)$  — sum of the gradients over full responses. Thus,
Definition 2 (New Definition of CAT). It will adaptively find a subset  $S$  of size  $T$  and the corresponding weight  $\{\gamma\}_{j}$  that approximates the full gradient: minimizing their difference for all the possible ability values of the optimization parameter  $\theta \in \Theta$ :
$$
\min  _ {| S | = T} \| \theta^ {T} - \theta^ {*} \| \Rightarrow \min  _ {| S | = T} \max  _ {\theta \in \Theta} \| \sum_ {j \in S} \gamma_ {j} \nabla l _ {j} (\theta) - \sum_ {i \in Q} \nabla l _ {i} (\theta) \|. \tag {4}
$$
Since we know nothing about the range of student's ability when optimizing, we consider the worst-case approximation error  $(\max_{\theta \in \Theta})$  instead of a particular  $\theta$ . After finding such  $S$  and the associated weights  $\{\gamma\}_{j}$ , the gradient updates on  $S$  will be similar to the gradient on  $Q$  regardless of the value of  $\theta$ , thus making the estimate close to the target  $\theta^{*}$ . In this way, CAT can be regarded as a subset selection optimization problem in a data-efficiency manner. Also, we find that it is consistent with recent Coreset techniques [20, 15, 21], which approximate the gradients of the full data, so that the model is trained only on the subset while still (approximately) converging to the optimal solution (i.e., the model parameters that would be obtained if training on the full data).
However, compared with the traditional Coreset problem, the biggest technical challenge is: the gradients on bank  $(\sum_{i\in Q}\nabla l_i(\theta))$  cannot be calculated without labels. Only the few questions that have been answered in previous steps (i.e.,  $S_{t}$ ) have the corresponding labels. Thus, to simplify the problem, we assume for the moment that student's full responses are available. In Section 3.1, we will further propose an expected approximate method to address this.
3 The BECAT framework
In this section, to solve the above optimization problem Eq.(4), we design a simple greedy algorithm in submodular functions. More importantly, we provide an upper bound on the expected error of the ability estimate when using our method.
**Optimization.** The above subset selection problem is NP-hard, thus, we transform it based on the recent Coreset method. It proves that the subset  $S$  that minimizes the error of estimating the full gradient is upper-bounded by a submodular facility location function that has been used in various summarization applications [22, 23]. Thus
$$
\begin{array}{l} \min _ {| S | = T} \max _ {\theta \in \Theta} \| \sum_ {j \in S} \gamma_ {j} \nabla l _ {j} (\theta) - \sum_ {i \in Q} \nabla l _ {i} (\theta) \| \Rightarrow \min _ {| S | = T} \max _ {\theta \in \Theta} \sum_ {i \in Q} \min _ {j \in S} \| \nabla l _ {i} (\theta) - \nabla l _ {j} (\theta) \| \\ \Rightarrow \max  _ {| S | = T} \sum_ {i \in Q} \max  _ {j \in S} w (i, j), \tag {5} \\ \end{array}
$$
where  $w(i,j) \triangleq d - \max_{\theta \in \Theta} \| \nabla l_i(\theta) - \nabla l_j(\theta) \|$  is the gradient similarity between response pair  $i = (q_i, y_i)$  and  $j = (q_j, y_j)$  for this student. The associated weight of the response  $j$ ,  $\gamma_j = \sum_{i \in Q} \mathbf{1}[j = \arg \max_{s \in S} w(i,s)]$ , is the number of responses in  $Q$  that are most similar to  $j \in S$ .
Given a subset  $S$ ,  $\sum_{i \in Q} \max_{j \in S} w(i, j)$  in Eq.(5) quantifies the coverage of the whole response data on  $Q$ , by summing the similarities  $w$  between every  $i \in Q$  and its closest item  $j \in S$ . The semantics of this optimization problem is shown in Figure 2(b). The larger the value of  $w(i, j)$ , the smaller their gradient difference in ability estimation for all the possible ability  $\theta \in \Theta$ , which means these two responses  $i$  and  $j$  have similar importance/influence on the student's ability estimation. Thus, the transformed problem in Eq.(5) is equivalent to selecting the most representative responses to form the subset  $S$ , which shares the same idea (i.e., selecting "representative" items) with previous selection algorithms [12, 9], active learning methods [24, 25] and unsupervised learning [26, 27].
Define a monotone non-decreasing submodular function — the facility location function  $F: 2^{Q} \to \mathbb{R}$ :  $F(S) = \sum_{i \in Q} \max_{j \in S} w(i, j)$ . The submodular optimization provides a near-optimal solution with a  $(1 - 1/e)$ -approximation bound [28], with simple greedy algorithm for selecting the  $t$ -th question:
$$
q _ {t} = \arg \max  _ {(q, y) \in Q \backslash S _ {t - 1}} \Delta ((q, y) | S _ {t - 1}). \tag {6}
$$
where  $\Delta((q, y) | S_{t-1}) = F(\{(q, y)\} \cup S_{t-1}) - F(S_{t-1})$  and  $S_{t-1}$  is the set of previous  $t-1$  responses of this student in CAT.
3.1 Expected Gradient Difference Approximation
However, the above selection algorithm is impractical in CAT. Because we cannot get student's full responses to bank  $Q$ , as a result, the gradient difference  $\| \nabla l_i(\theta) - \nabla l_j(\theta) \|$  in  $w(i,j)$  can not be calculated without related answer correctness labels. Actually, at step  $t$ , only the responses of previous  $t-1$  steps (i.e.,  $S_{t-1}$ ) can be obtained. Therefore, we propose an expected gradient difference approximation method to replace the original to measure their similarity, then the new similarity function  $\widetilde{w}(i,j)$  is:
$$
\widetilde {w} (i, j) \triangleq d - \max  _ {\theta \in \Theta} \mathbb {E} _ {y \sim p _ {\theta t}} \left[ \| \nabla l _ {i} (\theta) - \nabla l _ {j} (\theta) \| \right], \tag {7}
$$
where the normed gradient difference is calculated as an expectation  $\mathbb{E}_{y\sim p_{\theta t}}$  over the possible labelings, since student's response labels  $y$  to the candidate questions are unknown in the selection step. Moreover, for more accurate approximation and to make full use of the available previous  $t - 1$  responses, in Eq.(7), the expectation is determined by the current estimate  $\theta^t$ . This method can be regarded as a gradient difference approximation based on "soft pseudo-labels". Thus, the selection of the next question  $q_{t}$  no longer requires the student's real answer correctness labels:
$$
q _ {t} = \arg \max  _ {q \in Q \backslash S _ {t - 1}} \Delta (q | S _ {t - 1}). \tag {8}
$$
where  $\Delta(q|S_{t-1}) = \widetilde{F}(\{q\} \cup S_{t-1}) - \widetilde{F}(S_{t-1})$ , and  $\widetilde{F}(S) = \sum_{i \in Q} \max_{j \in S} \widetilde{w}(i,j)$ . Also, we uncover some important conclusions about this simple expected approximation:
Algorithm 1: The BECAT framework
Require:  $Q$  - question bank,  $f$  - IRT or neural network methods.
Initialize: Initialize the responses data  $S_0 \gets \emptyset$ .
1 for  $t = 1$  to  $T$  do
2 Select question  $q_{t}$  based on  $\widetilde{w}(i,j)$ :  $q_{t} \gets \arg \max_{q \in Q \setminus S_{t-1}} \Delta(q|S_{t-1})$ .
3 Get student's related answer correctness label  $y_{t}$  ..  $\bar{S}_t\gets S_{t - 1}\cup \{(q_t,y_t)\} .$
4 Update the weights  $\{\gamma_j\}_{j = 1}^t$  ..  $\gamma_{j}\gets \sum_{i\in Q}\mathbf{1}[j = \arg \max_{s\in S_{t}}\widetilde{w} (i,s)]$
5 Update student's ability estimate:  $\theta^t\gets \arg \min_{\theta \in \Theta}\sum_{i\in S_t}\gamma_i l_i(\theta).$
Output: The student's final ability estimate  $\theta^T$ .
Lemma 1. When we replace the original gradient difference in  $w(i,j)$  with  $\widetilde{w}(i,j)$ , the corresponding designed selection algorithm using submodular function  $\widetilde{F}$  is actually approximately solving the following optimization problem:
$$
\min  _ {| S | = T} \max  _ {\theta \in \Theta} \mathbb {E} _ {y} \left[ \| \sum_ {j \in S} \gamma_ {j} \nabla l _ {j} (\theta) - \sum_ {i \in Q} \nabla l _ {i} (\theta) \| \right] \tag {9}
$$
Based on the conclusion in Lemma 1, we can assume that, after optimization, the preconditioned expected gradient can be approximated by an error of at most  $\epsilon$ :  $\mathbb{E}_y\left[\| \sum_{i\in S}\gamma_j\nabla l_j(\theta) - \sum_{i\in Q}\nabla l_i(\theta)\|\right]\leq \epsilon$ . Then we find the theoretical guarantees for ability estimation when applying gradient-based estimation method to the subset  $S$  found by it:
Theorem 1 (Expected estimation error bound). Assume that the loss function for ability estimation is  $\alpha$ -strongly convex (e.g.,  $IRT$ ). Let  $S$  be a weighted subset obtained by the proposed method. Then with learning rate  $\frac{1}{\alpha}$ , ability estimation in gradient descent applied to the subsets has the following expected estimation error bound:
$$
\mathbb {E} \left[ \| \theta^ {t + 1} - \theta^ {*} \| ^ {2} \right] \leq \frac {2 \epsilon D \alpha + \sigma_ {l} ^ {2} + 2 \sigma_ {f} D \alpha H _ {p} \left(\theta^ {t} , \theta^ {*}\right)}{\alpha^ {2}} \tag {10}
$$
$$
w h e r e \quad H _ {p} \left(\theta^ {t}, \theta^ {*}\right) = \mathbb {E} _ {\left(q, y\right) \sim p _ {\theta^ {t}}} \left[ \frac {1}{p _ {\theta^ {*}} (q , y)} \right] \tag {11}
$$
where  $\theta^{*}$  is the optimal estimate using full responses,  $\sigma$  is an upper bound on the norm of the gradients, and  $D = \max_{\theta} \| \theta - \theta^{*} \|$ .
All the proofs can be found in Appendix. The above theorem shows that despite not being able to obtain student's full response, such simple expected gradient difference approximation can make the estimate error upper bounded at each step. This theorem is attainable for the case where the loss is strongly convex, such as the cross-entropy loss of the classic L2-regularized IRT [29]. We will further verify the performance of other cases (e.g., neural network-based methods) in experiments.
Theorem 1 also suggests that, to minimize the expected error bound, the CAT systems should try to minimize  $H_{p}(\theta^{t},\theta^{*})$  that can be regarded as a type of statistical distance: measuring how probability distribution  $p_{\theta^t}$  is different from  $p_{\theta^*}$ . Moreover, we find that with the help of the consistency estimation (i.e., binary cross-entropy) at each step,  $H_{p}(\theta^{t},\theta^{*})$  can reach its theoretical minimum: