SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 2. Assume that  $\theta^t$ , estimated by the cross-entropy loss in Eq.(1), can minimize the empirical risk i.e.,  $\sum_{i\in S_t}l_i(\theta^t) = 0$ . Then  $H_{p}(\theta ,\theta^{*})$  can take its minimum when  $\theta = \theta^t$ , that is
$$
H _ {p} \left(\theta^ {t}, \theta^ {*}\right) \leq H _ {p} \left(\theta , \theta^ {*}\right), \quad \forall \theta \in \Theta \tag {12}
$$
Therefore, the ability estimation methods commonly used in CAT can actually help minimize this upper bound. The proofs and related experiments can be found in the Appendix Cand E.4.
Complexity Analysis of BECAT. Algorithm 1 presents the pseudo-code of our BECAT framework. A naive implementation of our selection algorithm in Eq.(8) has the complexity of  $O(|Q|^2 |\Theta|)$ , because at each step we have to: (1) find the question from the bank  $O(|Q|)$  that (2) maximizes
the marginal gain  $\Delta (q|S_{t - 1}) = \widetilde{F} (\{q\} \cup S_{t - 1}) - \widetilde{F} (S_{t - 1})$  with complexity  $O(|Q||\Theta |)$ . To make BECAT faster and more scalable from the above two aspects, we adopt two speed-up tricks: lazy evaluations [30, 31] and multifaceted estimation [32] (See Appendix D for implementation details). Also, we compare the time (second) spent on question selection by different methods in Appendix E.
Evaluation Method. The goal of CAT is to estimate the student's ability accurately with the fewest steps. Therefore, there are usually two tasks to verify the performance of different CAT methods following prior works [9, 12]: (1) Student Score Prediction: To evaluate the ability estimate output by CAT, the estimate can be used for predicting the student's binary response (correct/wrong) on the questions he/she has answered in the held-out response data. Thus, Prediction Accuracy (ACC) and AUC are used for evaluations [33]; (2) Simulation of Ability Estimation: This is CAT's traditional evaluation methods. Since the ground truth of student ability  $\theta_0$  is not available, we artificially generate the  $\theta_0$  and further simulate student-question interaction process. Thus, we can use Mean Square Error (MSE) metric. See Appendix E for the details of these two evaluation methods.
Datasets. We conduct experiments on three educational benchmark datasets, namely ASSIST, NIPS-EDU, and EXAM. ASSIST [34] is collected from an online educational system ASSISTments and consists of students' practice logs on mathematics. NIPS-EDU [35] refers to the large-scale dataset in NeurIPS 2020 Education Challenge, which is collected from students' answers to questions from Eedi (an educational platform). The EXAM dataset was supplied by iFLYTEK Co., Ltd., which collected the records of junior high school students on mathematical exams. The statistics of the datasets are shown in appendix. The code can be found in the github: https://github.com/bigdata-ustc/EduCAT.
Compared Approaches. To verify the generality of BECAT, in addition to the traditional IRT, we also compare the neural network-based model NeuralCDM [36]: It can cover many IRT and cognitive diagnosis models, such as MIRT [37] and MF [38, 39]. For the selection algorithm, we mainly use the following SOTA algorithms as baselines: Random: The random selection strategy is a benchmark to quantify the improvement of other methods; FSI [7] and KLI [8] select the question with the maximum Fisher/Kullback-Leibler information, which measures the amount of information that a question carries about the unknown parameter  $\theta$ . They are specially designed for IRT. MAAT [9] utilizes Active Learning [40] to measure the uncertainty caused by each candidate question. BOBCAT [12] and NCAT [13] recast CAT as a bilevel optimization and Reinforcement Learning problem respectively, and train a data-driven selection algorithm from student response data.
4.1 Results and Discussion
In this section, we compare the performance on two classic CAT tasks introduced above to evaluate the effectiveness and efficiency of our proposed BECAT framework. Also, we conduct a qualitative investigation of the characteristics of the selected questions, and gain deeper insights on why BECAT leads to more accurate ability estimation.
Task1: Student Score Prediction. Following prior work [13], we also fix the max length  $T = 20$  and calculate the ACC and AUC at step 5, 10 and 20 on three datasets for Student Score Prediction task and the results are shown in Table 3. We find that:
(1) The explicit BECAT framework achieves the best overall performances on the three datasets. It performs significantly better than all the other methods, where the relative performance improvements are as least  $1.5\%$  with respect to ACC@20 and  $1.1\%$  with respect to AUC@20 on average on ASSIST. This result indicates that BECAT can provide accurate ability estimates at the end of the exam. Also, it even surpasses the implicit selection algorithms based on deep learning, such as NCAT and BOBCAT. This phenomenon shows that compared to focusing on modeling complex student-question interactions, targeting the accuracy of estimation indeed achieves amazing results.
(2) BECAT's performance on large-scale datasets (e.g., NIPS-EDU) is better. From Table 3, on NIPS-EDU dataset (the bank size is 27613), BECAT can achieve  $2.48\%$  AUC gain (on average) above the famous FSI baseline. On the other two datasets ASSIST and EXAM, the average improvement is only
Table 1: The performance of different methods on Student Score Prediction with ACC and AUC metrics. “-” indicates the information/uncertainty-based selection algorithms (e.g., FSI) cannot be applied to the deep learning method. The boldfaced indicates the statistically significant improvements (p-value  $< 0.01$ ) over the best baseline.
(a) Performances on ASSIST
CDM	IRT	NeuralCDM
Metric@Step	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20
Random	71.01/70.68	72.20/71.91	73.07/72.61	71.52/71.19	72.66/72.06	72.67/72.83
FSI	71.77/71.33	72.94/72.48	73.24/73.54	-	-	-
KLI	71.93/71.38	72.73/72.52	73.17/73.57	-	-	-
MAAT	72.20/71.54	72.33/72.58	73.22/73.08	72.36/70.98	72.52/72.33	71.74/72.27
BOBCAT	72.31/71.68	72.36/72.28	73.70/73.39	72.69/71.45	72.89/72.84	73.87/72.84
NCAT	72.28/71.53	72.55/72.31	73.81/73.50	72.28/71.59	72.63/72.37	73.90/73.59
BECAT	71.92/71.44	73.01/72.73	73.96/73.61	72.30/71.60	73.11/72.97	74.13/73.70
(b) Performances on NIPS-EDU
CDM	IRT	NeuralCDM
Metric@Step	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20
Random	66.45/69.05	68.23/71.66	70.23/74.82	67.19/69.32	68.44/71.56	70.57/74.99
FSI	67.70/70.60	69.62/73.62	71.03/76.24	-	-	-
KLI	67.09/69.79	69.27/73.30	70.42/75.73	-	-	-
MAAT	66.70/70.32	69.13/72.41	69.07/74.46	67.86/70.12	70.07/72.58	70.66/75.83
BOBCAT	69.51/74.42	70.94/75.73	71.73/76.58	71.13/76.00	72.52/77.87	73.47/79.00
NCAT	67.30/72.11	70.68/75.80	71.91/76.66	70.47/74.10	72.81/77.99	73.47/79.12
BECAT	66.98/73.15	71.61/75.87	72.00/76.82	71.33/76.30	73.09/78.34	73.58/79.36
(c) Performances on EXAM
CDM	IRT	NeuralCDM
Metric@Step	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20	ACC/AUC@5	ACC/AUC@10	ACC/AUC@20
Random	77.58/70.34	78.59/71.91	80.40/74.22	79.80/72.58	79.80/74.81	79.80/78.40
FSI	77.37/70.57	78.79/72.21	81.01/74.89	-	-	-
KLI	77.37/70.57	78.79/72.21	81.01/74.70	-	-	-
MAAT	76.97/70.38	78.79/72.12	80.61/74.65	82.82/70.32	82.83/74.11	83.82/79.44
BOBCAT	80.81/68.17	83.84/72.04	83.43/72.88	78.18/78.24	78.19/81.47	78.18/79.49
NCAT	80.92/70.72	83.99/72.71	84.02/74.29	82.30/78.77	83.19/81.47	81.53/79.49
BECAT	80.99/70.74	83.85/72.88	84.29/75.00	82.84/78.75	83.22/81.49	84.77/79.70
0.32%. This finding inspires us: BECAT is more adaptable to practical large-scale testing situations, and can retrieve the most suitable questions from the massive candidate questions. However, it cannot be ignored that the BECAT cannot surpass all other methods at the beginning of the exam. For example, on NIPS-EDU dataset, it is about 2.53% behind BOBCAT on ACC@5. This is because the student's response data available in the initial stage of exam is limited, and the data-driven methods (e.g., BOBCAT and NCAT) can be pre-trained on large-scale student response datasets to learn the interaction patterns, thus addressing this cold-start problem [41]. Thus, adapting the proposed explicit algorithm to data-driven frameworks is a very promising future work.
Task 2: Simulation of Ability Estimation. The goal of a practical CAT system is to accurately estimate student's ability. We conduct the Simulation of Ability Estimation experiment on the EXAM dataset using the mean square error  $\mathbb{E}[\| \theta^t -\theta_0\| ^2 ]$  between the ability estimate  $\theta^t$  and the true ability  $\theta_0$  at each step. Figure 3(a) reports the results of different methods on IRT. As the number of questions selected increases, we find that the BECAT method can always achieve much lower estimation errors, especially in the middle stage. Some implicit methods that do not aim at estimation accuracy perform better in the initial stage (e.g., NCAT), but the final accuracy still lags behind BECAT framework. Also, compared with the widely used FSI, the proposed BECAT can reach the same estimation error using up to  $20\%$  less questions. On average, it can reach the same estimation accuracy using  $15\%$  less questions, which demonstrates its efficiency in ability estimation, i.e., reducing test length.
The Characteristics of the Selected Questions. To gain deeper insights on why BECAT leads to more accurate estimation, we take a close look at the characteristics of the selected questions. First, for IRT, we output the difficulty and discrimination parameters of the selected questions and draw a scatter chart in Figure 3(b). We find that it tends to choose those questions with high discrimination,
(c)
Figure 3: (a) The error of ability estimation on EXAM dataset. (b) The characteristics (i.e., discrimination and difficulty) of the questions selected for 10 students in IRT, where grey dots represent all the questions in the bank, and the “*” represent the ones selected by BECAT. (c) The Jaccard similarity coefficient of the selected questions.
and their difficulty is scattered and roughly concentrated in the middle difficulty area, which may be caused by the fact that most of the students are of middle-ability [42]. Second, for NeuralCDM, to gain a better insight into the knowledge concepts (e.g., Geometry in mathematics) covered by the selected questions, and the association between BECAT and other methods. Figure 3(c) shows the Jaccard similarity coefficient of questions' concepts. Questions selected by the same type of method have a high overlap in knowledge concepts, such as FSI and KLI, BOBCAT and NCAT. MAAT and FSI have the highest similarity scores with BECAT: 1) Although BECAT does not directly adopt the concept features in the selection, it has a high score to MAAT that directly targets knowledge concept coverage/diversity, thus making the measurement more comprehensive. 2) The high similarity (with FSI) proves that BECAT is not only general but also capable of selecting informative items.
5 Related Works
Computerized Adaptive Testing Computerized Adaptive Testing (CAT) technology has been widely used in many standardized tests, such as GMAT, and the multistage testing in GRE is also its special case of CAT [17]. It is an iterative procedure, mainly including Item Response Theory and a question selection algorithm. The following reviews these two components separately:
(1) Item Response Theory (IRT). It is built on psychometric theory and has become popular in educational assessment to provide more individualized feedback about a student's latent ability [43, 44]. It assumes that the examinee's ability is unchanged throughout a test, thus the ability can be estimated using his/her previous response on questions in gradient-based optimization [32]. The classic form is the two-parameter logistic (2PL):  $p$  (the response to question  $j$  is correct) = sigmoid  $(a_{j}(\theta - b_{j}))$ , where  $a_{j}, b_{j} \in \mathbb{R}$  represent each question's discrimination and difficulty respectively that are pre-calibrated before testing [29], and  $\theta \in \mathbb{R}$  is student's ability to be estimated. Recently, many studies [36, 45, 46] combine cognitive diagnosis and utilize neural networks to model such student-question interaction (e.g., NeuralCDM [36]).
(2) Selection Algorithms. The selection algorithm is the core component to realize CAT's adaptivity – accurately estimating student's ability with the fewest test steps. Traditional algorithms are based on some uncertainty or information metrics, e.g., the famous Fisher Information (FSI). Based on it, many methods [8, 47, 48, 49] have been proposed to introduce additional information in selection. Since they are not general and not applicable to recent neural network methods, MAAT [9] uses active learning to select diverse and representative items in question's feature space. Recently, BOBCAT [12] and NCAT [13] regard CAT as a Reinforcement Learning (RL) problem and train selection algorithms directly from large-scale student response data. Due to the unknown of the  $\theta_0$ , their goal is to minimize the student performance prediction loss of the estimate on the held-out responses data, which is also implicit and prone to biases in training data. In this paper, BECAT is general and explicitly targets the accuracy and efficiency of ability estimation. Compared with previous implicit methods, we find that it exhibits superior performance both theoretically and experimentally. However, various biases, such as those introduced in test item design and respondent pool selection, can affect the validity of estimating a student's true ability [50]. While our approach seeks to improve
the efficiency with which student ability is estimated, it does not diminish the need for test designers to mitigate sources of bias introduced outside of the model fitting process.
Data Efficiency. Another closely related literature is data efficiency (or data summary) [51, 15, 52]. To alleviate various costs (computational [53, 54] or labeling costs [40]), data efficiency is used to carefully select or generate some samples from dataset on par with the full data. Its specific implementation methods include Coreset [15, 21, 16, 20], Active Learning [40, 55, 56], Data Distillation [57], etc. For example, recent Coreset approaches [15, 21, 20] try to find a subset that closely approximates the full gradient, i.e., the sum of the gradients of the whole training samples. In this paper, Coreset helps us transform our optimization problem in Section 2.2, but the gradient calculation requires labels, which is obviously not applicable to the CAT scenario (student's response labels cannot be obtained before the question selection). Therefore, we improve it and design an expected gradient difference approximation method and provide good upper-bound guarantees to the optimal solution, which is one of the main contributions of this paper.
This paper focuses on the explicit approach for accurate and efficient estimation of student's true ability  $\theta_0$ . Given that the ground truth  $\theta_0$  is unavailable, we find its theoretical approximation: the ability estimated by the full responses to the question bank, and use it as the optimization goal to design a Bounded Ability Estimation CAT framework (BECAT). For practical use in CAT scenario, we propose a simple but effective expected gradient difference approximation in the greedy selection algorithm. We further analyze its theoretical properties and prove the error upper-bound of the ability estimation on questions found by BECAT. Through extensive experiments on three real-world education datasets, we demonstrate that BECAT can achieve the best estimation accuracy and outperform existing CAT methods at reducing test length.
This research was partially supported by grants from the National Key Research and Development Program of China (No. 2021YFF0901003), the National Natural Science Foundation of China (Grants No. U20A20229, No. 62106244), and UC Berkeley MicroGrant from the Vice Provost of Undergraduate Education.
[1] Zachary A Pardos. Big data in education and the models that love them. Current opinion in behavioral sciences, 18:107-113, 2017.
[2] Jill-Jenn Vie, Fabrice Popineau, Éric Bruillard, and Yolaine Bourda. A review of recent advances in adaptive assessment. Learning analytics: Fundamentals, applications, and trends, pages 113-142, 2017.
[3] Wim J Van der Linden and Peter J Pashley. Item selection and ability estimation in adaptive testing. In Computerized adaptive testing: Theory and practice, pages 1-25. Springer, 2000.
[4] Andrew S Lan, Andrew E Waters, Christoph Studer, and Richard G Baraniuk. Sparse factor analysis for learning and content analytics. Journal of Machine Learning Research (JMLR), 2014.
[5] Wynne Harlen. The Assessment of Scientific Literacy in the OECD/PISA Project, pages 49-60. Springer Netherlands, Dordrecht, 2001.
[6] Zheng Zhang, Qi Liu, Hao Jiang, Fei Wang, Yan Zhuang, Le Wu, Weibo Gao, and Enhong Chen. Fairlisa: Fair user modeling with limited sensitive attributes information. Advances in Neural Information Processing Systems, 2023.
[7] Frederic M Lord. Applications of item response theory to practical testing problems. Routledge, 2012.
[57] Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A. Efros. Dataset distillation, 2018.
[58] Angela Nedic and Dimitri Bertsekas. Convergence rate of incremental subgradient algorithms. In Stochastic optimization: algorithms and applications, pages 223-264. Springer, 2001.
[59] George B Arfken and Hans J Weber. Mathematical methods for physicists, 1999.
[60] Satoru Fujishige. Submodular systems and related topics. In Mathematical Programming at Oberwolfach II, pages 113-131. Springer, 1984.
[61] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In *ICLR (Poster)*, 2015.
[62] Andrew P Bradley. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7):1145-1159, 1997.
[63] Ying Cheng. When cognitive diagnosis meets computerized adaptive testing: Cd-cat. Psychometrika, 74(4):619-632, 2009.
A Proofs of Lemma 1
We first proof the following Lemma
$$
\min  _ {| S | = T} \max  _ {\theta \in \Theta} \mathbb {E} _ {y} \left[ \| \sum_ {j \in S} \gamma_ {j} \nabla l _ {j} (\theta) - \sum_ {i \in Q} \nabla l _ {i} (\theta) \| \right] \tag {13}
$$
Proof. Following the theoretical analysis in the recent Coreset method CRAIG [15], we first define a mapping function  $h$  from set  $Q$  to  $S$  to a mapping function:  $\forall i \in Q, h(i) \in S$ . It assigns every response data point  $i \in Q$  to one of the elements  $j$  in  $S$ . Then, for any arbitrary ability parameter  $\theta \in \Theta$  we can write
$$
\begin{array}{l} \sum_ {i \in Q} \nabla l _ {i} (\theta) = \sum_ {i \in Q} \left[ \nabla l _ {i} (\theta) - \nabla l _ {h (i)} (\theta) + \nabla l _ {h (i)} (\theta) \right] (14) \\ = \sum_ {i \in Q} \left[ \nabla l _ {i} (\theta) - \nabla l _ {h (i)} (\theta) \right] + \sum_ {j \in S} \gamma_ {j} \nabla l _ {j} (\theta) (15) \\ \end{array}
$$
Subtracting and taking the expected norm of the both sides, we get an upper bound on the error. According to the triangle inequality, we have
$$
\mathbb {E} \left[ \| \sum_ {i \in Q} \nabla l _ {i} (\theta) - \sum_ {i \in S} \gamma_ {j} \nabla l _ {j} (\theta) \| \right] \leq \sum_ {i \in Q} \mathbb {E} \left[ \| \nabla l _ {i} (\theta) - \nabla l _ {h ^ {\prime} (i)} (\theta) \| \right]. \tag {16}
$$
When the mapping function  $h$  is to map each element in  $Q$  to the one in  $S$  that is closest to its expected gradient, the right side of inequality (16) is minimized, or minimum expected distance between the gradient:  $h(i) = \arg \min_{j\in S}\mathbb{E}\left[\| \nabla l_i(\theta) - \nabla l_j(\theta)\| \right]$ . Therefore, the upper bound of the expected gradient difference can be further constrained:
$$
\min  _ {| S | = T} \mathbb {E} \left[ \| \sum_ {i \in Q} \nabla l _ {i} (\theta) - \sum_ {i \in S} \gamma_ {j} \nabla l _ {j} (\theta) \| \right] \leq \sum_ {i \in Q} \min  _ {j \in S} \mathbb {E} \left[ \| \nabla l _ {i} (\theta) - \nabla l _ {j} (\theta) \| \right]. \tag {17}
$$
Next, define a similarity function  $\widetilde{w}(i,j)$  which measures the expected gradient similarity between response pair  $i$  and  $j$ :  $\widetilde{w}(i,j) = d - \max_{\theta \in \Theta} \mathbb{E}\left[\|\nabla l_i(\theta) - \nabla l_j(\theta)\|\right]$ , and  $d = \max_{i \in Q, j \in S} \max_{\theta \in \Theta} \|\nabla l_i(\theta) - \nabla l_j(\theta)\|$  is the maximum pairwise gradient distance. Thus, the optimization problem (Eq.(13)) can also be transformed as:
$$
\max  _ {| S | = T} \sum_ {i \in Q} \max  _ {j \in S} \widetilde {w} (i, j). \tag {18}
$$
Following the same way of origin problem, its corresponding submodular  $\widetilde{F}(S) = \sum_{i \in Q} \max_{j \in S} \widetilde{w}(i,j)$ , which is the same with our proposed method. Thus, the designed selection algorithm is the greedy algorithm of the optimization problem (Eq.(13)).
B Proofs of Theorem 1
Theorem 1 (Expected estimation error bound). Assume that the loss function for ability estimation is  $\alpha$ -strongly convex (e.g., IRT). Let  $S$  be a weighted subset obtained by the proposed method. Then with learning rate  $\frac{1}{\alpha}$ , ability estimation in gradient descent applied to the subsets has the following expected estimation error bound:
$$
\mathbb {E} \left[ \| \theta^ {t + 1} - \theta^ {*} \| ^ {2} \right] \leq \frac {2 \epsilon D \alpha + \sigma_ {l} ^ {2} + 2 \sigma_ {f} D \alpha H _ {p} \left(\theta^ {t} , \theta^ {*}\right)}{\alpha^ {2}} \tag {19}
$$
$$
w h e r e H _ {p} \left(\theta^ {t}, \theta^ {*}\right) = \mathbb {E} _ {\left(q, y\right) \sim p _ {\theta t}} \left[ \frac {1}{p _ {\theta *} (q , y)} \right] \tag {20}
$$
Proof. We now provide the expected estimation error bound for strongly convex functions building on the analysis of [15, 58]. Let  $g^{t} = \frac{1}{|Q|}\sum_{i\in Q}\nabla l_{i}(\theta^{t})$ ,  $g_{S}^{t} = \sum_{i\in S}\gamma_{i}\nabla l_{i}(\theta^{t})$ , and normalize the subset weights at every iteration i.e.,  $\sum_{j\in S}\gamma_{j} = 1$ . Let  $L(\theta) = \sum_{i\in S}\gamma_{i}l_{i}(\theta)$  be the weighted subset training loss parameterized by ability parameters  $\theta$ , and we have:
$$
\begin{array}{l} \left\| \theta^ {t + 1} - \theta^ {*} \right\| ^ {2} = \left\| \theta^ {t} - \eta g _ {S} ^ {t} - \theta^ {*} \right\| ^ {2} \\ = \| \theta^ {t} - \theta^ {*} \| ^ {2} - 2 \eta \left(g _ {S} ^ {t}\right) ^ {\top} \left(\theta^ {t} - \theta^ {*}\right) + \eta^ {2} \| g _ {S} ^ {t} \| ^ {2} \\ \leq \left\| \theta^ {t} - \theta^ {*} \right\| ^ {2} - 2 \eta \left[ L \left(\theta^ {t}\right) - L \left(\theta^ {*}\right) \right] + \eta^ {2} \left\| g _ {S} ^ {t} \right\| ^ {2} \\ \leq \| \theta^ {t} - \theta^ {*} \| ^ {2} - 2 \eta \left[ \left(g _ {S} ^ {*}\right) ^ {\top} \left(\theta^ {t} - \theta^ {*}\right) + \frac {\alpha}{2} \| \theta^ {t} - \theta^ {*} \| ^ {2} \right] \\ + \eta^ {2} \| g _ {S} ^ {t} \| ^ {2} \quad (\alpha - \text {s t r o n g l y c o n v e x}). \tag {21} \\ \end{array}
$$
According to Cauchy-Schwarz inequality, we have
$$
\left| \left(g _ {S} ^ {*}\right) ^ {\top} \left(\theta^ {t} - \theta^ {*}\right) \right| \leq \left\| g _ {S} ^ {*} \right\| \left\| \theta^ {t} - \theta^ {*} \right\|. \tag {22}
$$
$$
\left\| \theta^ {t + 1} - \theta^ {*} \right\| ^ {2} \leq \left\| \theta^ {t} - \theta^ {*} \right\| ^ {2} + 2 \eta \left[ \| g _ {S} ^ {*} \| \| \theta^ {t} - \theta^ {*} \| \right] - \eta \alpha \| \theta^ {t} - \theta^ {*} \| ^ {2} + \eta^ {2} \| g _ {S} ^ {t} \| ^ {2} \tag {23}
$$
Taking expectation with respect to the randomness in the label (i.e., the correctness of response) decided by  $\theta^t$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \| \theta^ {t + 1} - \theta^ {*} \| ^ {2} \right] \leq (1 - \eta \alpha) \mathbb {E} \left[ \| \theta^ {t} - \theta^ {*} \| ^ {2} \right] \\ + 2 \eta \mathbb {E} \left[ \| g _ {S} ^ {*} \| \| \theta^ {t} - \theta^ {*} \| \right] + \eta^ {2} \mathbb {E} \left[ \| g _ {S} ^ {t} \| ^ {2} \right]. \tag {24} \\ \end{array}
$$