Tracing Training Progress: Dynamic Influence Based Selection for Active Learning

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 11:20:06

Abstract: Active learning (AL) aims to select highly informative data points from an unlabeled dataset for annotation, mitigating the need for extensive human labeling effort. However, classical AL  methods heavily rely on human expertise to design the sampling strategy, inducing limited scalability and generalizability. Many efforts have sought to address this limitation by directly connecting sample selection with model performance improvement, typically through influence function. Nevertheless, these approaches often ignore the dynamic nature of model behavior during training optimization, despite empirical evidence highlights the importance of dynamic influence to track the sample contribution. This oversight can lead to suboptimal selection, hindering the generalizability of model. In this study, we explore the dynamic influence based data selection strategy by tracing the impact of unlabeled instances on model performance throughout the training process. Our theoretical analyses suggest that selecting samples with higher projected gradients along the accumulated optimization direction at each checkpoint leads to improved performance. Furthermore, to capture a wider range of training dynamics without incurring excessive computational or memory costs, we introduce an additional dynamic loss term designed to encapsulate more generalized training progress information. These insights are integrated into a universal and task-agnostic AL framework termed Dynamic Influence Scoring for Active Learning (DISAL). Comprehensive experiments across various tasks have demonstrated that DISAL significantly surpasses existing state-of-the-art AL methods, demonstrating its ability to facilitate more efficient and effective learning in different domains.

Main paper: 
 introduction: 
 Active learning (AL) is a machine learning paradigm that focuses on selecting more informative data points from an unlabeled dataset for annotation [24] . By identifying an efficient training data subset, AL has demonstrated to be an effective method for alleviating the Figure 1 : The red and blue points represent the data samples selected at Epoch 10 and 100, while employing the same AL method with 10% selecting rate. Subsets selected from different checkpoints tend to exhibit significant fluctuation.bottleneck challenge of data annotation [1, 20] , enabling faster iteration in model development. Since current deep learning models are data-hungry [18] , particularly in the context of large foundation models, where the quality of labeled data also significantly impacts model performance [3, 40] , actively selecting data has gained increasing attention in recent years [33] .Existing AL approaches can be generally divided into two main categories: uncertainty-based [9, 31] and feature distribution-based [2, 27] . The former utilizes uncertainty estimation to identify hardto-learn samples, while the latter aims to understand the underlying feature distribution and identify representative and diverse samples. Additionally, a variety of hybrid strategies have been proposed to combine the above these two criteria for AL [4, 36, 39] . While they define the state-of-the-art baselines, the design of these AL strategies are mainly based on human experience, lacking interpretability. Moreover, they are often tailored for specific tasks or labeling budget [4, 11] .To better explain how each selected instance impacts the model performance, gradient-based influence function [16] has been introduced in AL to directly make a direct quantitative evaluation [21, 35] . For example, Liu et al. [21] estimate its expected gradient to calculate unlabeled samples' impact on model performance. Similarly, Wang et al. [35] employ two alternative schemes to calculate the gradient norm, leading to a lower upper-bound of the test loss. Despite of the significant progresses, existing approaches only consider one checkpoint into influence estimation, neglecting the evolving dynamics of training, which has been shown to provide important clues for scoring each data influence [12, 23] . Specifically, the model behavior on each instance may vary during the progress of training via stochastic gradient descent, resulting in the importance scores of each sample to fluctuate throughout the optimization path of the deep networks. As illustrated in Figure 1 , the subsets selected at different training checkpoints clearly contain various samples. Therefore, relying on static information from a single checkpoint for sample selection can lead to suboptimal subset construction and potential overfitting.To address aforementioned problems, this work focuses on exploring a dynamic influence based data selection strategy. To do so, we leverage the gradient-based dynamic influence estimation [23] to trace per-sample contribution across the model training procedure. Our theoretical analysis reveals that, for each checkpoint, selecting instance of higher projected gradient along the accumulated optimization direction leads to a better test performance. Specifically, given the unavailability of ground truth labels in AL, we rely on Kullback-Leibler (KL) divergence given by model outputs between checkpoints to measure gradient projection, eliminating the introduction of biased predicted label. Moreover, to extend our theoretical findings to a broader range of training progress, a straightforward approach is to save checkpoints and then calculate the gradient at each checkpoint [23] . Since expensive cost required for tracing massive unlabeled data in the AL setting, we propose to introduce an additional loss term to integrate the dynamics throughout the training progress, thus enhancing selection generalization. By incorporating these designs, we propose an universal AL framework, named Dynamic Influence Scoring for Active Learning DISAL, illustrated in Figure 2 .Extensive experiments on different tasks proves the validity and task-agnostic nature of our framework, including balanced and imbalanced image classification, semantic segmentation. We prove that our AL framework shows superior performances in comparison with other state-of-the-art baselines. Notably, our framework DISAL, utilizing 40% annotation budget on Caltech101 dataset [7] , reaches a 93.58% accuracy, exceeding the model training based on the full training set at 92.59%.We summarize our main contributions:• We present a theoretical analysis demonstrating that the projected gradients on the accumulated optimization direction contribute to a valid influence scoring of unlabeled samples, which naturally leads to a criterion for data selection.  
related work 2.1 active learning: 
 The key objective of AL is to construct a data subset through designing an effective AL strategies, including mainstream feature distribution-based approaches and model uncertainty-based approaches.As to the former, numerous studies have introduced k-means clustering and its variants to overcome data redundancy [2, 11] . Moreover, CoreSet [27] focuses on selecting a subset from an unlabeled data set that represents the whole set well. DAL [10] conceptualizes AL as a binary classification problem, labeling undistinguishable examples. VAAL [29] introduces an adversarial network to discriminate between labeled and unlabeled samples within the latent space encoded by the VAE [15] .As for uncertainty-based AL approaches, uncertainty has been widely used in AL to estimate samples' importance. Contrast to traditional uncertainty-based methods directly using the posterior probability [22, 22, 28, 34] , DBAL [9] leverages Bayesian deep learning within AL, using Dropout for computing model uncertainty. Tran et al. [31] calculate the gradient with respect to the final layer of the deep network, prioritizing the annotation of diverse unlabeled samples with higher gradient norms. LL4AL [37] offers a more direct strategy by integrating an additional network to precisely predict each sample's loss. Furthermore, generative methods [6, 31, 41] have also been adopted in AL to generate samples with high uncertainty.While existing two main categories of AL methods define the state-of-the-art baselines, they do not directly show how selected data impacts model performance. Several influence estimationbased methods aims to address this problem Liu et al. [21] , Wang et al. [35] . Specifically, Liu et al. [21] prioritize the selection of unlabeled samples that can provide more positive influence, calculated by estimating its expected gradient. Similarly, drawing inspiration from Koh and Liang [16] , Wang et al. [35] explore such an impact by offering theoretical evidence that the selection of unlabeled data with a higher gradient norm result in better test performance.Influence estimation is designed to assess the impact of specific training instances on a model's prediction for a given input. For models trained with gradient descent, the influence of training data is exerted solely through training gradients. Static gradient-based influence estimators determine influence by referencing only the final model parameters. A notable example is the influence function presented in [16] , which define influence as the change in test loss caused by a minor perturbation to a training instance loss's weight. [25] extend this application of the influence function to assess unlabeled samples in semi-supervised learning scenarios. These static gradient-based influence estimation methods, however, only consider a single point in the procedure of training to estimate training instance's possible effect. Dynamic methods, however, provide a more comprehensive view by capturing the entire "story" of an instance's influence. TracIn [23] calculates influence by "tracing" gradient descent, aggregating changes in test loss each time training instance's gradient updates parameters.Despite these advancements, the context of influence estimation in the setting of AL is entirely distinct. As far as we know, we are the first to introduce dynamic influence estimation to select unlabeled samples. 
methodology: 
 Despite influence function-based methods establish a connection between sample contribution and model training loss, the single checkpoint-based selection hinders the construction of a subset with strong generalization. In this section, inspired by [23] , we propose the Dynamic Influence Scoring for Active Learning (DISAL) framework as shown in Figure 2 . In Section 3.1, We begin with the problem preliminary of AL. Following this, in Section 3.2, we provide a detailed theoretical derivation to evaluate the unlabeled sample at checkpoints by projecting gradient on accumulated optimization direction. Section 3.3 extends the contribution evaluation to the entire training procedure, so as to provide a more comprehensive dynamic influence scoring. Finally, in Section 3.4, we show our proposed AL framework, DISAL.We define the AL procedure within the framework of standard supervised learning as follows: consider X to represent the input, and Y denote the corresponding output. Let 𝐷 = {X, Y} be the large pool of unlabeled data. A labeled training dataset is denoted as 𝐷 𝐿 = {(𝑥 1 , 1 ), (𝑥 2 , 𝑦 2 ), ..., (𝑥 𝑛 , 𝑦 𝑛 )}. We denote 𝐷 𝑈 ⊆ 𝐷 as the subset of unlabeled datasets. Throughout the AL procedure, a fixed number of samples are iteratively chosen from the unlabeled pool according to the acquisition strategy until reaching the sampling budget. In our method, we leverage the dynamic influence scoring as the criterion to select the most impactful instances on model performance. After adding the newly-annotated samples to labeled set, the optimization goal of neural network is to find the parameters 𝜃 that minimizes the training loss. Here we take the cross-entropy (CE) loss applied to image classification as an example, our goal is to train a label-efficient deep model 𝑓 (•; 𝜃 ) where 𝜃 represents the parameters of the deep model to be optimized. Here the loss function is:EQUATIONThe ultimate objective of AL is to identify and annotate the most informative data, thereby enhancing the performance of the model.To calculate the dynamic influence score, we firstly explore how the unlabeled instance impact the model performance at checkpoints and propose a detailed theoretical derivation to evaluate the reduction in test loss. We firstly assume that the model is updated via vanilla stochastic gradient descent, based on an idealized definition. Each training minibatch, denoted as 𝐵(𝑡), consists of a single instance, and gradient updates occur without momentum [26] . We first evaluate the influence of incorporating a single sample 𝑥 𝑛 to the training set on the change in test training loss. Each iteration 𝑡 affects only the model parameters, updating the parameter vector from 𝜃 𝑡 to 𝜃 𝑡 +1 using the training example 𝑥 𝑛 . In this context, the idealized influence of the training example 𝑥 𝑛 on the test example 𝑥 ′ is denoted as the total reduction in loss:EQUATION)Although the definition of ideal influence has a strong theoretical motivation, it relies on single-instance batch setting and vanilla stochastic gradient descent, failing to work in practical scenarios. In fact, models are trained on batches containing hundreds or more of instances to ensure feasible training times. According to [23] , to handle minibatches of size 𝑏 ≥ 1, a first-order Taylor approximation is used to approximate the impact of each training instance within a minibatch 𝐵(𝑡).Given the typically small step sizes in parameter updates during training, the loss change for a test example within a given iteration can be estimated using a simple first-order Taylor expansion:ℓ 𝑓 𝜃 𝑡 +1 𝑥 ′ , 𝑦 ′ = ℓ 𝑓 𝜃 𝑡 𝑥 ′ , 𝑦 ′ + ∇ℓ 𝑓 𝜃 𝑡 𝑥 ′ , 𝑦 ′ • (𝜃 𝑡 +1 -𝜃 𝑡 ) + 𝑂 ∥𝜃 𝑡 +1 -𝜃 𝑡 ∥ 2 .(3)Here, the gradient ∇ℓ 𝑓 𝜃 𝑡 (𝑥 ′ ), 𝑦 ′ is calculated with respect the model parameters 𝜃 𝑡 . While stochastic gradient descent is utilized in training the model from 𝜃 𝑡 to 𝜃 𝑡 +1 , the formula for the change in model parameters is given byEQUATIONwhere 𝜂 𝑡 represents the step size at iteration t. Ignoring the higherorder term, which is on the order of 𝑂 (𝜂 2 𝑡 ), we derive a first-order approximation for the change in loss:EQUATIONConsequently, for each training data 𝑥 𝑛 within the minibatch 𝐵 𝑡 , we attribute the influence of 𝑥 𝑛 on the test point 𝑥 ′ as the portion1 𝑏 𝜂 𝑡 ∇ℓ 𝑓 𝜃 𝑡 (𝑥 ′ ), 𝑦 ′ ) • ∇ℓ 𝑓 𝜃 𝑡 (𝑥 𝑛 ), 𝑦 𝑛 ,effectively quantifying the contribution of each training instance with a first-order approximation.However, the inherent randomness in training processes can negatively affect the above inference of sample influence. For instance, although intuitively identical training samples should receive the same influence score, their inconsistent appearance in the same minibatch can result in varying scores. We overcome it by evaluating the influence of each instance across the entire training data, not limiting the analysis to only the batches. In our context, during each iteration 𝑡 ∈ 𝑇 , all training instances are visited exactly once, updating the model parameter from 𝜃 𝑡 to 𝜃 𝑡 +1 . We assume a consistent step size 𝜂 𝑡 between iterations. [23] provide a practical heuristic influence estimation at each iteration 𝑡:EQUATIONWe assess example influence scoring on all the test sets. In AL settings, we assume the test set is 𝐷 𝑡𝑒𝑠𝑡 . For any instance 𝑥 𝑛 involved in model training at iteration 𝑡, the influence on the test loss 𝐼 (𝑥 𝑛 , 𝐷 𝑡𝑒𝑠𝑡 ) can be computed as:EQUATION)The influence score 𝐼 (𝑥 𝑛 , 𝐷 𝑡𝑒𝑠𝑡 ) indicates each training sample's potential to change test model performance. A higher positive influence score suggests a greater ability to reduce test loss. In contrast, the examples with a negative influence score are likely to be discarded as they contribute to an increase in test loss. It's notable that the test instance in question does not necessarily need to be in the test set but refers to any example whose prediction is being explained [23] . Given that the test set 𝐷 𝑡𝑒𝑠𝑡 is unavailable at training phase, we use all labeled samples to act as an alternative, representing the data distribution of current tasks. Consequently, we define a practical scheme for influence estimation at iteration 𝑡 as follows:EQUATION)Equation 8 indicates that per-sample contribution can be estimated by projecting its gradients onto the accumulated optimization gradient. Thus, we propose to utilize projected gradient to score sample influence at checkpoints. Compared with existing static gradient-based AL methods [35] , which overlook optimization context, we contribute to a more valid contribution evaluation, which has been proved in the experiments in subsection 4.5.1.However, Equation 8 involves calculating per-sample gradients over all training samples, which is not feasible due to computation and memory constraints. According to model optimization 𝜃 𝑡 +1 = 𝜃 𝑡 -𝜂 𝑡 𝑥 ′ ∈𝐷 𝐿 ∇ℓ 𝑓 𝜃 𝑡 (𝑥 ′ ), 𝑦 ′ at iteration 𝑡, the accumulated gradient is equal to 1 𝜂 𝑡 (𝜃 𝑡 -𝜃 𝑡 +1 ). Thus the influence described in Equation 8 is further reformulated in terms of the loss difference as follows:EQUATIONwhich can be derived from the first-order Taylor expansion. Then the contribution of each sample to model performance can be assessed through changes in its loss. While ℓ is commonly represented by CE loss in the context of classification, the loss divergence can be defined as:EQUATIONUnfortunately, as discussed earlier, the one-hot encoding 𝑦 𝑛 is unavailable for unlabeled data in the setting of AL. We overcome the challenge by replacing 𝑦 𝑛 with predicted probability 𝑓 𝜃 𝑡 (𝑥 𝑛 ). So KL divergence is employed to capture a more comprehensive sample contribution information:EQUATIONIn our experiments (refer to 4.5.2), we have observed that KL divergence outperforms the CE loss difference. We hypothesize that this superiority stems from the fact that the one-hot encoding of 𝑦 𝑛 neglects non-target probabilities, thereby causing a substantial loss of information.Since that exploring long-range training progress contributes to a more comprehensive sample evaluation, in this section, we simply extend our proposed influence scoring for each checkpoint in Section 3.2 to a dynamic influence scoring by introducing an additional loss. By capturing the entire "story" of influence, well-generalized samples are more likely to be identified.To scale to long training processes, [23] replay the procedure by storing checkpoints at regular intervals, and sums up the influence throughout the model optimization. However, applying this approach to large-scale unlabeled data is computationally impractical, as it requires model inference on all the unlabeled data at each checkpoint. To address this, we devise an efficient method by directly plugging the dynamic information into model training. Thus by the end of model training, more comprehensive sample-wise gradient projection would be obtained to select samples at the last iteration, via incorporating the entire of training progress. Specifically, we develop a dynamic loss term, trained with dynamic prediction probabilities of labeled data, which is available during the training process. For simplicity, we construct the dynamic predicted probabilities 𝑓 (𝑡 ) (𝑥 𝑛 ) by averaging the predicted outputs during 𝑡 iterations [30] . Then the dynamic predictions is updated at 𝑡 iteration as follows:EQUATION)To learn the dynamic information effectively, the dynamic loss term is optimized by minimizing the KL divergence loss between the actual prediction 𝑓 𝜃 𝑡 (𝑥 𝑛 ) and the dynamic predicted probability 𝑓 (𝑡 ) (𝑥 𝑛 ) at iteration 𝑡:EQUATIONBy integrating the main task loss in Equation 1 and dynamic loss, we minimize an overall objective function as follows:EQUATIONwhere 𝜆 is a trade-off weight to control the effect of dynamic loss terms. The setting of 𝜆 is examined in Section 4.5.3. Finally, after model training, a more generalized dynamic influence scoring of each sample can be obtained at the final iteration 𝑇 , by which point the entirety of the training dynamics are considered for sample selection.𝑛 ) = 𝑓 𝜃 𝑇 -1 (𝑥 𝑛 ) ⊤ • log 𝑓 𝜃 𝑇 (𝑥 𝑛 ) 𝑓 𝜃 𝑇 -1 (𝑥 𝑛 ) (15)Based on theoretical insights, we have formulated a universal AL framework integrating the dynamic influence scoring discussed in the preceding section. The framework begins with the selection of randomly annotated data to initiate training of the task model. Subsequently, after each training cycle, dynamic influence scoring is applied to select and annotate unlabeled data, which are used for retraining the task model. Notably, our framework introduces only minimal adjustments to the conventional AL workflow, ensuring its straightforward applicability across diverse scenarios. For a comprehensive understanding of our framework, please refer to Algorithm 1 and the Figure 2 . 
experiments: 
 Considering the sensitivity of AL sampling algorithms to taskspecific scenarios, we experimentally verify the effectiveness of our framework on two widely recognized visual tasks: image classification and semantic segmentation, demonstrating its broad applicability. Additionally, we explored its performance in class-imbalanced settings by using modified versions of the Cifar10 dataset. TheseInput: Initial labeled dataset 𝐷 𝐿 ; Unlabeled dataset 𝐷 𝑈 ; AL cycles 𝐾; Iterations 𝑇 ; Dynamic influence scoring 𝐼 (•).1: for 𝑘 = 1, • • • , 𝐾 do 2: for 𝑡 = 1, • • • ,𝑇 do 3:Record predicted probabilities 𝑓 𝜃 𝑡 (𝑥 𝑛 ), 𝑥 𝑛 ∈ 𝐷 𝐿 4:Update dynamic predicted probabilities 𝑓 (𝑡 ) (𝑥 𝑛 ) 5:⊲ Defined in Equation 126:Train model integrating main task loss with dynamic loss 7:⊲ Defined in Equation 148:if 𝑡 = 𝑇 then 9:Calculate dynamic influence scores 𝐼 (𝑥 𝑛 ) 10:⊲ Defined in Equation 1511:end if 12:end for 13:Q 𝑡 ←Sample queries 𝑄 𝑡 by scores 14:Q 𝑡 ←Label queries 𝑄 𝑡 by annotator 15:𝐷 𝐿 ← 𝐷 𝐿 ∪ Q 𝑡 16:𝐷 𝑈 ← 𝐷 𝑈 ∪ Q 𝑡 17: end for experiment results are reported across three trials, each employing different initial network weights and labeled data pool. Finally, the ablation study is conducted to provide deeper insights.Dataset. In our experiments, we employ four image classification datasets: Cifar10 [17] , Cifar100 [17] , SVHN [22] and Caltech101 [7] , with varying size and number of categories. Cifar10 and Cifar100 include 50,000 training and 10,000 testing images across 10 and 100 categories, respectively, with a resolution of 32 × 32 pixels. SVHN, similar in class count to Cifar10, includes 73,257 training and 26,032 testing images, without utilizing additional training data for fair comparison with other methods. Caltech101 comprises 9,146 images (300 × 200 pixels) across 101 semantic categories, split into 8,046 for training and 1,098 for testing.Baselines and implementation details. We compare DISAL with state-of-the-art AL baselines, including Dropout [8] , Learning Loss [37] , CoreSet [27] , VAAL [29] , CoreGCN [37] , Boosting [35] , TOD [14] and TiDAL [19] . In addition, the baseline methods also include the random sampling ("Random") and the model training based on the full training set ("Full Training").Following conventional practices in AL [29] , we employ ResNet-18 [13] as the backbone network for the primary task learner. Ninety percent of the images are allocated for training, while the remaining ten percent are reserved for testing purposes. The model is trained using the SGD optimizer, incorporating a momentum of 0.9 and a weight decay of 5 × 10 -4 . For Cifar10 and Cifar100, the models undergo 200 epochs of training with a batch size of 128 and an initial learning rate of 0.1. Conversely, Caltech101 is trained for 50 epochs with a batch size of 64 and an initial learning rate of 0.01. Input images are standardized to dimensions of 224 × 224 pixels, accompanied by preprocessing techniques including random resizing, cropping, and horizontal flipping. For further elaboration, please refer to the supplementary material.Active learning setting. For fair comparison, We follow the same AL setting and practice with baseline methods [29, 35] . In all tasks, we conduct 7 cycles of data annotation, expanding from 10% to 40% in 5% increments to cover various annotation budgets. In the initial cycle, we populate the labeled pool 𝐷 𝐿 0 by randomly selecting 10% instances for annotation from the unlabeled dataset to train an initial task learner. In each subsequent AL cycle, 5% of the data is selected from the unlabeled data pool 𝐷 𝑈 for model retraining, according to the specific sampling strategy of each method. This process is repeated until the labeled portion of the dataset reaches 40%.Results. Figure 3 illustrates the performance in terms of average test accuracies across different AL methods. Our approach, DISAL consistently outperforms baseline methods on all the datasets with a significant margin. Additionally, we make several observations that highlight the effectiveness of our proposed DISAL. First, DISAL achieves higher classification accuracies compared to other methods at various cycles, indicating that given different annotation budget such as 20%, our proposed DISAL still works best. Second, DISAL stands out in handling complex datasets like Cifar100 and Caltech101, which present greater challenges due to their larger number of classes. The superior performance and smoother curves in our method demonstrate its robustness. Remarkably, with 20,000 labeled instances, DISAL reaches a 69.64% accuracy in the final iteration, exceeding the next at 68.49%. Third, our model, when trained on 40% of the data using DISAL, outperform the "Full Training" on datasets including Cifar10, SVHN and Caltech101, e.g., 94.79% vs. 93.12% on Cifar10. It's also worth noticing that DISAL is the only method to achieve higher test accuracy than "Full Training" results on challenging Caltech101 benchmark, despite using only 40% labeled training data. Last, we also compare DISAL with semisupervised method, TOD, which introduces a semi-supervised loss to learn from unlabeled pool. Surprisingly, our proposed DISAL surpasses TOD in all datasets, even though TOD leverages additional unlabeled information and more training resources.Dataset and active learning setting. To prove the effectiveness of AL methods in complex, large-scale environment, we evaluate our method on the widely recognized semantic segmentation benchmark dataset, Cityscapes [5] . Semantic segmentation task involves pixel-level classification, thus incurring significantly higher annotation costs. The Cityscapes dataset is a large-scale driving video dataset, consisting of video sequences of urban street scenes from 50 cities. The dataset includes 2,975 training and 500 validation images. For fair comparison, only the standard training and validation sets are utilized. In each AL iteration, 10% images are selectively sampled for the labeled training set until reaching the budget of 40%.Baselines and implementation details. We choose the stateof-the-art AL methods for this experiment: Dropout [8] , Learning Loss [37] , CoreSet [27] , VAAL [29] , CoreGCN [37] , Boosting [35] and TiDAL [19] . Following the widely used settings in [29] , we adopt the 22-layer dilated residual network (DRN-D-22) [38] as the task model. The performance of each method was evaluated using the mean Intersection-over-Union (mIoU) metric. We crop the input images to a resolution of 688 × 688 pixels without the application of data augmentation. The dataset contains 19 categories. Detailed information regarding other hyper-parameters is available in the supplementary material.Results. The superior results in Figure 4 demonstrates the competence of DISAL in the challenging semantic segmentation task measured by mIoU. It is important to note that Cityscapes images are significantly larger than those used in classification benchmarks and require pixel-level classification, which indicates our method's robustness to complex data. Furthermore, DISAL's independence from task-specific knowledge eliminates the need for adjusting the training framework with changes in tasks, making it universally applicable. However, we observe that DISAL exhibits slightly lower performance than other premier methods in the first data selection. This discrepancy is likely due to the inherent instability of model training on challenging tasks with extremely limited data, where learning the dynamic information could potentially impact the model training.Dataset and Implementation Details. To evaluate DISAL's performance in a more class-imbalanced context, we use the imbalance ratio (IR) of 10 to recompose the Cifar10 dataset. The distribution of images per class is as follows: airplane-500, automobile-1,000, bird-1,500, cat-2,000, deer-2,500, dog-3,000, frog-3,500, horse-4,000, ship-4,500 and truck-5,000. The experiments are conducted over the same seven AL cycles, varying from 2𝑘 of the labeled pool to 8𝑘. All additional experimental details are consistent with those for the balanced Cifar10, described in Section 4.1.Results. Figure 5 shows the performances between DISAL and other baselines on the synthetically imbalanced Cifar10. Under the imbalanced settings, our proposed DISAL still clearly outperform all the baselines. Additionally, our findings reveal intriguing insights: Contrary to results from experiments on balanced Cifar10 and Cifar100 datasets, data distribution-based AL methods, such as As described in Section 3.3, the extra computation for DISAL, apart from the task model training with an additional loss, is calculating the KL divergence of models outputs between checkpoints at the last iteration for selecting data. To compare computational overheads fairly, we assess one cycle duration of AL, which contains model training and a single-pass data selection, on Cifar10 and Ci-far100 using various methods. As illustrated in Table 1 , while DISAL incurs no additional memory costs such as saved checkpoints for dynamic influence estimation, its computational overheads are also comparable to those of other approaches under the same experimental settings. We attribute this advantage to its simple design, which relies solely on KL loss, without the requirement for additional model [19, 29] or costly clustering [27, 37] .4.5.1 Analysis on sampling strategy and dynamic loss. To further understand the remarkable performance of DISAL, ablation studies are conducted to evaluate the contribution of its two main designs, influence scoring via checkpoints and dynamic loss.As can be seen from the Table 2 , our components notably enhance performance and robustness. For "DISAL w/o D", a variant that operates without dynamic loss, achieves comparable performance compared with other baselines, demonstrating the high effectiveness of influence scoring via checkpoints. And it's worth noting that on all the datasets, "DISAL w/o D" achieve consistent better performance than Boosting. Since Boosting doesn't take accumulated optimization direction into consideration while calculate gradients, indicating that employing projected gradient could contribute to a more valid single samples influence contribution.Despite comparable results, integrated with the dynamic loss, the accuracy of DISAL increases by 2.5% and 1.1% respectively against "DISAL w/o D" on Cifar100 and Caltech101. This highlights the need for the dynamic loss to trace training dynamics and evaluate more generalized sample contribution. To deepen our comprehension of dynamic influence scoring, we also show t-SNE [32] embeddings of the Cifar10 data, using DISAL and "DISAL w/o D". We use the model trained on these AL strategies to compute the representation of all unlabeled samples and those selected in the first AL cycle. Observations from Figure 6 reveal that both DISAL and "DISAL w/o D" strategies select samples that cover the entire data distribution of each class. However, selected samples from DISAL exhibit a broader distribution, particularly at class boundaries. As those samples on class boundaries are more uncertain with respect to the task model, they can significantly boost model training in the subsequent cycle. It is the reason why a substantial performance improvement has been observed while introducing the dynamic loss. 4.5.2 Ablation on KL divergence loss. As discussed in Section 3.2, we utilize the KL divergence loss to provide a more comprehensive assessment while the label of unlabeled data are unavailable. To evaluate its effectiveness in characterizing sample contribution, we compare KL divergence loss with CE loss difference, which temporarily "observes" its label. As illustrated in Figure 7 , although CE loss difference is blessed with labels which is unreasonable in AL, KL loss still exhibits an absolute advantage at all the cycles on Cifar10 and Cifar100, proving the effectiveness of our KL divergence loss. The results are presented in Table 3 . Notably, when 𝜆 is equal to 1, our AL framework DISAL attains optimal results across various annotations. 
conclusion: 
 This paper first presents a theoretical analysis demonstrating that selecting samples with higher projected gradients on the accumulated optimization direction throughout training can lead to enhanced performance. By integrating dynamic model training, we propose a task-agnostic framework called Dynamic Influence Scoring for Active Learning (DISAL), evaluating per-sample contribution throught the training procedure. Our extensive experiments across diverse tasks confirm the validity of our theoretical findings and showcase the effectiveness of the proposed framework. Looking ahead, our future work will delve into incorporating a learnable trade-off weight to more adeptly capture dynamic information, thereby advancing the field of AL even further.  
 
 Appendix:  


Decision: This work aims to improve active learning performance by dynamically evaluating the influence of unlabeled samples throughout the training process. The proposed method argues that dynamic selecting strategies are preferring during the training procedure. Gradient projections at multiple checkpoints are demonstrated to be good indicator for sample selection. A dynamic loss term is introduced to capture global influence. The proposed method outperforms competing methods in both classification and segmentation tasks.

Pros:
- The concept of dynamically evaluating sample influence and integrating a dynamic loss term is innovative. This approach provides a new perspective on active learning (AL) by focusing on the influence of samples during the training process​.

- This work includes extensive experiments across multiple datasets and tasks, demonstrating significant improvements over existing state-of-the-art AL methods. 

Cons:
- The reliance on dynamic gradient tracking and additional loss terms might complicate implementation and increase computational overhead. 

- The framework is mainly tested on image data, while the authors claim it to be task-agnostic. Additional evaluations on other modalities is recommended to confirm its generalizability. During the rebuttal, authors presented additional experiments on text classification tasks, which alleviated the above concerns.

Overall, this paper proposed novel ideas for active learning with solid empirical results on multiple tasks. Reviewers are generally positive after reading the rebuttal. Therefore, it is recommended for acceptance.

Reviews: review: paper_summary: paper_summary: The paper introduces a novel active learning method that dynamically tracks the training process to select samples based on the current model status. The authors review previous works and argue that optimal selection strategies can change throughout training. Theoretical analyses demonstrate that selecting samples with high projected gradients can be beneficial. Building on this, the authors propose DISAL, which dynamically evaluates the influence of unlabeled samples on the current model and successively selects the most important ones. Additionally, a dynamic loss term is introduced to capture global influence. Experiments show that DISAL outperforms competing methods in both classification and segmentation tasks.
strength_weakness: strength_weakness: 1. The motivation behind the idea is both sound and clear, demonstrating the insight that optimal selection strategies can evolve throughout the course of training. The empirical results presented in Figure 1 convincingly support this concept.
2. Theoretical analyses in Section 3.2 offer a stronger theoretical guarantee for the proposed method compared to heuristic approaches. This robust theoretical foundation is particularly crucial for the reliability and efficacy of an active learning method.
3. The experimental evaluation is comprehensive, encompassing a variety of tasks, including both classification and segmentation. It also addresses both balanced and imbalanced settings, ensuring the method's versatility. The visualization results, such as those presented in Figure 6, are especially interesting and provide valuable insights that help to better understand and appreciate the method's effectiveness.
4. The organization of the paper is logical and coherent, with writing that is clear and fluent. This makes it easy for readers to understand the content and follow the arguments presented, enhancing the overall readability and impact of the paper.
limitations: limitations: 1. The concept of "Dynamic Loss" introduced in Section 3.3 could use some additional clarification. For instance, what exactly does "capturing the entire 'story' of influence" mean? Additionally, it would be helpful to understand why minimizing the KL divergence in Equation 13 is beneficial. Could this process slow down training by constantly aligning with the previous predictions?
2. The experiments seem to indicate that DISAL is effective for unimodal tasks. However, is there any indication or speculation on whether it could extend its benefits to multimodal classification tasks?
paper_summary: paper_summary: Authors introduce the DISAL to dynamically evaluate sample influence and includes effective dynamic loss term to trace model training progress. Authors projected gradients on the accumulated optimization direction contribute to a valid influence scoring of unlabeled samples.
strength_weakness: strength_weakness: [1] The method proposed in this paper is simple and effective.
[2] This paper is clearly laid out.
limitations: limitations: [Major] Whether this method truly fits within the active learning framework, as AL may require a labeling process after the selection and before further training. This method may not be applied at all in practice.
[Minor] I acknowledge that the authors have indeed made novel contributions: 1) proposing to calculate gradient projections at multiple checkpoints instead of a single point to capture training dynamics; 2) introducing an additional dynamic loss term to more efficiently integrate training dynamics information. However, these contributions are largely built upon the existing theoretical proof that selecting unlabeled data with higher gradient norms leads to better test performance.
paper_summary: paper_summary: The paper is about a task-agnostic framework DISAL, which integrates dynamic model training to evaluate per-sample contribution throughout the training procedure. The paper firstly analyzes that the unlabeled data's projected gradient on the accumulated training gradient can be interpreted as its influence score, then the framework aims to enhance performance by selecting samples with higher projected gradients on the accumulated optimization direction. Besides, an auxiliary dynamic loss is introduced to help trace the long term progress.
strength_weakness: strength_weakness: 1. The derivation of the key equation 11 is interesting and reasonable.  
2. The experiments are relatively comprehensive. Besides the image classification task, segmentation task is also included to verify the effectiveness. Imbalanced dataset which is more difficult is also studied. 
3. According to the ablation studies, the two introduced method --- sampling strategy and dynamic loss, bring large bonus on the baseline accross different datasets. 
4. While bringing improvements on the accuracy, the proposed DISAL still maintains a relatively fast inference speed.
limitations: limitations: 1. Some expressions in the paper seem to be confusing and need further refinement or illustration. For example, what does the statement "For instance, although intuitively identical training samples should receive the same influence score, their inconsistent appearance in the same minibatch can result in varying scores." mean?
2. Although the ablation study shows the introduced dynamic loss helps, it is somehow confusing. The alignment between past progress and the prediction at present does bring the dynamic information, but the future progress would also be restricted. This may occur when more fluctuating training like training on the imbalanced datasets or other modal data. 
3. Although the author state that the method is task-agnostic and can be generalized, it is only tested on image modal. The verification on other modalities is highly recommended and encouraged.
paper_summary: paper_summary: The paper presents a novel active learning framework, "Dynamic Influence Scoring for Active Learning" (DISAL), which aims to enhance model performance by dynamically tracing the impact of unlabeled instances on model performance throughout the training process. The authors argue that classical active learning (AL) methods often rely on human expertise to design sampling strategies, which limits their scalability and generalizability. The proposed DISAL framework employs gradient-based dynamic influence estimation to trace per-sample contribution across the model training procedure. The theoretical analysis suggests that selecting instances with higher projected gradients along the accumulated optimization direction at each checkpoint leads to improved performance. The framework is tested across multiple tasks, demonstrating significant improvements over existing state-of-the-art AL methods.
strength_weakness: strength_weakness: The proposed framework is universal and task-agnostic, making it broadly applicable to various domains.

The paper presents extensive experiments across multiple datasets and tasks, demonstrating the efficacy of the DISAL framework.
limitations: limitations: 1. **Complexity in Implementation:** The method’s reliance on dynamic gradient tracking and additional loss terms could complicate implementation and increase the computational overhead, despite claims of computational efficiency.
2. **Limited Discussion on Scalability:** While the method is shown to be effective on benchmark datasets, the paper lacks a detailed discussion on scalability, especially for very large datasets or in resource-constrained environments.
3. **Dependency on Hyperparameters:** The performance of DISAL seems to be sensitive to specific hyperparameters like the trade-off weight λ, but the method for selecting these parameters isn’t comprehensively discussed, which could affect reproducibility and practical deployment.