RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
(a); (b); Ablation studies on CIFAR100 showing: (a) the variation of accuracy with the number of Transformer decoder layers in the VGA module, (b) the variation of accuracy with the number of linear layers in the task-specific mean and standard deviation encoders.
Figure 11 : Evolution of the loss value \mathcal{L} during the first 100 training iterations of each task on CIFAR100. Training with our proposed weight initialization strategy consistently leads to lower training losses thus bridging the stability gap [ 37 ] in CL.; Evolution of the loss value \mathcal{L} during the first 100 training iterations of each task on CIFAR100. Training with our proposed weight initialization strategy consistently leads to lower training losses thus bridging the stability gap [ 37 ] in CL.
Figure 12 : Test accuracy without weight initialization on the first task for the initial 100 iterations of incremental training on all ten tasks of CIFAR100. The green lines are the means over three different runs, the orange shades denote \pm 1 standard error of the mean. The labels to the vertical bars denote the accuracy values for the first iteration of training on each task.; Test accuracy without weight initialization on the first task for the initial 100 iterations of incremental training on all ten tasks of CIFAR100. The green lines are the means over three different runs, the orange shades denote \pm 1 standard error of the mean. The labels to the vertical bars denote the accuracy values for the first iteration of training on each task.
Figure 13 : Test accuracy with weight initialization on the first task for the initial 100 iterations of incremental training on all ten tasks of CIFAR100. The green lines are the means over three different runs, the orange shades denote \pm 1 standard error of the mean. The labels to the vertical bars denote the accuracy values for the first iteration of training on each task.; Test accuracy with weight initialization on the first task for the initial 100 iterations of incremental training on all ten tasks of CIFAR100. The green lines are the means over three different runs, the orange shades denote \pm 1 standard error of the mean. The labels to the vertical bars denote the accuracy values for the first iteration of training on each task.
(a) L2 norm of mean of past task heads; (b) L2 norm of standard deviation of past task heads; Evolution of mean and standard deviation of past task encoders with training iterations.
(a) L2 norm of mean of current task heads; (b) L2 norm of standard deviation of current task heads; Evolution of mean and standard deviation of task encoders (recorded at the step where they were first introduced) with training iterations.
Figure 16 : Performance comparisons for post-hoc novel data detection averaged over 3 runs on CIFAR100: FPR95 ( left ), AUROC ( middle ), and AUPR ( right ). The evaluations are carried over all but the last incremental test step.; Performance comparisons for post-hoc novel data detection averaged over 3 runs on CIFAR100: FPR95 ( left ), AUROC ( middle ), and AUPR ( right ). The evaluations are carried over all but the last incremental test step.
Performance comparison of different methods averaged over three runs. Best scores are in bold . Second best scores are in blue . The results for L2P, DualPrompt, and PROOF are taken from [ 92 ] . See App. Table 14 for std. dev. scores.
Performance comparison on the CDCL setting [ 74 ] . All CLIP-based methods use the ViT-L/14 backbone.
Ablations of the key components of CLAP 4 CLIP on CIFAR100.
Performances of different priors averaged over 3 runs on CIFAR100.
PhNDD performances averaged over 3 runs on CIFAR100. Best scores for each variant are in bold .
Entropy-based exemplar selection results for different methods on CIFAR100.
Benchmark datasets and their details.
Hyperparameter tuning : we run a gridsearch on the CIFAR100 setup with a validation set comprising 10\% of the training set. The chosen values are reused across all other setups.
Accuracy vs. Training epochs
Accuracy vs. Finetuning epochs
Accuracy vs. weight “ \gamma ” for \mathcal{L}_{\text{KD}}
Accuracy vs. weight “ \lambda ” for \mathbb{D}_{\text{KL}}
Performance comparison of VT-CLIP styled VGA with Ours on CIFAR-100.
Standard deviation (std. dev.) scores comparison for Avg. accuracy scores of Table 1 between our variants and their corresponding baseline prompt-based finetuning methods over three runs. In general, our std. dev. scores are comparable to or lower than the corresponding baseline methods and are thus statistically significant.
Backward Transfer (BwT) scores \uparrow comparison between our variants and their corresponding baseline prompt-based finetuning methods averaged over three runs. Best scores across each pair is highlighted in bold .
Transfer scores [ 89 ] \uparrow comparison between our variants and their corresponding baseline prompt-based finetuning methods averaged over three runs. Best scores across each pair is highlighted in bold .
Expected Calibration Error (ECE) scores \downarrow (computed over 15 bins) comparison between our variants and their corresponding baseline prompt-based finetuning methods averaged over three runs. Best scores across each pair is highlighted in bold .
Average inference time for different finetuning methods on CIFAR100.
Performance comparison on CIFAR100 using hand-crafted vs. LLM-generated prompts for encoding language-aware CLIP knowledge. The results reported are averages over 3 runs. Best results across the metrics are highlighted in bold .
Single-task learning performance on the VQAv2 subset of the CLiMB dataset.
Influence of the context set size used to derive the data-driven prior on CIFAR100.