Bridging between Pool- and Stream-Based Active Learning with Temporal Data Coherence
Authors: Sebastian Schmidt
Stephan Günnemann
Venue: Submitted to ICLR 2023 (2023)

Abstract: Active learning (AL) reduces the amount of labeled data needed for training a machine learning model by choosing intelligently which instances to label. Classic
pool-based AL needs all data to be present in a datacenter, which can be challenging with the increasing amounts of data needed in deep learning. However,
AL on mobile devices and robots like autonomous cars can filter the data from
perception sensor streams before it ever reaches the datacenter. In our work, we
investigate AL for such image streams and propose a new concept exploiting their
temporal properties. We define three methods using a pseudo uncertainty based
on loss learning (Yoo & Kweon, 2019). The first considers the temporal change
of uncertainty and requires 5% less labeled data than the vanilla approach. It is
extended by the change in latent space in the second method. The third method,
temporal distance loss stream (TDLS), combines both with submodular optimization. In our evaluation on an extension of the public Audi Autonomous Driving
Dataset (Geyer et al., 2020) we outperform state-of-the-art approaches by using
1% fewer labels. Additionally, we compare our stream-based approaches with
existing approaches for AL in a pool-based scenario. Our experiments show that,
although pool-based AL has access to more data, our stream-based AL approaches
need 0.5% fewer labels.

Under review as a conference paper at ICLR 2023
BRIDGING BETWEEN POOL- AND STREAM-BASED AC- TIVE LEARNING WITH TEMPORAL DATA COHERENCE
Anonymous authors Paper under double-blind review
ABSTRACT
Active learning (AL) reduces the amount of labeled data needed to train a ma- chine learning model by choosing intelligently which instances to label. Classic pool-based AL needs all data be present lenging with the increasing amounts of data needed in deep learning. However, AL on mobile devices and robots like autonomous cars can filter the data from perception sensor streams before it even reaches the datacenter. In our work, we investigate AL for such image streams and propose a new concept exploiting their temporal properties. We define three methods using pseudo uncertainty based on loss learning YYoo & Kweon 2019). The first considers the temporal change of uncertainty and requires 5% less labeled data than the vanilla approach. It is extended by the change in latent space in the second method. The third method, temporal distance loss stream (TDLS), combines both with submodular optimiza- tion. In our evaluation on an extension of the public Audi Autonomous Driving Dataset (Geyer et al. 2020) we outperform state-of-the-es 1% fewer labels. Additionally, we compare our stream- based approaches with ex- isting approaches for AI. in pool-based scenario. Our experiments show that, although pool-based AL has more data access, our stream-based AL. approaches need 0.5% fewer labels.
datacenter, which can be chal-
approaches by using
INTRODUCTION
Active learning (AL) is a technique to minimize the labeling effort, in which a machine learning model chooses the data to be labeled by itself. It can be divided into two main scenarios, pool-based and greeambasedd AL (Settles) 2010 Pool-based AL is a cyclic process of selecting batches of the most promising samples from pool of data based on query function. The model is retrained after the selection to start the next iteration of the AL cycle. The data pool is stored such that all samples are always accessible. In contrast, stream-based AL assumes an inflow of samples classic stream-based AL
stream and
the model decides if a sample should be saved and labeled or disposed. the model is trained with each selected sample (Settles) 2010). However, in deep learning samples are usually selected batches, due to the long training time of the models. This comes with the risk of selecting samples with equal information gain. Most approaches ignore this fact or solve it by using a small selection batch size.
Besides the scenarios, the selection method, also called querying strategy, is another important fac- tor of AL methods. There are three main categories of AL algorithms: amcertainty-based diversity- based and learning-based AL (Ren et al.) 2022p. The first group are uncertainty-
AL methods, including for example Monte Carlo (MC) dropout methods (Gal & Ghahramani 2016) or meth- ods approximating the uncertainty by using ensembles (Beluch et al. 2018). The second group are diversity- based methods like Coreset (Sener & Savarese, 2018) or diverse embedded gradients (Ash al., (220)p These methods select samples based on the dataset coverage. The third group are learning- based approaches. These methods, like loss learning (Yoo & Kweon, 2019), train an addi- tional model, which either predicts a value, determining the usefulness sample should be selected directly. Recent approaches from this category often include unlabeled data for unsupervised training. Other approaches taking diversity into account usually perform an optimization, which requires constant access to the complete labeled and unlabeled dataset. This decreases the number of needed samples as intended, but the access to unlabeled data makes the transfer to stream-based scenario impossible.
sample, or decides if a

---

Under review as a conference paper at ICLR 2023
A large body of research in the perception domain focuses pool-based AL, which requires the transfer of all data to a datacenter. Especially in autonomous driving AL is already an important research topic (Feng al., 2019) (Hekimoglu et al.) 2022). However, the data logistics and data preparation limits the possibilities to apply and scale this approach to open world perception prob- lems, where lot of data is required. These perceptions task including autonomous driving and robotic perception and environmental sensing. In contrast to pool-based AL, stream- based AL can run directly on mobile devices used in these applications and enables data collection through large number of agents without a prior transfer to the data center. By performing AL on a mobile robot, it can be applied on temporally coherent camera streams directly, which reduces preprocessing efforts. Based on these considerations we focus on stream-based AL for temporally coherent data.
Our contribution can be summarized as follows: We suggest a novel concept of incorporating tem- poral information into AI., especially stream-based AL. Our concept exploits the temporal change of uncertainty and distance in latent space. Based on this we propose three methods and compare them with state-of-the-ar methods in classification task; the most commonly used task to bench- mark AL. We evaluate our methods against other state-of-the-art methods. Therefore, we create a operational domain detection dataset by adding scene annotations to the Audi Autonomous Driving Dataset (A2D2) (Geyer et al., 2020). Further, we give an overview of the necessary steps to trans- form pool-based scenario in a stream-based scenario and perform, the best of our knowledge, the first direct comparison between stream-based and pool-based AL. methods.
2 RELATED WORK
While a lot of authors did great research in the field of pool-based AL, stream-based AL has become unpopular with the rise of deep learning. However, the number of vision sensors receiving constant data streams is increasing, search of stream- atream-based AL techniques interesting, as not all data can be transferred to the datacenter to perform pool-based AL.
will the cost of transferring these data to datacenter. This makes
2.1 POOL-BASED ACTIVE LEARNING
Sener & Savarese 2018) defined AL as core set selection problem. The authors aim select samples that minimize the maximum distance to other not selected points. In this way, it can be formulated as a K-center problem. Solving this quite costly, so the authors suggested to use a greedy algorithm to approximate the K-center problem. The method will be further denoted as Coreset. In Bayesian active learning with diverse gradient embedding (Badge) Ash et al. (2020) the diversity idea has been extended by taking the prediction uncertainty into account. The authors combined a distance representation of the latent space with pseudo labels based on the highest one- hot encoded value to generate gradients. These are created for every class such that the dimension of the embedding is higher than the Coreset (Sener Savarese 2018) approach. The optimal set is estimated using greedy optimization algorithms.
An uncertainty-base approach is MC dropout as a Bayesian approximation (Gal & Ghahramani 2016). The method uses several dropout layers which are active during the prediction phase. By performing multiple forward passes a distribution over the class predictions is generated where the authors applied the mutual information function in order to calculate the uncertainty of the samples. This is often combined with the Bayesian active learning by disagreement (Houlsby et al. [201T) metrics, considering the mutual information of the multiple forward passes. Their approach has been modified by Kirsch et al. (2019) to take the diversity of the selected batch into account by calculating the joint mutual information. With their BatchBald approach, the authors reduced the selected samples with redundant information in batch. In contrast to sampling- based approaches, loss learning YYoo & Kweon 2019) a learning-basse based approach that needs only one forward pass. By adding loss module to specific layers of the prediction network, the authors predicted the network's 's loss and used it as pseudo uncertainty for sample selection. However, the loss module can only predict relative loss. The authors showed the flexibility of the approach for several tasks, which makes it quite popular. Novel learning-based methods like variational adversarial active learn- ing (VAAL) (Sinha et al. 2019) use the unlabeled data as well. An autoencoder is trained to learn a latent space representation of the data based on the labeled and unlabeled set. Based on the latent space encoding, discriminator model is trained to discriminate between labeled and unlabeled data.
2

---

Under review as a conference paper at ICLR 2023
The selection is based on the lowest prediction confidence of the discriminator out of the unlabeled dataset predictions. The authors outperformed algorithms like Coreset (Sener & Savarese, 2018 and MC dropout (Gal & Ghahramani] 2016) methods. Kim et al. (021) extended VAAL (Sinha et al. 2019) by adding a loss prediction module to the task model and the predicted loss to the latent space such that it is included in the input vector of the discriminator model. A more diversity-oriented learning-based approach using unlabeled data is sequential graph convolutional network for active learning (Caramalau et al. 2021). The authors used the distance between the features of the task model to calculate an adjacency matrix for graph containing labeled and unlabeled data. Based on this matrix a graph neural network is trained. By using message passing the network should predict the nodes' value for being labeled. With this approach, further denoted as CoreGCN, the authors achieved a good performance on classification datasets.
2.2 STREAM BASED ACTIVE LEARNING
Stream- based AL has rarely been used for perception tasks so far, especially with deep learning mod- els. In the field of perception Narr al. (2016) selected data by using mondrian forests stream-based AL and trained them in an online learning fashion. For non-deep neural network models online and incremental learning (Chiotellis et al. 2018 is often combined with AL as classical stream-based AL models are retrained after each selection. Another challenge is dealing with conceptual drifts (Eukasz Korycki al.] 2019). (Pham et al. (022) in which the underlying distribution chang- ing over time. Especially in treeam-based AI., the selection is seen as a submodular optimization problem where the value of an added labeled sample is dependent on the labels already present. As solving these problems is computationally expensive, stream-based greedy algorithms are im- portant field of research (Fujii & Kashimal (2016). A method for solving submodular optimizations (Kazemi al. 2019). The concept of submodular optimization opens many possibilities for future work. Sieve-Strent Strve--treaing+++ has been used by Senzaki & Hamelain (222I) explicitly for AL on edge devices (ALED). The authors tried different semi cemi-positive defi- nite kernels with the prediction confidence as a value function. Temporal and stream properties are neglected in their work.
problems is Sieve-Streamintt
In our work we will neglect online learning and concept drifts and focus on connecting pool- and stream-based AL for perception.
2.3 ACTIVE LEARNING TEMPORAL DATA
Although temporal coherence is an important property of stream, this property is only used for pool-based AL in previous works. [Bngaar et al. (2019) used the object detection false positive (FP), true positive (TP), false negative (FN) and true negative (TN) metries to build temporal graph and select samples with energy minimization. As this approach requires ground truth, it can be only used theoretical baseline. Besides, the authors provided the SYNTHIA-AL dataset, based on the SYNTHIA (Ros et al. 2016) dataset created for AL purposes. Due to the short snippets and high sampling rate, the dataset mostly targets semantic segmentation object detection applications. Schmidt et al. (2020) used the object detection classification uncertainty estimated by the entropy over time horizon. To do so the authors used preceding and succeeding images of each Kitti (Geiger et al.) 2012) sample. By using this approach comparable uncertainty can be estimated, avoiding the usage of ensembles (Beluch et al. 2018) or MC dropout (Gal & Ghahramani (0066) methods. Nevertheless, the authors only described this approach for pooi-based AI., HHang et al. (2018) used temporal information to avoid multiple MC dropout forward passes (Gal Ghahramani (0166) for semantic segmentation by combining forward pass uncertainty prediction with flow network to calculate the uncertainty as moving average over a time horizon.
Although many topics have been covered, in particular in pool-based AL, temporal properties have only been used to save computation for MC dropout (Gal & Ghahramani 2016) passes. Temporal properties for stream-basee based AL still appear to be a relatively unexplored research topic. We want to investigate the change of uncertainty and diversity over time, especially for the seldomly covered stream-based AL..
3

---

Under review as a conference paper at ICLR 2023
|
Train Nitural Network
M|
Train Niuuaal
Nenwork
Neural
|00
Train Neural
Untabeled Streams
3.6
3.4
Selecctoon
Unrabeted
Pool W
Human Annotator
Label
Urtabeted Pool
Human Annorttor
Label
Humaa
Pool
Ladaa
Pool
Armotattr
Unabe ed Streams
Additional Steams
(a) Pool
(b) Pool stream
(c) Stream batch
(d) Stream
Figure 1: Active learning scenarios from pool to stream
FROM POOL-BASED TO STREAM BASED-ACTIVG LEARNING
As pool-based and stream-based AL are quite detached, we want to bridge the gap and enable com- parisons by adding two intermediate scenarios. Namely the pool stream and stream batch scenario. A collection of the different scenarios is depicted in Figure [ All scenarios start with small la- beled dataset (I) which is used to train a model (2). In the classic pool-base scenario, shown in selected from constant unlabeled pool (3) and sent to oracle (4) for labeling. All samples including the unlabeled ones can be seen and used multiple times. The second scenario depicted in Figure [b] reflects a continuous data selection, in which the unlabeled pool (3) is extended at every cycle. To reflect limited recording and transferring capabilities we change the scenario stream-based one. We remove the pool to create the stream batch scenario depicted in Figure Ic] where the current stream is visible for the model only once. In contrast to the classical stream scenario, batch B (3) of a maximum size of b can be selected from each stream. the clas- sic stream scenario depicted in Figure ld the samples need to be chosen or disposed immediately. This adds the challenge of defining a threshold function classifying useful and useless samples.
Figure Ta the samples
Having introduced the four scenarios, we define four categories for the AL querying strategies and evaluate the possibility of using them in stream scenarios. The first category contains methods evaluating samples individually like loss learning (Yoo & Kweon 2019), ensembles (Beluch et al.) 2018) and MC dropout (Gal Ghahramani (016) methods. These can be used for stream and pool scenarios without any adaptation. The second category describes methods performing optimiza- tion which requires access to all unlabeled data during the optimization. This category contains mostly the diversity- based methods Coreset (Sener & Savarese 2018), Badge (Ash et al. 2020) and BatchBald (Kirsch et al. 2019). which cannot be used for stream- can be used if the greedy optimization can be transformed work on streams. The third category contains methods that use unlabeled data for training, such as CoreGCN (Caramalau et al. 2021) I) or
AL directly. However, they
VAAL VAAL(Sinhaa et al. 2019). and cannot be transferred to stream-based scenarios. The fourth category contains methods that are stream- based, such as the video summarization (Kazemi et al.) (211)) and ALED (Senzaki Hamelain 2021). As current AL research in perception focusing on the third category, the number of methods that can be transferred to a stream scenario is quite limited. Only the first and fourth category can be used in all AL. scenarios.
TEMPORAL INFORMATION IN PERCEPTION DATA
Most perception datasets do not contain temporal data. Since these datasets are meant for classi- fication, object detection or semantic segmentation tasks this information naturally of lower im- portance for the task at hand. The commonly used datasets Kitti (Geiger et al. 2012) or Cityscapes (Cordts et al. 2016) aim to have a good diversity to be highly generalizable. Classification datasets like C0 Krizhevsky et al.] 2009), which is often used to benchmark AL, are not temporally ordered data streams. Benchmarking AL on these datasets is sub-optimal and only shows potential label savings datasets that have been manually designed for diversity. Instead, propose that AL shall be benchmarked on camera or sensor streams directly such that no additional manual work besides labeling is needed.
4

---

Under review as a conference paper at ICLR 2023
We create our benchmark dataset based on A2D2 | Geyer al. 2020) which provides temporally coherent frames structured different drives. We assign the classification labels urban, highway, country road and construction site describing the driving environment to create operation domain detection task. This task is important in mobile robotics to estimate if action can be executed safely. The dataset contains several recorded drives in southern Germany, with around 680 frames aver- age per recording. The data is temporally clustered in the latent space by the nature of the drives, which can be seen in Figure 2 Further details can be found in Appendix A
VALUE OF TEMPORAL INFORMATION FOR ACTIVE LEARNING
By defining the drives of the dataset as consecu- tive order streams, properties like the predictive uncertainty dp can be represented as function of time I. As sampling-based approaches are problematic for stream-based applications, due to increased computational cost, use loss module fc (Yoo & Kweon 2019) to estimate
75
Drive
50
Drive
Drive
Drive3
the predictive uncertainty. The predicted loss ć (pseudo uncertainty) of sample can be de- fined as in Equation
Drive
Drive
Drive 9
Drive 10
Op ~ L. sc(w] [2(
(1)
The loss module, as well as latent space repre- sentations depend on the current selected train- ing set and are updated after each AL cycle. As time strictly increasing, the time derivative of the predicted loss & exists. By taking the temporal coherence properties of sensor stream into account and assuming real world (or their simulations) situations, the change between two samples is naturally limited. This effect can be observed in Figure 2 where the different drives form natural clusters. We use the existence of time derivative to propose our first method! Temporal Predicted Loss (TPL):
-25
100
-75
sio
Figure 2: T-SNE analysis with perplexity 30 of the different recorded drives from the training set.
d
A;
= dt
(2)
Based on the selection value A we select the sample with the highest absolute temporal change in the predicted loss, instead of selecting samples with the highest predicted loss for the next AL cycle. By taking the temporal change into account the method can easily filter similar samples as they have close temporal relation well similar uncertainty values. In addition, the method is very sensitive to samples having sudden changes which cause a change in uncertainty, which can be challenging for the model. In Figure 3 we compare the latent space coverage of loss learning and TPL using t-SNE plots. While the vanilla loss learning approach mostly covers only one corner, our approach selects samples all over the latent space.
Our second method, Temporal Distance Loss Change (TDLC), is also taking the diversity of the dataset into account by analyzing the change in the latent space representation of the samples. As Figure 2 shows, the samples of one drive are often grouped in clusters. We want to investigate if the change of distance in latent space a suitable metric to increase the performance of selection query. Thus, we formulate Equation to combine the temporal change of the predicted loss d L, with the temporal change in latent space A. scaled by the factor 5 which is set to one. In this equation, i denotes the sample and X; its selection value. As the magnitude of the learned loss and distance in latent space can be different, we calculate the mean and standard deviation of both values on the fly denoted as mean and std to combine the zero mean unit variance value of both flows.
de
mean
de
df.
mean
8
(3)
std
de
std df
Will be made available upon acceptance to preserve authors' privacy.
5

---

Under review as a conference paper at ICLR 2023
60
Labeled Uniabeled Selected
60-
Labeled Selected
40-
20
20
0
-20
-20
-40
-00
-00
-00
-40
-20
60
80
-80
-00
-00
-0
20
80
Loss learning
(b) Temporal predicted loss
Figure 3: Latent space comparison between loss learning and temporal predicted loss using t-SNE with perplexity 30 for drive 20181 108 084007.
The last method is following the idea of submodular optimization, as these methods take the relations inside the batch into account. We base our method on the srove-Steeamin+++ algorithm and follow the idea of (Kazemi al. 2019) and (Senzaki & Hamelain 2021 use the determinant of a positive semi-definite kernel. We choose to evaluate the distance of the selected samples with the dot product of the feature vectors, which have also been evaluated by (Senzaki & Hamelain, 2021). In contrast to the distance matrix used in (Kazemi et al. 2019) the dot product of the vectors takes the direction in latent space into account. The resulting matrix has the squared norm of the selected vectors the main diagonal, while the other values depend on the orientation of the data points from the origin of the latent space. These mathematical properties lead to a higher diversity, compared to a regular distance matrix. We integrate the temporal change of the predicted loss di L, of the i-th sample with the matrix product of the feature vectors into submodular optimization. The value Ai of the selected set with j elements
Equation 4 to be maximized. Where j is constrained by batch size b to j < b. Fix" denotes the matrix of j latent space feature vectors with equals the feature dimension. We followed Senzaki Hamelain (202I) and set the scaling factor ò to 0.5. This method is further denoted as Temporal Distance Loss Stream (TDLS).
L. log(det(FF' 1,))
As ablation study, we replace Fixm with the gradient embedding based on Ash et al. (000)) Eux(n-c) with e being the number of classes, further referred Temporal Embedded Gradient Loss Stream (TEGLS). This adds information on possible loss directions to the optimization.
6 EXPERIMENTS AND RESULTS
For our experiments, we use the A2D2 object detection dataset labeled as described in Section 4 We start with an initial training images remain unseen to be used as streams in the AL cycles. For the testing and validation set, we use three drives each, with a total of 2776 and respectively 2577 images. All splits and further detail can be found in Appendix A As the streams differ from each other in length, we use percentage selection size for each stream instead of fixed one. At each AL cycle, indicated as marker in the plot, a new stream will be added according to the specified scenario from Figure [ In the results figures we plot the accuracy over the percentage used from the initial training set and the unlabeled pool. As baselines, selections, as well as a random selection strategy. Besides these two commonly used baselines, we introduce fixed step selection strategy which is often used strategy reduce the number of samples in a recording. We use a ResNet18 (He et al.] 2016) model for most experiments as it is the most common model in the related work. Further, extend the classification head three
of two drives with 1674 images. Another nine drives with 4578
use neural network trained on the whole training set including all possible

---

Under review as a conference paper at ICLR 2023
fully connected layers with dropout layers in between, such that it can be compared with sampling methods like BatchBald. For the convolutional layers the pre-trained ImageNet (Deng et al. (009) weights provided by PyTorch (Paszke al.) 2019) are used. As noticed a positive effect of the joint loss from the loss learning module, we add this module to all models for a fair comparison. All hyprrparaeeters and model details listed in Appendix [ After each selection cycle, the model is trained from scratch. As to our knowledge no other comparable datasets are available, we variate the order of streams ingested in the cycles prove the robustness of our approach. The three tested orders are shown alternately in the figures such that each order is shown twice. The same five seeds are used for the different methods. At first, we compare our methods in the ttream-based scenario introduced in Figure Ic and finally we relate them to the pool stream scenario from Figure lb
6.1 TEMPORAL COHERENCE FOR BATCH DIVERSIFICATION
In our first experiment, we want to show the influence of the temporal relation. In Figure 4 we com- pare loss learning with TPL in the stream batch scenario from Figure Ic. We show both approaches for the three selection sizes 5%, 10% and 20%. Besides the ResNet18 model, approaches with the ResNet34 model to prove flexibility. can be seen that our approach outper- forms the vanilla loss learning approach for different models and selection sizes. For all parameters reaches a higher accuracy score with the same amount of labeled data. For ResNet18 shown in Aagure4ajonty our approach reaches the performance of the network trained all data. In Figure 4b our approach clearly reaches the standard deviation region of the fully trained model for selection size of 20%, while the vanilla loss learning approach reaches it in the last step with 5% additional data selected. The overall performance of the loss learning approach is lower for ResNet34 which influences our approach well. Qualitatively the increased diversity of our method can be seen in Figure 5b in comparison to loss learning shown in Figure 5a
compare both
0.80
Accuaty
Accuary
0.76
TPL (ours) 5%
TPL (ours) 5%
10%
0.76
TPL (ours) 20%
(ours)
loss leannnng 5%
0.74
loss learning 5%
10%
learning 10%
0.72
Fully trained
karning 20%
0.72
Fully trained
learning
Trained
0.70
Fully Trained Std
training 196]
training in
(a) ResNet18
(b) ResNet34
Figure 4: Comparison of loss learning with TPL for different selection sizes and models. Colored areas indicate standard errors. The lower standard deviation of the fully trained model is dotted.
In general, it can be seen that approach outperforms vanilla loss learning for different selection sizes and models. The effect is reduced with a larger selection size, which was expected. In addition, loss learning seems not the perfect method estimate the (pseudo) uncertainty of sample as its performance varies between the models in Figure 4 However other approaches require either multiple models or forward passes, which increases computation cost and therefore problematic for streams. This is important if the selection is performed on mobile device directly.
6.2 STREAM-BASED ACTIVE LEARNING
In these experiments, we compare methods introduced in Section 5 with state-of-the-art methods for batch stream-based AL. All experiments in this section are conducted according to the stream batch scenario from Figure Ic After an ablation study we select our TDLS and TEGLS method for further comparison, as state-of-thed -he--rtrrttttte approaches mostly combine diversity and uncertainty as well, details can be found in Appendix

---

Under review as a conference paper at ICLR 2023
(a) Qualitative results loss learning
(b) Qualitative results of TPL
Figure 5: Section of selected images by TPL and loss learning in consecutive order from left right for drive 20181 00880844077
0.81
0.80
Acoaty
Acouary
TDLS (ours)
TDLS (purst
ALED
0.75
ALED
Fixed
0.74
Fixed
Fully hed
Fully Trained
0.73
28
20
30
31
32
33
34
27
20
30
32
33
34
Used
Used training
(a) ResNet18 order
(b) ResNet18 order
Figure 6: Comparison of different stream-base based methods in the stream batch scenario (Figure Ic
Colored areas indicate standard errors.
In Figure we compare TDLS and TEGLS with Random, Fixed and ALED for two different or- ders. In Figure 6b both methods need one stream selection less to cross the fully trained network's s performance at 33% and use 0.5% labels than ALED. In Figure 6a TEGLS cross the fully trained network's 's line as 31.8% while ALED needs about 1% more labels to achieve the fully trained net- works performance at 32.8%. It can be seen that combining the temporal change uncertainty with the most informative latent space encoding works best and outperforms random well as state-of-thh-aat selection methods.
6.3 FROM STREAM BASED TO POOL-BASED ACTIVE LEARNING
The main body of work in the field is focused pool-based AL only. We want compare the two scenarios stream batch (Figure Tc) and pool stream (Figure (b) and close the gap. To the best of our knowledge, we the first compare the different scenarios explicitly. We use the stream batch and the pool stream scenario introduced in Section 3 for this experiment as otherwise, the pool-based methods could select samples from future streams. Pool-based methods can use information that is not available for ftream-basel methods like using data from the unlabeled pool for unsupervised training or optimization approach where the data is visited at each optimization step. Therefore pool- based methods are expected to outperform the brread methods. The goal of these experiments to investigate the decrease in performance that comes along with changing from a pool- wol--bsed to stream-basee setup. As pool-based scenarios require more data logistics and computationally more expensive, a certain decrease performance might be acceptable. Figure 7 shows the results for different orders.
8

---

Under review as a conference paper at ICLR 2023
0.81
0.80
0.79
0.78
0.78
Accuary
0.77
0.76
TEGLS (ours)
(ours)
0.76
TDLS (ours)
TDLS (ours)
CoreGCN
CoreSON
0.75
BatchBald
- Badge
- Badge
Fully Trained
Fully Trained
28
29
31
32
34
27
29
30
Used training data in 194
Used training data in 1961
(a) & order
(b) ResNet18 order
Figure 7: Comparison of stream-based and pool-based active learning. Pool-based methods indicated by a dashed line. Colored areas indicate standard errors.
In Figure 7a TEGLS even reaches the fully trained network's 's performance one iteration before the pool-based methods and can achieve data saving of around 1%. Figure [bb our method does not suffer from any performance loss due to the disadvantage in the scenarios and reaches the fully trained network's line together with other methods 31.5% used data. The performance better than the fully trained network can be explained by the independence of drives in the different sets. So subset can have better distribution
To the best of our knowledge we are the first showing that a stream-based method can compete with pool-based approaches terms of performance. Our proposed method offers data savings like state- of-the-art pool-based methods, while offering the reduced data logisties of stream-basel based approaches. As most perception problems start from sensor (e.g. camera) streams our method can be used on the mobile device directly for many applications like roboties, autonomous driving environmental surveillance. This saves additionally computational costs and the data prepossessing efforts.
Although our ablation study in Appendix C showed that the combination with diversity based ap- proaches work quite well, TPL. and TDL.S do not need any complex matrix operations or submodular optimization techniques.
7 CONCLUSION AND FUTURE WORK
our work, we investigated stream-based AL for temporally coherent data. The proposed theoret- ical modifications that make it possible to exploit the temporal information resulted in three classes of methods. To evaluate these scenarios, we created a classification dataset with temporally coherent data including timestamps based the A2D2 autonomous driving dataset for this purpose, which we made publically available. In our first experiment, we showed that our modifications applied to loss learning outperform the vanilla approach by saving up to 5% more labeled data. Our second experiment proved that our methods combining temporal changes in (pseudo) uncertainty with di- versity lead 1% additional data saving in comparison state-of-the-art methods in stream batch AL. In the last experiments conducted, we provided, to our knowledge, the first comparison between based and pool-based AL using pool stream and stream batch scenarios. These experiments could prove that our stream-basee methods achieve with 0.5% more data savings the same per- formance as pool-based methods for temporal data, so we bridged the gap between them. Given the additional effort, pool-based scenarios require in terms of data logistics, this is major point for enabling large scale AL.
In future work, want to focus alternative uncertainty estimation methods. For one, the uncer- tainty estimation can be integrated more easily with a diversity measurement, which already showed good results in pool- mool-baeed approaches. After proved the benefit of exploiting temporally co- herent vision data, we want to extend our approach to semantic segmentation and object detection. Additionally plan create dataset for steam-based in mobile robot perception.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
To ensure reproducibility we use the five seeds 1, 42, 64, 101 and 999 and set cudnn to "determin- istic". We used the Py PyTorch modelzod implementation of the ResNet18 model and mention all modifications in Appendix B An exact data preprocessing as well as training parameters can be also found in Appendix B The exact dataset detail including all training, validation and test split as well as the split between the initially labeled and unlabeled pool are given in Appendix A
ETHICS STATEMENT
In our work, we deal with data selection and recording on mobile devices. Nevertheless, such recordings are necessary and already conducted, especially in autonomous driving data is collected by different institutions. To respect personal data privacy these recordings are strictly regulated by governmental authorities. We strongly encourage to respect these regulations. Nonetheless, such methods can be used to collect data unauthorized. However, we think that the benefit our stream- based AL approach and data collection mobile devices can create to increase the perception and reliability of these mobile devices, robots and autonomous cars outweigh this risk.
REFERENCES
Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations, 2020.
William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. The power of ensem- bles for active learning image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Javad Zolfaghari Bengar, Abel Gonzalez-Gareia, Gabriel Villalonga, Bogdan Raducanu, Hamed H. Aghdam, Mikhail Mozerov, Antonio M. Lopez, and Joost van de Weijer. Temporal coherence for active learning in videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2019.
Razvan Caramalau, Binod Bhattarai, and Tae-Kyun Kim. Sequential graph convolutional network for active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9583-9592, 2021.
Ioannis Chiotellis, Franziska Zimmermann, Daniel Cremers, and Rudolph Triebel. Incremental semi-supervised learning from streams for object classification. In In proceedings of the IEFE/RSI International Conference on Intelligent Robots and Systems (IROS), pp. 5743-5749, 2018.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Ro- drigo Benenson, Uwe Franke, Stefan Roth, Bernt Schicle, Daimler Ag R&d, and Darm- stadt. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. URL
www. cityscapes-dataeet.com
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Eecognition(CCVPP pp. 248-255, 2009.
Di Feng, Xiao Wei, Lars Rosenbaum, Atsuto Maki, and Klaus Dietmayer. Deep active learning for efficient training of lidar 3d object detector. In IEEE Intelligent Vehicles Symposium (IV), pp- 667-674, 2019.
Kaito Fujii and Hisashi Kashima. Budgeted beeam-based active learning via adaptive submodular maximization. In In proceedings of Neural Information Processing Systems, 2016.
10

---

Under review as a conference paper at ICLR 2023
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In Proceedings the 33th International Conference on International Conference on Machine Learning, pp. 1050-1059, 2016.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), 2012. URL www. cvlibs net dtt.aatts/ssssee///eeea..
Jakob Geyer, Yohannes Kassahun, Mentar Mahmudi, Xavier Ricou, Rupesh Durgesh, Andrew S. Chung, Lorenz Hauswald, Viet Hoang Pham, Maximilian Mühlegg, Sebastian Dorn, Tiffany Fer- nandez, Martin Jänicke, Sudesh Mirashi, Chiragkumar Savani, Martin Sturm, Oleksandr Voro- biov, Martin Oelker, Sebastian Garreis, and Peter Schuberth. A2d2: Audi autonomous driving dataset. arXiv, 2004.06320, 2020. URL httpt: //ww:..:/wwww... a2d2. audi
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2016.
Aral Hekimoglu, Michael Schmidt, Alvaro Marcos-Ramiro, and Gerhard Rigoll. Efficient active learning strategies for monocular 3d object detection. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), 2022.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv, 1112.5745, 2011.
Po Yu Huang, Wan Ting Hsu, Chun Yueh Chiu, Ting Fan Wu, and Min Sun. Efficient uncertainty estimation for semantic segmentation in videos. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
Ehsan Kazemi, Marko Mitrovic, Morteza Zadimoghaddam, Silvio Lattanzi, and Amin Karbasi. Submodular streaming in all its glory: Tight approximation, minimum memory and low adap- tive complexity. In Proceedings of the 36th International Conference on Machine Learning, pp- 3311-3320. PMLR, 2019.
Kwanyoung Kim, Dongwon Park, Kwang Kim, and Se Young Chun. Task-aware variational adversarial active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8166-8173 8175, 2021.
Andreas Kirsch. Joost Van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acqui- sition for deep bayesian active learning. In In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 7026-7037, 2019.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Canadian Institute for Advanced Research, 2009. http://w////
cs. es.toronto..du//keeee
html
Alexander Narr, Rudolph Tricbel, and Daniel Cremers. Stream- based active learning for efficient and adaptive classification of 3d objects. In proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' M'Alché-Buc, E. Fox, and R. Garnett (eds.), In Proceedings of the International Conference on Neural Information Processing Systems, pp. 8024-8035. Curran Associates, Inc., 2019.
Tuan Pham, Daniel Kottke, Georg Krempl, and Bernhard Sick. Stream-based active learning for sliding windows under the influence of verification latency. Machine Learning, 111(6):2011- 2036, Jun 2022. ISSN 1573-0565. doi: 10. 12.10077/1109944-11-009999 URL https: / d/o... org/ 1007 0994 021 06099-2

---

Under review as a conference paper at ICLR 2023
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Yao Huang, Zhihui Li, Xiaojiang Chen, and Xin Wang. A survey of deep active learning, ACM Computing Surveys (CSUR), 54:1 - 40, 2022.
German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M. Lopez. The syn- thia dataset: A large collection of synthetic images for semantic segmentation urban scenes. In Proceedings of the IEEE Computer Society Conference Computer Vision and Patlern Recognition, volume 10le-December, pp. 3234-3243. IEEE Computer Society, 12 2016. ISBN 9781447388504, doi: 10. 00.1109/CVPR.2216.3322
Sebastian Schmidt, Qing Rao, Julian Tatsch, and Alois Knoll. Advanced active learning strategies for object detection. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), pp. 871-876 2020.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In Proceedings of the International Conference on Learning Representations, 2018.
Yuya Senzaki and Christian Hamelain. Active learning for deep neural networks on edge devices.
ArXiv, 2106. 10836, 2021.
Burr Settles. Active learning literature survey, Technical report, University of Wisconsin-Madison, 2010.
Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. Variational adversarial active learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Lukasz Korycki, Alberto Cano, and Bartosz Krawezyk. Active learning with abstaining classifiers for imbalanced drifting data streams. In In Proceedings of the IEEE International Conference on Big Data, 2019.
A DATASET DESCRIPTION
In experiments, used the object detection part of the A2D2 datase The dataset contains 17 different drives in southern Germany. The frames are timestamped with high frequency of up to 10 Hz so that the temporal change of the samples can be evaluated meaningfully. Due to sensor synchronization the rate not constant, but the optical flow does not get lost. This high frequency brings the risk of selecting redundant samples in batch. The recording drives labeled pool and unlabeled pool for training as well as validation and test set as shown in Table In the stream-based setups, the unlabeled drives fed streams into the AL algorithm. The images have been resized with preserved aspect ratio to 240 x 151 pixels. For the training, the images have been normalized with the mean and the standard deviation of the currently selected samples. The training dataset has been shuffled.
split into initial
B DETAILED EXPERIMENT DESCRIPTION
We used for all experiments Py PyTochh for the existing reference methods used the code published by the authors Kirsch et al. (2119 and Caramalau et al. (2021 [5] The submodular optimization approaches are implementation using has been modified with a three fully connected layer classification head with inner dimensions of 256 and 128. In front of each fully connected layer, we added dropout with a probability of 0.3. These layers remained active for the MC dropout based methods with ten forward passes. A softmax activation is attached the last layer. For the convolutional layers the pre-trained weights provided
(Kazem al. 2019 The ResNet18
htttass//ltaaaccco////cccaaaaaat redux
12

---

Under review as a conference paper at ICLR 2023
Table 1: This table shows the dataset split into internal labeled and unlabeled pool training set as well as validation and test set.
Assignment
Sessions
initial labeled
20181 107_113303 20181108 000800099955
20181107 133258 20180807 145028 20180925 135056
unlabeled
20181107 132300 20181204 170238 20181108 084007
order
20180810 20181008 1090221 20181204.154421
20181204 20181204_154421
20181107-132300
unlabeled
20181008 _095521 20180925 -135056 20180810.142822
order 2
20180807 _145028 20181108
01811000084007 084007 20181107_133255
20181 107_13358333 20181108 084007 20180807 1450288
unlabeled
201 10810_142822 20180925 135056 20181008 095521
order 3
20181 107_132300 20181204 154421 20181204 170238
validation set
201 10118899_51155 20181016 20181204_135952
test set
20180925 10181108_1237551 20181108_103155
by PyTorch has been used and freezed. We trained each model with the SGD optimizer using PyTorch 1.11.0 with a learning rate of 0.0001, a momentum of 0.9 and a weight decay of 0,0005 for a maximum of 200 epochs. To ensure convergence we use early stopping on the validation accuracy with patience of 30. The batch size is set to 128. Due to the drive concept of A2D2, the dataset contains samples very close to each and can contain only small changes the image. So the parameters have been selected, such that the model does not overfit the initial set. As performance increase has been observed when training with an attached loss learning module, this module has been attached to all models. This effect
only observed for few amounts of data. The loss learning weight in the loss function set to 1. The learning rate is decreased by a factor of ten, after 160 epochs. The parameters for CoreGCN taken from the authors' implementation Caramalau et al. (202T). The experiments are conducted on Nvidia V100 Graphic cards.
C ABLATION STUDY
0.81
0.80
0.79
0.78
0.78
Accuary
0.77
0.70
0.76
TEGLS (ours)
TEGLS [ours]
0.75
TOLS fours)
TDLS (ours)
(ours)
0.74
TOLC lours)
TDLC tours)
0.73
Fully Trained
0.72
Fully Trained
28
training
training
(a) ResNet18 order 2
(b) ResNet18 order
Figure 8: Comparison of the introduced stream-based methods in the stream batch scenario (Figure Ic). Colored areas indicate standard errors.
We compare our methods introduced in Section with each other an ablation study to investigate show the comparison for two different stream orders. can be seen, that the temporal change in latent space distance (TDLC) without submodular optimization, does not generate huge improvement, only minor one be in Figure Sal As the distance reflect already the change latent space, the derivative of this change does not seem add much value. However, the combination of the distances (TDLS) embedded gradients
the effect of our proposed adaptions. In Figure S
13

---

Under review as a conference paper at ICLR 2023
(TEGLS) with the temporal loss submodular optimization problem seems to improve our base approach.
14


Reviews:
[{"review_id": "yb3WgBKWfLP", "reviewer": "Reviewer_aUcz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose an Active Learning method for AL in a streaming context for an application of AL on edge (sensor) devices such as a fleet of cars or robots. Additionally, they define two contexts with different configurations of pool-based batching and streaming of data. Then, they introduce and evaluate variants of loss learning (Yoo & Kweon, 2019) on a new 4-class classification task defined on the Audi Autonomous Driving Dataset (Geyer et al., 2020) and ResNet-based architectures.", "summary_of_the_review": "The paper addresses the field of stream-based active learning which is definitely relevant for a lot of field applications where actual systems are being deployed. However, as the paper misses a lot of related work and does not support its central claims neither theoretically nor experimentally, I do not see ICLR as an appropriate venue for this work. Moreover, writing needs to significantly improve, making this paper not mature enough at this point.", "strength_and_weaknesses": "**Strengths:**\n- Addresses a relevant problem: early pre-processing of data close to sensors\n\n**Weaknesses:**\n- Language in-correctness and lack of clarity (see below)\n- Motivation of the method (why is the classification of highways vs. construction sites important?) \n- Claims are not well supported by experiments nor by theory\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "aJ11ei6HV8", "reviewer": "Reviewer_SVPd", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper applies stream-based active learning to datasets with time stamps. The main idea is to select data points with a large loss change $\\left\\lVert \\frac{d}{dt} \\hat{\\mathcal{L}} \\right\\rVert$, where $\\hat{\\mathcal{L}}$ is an estimated loss. The authors propose three methods by following this policy, and validate their empirical quality in object detection experiments.", "summary_of_the_review": "Stream-based active learning with temporal information is an important problem and the proposed algorithms based on the loss change is practically useful, but it would be better to discuss in more detail the advantages of the proposed algorithms.", "strength_and_weaknesses": "Strengths:\nAs mentioned in the paper, stream-based active learning is not much studied compared with pool-based active learning. However, since data are often collected in a streaming fashion (e.g. autonomous driving), the importance of stream-based active learning is increasing. This paper proposes practical ideas for stream-based active learning, and then the results might be used in future applications.\n\nWeaknesses:\nThe support for the superiority of the proposed algorithms is not so strong. There is no theoretical guarantee, and why the authors choose the proposed criteria (objectives (1) and (2)) is not explained clearly. Although the authors empirically show that the proposed algorithms perform well in some object detection tasks, but only for the specific task with specific datasets. I think more experiments are needed if the authors claim that these algorithms perform well for general stream-based active learning with time stamps.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YrQvkjcrtMP", "reviewer": "Reviewer_63pA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper shows several methods on how to use temporal information in stream-based active learning. It shows why classical pool-based AL methods cannot be used in this domain. It also shows that some methods can even outperform pool-based methods in terms of data savings. This paper closes the gap between pool and stream-based active learning. The authors performed several experiments on the public Audi Autonomous Driving Dataset and showed marginal improvements over state-of-the-art approaches by using 1% fewer labels.", "summary_of_the_review": "Overall this is a concise paper but some parts could be revised to further enhance reproducibility. \nAlgorithm descriptions would be very helpful for the reader. While some related research on stream-based AL is given it does not give a correct representation of the actual research that has been done in this domain. The obtained results should be discussed in more detail to show the usefulness of the proposed approach. In its current form, it is hard to judge its value.", "strength_and_weaknesses": "+ The paper is written clearly and easy to follow. \n+ The paper gives a view on how to transform pool-based methods into stream-based methods.\n+ The custom dataset can be easily reproduced by the dataset description or downloaded as it is publicly available, and all hyperparameters needed to replicate the results are given.\n\n- related work section can be improved a lot as it does not cover the current state of the art. Good literature overview of existing pool-based query methods and their (dis)advantages. \nAn overview of the properties of stream-based data, as opposed to pool-based data, is missing. For example, the non-stationary nature and possible concept drift of stream-based data are very important properties that can be solved by (stream-based) Active Learning.\n\n- Paragraph 2.2 only contains one Stream based active learning technique. There are many more that could be discussed to give a better view of previous research. For example, it would be good to add some info about “RAL - Improving stream-based AL by RL”. There is much more research on stream-based active learning than this paper suggests. \n\n- The obtained results can be discussed further to show the real value of the proposed approach. In the current form, it is very hard to see why the proposed approach required 1% less data than other ALs (sec. 6.2, and 6.3).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject