# Active Learning Based on Crowdsourced Data
## 4 The Proposed Model Improvement Process
### Iteration Time Optimization
#### 4.3.2 Training Time  
Training time highly depends on the hardware used and the number of other processes running in the system. For this reason, we decided to consider training time as the number of gradient calculations that need to be done during the learning process. The number of calculations for 10-epoch training for a different number of added samples can be seen in Figure 8. During the first iterations, especially for small batch sizes, the training time is relatively short. As we can observe, however, it quickly rises as new samples are added to the training set. We followed observations found in [30] and decided to use a constant number of training samples in the following iterations. For that purpose, we modified our algorithm once again. We introduced a new training set, a global training set, containing all selected training samples. The training set actually used for training contains new randomly  
Figure 5: Ratio of informative samples to all samples in the training pool for given batch size.  
selected images and a constant amount of random images from the global training set. This way, we aim at training the network with new, informative images and a subset of previously used samples, thus limiting the time needed for training and eliminating the overfitting effect that can occur when we would use only new images for a given iteration.  
Figure 8: Gradient calculations for given batch size.  
Figure 6: AP for network trained with batch 100 and network trained with modified algorithm with non-informative samples removed.  
Figure 7: Number of detections done during training of the network with batch 100 and training of the network using modified algorithm with non-informative samples removed.  
The crucial aspect of this modification is the selection of proper ratios between new samples and those from the global training set. We tested the following configurations:  
* 50:50--50 known samples and 50 new samples,
* 150:50--150 known samples and 50 new samples,
* 300:50--300 known samples and 50 new samples.  
As a reference, we used a network trained using the original approach with a batch size of 50. To eliminate the problem of not enough images in the global training set, we first trained the network for five iterations with a batch size of 100 using the original approach and used the training set created by this approach as the global training set in following iterations, which were done with the aforementioned configurations.  
As we can see in Figure 9, not using all previous training samples does not introduce a heavy impact on the training process, and the AP is on similar levels. The best results were obtained during training of the 300:50 network. They were similar to full training of a 50-batch network but required 62% fewer gradient calculations. The remaining 150:50 and 50:50 networks achieved similar results with AP around 98% and 97.6% of the reference network, respectively. During the training of the aforementioned networks, we had to do 78% and 22% fewer gradient calculations, respectively.  
The aforementioned changes increased the number of informative samples slightly (Figure 10), especially for 50:50 and 150:50 configurations.  
Figure 10: The number of informative samples in the pool for networks trained with only part of the samples used in previous iterations.