A stepped sampling method for video detection using LSTM
Authors: Dengshan Li
Venue: ICLR 2022 Submitted (2022)
Keywords: stepped sampler
LSTM
video detection
psychology
human memory rule

Abstract: Artificial neural networks that simulate human achieves great successes. From the perspective of simulating human memory method, we propose a stepped sampler based on the “repeated input”. We repeatedly inputted data to the LSTM model stepwise in a batch. The stepped sampler is used to strengthen the ability of fusing the temporal information in LSTM. We tested the stepped sampler on the LSTM built-in in PyTorch. Compared with the traditional sampler of PyTorch, such as sequential sampler, batch sampler, the training loss of the proposed stepped sampler converges faster in the training of the model, and the training loss after convergence is more stable. Meanwhile, it can maintain a higher test accuracy. We quantified the algorithm of the stepped sampler. We assume that, the artificial neural networks have human-like characteristics, and human learning method could be used for machine learning. Our code will be available online soon.

Under review as a conference paper at ICLR 2022
A STEPPED SAMPLING METHOD FOR VIDEO DETECTION USING LSTM
Anonymous authors Paper under double-hlind review
ABSTRACT
Artificial neural networks are considered to simulate the human neural network- s, and achieves great progress on object detection, natural language processing (NLP), image generation, etc. Hermann Ebbinghaus proposed the law of human memory and how to improve human learning 1885. Inspiring from Ebbing- haus" work, we propose a stepped sampler based on the "repeated input", which is Ebbinghaus' contribution that how to strengthen the learning. We repeatedly in- putted data to the LSTM model stepwise in a batch. The stepped sampler is used to strengthen the ability fusing the temporal information in LSTM. We tested the stepped sampler the LSTM offered by PyTorch. Compared with the traditional sampler of PyTorch, such as sequential sampler, batch sampler, the training loss of the proposed stepped sampler converges faster in the model training, and the training loss after convergence is more stable, which means that there is no large jitter after the convergence. Meanwhile, can maintain a higher test accuracy. compared with the traditional sampler. We quantified the algorithm of the stepped sampler. We assume that, the artificial neural networks may have human char- acteristics, and human learning method could be used for machine learning. Our code will be available online soon.
INTRODUCTION
The emergence of convolutional neural networks (CNN) (LeCun et al. 1989) has improved the self- learning ability of artificial neural networks. Recurrent Neural Network (RNN) (Mikolov et al.) 2010) is used to process the temporal information data. RNN takes the output of the previous time period as the input of the next time period, effectively using the temporal information of the input sequence.
RNN sometimes may have the problem of gradient disappearance gradient explosion. Hochre- iter et al. (Hochreiter & Schmidhuber 1997) proposed LSTM. LSTM adds gates to RNN, thus it can effectively avoid the problem of gradient disappearance or explosion. These gates include the forgetting gates, the input gates, and the output gates. The forgetting gate seems to be the most important among them. LSTM may simulate the memory process of human brain. Human brain selectively forgets some information for learning better.
Consider that one of the principles of neural networks may be learned from biological neural net- works, for those artificial neural networks with the memory effects, such as LSTM, learning from the memory method of human, which is the repeated input and timely review, we study the effect of this method with repeated input on LSTM detection results, without considering changing the LSTM network structure.
In this study, we learn the effect of the proposed input method on neural networks with memory characteristics, such as LSTM. Specifically, it is to repeatedly input training data by simulating the "repeated input and timely review" method of the human memory, and the "repeated input and timely review" method is proposed by Hermann Ebbinghaus (Ebbinghaus) 1913) in 1885, which is the "Increasing Memory Strength and Rate of Learning in his literature.

---

Under review as a conference paper at ICLR 2022
1.1 OUR CONTRIBUTION
Our views in this paper mainly include the following 3 aspects:
a) A novel sampler is proposed, which implements sampling in a circular and stepwise manner. Compared with the traditional sampler, the loss curve of the LSTM model using this stepped sampler converges faster in training, and is more stable after the convergence, namely there is no large jitter after the convergence. Moreover, its test accuracy curve is more stable either, which has no jitter. When the batch size is 15, the test accuracy of the stepped sampler LSTM is much higher than that of the traditional sampler with the same parameters.
b) The idea of this sampler comes from the laws of human memory, which was proposed by Ebbing- haus (Ebbinghaus) 1913). We courageously assume that, other human learning methods can also be applied to machine learning. One example is the proposal of the attention mechanism (Vaswani al. 2017). Moreover, we believe that artificial neural networks have human-like characteristics from the experimental performance.
<) We try to use mathematical language to describe the temporal information of the video frames. We try to apply the mathematical equations to our experimental results, and analyze that the test accuracy in the experiment is the temporal information between video frames. The derivation process is shown in Aapeedix/AAandee Appendix B
RELATED WORK
Gibbs sampling is one of the earlier data sampling algorithms, which is proposed by Geman et al. (Geman & Geman 1984) in 1984. Gibbs sampling is to make the probability of the data sample approximately equal to the required probability distribution via iterations. Gibbs sampling random- ly selects data from an initial input sequence, and iterates according to the specified conditional probabilities, which are related to the required probability distribution of the final sampling data. After iterations, Gibbs sampling generates data which is consistent with the required probability distribution. Hu et al. (Hu et al. 2018) used neural networks to generate sampler, which transfer the initial data distribution to the target distribution. The method can generate the sampling data at the same time of training. This method works with the un-normalized probability density func- tion. Wang et al. (Wang et al.] 2018) used Generative Adversarial Nets (GAN) (Goodfellow et al.] 2014) to generate the negative samples. The approach is the first to combine GAN with the negative sampling method, which improves the training effect of the streaming recommend system. Chu et al. (Chu al., 2019) proposed novel sampler that can sample both the positive and the negative data from the input data sequences, so as to let the classifier utilize the Regions of Interests and the background of the data. The sampler is used in the few-shot image classifier, which uses the rein- forcement learning method. The reinforcement learning algorithm (Kaelbling et al. 1996) needs to continuously select the regions of interests from the images, subsequently to recognize the content of the Regions of Interests. Sampling these Regions of Interests can improve the efficiency of reinforce- ment learning, for the reason of the reduction of the input samples. Muhammad et al. (Muhammad et al.) ) proposed bi-directional long short-term memory (BiLSTM) with attention mechanism and a dilated convolutional neural network (DCNN) to perform action recognition, which outper- formed the state-of ofute-of-thh-arr the methods. Kwon et al. (Kwon et al.) 2021) proposed opatio-tempone neighbourhood learning method on action recognition, which performed the state-of-the-art.
3 MATERIALS AND METHODS
This paper is from the perspective of data input, rather than the neural network structure, and study the impact of the memory effect the temporal sequence neural networks (such as LSTM). The process simulates the method of enhancing the memory process of human brain, repeats the input data in stepped way. The method is proposed by Hermann Ebbinghaus (Ebbinghaus) 1913) called "Increasing rate of learning" in his book. The specific mode we used was the wheel tactic (Smith 1994) when we recited words, by establishing novel data sampler in the LSTM model training. The dataset in the experiment is UCF101 (Soomro et al. 2012), which is a human action recognition video dataset. The name of each folder indicates the annotation of the video.
2

---

Under review as a conference paper at ICLR 2022
3.1 EBBINGHAUS FORGETTING CURVE
Ebbinghaus forgetting curve (Ebbinghaus 1913) describes the memory effect of human brain time, which was proposed by Hermann Ebbinghaus in 1885. This theory reveals the human mem- ory law. It is also the law of human learning. That is, the loss of human memory when learning new knowledge is drop fast first and slow later. Ebbinghaus also pointed out that, timely review and repeated input are the key point to prevent forgetting, consolidate knowledge, and learn better. Figure illustrates Ebbinghaus forgetting curve, and timely review can reduce the knowledge for- getting, which makes the learning better. Based on Ebbinghaus forgetting curve on the human brain, we simulated Ebbinghaus' method on machine learning. We believe that the experimental results in Section could prove that there is a certain correlation between human learning and machine learn- ing, since the machine learning method with timely review and spaced repeat has a faster learning effect, compared with the machine learning without the human-like method.
Ebbinghaus also found that, making good use of the correlations between knowledge is another key point for enhancing learning. We definite these correlations are temporal information in Appendix[A] Thereby enhancing the use of temporal information is the key to video detection, natural language processing (NLP), etc. We believe that, the partly repeated input of the stepped sampler enhances the correlation and the temporal information.
3.2 LSTM
The LSTM architecture we used in this paper is to start with CNN backbone. The CNN backbone has four convolu- tional layers. The dimension of convo- lution kernels of each convolutional lay- er is 32, 64, 128, 256. The size of each convolution kernel is 5 x 5,3 x 3.3 x 3, x,3, the stride of the convolution ker- nel is 2, and the padding is O. Each con- volutional layer is followed by batch normalization (BN) (loffe & Szegedy 2015) layer and ReLU layer. The last part of the CNN model is 3 fully con- nected (FC) layers, which use dropout function. The dimensions of the 3 FC layers are 1024, 768 and 512 respective. ly. The LSTM used in the paper is the model existed in PyTorch. The input dimension of LSTM is 512, the hidden layer dimension is 512, and the number of hidden layers is 3. The next is two fully connected (FC) layers followed by dropout function. The dimension of the FC layers is 256. The dropout rate of the CNN backbone and LSTM are both 0.3.
Figure The illustration of Ebbinghaus forgetting curve. The red curve is without timely review (repeated input), the blue curve is with I day, 2 days and days spaced repetition. It can be seen that, with the spaced repetition, the learning effect is better.
laye..
3.3 THE STEPPED SAMPLER
Our experiment is compared with the common sampler. Common sampler in PyTorch (Paszke et al. 2019) include random sampler, weighted random sampler, batch sampler, etc. Batch sampling nearly the most commonly used. The previous research is to add memory units to deep learning networks, such as RNN, LSTM, etc. Analogous to human learning, an important point repetition. And the sampler should be an appropriate way to simulate the "repetition", since the data in each batch can be designed be input repeatedly. We suppose that, "repetition" important, not only for human beings, but also for computers. To make computers better the "repetition", analogizing the way how we recite words, we propose a "stepped" repetition input method, which is the stepped sampler.
The structure of the proposed stepped sampler is illustrated in Figure 2 It is established on the batch sampler. The stepped sampler divides batch into some sub-batches. Like human memory, this sampler adopts the principle of adjacent repetition (Crowder) 1968). namely, the back of the previous sub-batch is the same with the front of the next sub-batch.
3

---

Under review as a conference paper at ICLR 2022
The structure of the stepped sampler shows that, the input data of different batches is partly dupli- cated. The repeated input seems to increase the redundancy, but the experimental results show that, with experimental environment, this method can accelerate the convergence of LSTM model. There is a stride between the previous sub-batch and the next sub-batch. The stride size n can be set manually. We helieve that this part repetition enhances the correlation of the input frames, thereby enhancing the temporal information of the imput frames, according to our definition of the temporal information in Appendix A Section 4 describes the comparative experiments on the sampler with different stride size.
3.4
THE ALGORITHM OF THE STEPPED SAMPLER
The stepped sampler is designed on the basis batch sampler. The algorithm designed to imple- ment stepped sampling within each batch via the batch sampler. The workflow of the algorithm is as follows: the data first goes through the sequential sampler of PyTorch, then, they are processed to batches by the batch sampler. Finally, the data in each batch are divided into sub-batches with the same strides by the stepped sampler.
As shown in Figure 2 assuming that the iteration number of the stepped sampler in a batch is d, it can be concluded from the figure:
L m + n x dd
It can be deduced that, the stepped sampler iteration number per batch, d is:
d m
(2)
Equation 2 is used as the iteration number within a batch in Algorithm [ d is computed by the algorithm when L, m, is determined. If d is not an integer, PyTorch will round down to ensure that is an integer. The number of batches calculated by the framework, and the number of epochs is set manually. The algorithm of the proposed sampler is shown in Algorithm []] The idea to implement the stepped sampler in each batch after the sequential sampler and batch sampler of PyTorch. Line 12 of Algorithm i that, after each previous stepped sub-batchi output, the starting coordinate is moved by n (step stride) data from the starting position of the previous sub-batch.
4
RESULTS
4.1 EXPERIMENT SETUP
The system used the experiment was a workstation with 32 GB CPU RAM and a NVIDIA GeForce 1080ti GPU. The processor was Intel i7 8700, the operating system was Ubuntu 16.04 64 bits. PyTorch version used in the experiment was 1.0. Python version was 3.6, Numpy ver- sion was 1.20.4, Sklearn version was 0.20.4, and Matplotlib, Pandas, tqdm were implemented as the software environment. The reason why we chose old version PyTorch is, the performance the old version may not be powelful, however, the experimental effect may be better, since the contrast of the results may be larger. And since the old version may not have too many functions, it can focus on the factor of "repeat input", without the interferences from other irrelevant factors.
We searched the relevant literature, and found that there may be no LSTM literature that applics human learning methods to machine learning for now. Therefore, the experiment we designed is a comparison experiment, which is an ordinary CNN-LSTM, with or without the stepped sampler, and the other parameters are all the same. One of the advantages of this is that, it reduce the influence of other irrelevant factors, and can specifically concentrate on the machine learning results of human learning methods. And the human learning methods is the timely review proposed by Ebbinghaus.
4.2 TRAINING
The detection accuracy evaluation and cross-entropy loss were used for the training of the models. The accuracy evaluation in the experiment used the accuracy score tool in the Skleam package of Python. The cross- loss used the function of PyTorc. The accuracy and loss were graphical- ly depicted in Figure 4
5 and 6 The accuracy and loss were computed every epoch. The dataset of
4

---

Under review as a conference paper at ICLR 2022
UCF101 was split into training set and test set by a ratio of 3:1. After training, an overall accuracy and loss were computed by the test set, to evaluate the performance of the models. The epoch was set to 150. We used Adam as the optimization algorithm. We experimented different batch sizes and step sizes, which was changing the size of I., m and the step stride n shown in Figure? 2
Our experiment is trained from scratch. Training from seratch may decrease the test accuracy, but it can eliminate interference and focus on the stepped sampler. The learning rate was set to 0.0001. The momentum was set to 0.01. The operators of batch normalization (BN) and ReLU activation were used after each convolutional layer in the CNN backbone. The CNN backbone is not shown in Figure 2 The data transformation was applied to enhance the network. The input frames were transformed into 256 x 342 pixels.
4.3 EXPERIMENT RESULTS
Figure are the visualized results. We tested the sampler of batch size 25. Figure 4 presents the experimental results. Each subfigure shows the training loss and test accuracy of the models. The difference of the models is only the sampler, for comparing the results only caused by the sampler. Figure (a) is the model of traditional sampler, which is the sequential sampler and the batch sampler in PyTTorc,, and the other in Figure 4 are the models of the proposed stepped sampler. Figure 4 (b), (c), (d), (e), (f) are only different from the step stride for comparing. From Figure we consider that step stride 2 (batch size 25, step size 20, in Figure 4 (c)) is the optimal. The training loss in Figure 4 (a) has many jitters, even when the epoch is more than 110, while the training loss in the other subfigures are much smoother, and can converge earlier than the traditional batch sampler model (Figure (a)). Nonetheless, the test accuracy score of the traditional batch sampler model (Figure 4 (a)) is slightly higher. The test accuracy of Figure [ (a) can be 0.656, while the test accuracy of the model with stride 2 stepped sampler (Figure 4 (c)) can be 0.603. The test accuracy of the models in Figure 4 is shown in Table
From Figure 4. the following could be concluded: a) In the model training. LSTM with the stepped sampler converges faster than LSTM with the traditional sampler, and the convergence effect is better, i.e., there is no large jitter after the drop. b) When the batch size and the step size are fixed, the smaller the step stride was, the worse the detection effect became. Similarly, the larger the step stride was, the worse the detection effect became either. If the batch size and the step size are fixed, the detection effect seems to be normal distribution of the step stride. e) However, LSTM with the traditional sampler whose batch size is 25 has a higher test accuracy on the test set. Although this value is not much higher than the optimal stepped sampler model (Figure (c)).
From Table T it can be concluded that, for the same batch size, the test accuracy of the LSTM with stepped sampler rises faster than the traditional sampler LSTM. This can also be seen in Figure 4 Figure 5 and Figure are the illustration of the traditional LSTM and the stepped sampler LSTM, when the batch sizes are all set to 20,15 respectively. The training loss of Figure 5 (c) and Figure 6
(c) converge faster than Figure (a) and Figure (a). which denotes that, our method may have
broad-spectrum effect on machine learning. The test accuracy score of Figure 6 (c) is higher than Figure
(a), (a), which denotes that, the stepped sampler L.TTM may have higher test accuracy than the traditional sampler LSTM, when the batch size is 15, step size is 10, step stride is 5 of the stepped sampler LSTM. and the traditional sampler LSTM is with batch size 15. It can be seen that, there is a large jitter when the epoch is about 100 in Figure (a). Figure (b) and Figure (c) have no large jitter after the epoch is about 60. The training loss Figure (c) drops faster than Figure 6 (a). The test accuracy of the three models is shown in Table 2 From Figure 6 we can see that, the training loss of the stepped sampler model still converges faster than the traditional sampler model.
In our experiments, most L.TTM models with the stepped sampler have a more stable convergence of training loss, compared with the traditional LSTM models with the same batch size. Figure (a) and Figure 6 (a) are the loss curves of the traditional batch sampler, it can be seen that, the loss curves have large jitters after the convergence. Other loss curves in Figure 4 and Figure 6 are stabler after the convergence. The stepped sampler LSTM may have higher test accuracy than the traditional sampler LSTM, in the same batch size, which can reach the value of 0.639 (Table
Our test uses the shuffle operation. ShuffleNet (Zhang al. 2018) proves that the shuffle opera- tion can improve the image detection mAP. We analyse that, the reason should be that the shuffle operation can reduce the correlation. According our definition in Appendix A the correlation
5

---

Under review as a conference paper at ICLR 2022
is the temporal information. Therefore, consider that the shuffle operation reduce the tem- poral information. If the shuffle operation is not used during detection, the frames are sequential. We believe that this continuity will have certain impact on the model with temporal information. Since the test data is shuffled, there should be less temporal information among the data, the test results may reflect the detection effect better. The literature (Zhou al., (018) proves that, shuffle operation makes little impact on UCFI01, and we think there would be no disadvantages for us to use the shuffle operation when testing.
R8'
3
N
x
x²
Figure 2: The illustration of the proposed stepped sampler. The upper half of the figure is the LSTM network, the lower half descrihes the workflow of the stepped sampler. The step size m the size of the sub-batch, which is set to 3 in the figure, the step stride is set to 1, the batch size I. is set to 5 for the illustration. We operated the stepped sampling of the input data within each batch. The car images from Yobtube-Objects dataset (Brox & Malik 2010).
Figure 3: The video detection results of UCF101, which uses the proposed stepped sampler.
4.4 THE TRAINING TIME
A batch is divided into multiple sub-batcher might prolong the training time. However, since the repeated data are the same, the training time of a sub-batch is much shorter than an ordinary batch. Therefore, for the total training time, the stepped sampler and the traditional sampler are almost the same. For example, when the batch sizes are all set to 15, the stepped sampler with the step stride of 5 and the traditional batch sampler both take about 60 hours for training with experimental conditions. Moreover, we believe that the training epoch which the stepped sampler needs might be less than that of the traditional sampler.

---

// traverse the elements in batch of the Under review as a conference paper at ICLR 2022
Algorithm 1:
The stepped sampler
Input:
Dataset, batch size L, step size m, step stride and L > m 2
Output: Stepped sub-batch of the dataset
Initialize the dataset by Sequential sampler of PyTorch;
for Batch = 2, |len (ennhachhsampllr) do // use batch sampler to traverse all data
Initialize empty set betcbatch!];
for idr ......... LL do
batch a sampler output the idx-th item batch[idx] into step batch[]; idx+=
if len(step len(step-batch// batch/ [) then // when the size step_batch reaches m Reset step [atch[] to empty set;
return step.batch[];
// output the sub-batch
idz = idr m + n; // move the coordinate to the next sub-batch
by stride
end
end
end
Figure 4: The training loss and test accuracy of the traditional sampler LSTM model and the stepped sampler LSTM model. The blue curves denote the training loss, and the yellow curves denote the test accuracy. The above six models are all with batch size 25. Subfigure (a) is a traditional LSTM model, which uses the batch sampler in PyTorch. Subfigure (b) is with the proposed stepped sampler, whose batch size is 25, step size is 20, step stride is 1. Subfigure (c), (d), (e), (f) are with the stepped sampler, whose batch size is 25, step size 20, step stride 3, 4, 5, respectively. Our purpose is to study the better step stride when the batch size and step size are fixed.
Table 1: The test accuracy of the six models in Figure BatchSampler denotes Figure aa, stride stepped denotes Figure bb), stride 2 stepped denotes Figure cc) etc.
model
epoch 10
epoch 50
epoch 100
epoch 120
epoch 1.50
BatchSampler
0.289
0.621
0.650
0.642
0.635
stride stepped
0.117
0.369
0.358
0.360
0.352
stride stepped
0.307
0.566
0.577
0.602
0.587
stride 3 stepped
0.133
0.471
0.506
0.505
0.514
stride stepped
0.170
0.502
0.506
0.546
0.521
stride 5 stepped
0.232
0.553
0.570
0.568
0.593

---

Under review as a conference paper at ICLR 2022
130
63
Figure 5: The training loss and the test accuracy of the traditional sampler and the stepped sampler model. The batch size of the three models are all 20. The blue curve denotes the training loss, and the yellow curve denotes the test accuracy. (a) is LSTM with the traditional batch sampler of PyTorch. (b) and (c) are LSTM with the proposed stepped sampler, using the stepped size 10, and the step stride of (b) is set to 2, and the step stride of (c) is set to 5. The training loss of (c) converges faster than (a), and the test accuracy of (c) is more stable than (a), when the accuracy of the both seems to be equal value.
Figure 6: The training loss and the accuracy of the traditional model and our stepped sampler model. The batch size of the three models are all 15. The blue curve denotes the training loss, and the yellow curve denotes the accuracy. (a) is LSTM with the traditional sampler, batch sampler in PyTorch. (b) and (c) are LSTM with the proposed stepped sampler, with the same stepped size 10, when the step strides of (b) and (c) are set and respectively. The training loss of (c) still converges faster than (a).
Table 2: The test accuracy of the three models in Figure BatchSampler represents Figure 6(a),
stride 3 stepped represents Figure bb), stride 5 stepped represents Figure c)c)
model
epoch 10 epoch 50 epoch 100
epoch 120
epoch 150
BatchSampler
0.303
0.585
0.607
0.622
0,616
stride 3 stepped
0.178
0.517
0.532
0.544
0.535
stride 5 stepped
0.282
0.593
0.631
0.639
0.637
8

---

Under review as a conference paper at ICLR 2022
5 DISCUSSION
The experiment is to study the detection effect of the proposed sampler, which simulates one of the human brain memory law, repeating the input, to use the temporal information (we think it is the correlation in/between frames) of videos.
As the data are partly repeated inputted, it may be equivalent to the timely knowledge review of hu- man brain, which strengthens the memory of the LSTM network, and reduces the information forget- ting. The process is very similar with human learning, which was revealed by Hermann Ebbinghaus, and it is illustrated in Subsection 3.T LSTM can selectively memorize the temporal information, which is human-like.
From Figure 4 the repeating times of the stepped sampler is not the more the better, as shown in Figure (b), the stride is 1, and the convergence speed of the model is not improved much. In addition, the repeating times of the stepped sampler is not the less the better. As shown in Figure 4 (f), the convergence speed of the model is even slower. Figure 3 (b) also seems to show this. The phenomenon is the same with human learning. Too much repetitive input and too little repetitive input would not improve the learning effect of human. The experiments seem to verify the similarity between machine learning and human learning. We assume that artificial neural networks have human-like characteristics. What is the best spaced repetition, is still need be studied. We assume that, it should be "one solution to one issue", just like human.
Temporal information also seems to have human-like characteristics. Temporal information is the correlation of temporal sequences, analogous to human learning, it is the correlation of knowledge. In human learning, one of the important learning methods is use the correlation of knowledge, and using the temporal information may also be one of the important learning methods of machine learning.
6 CONCLUSION
We refer to the human memory rules, and propose the stepped sampler, repeating input method which uses the timely review approach. The timely review approach was proposed by Ebbinghaus, and is used to strengthen human memory and learning. In our experiments, this method has better promotion on the detection effect of LSTM.. The experimental results show that, compared with the traditional sampler, the training loss of the stepped sampler converges faster, and is more stable after the convergence, i.e., there is no large jitter after convergence. The test accuracy of the model with the stepped sampler also reaches a high point faster and is more stable either. When the batch size is 15, the test accuracy of the stepped sampler LSTM is significantly higher than that of the traditional sampler with the same batch size. We analyzed the algorithm of stepped sampler and got several equations. Ebbinghaus also pointed out that utilizing the correlations between knowledge is another key to learn better. We believe that this part repetition of the sampler enhances the correlation of the input frames, thereby enhancing the temporal information of the input frames, from our definition of the temporal information in Appendix A
We try to use the mathematical language describe the temporal information of video frames, which is shown in Appendix A Since these mathematical descriptions do not involve specific artificial neural networks, the parameters of neural network are not added the equations.
We try to use some human learning methods to study artificial neural networks. Compared with the traditional sampler, the stepped sampler LSTM has a faster learning effect, and has a higher test accuracy under certain parameters. The results show that, there may be a close relationship between biological neural network and artificial neural network, whatever in structure and even in principle. How to improve human learning is different for each individual, and the test accuracy of our ex- periment may illustrate this point, and we believe that is why not all the stepped sampler LSTM's
test accuracy is higher than the traditional sampler. The attention mechanism (Vaswani et al. 2017) may be also inspired by human learning methods. Transfer learning (Bozinovski & Fulgosi 1976)
is using old knowledge to learn new knowledge, which may be inspired by human learning meth- ods either. We believe that, artificial neural networks seem to have human-like characteristics, and human learning and machine learning seem have some similarities.
9

---

Under review as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
This work was supported in part by the National Natural Science Foundation of China under Grant 61773360.
REFERENCES
Thomas Bayes and David Hume. Bayes's theorem. In Proceedings ofthe British Academy, volume 113, pp. 91-109, 1763.
S Bozinovski and A Fulgosi. The influence of pattern similarity and transfer learning upon training of base perceptron b2. In Proceedings of Symposium Informatica, pp. 3-121, 1976.
Thomas Brox and Jitendra Malik. Object segmentation by long term analysis of point trajectories. In European conference on computer vision, 282-295, Springer, 2010.
Wen-Hsuan Chu, Yu-Jhe Li, Jing-Cheng Chang, and Yu-Chiang Frank Wang. Spot and learn: A maximum-cntroppy patch sampler for few-shot image classification. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6251-6260, 2019.
Robert G Crowder. Intraserial repetition effects in immediate memory. Journal of Verbal Learning and Verbal Behavior, 7(2):446-451, 1968.
Hermann Ebbinghaus. Memory: A contribution to experimental psychology. 1885. New York: Teuchers College, Columbia University 1913.
Stuart Geman and Donald Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on pallern analysis and machine intelligence, (6): 721-741, 1984.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Tianyang Hu, Zixiang Chen, Hanxi Sun, Jincheng Bai, Mao Ye, and Guang Cheng. Stein neural sampler. arXiv preprint arXiv:/810.085 2018
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference machine learning, pp. 448-456. PMLR, 2015.
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237-285, 1996.
Heeseung Kwon, Manjin Kim, Suha Kwak, and Minsu Cho. Learning self-similarity space and time generalized motion for video action recognition. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pp. 13065-13075, 13075, October 2021.
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard Howard, Wayne Hub- bard, and Lawrence Jackel. Backpropagation applied handwritten zip code recognition. Neural computation, 1(4):541-551, 1989.
Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černocký. and Sanjeev Khudanpur. Recurrent neural network based language model. In Eleventh annual conference of the international speech communication association, 2010.
Khan Muhammad, Amin Ullah, Ali Shariq Imran, Muhammad Sajjad, Mustafa Servet Kiran, Gio- vanna Sannino, Victor Hugo de Albuquerque, et al. Human action recognition using attention based Istm network with dilated enn features. Future Generation Computer Systems, 125:820- 830, 2021.
10

---

Under review as a conference paper at ICLR 2022
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga. et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32: 8026-8037, 2019.
Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379-44 1948.
Steven M Smith. Theoretical principles of context-dependent memory. Theoretical aspects of mem- ory, 2:168-195, 1994.
Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint 1212.000, 2012.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Qinyong Wang, Hongzhi Yin, Zhiting Hu, Defu Lian, Hao Wang, and Zi Huang. Neural memory streaming recommender networks with adversarial training- In Proceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 2467-2475, 2018.
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 6848-6856, 2018
Bolei Zhou, Alex Andonian, Aude Oliva, and Antonio Torralba. Temporal relational reasoning in videos. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 803-818, 2018.

---

Under review as a conference paper at ICLR 2022
A VIDEO TEMPORAL INFORMATION DESCRIBED IN MATHEMATICAL LANGUAGE
We consider that, video temporal information is the correlations of the objects between frames, or within a frame. We also consider that, this kind of correlation could be analogized to the correlation of human knowledge, and machine learning may be similar with human learning. From the point, we derive the equations of video temporal information from Bayes' theorem, and mutual information of information theory.
A.1 THE TEMPORAL INFORMATION BETWEEN THE FRAMES
Bayes' theorem Bayes & Hume 1763), i.e. a conditional probability, can be described as:
P(AB) = P(ANB)
(3)
P(B)
where P(A[B) is the probability of event A occurring, under the condition that event B occurs. P(ANB) B) is the probability of event A and event B occurring at the same time. P(B) is the
probability of event occurring. P(B) *
We consider that, Bayes' theorem reflects a temporal correlation, that is, event B occurs first, and then event occurs. The video frame could be analogized kind of Venn diagram, and the objects in the frame could be analogized to the events in the Venn diagram. Thus, we apply this event temporal correlation to the temporal information between frames.
We use the area ratio of object in the video frame R(A) to replace the occurrence probability of the event. The area ratio is defined as:
R(A) area of object area of frame
Refer Bayes' theorem, according to the sequential relations of the temporal information between frames, i.e., the state of the object in the next frame is derived from the state the object in the previous frame, the temporal information between the frames the object A is:
Ta(nsles) R(Aps AAa))
(5)
R(App)
where "nf" represents the next frame, "pf" represents the previous frame, (n (pff denotes the temporal information between the frames of the object A, R(Aps n Anf) represents the overlapping area ratio of the previous frame and the next frame of the object R(Aps) represents the area ratio of the object A in the previous frame.
Similarly, the temporal information between the frames of the object B is:
Ta(nflef) R(Bps BBa) R(Bp))
(6)
And the temporal information between the frames can be described as:
Tof = (p(f(pp)) Te(aslp/) + +.. R(Aps Anss)
+ R(Bps n Bas) R(Bps)
+...
(7)
R(Apr)
where Toy denotes the temporal information between the frames.
A.2
THE TEMPORAL INFORMATION WITHIN A FRAME
The temporal information within frame reflects the correlations of objects in the same image (frame). The correlations within the same image can represented by mutual information (Shan-) non 1948) in information theory. The discrete equation of mutual information is
I(x;Y) = -E p(z, voog p(.. v)
(8)
p(z)p(y)
WEY TEX
where p(a, y) is the joint probability function of and Y, and p(x) and p(v) are the marginal probability functions of X and Y.
12

---

Under review as a conference paper at ICLR 2022
Referring to the discrete equation of mutual information, propose the temporal information with- frame as: when there is an overlapping area between object A and object in the frame,
A
B R(ANB) R(A)R(B)
T(A B)
Σ E R(A MAAA BEF AEF
where ((AN B) represents the overlapping areas, F represents all objects in the frame, R(AN represents the overlapping area ratio of object A and object B in the frame, R(A) and R(B) respectively represent the area ratio of the object A and the object B in the frame.
temporal information when object A and object B have
Correlated objects in frame generally have overlapping areas. For example, blue sky and white clouds, grass and pets, roads and cars, etc. For the correlation of objects in a frame without over- lapping areas, since most of the correlated objects in frame may have overlapping areas, the math- ematical representation of the correlation of objects in a frame without overlapping areas can be weakened by a certain form.
We realize this form by taking the logarithm. Then, its mathematical expression is
B) E Σ log (R(AUB)oOR) Blog R(AUB) B) R(A)R(B)
R(AUR
(10)
BEF AE
where B) represents the temporal information in a frame with no overlapping area between object A and object B, F represents the all objects in the frame, and R(AU UU)) B) represents the total area ratio of object A and object B in a frame. R(AA represents the area ratio of the object A in the frame, and R R)B) represents the area ratio of the object B in the frame.
Since the above equation
R(AU R(AUB)) ) logarithmic operator, when R(AU R(A)R(B)
1, log (R(AU loogg log Bloog RAUB) R(A)EE
R(A)R(B)
RIAUBL
R(AUB) 0: when R(AUB)log( R(A)R(B)
1,
0. This kind of positive and negative value realizes the reduction the correlation of non-overlapping objects in frame. Moreover, such negative values very small. We consider that, the logarithmic operator is suitable for the correlation of non-overlapping objects a frame.
And the temporal information within the frame can be the sum of the overlapping areas and the non-overlapping areas. It can be expressed in mathematical expression as:
Tw. = T(A T(AN B) + T(A
= EΣΣ R(A Bog R(ANB) B)
+ log (RIAU R(AUB)
(11)
R(A)R(B)
R(A)R(B)
BEF
BEF AEF
where fwy denotes the temporal information within a frame.
A.3 THE EQUATION OF VIDEO TEMPORAL INFORMATION
The video temporal information is the sum of the one between the frames and the one within frame. Then, the temporal information of the frame is:
T = Tay + Two
R(Aps N R(Bps (aasBBat Bus) R(Aps)
R(Bp1)
ΣΣ BEF AEF
R(A Byllo(
R(AN
(12)
H(A)R(B)
ANB
R(AUB) R(A)R(B)
log R[AU Bog
BEF
AUB
where T denotes the video temporal information. The equations we proposed try to describe the video temporal information in mathematical language. Since the analysis process is without involy- ing the specific neural networks, the equations are only from the classic equations, without adding the parameters optical flow and Convolutional Neural Network (CNN).
13

---

Under review as a conference paper at ICLR 2022
B THE APPLICATION OF THE EQUATIONS OF THE TEMPORAL INFORMATION IN THE EXPERIMENT
In this section, try apply the equation in Appendix A to analyze the experimental results in Subsection
The accuracy in the experiment the detection result, and the result is of different frames sent into the model. Thus, the test accuracy can be approximately regarded as the temporal information between frames, i.e., the test accuracy s Tof.
The analysis is as follows. Since the UCF101 dataset one object one video, Equation 7 can be transformed into Tis = (n(fllpf = (tAArOAnd The test accuracy in our experiment is essentially the Intersection over Union (IoU) of the bounding boxes. Therefore,
R(A
Area(Apr Aagz)
Area(Ap) [
R(Aps 0 Anr) R(Aps Ans)
The test accuracy IoU
aven of Frame Area(A, UAa))
Area( Ans)
area of frame
(13)
R(App Ans) To5
R(Aps)
In the above equation, since the position of the objects in the UCFI01 dataset does not change much between the previous and the next frames, for the area occupied by the objects, the union set of the objects between the previous and the next frames is approximate the same the previous frame, which can be approximately regarded as R(Aps U Anf) z R(Apr)-
14

---

Under review as a conference paper at ICLR 2022
C THE ROLE OF APPENDIX
A IN THE PAPER
In Appendix A one of our basic view points is that temporal information is kind of correlation. In Subsection 3.1 Ebbinghaus proposed that one the ways to improve human learning is make good use of the correlations between knowledge. When this correlation transfers to the machine learning of video, should be the video temporal information. Therefore, making good use of video temporal information is the key point to video detection. The sampler we proposed can enhance the temporal information.
Moreover, in Subsection we analyzed the shuffle operation and also applied this view. Since shuffle can reduce correlations between the input data, can also reduce the temporal information. Therefore, the interference of the temporal information to the test can also be reduced. The temporal information is undoubtedly helpful for training, since training needs the temporal information to enhance learning. However, if it is also helpful for testing, it will undoubtedly increase the test accuracy. Therefore, we added shuffle operation to the test, for reducing the test accuracy and make the results more objective.
In Section 5 we applied this view either, and pointed out that the key of knowledge engineering lies in knowledge correlation, and for video detection and language processing, it lies in the tem- poral information. The above is the reason why we put forward the appendix of video temporal information.
15


Reviews:
[{"review_id": "rep5FKHUaK", "reviewer": "Reviewer_JLJ8", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper presents many substantial flaws, which I detail below.\n\n# Anachronistic temporal information representation\n\nThe authors start providing a 3 page long formulation of “video temporal information”. The authors base their formulation on the correlation of objects between frames. While this might make sense for some actions, such correlation is solely modelled as the ratio between an object bounding box and the video frame size. The authors then go on with many equations (10 of them) based on the Bayes theorem and Mutual information theory to “model video temporal information”, again based only on bounding boxes ratios. \n\nThis approach is completely oblivious of what has been done in the past 20 years to model actions. No optical flow, no visual features, no CNNs are mentioned at all. In fact, the authors do not include a single paper related to action recognition in their related work. Moreover, it is not clear how this temporal information formulation is even used, if it is used at all in this work. Indeed, after presenting their formulation, the authors continue saying they use a simple CNN with an LSTM to recognise actions. My guess is that the object-box-based formulation is not used, and that CNN features are instead employed, which leaves me to wonder why such formulation is presented in the first place. \n\nTo summarise this point, Section 3.2 (Video Temporal Information) is entirely disconnected to the rest of the paper. The temporal information formulation is overly simplistic and completely unaware of any previous work in the field. It is not clear whether this formulation is even employed.\n\n# Insufficient contribution\n\nThe core idea of this work is to repeat frames in a batch when training an LSTM to recognise actions. This very simple idea does not constitute a sufficient contribution, because it essentially corresponds to augmenting the training data with more frames, which naturally speeds convergence. In short, no novel contribution is presented in any way since the method is just a sampler that repeats frame in a trivial way.\n\n# Flawed and incomplete evaluation\n\nQuoting from page 6: \n\n> Our test uses the shuffle operation, to make the test results more objective. Since the test data is shuffled, there should be less temporal between the data, the test results may depend on the sampler and model only, which could reflect the detection effect well.\n\nFirstly, it is unclear how shuffling would make test results more “subjective”. Secondly, it is arguably incorrect to test a model shuffling the data as this introduces noise in the evaluation. Finally and most importantly, it is not clear why the method would perform better or worse by shuffling data. This confusion stems from the fact that the authors do not specify whether they shuffle the batch (in which case shuffling makes no sense during inference) or whether they shuffle frames within a sequence (which would be a major flaw since the videos would be altered).\n\nBesides, the method is evaluated only on a single dataset without comparison to previous work. Results show that the proposed method under-performs in many cases as well.", "summary_of_the_paper": "This paper proposes a sampler for LSTM video models. The sampler works by repeating frames in a training batch. The method is evaluated for the task of action recognition on the UCF 101 dataset. ", "summary_of_the_review": "This paper presents major problems: it is severely unaware of previous work in the field, does not present a sufficient contribution and it is not well evaluated.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "J1LHfFPLp5P", "reviewer": "Reviewer_hB9n", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1, This paper gives very detailly derivation processes, equations, algorithms for the idea. The illustrated graph clearly states its idea.\n\nWeaknesses:\n1, Performances: It is useless to discuss the convergence speed without getting the optimal convergence value.  For methods comparison in RESULTS part, authors can only achieve better results when batch size equal to 15. And authors don't provide details to illustrate why choose 25 or 15 batch size. The parameters selection seems randomly. The results cannot be predicted and repeated in other situations.\n2, As this paper limits its usage and performances under very narrow scope (for certain task Video detection and certain network structure LSTM) and the results are not good and seem to be unpredictable in further reproduction, there is no need to discuss other weakness. The method is useless.", "summary_of_the_paper": "This paper mainly provides a stepped sampling method only for video detection and only adopted in LSTM structure. This idea is inspired from human's repeatedly memory working mechanism. This method can achieve fast convergence during training, smooth the training loss curves and get a good test accuracy when batch size equal to 15.", "summary_of_the_review": "This paper proposes a very simple idea: repeatedly inputting previous frames into the same units in LSTM structure attempting to imitate humans' repeatedly memory system. But the results seem ugly, the training parameters (batch size) seem to be randomly selected. This method seems impossible to generalize to other training parameters, tasks and network structures.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "2CPdvg8UoVA", "reviewer": "Reviewer_BB9f", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n1. The proposed methodology is technically sound and clearly presented.\n2. The experiments show promising results, lending good empirical credibility to the authors' claims.\n\n### Weaknesses\n1. My main concern is that while the authors argue that their proposed sampling technique is more \"human-like\" or closer to \"how humans learn from data\", I did not find any references in the paper backing up this claim. Are there any studies in psychology, neuroscience, or any other relevant field that grounds the key intuition for stepped sampling, which is repeating the parts of the same input in different batches? Without such grounding, the claim can come across as being made post-hoc, following the experimental success of stepped sampling.\n\n2. Since $d$ in Eq. 11 is called the iteration number, I presumed it is a (non-negative) integer. How, then, is $d$ determined for $L=25$, $m=20$, and strides $n=2, 3, 4$ in the results shown in Figure 3?\n\n3. The authors make claims of stepped sampling leading to stabler convergence compared to traditional sampling techniques in a few places in the paper (e.g., last paragraph in Section 4.3). Is there a way to quantify this claim, e.g., by plotting the rate of change of the loss function across the training epochs?\n\n### Minor comments\n1. Is there any specific reason the authors worked with PyTorch version 1.0.1 (Section 4.1), which came out in early 2019, when much newer versions such as 1.8.0 have been available since early 2021?\n\n2. For the sake of completeness, please specify that $L>m \\geq n$ in Algorithm 1.", "summary_of_the_paper": "The authors present a stepped sampling to improve the learning capabilities of neural network models such as LSTM. Specifically, the stepped sampling procedure repeats the same input data in multiple batches, in other words, the batches \"overlap\" with one another in terms of the contained input data. This follows from the authors' argument that repeatedly providing the same data leads to faster and stable convergence in training, as well as higher accuracy in testing. The authors experimentally show these benefits of their stepped sampling procedure over traditional sampling techniques (e.g., random sampling) in the context of action detection from videos using LSTMs.", "summary_of_the_review": "Overall, the authors propose a technically sound sampling technique and experimentally show its benefits over traditional sampling techniques, such as faster and stable convergence and higher test accuracy, in the context of action detection from videos using LSTMs. However, the authors also attribute the superiority of their technique to the claim that it is closer to how humans learn from data, but this is not grounded with any relevant prior studies or observations. Without such grounding, the claim comes off as post-hoc and the technique comes off as a successful engineering effort rather than a novel research contribution, thus making me lean towards rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a stepped sampler for LSTM-based video detection. However, reviewers raised a series of issues of this paper, including the weakness in novelty, experiment evaluations, and generalizability of the method. Considering the limited contribution of this paper, and limited experiment evaluations, the AC agrees with the reviewers and recommends reject for this paper.
Decision: Reject