Omni-Scale CNNs: a simple and effective kernel size configuration for time series classification
Authors: Wensi Tang
Guodong Long
Lu Liu
Tianyi Zhou
Michael Blumenstein
Jing Jiang
Venue: ICLR 2022 Poster (2022)
Keywords: Time series classification

Abstract: The size of the receptive field has been one of the most important factors for One Dimensional Convolutional Neural Networks (1D-CNNs) on time series classification tasks. Large efforts have been taken to choose the appropriate receptive field size, for it has a huge influence on the performance and differs significantly for each dataset. In this paper, we propose an Omni-Scale block (OS-block) for 1D-CNNs, where the kernel sizes are set by a simple and universal rule. OS-block can efficiently cover the best size of the receptive field across different datasets. This set of kernel sizes consists of multiple prime numbers according to the length of the time series. We experimentally show 1D-CNNs built from OS-block can consistently achieve the state-of-the-art accuracy with a smaller model size on five time series benchmarks, including both univariate and multivariate data from multiple domains. Comprehensive analysis and ablation studies shed light on how our rule finds the best receptive field size and demonstrate the consistency of our OS-block for multiple 1D-CNN structures.

Published as a conference paper at ICLR 2022
OMNI-SCALE CNNS: A SIMPLE AND EFFECTIVE KER- NEL SIZE CONFIGURATION FOR TIME SERIES CLASSIFI- CATION
Wensi Tang', Guodong Long', Lu Liu " ,Tannii Zhou Michael Blumenstein', Jinn Jiang' Australian Artificial Intelligence Institute, University of Technology Sydney, Google
3 University of Washington, Seattle, 'University of Maryland, College Park (wnnsi.tang lu. 10) @ wtt..uu.ssuuuuuu.... londonng.ong, blument.b, jing.jiang } @ @uts.cdu.au, tianyizh@uw.ede
ABSTRACT
The Receptive Field (RF) size has been one of the most important factors for One Dimensional Convolutional Neural Networks (1D-CNNs) on time series classi- fication tasks. Large efforts have been taken choose the appropriate size be- cause it has a huge influence on the performance and differs significantly for each dataset. In this paper, we propose an Omni-Scale block (OS-block) for ID-CNNs, where the kernel sizes are decided by simple and universal rule. Particularly. it is set of kernel sizes that can efficiently cover the best RF size across differ- ent datasets via consisting of multiple prime numbers according to the length of the time series. The experiment result shows that models with the OS-block can achieve similar performance
models with the searched optimal RF size and due to the strong optimal RF size capture ability, simple ID-CNN models with OS-block achieves the state-of-the-ar performance marks, including both univariate and multivariate data from multiple domains. Comprehensive analysis and discussions shed light on why the OS-block can cap- ture optimal RF sizes across different datasets. Code available here
four time series bench-
INTRODUCTION
One of the most challenging problems for Time Series Classification (TSC) tasks how to tell models in what time scales to extract features. Time series (TS) data is a series of data points ordered by time or other meaningful sequences such as frequency. Due to the variety of information sources (e.g., medical sensors, economic indicators, and logs) and record settings (e.g., sampling rate, record length, and bandwidth), TS data is naturally composed of various types of signals on various time scales (Hills al. 2014; Schäter 2015 Dau et al.) 2018). Thus, in what time scales can model "see" from the TS input data has been a key for the performance of TS classification.
Traditional machine learning methods have taken huge efforts capture important time scales, and the computational resource consumption increase exponentially with the length of TS increase. For example, for shapelet methods (Hills al. 2014: Lines al. 2012)) whose discriminatory feature is obtained via finding sub-sequences from TS that can be representative of class membership. the time scale capture work is finding the proper sub-sequences length. To obtain the proper length, even for dataset with length 512, (Hills et al. 2014) has to try 71 different sub-sequence lengths. For other methods, such as (Berndt & Clifford 1994 Schäfer [015]] Lucas et al., 2019), despite the time scale capture might be called by different names such as finding warping size or window length. They all need searching works to identify those important time scales. More recent deep learning based methods also showed that they had to pay a lot of attention to this time scale problem. MCNN (Cui et al. (016) searches the kernel size to find the best RF of a ID-CNN for every dataset. Tapnet (Zhang al. 2020) additionally considers the dilation steps. Chen & Shi (2021) also take
https: : / /i thub. com /aagg/ss-CCM II has different names for different methods. Generally, it refers to the length of the time series sub- sequence for feature extraction.

---

Published as a conference paper at ICLR 2022
The tank UCR line dariantt arthive actieved by mocels difee
BS time datanets archhre
25
20 15
Number
10
Figure I: Left: A model'ss accuracy on the UCR 85 datasets changes by tuning the model's receptive field sizes from 10 to 200. Right: The average rank results of each receptive field size are pretty similar, which means that no single receptive field size can significantly outperform others on most datasets.
the number of layers into considerations. These are all important factors for the RF of CNNs and the performance for TSC.
Although number of researchers have searched for the best RF of ID-CNNs for TSC, there is still no agreed answer to 1) what size of the RF the best? And 2) how many different RFs should be used? Models need to be equipped with different sizes and different numbers of RFs for a specific dataset. Using the same setup for every dataset can lead to significant performance drop for some datasets. For example, "bake off"" datasets in Figure Ta. the accuracy of most datasets can have a variance of more than 5% just by changing the RF sizes of their model while keeping the rest of the configurations the same. As also shown in Figure Db. no RF can consistently perform the best over different datasets.
shown by the statistics the University of California Riverside (UCR) 85
To avoid those complicated and resource-consuming searching work, we propose Omni-Scale block (OS-block), where the kernel choices for 1D-CNNs are automatically set through a simple and universal rule that can cover the RF of all scales. The rule inspired by Goldbach's conjecture, where any positive even number can be written as the sum of two prime numbers. Therefore, the OS-block uses set of prime numbers as the kernel sizes except for the last layer whose kernel sizes are 1 and 2. In this way, a ID-CNN with these kernel sizes can cover the RF of all scales by transforming TS through different combinations of these prime size kernels. What's more, the OS-block is easy to implement to various TS datasets via selecting the maximum prime number according to the length of the TS.
In experiments, we show consistent state-of-the-art performance on four TSC benchmarks. These benchmarks contain datasets from different domains, i.c., healtheare, human activity recognition, speech recognition, and spectrum analysis. Despite the dynamic patterns of these datasets, ID- CNNs with our OS- robustly outperform previous baselines with the unified training hyper- parameters for all datasets such as learning rate, batch size, and iteration numbers. We also did comprehensive study to show our OS-block, the no time scale search solution, always matches the performance with the best RF size for different datasets.
MOTIVATIONS
Two phenomena of ID-CNNs inspire the design of the OS-block. In this section, we will introduce the two phenomena with examples in Figure 2 and more discussions can he found in Section 4.6)
Firstly, found that, although the RF size is important, the ID-CNNs are not sensitive to the specific kernel size configurations that we take to compose that RF size. An example is given in the right image of the Figure
Secondly, the performance of ID-CNNs is mainly determined by the best RF size it has. To be specific, supposing we have multiple Single-RF-size-modee layer numbers, but each of them has a unique RF size. Let's denote the set of those RF sizes as S. When testing those models on a dataset, we will have set of accuracy results A. Then, supposing we have a multi-kernel model which has multiple RF sizes and set of those sizes is also S. Then,
which are of similar model size and
A detailed discussion about how to calculate RF sizes for ID-CNNs with multiple kernels in each layer be found in Section 3
2

---

Published as a conference paper at ICLR 2022
(Recestive fidle 2.00.
Set of recoptive fald size
zach layor
(9)
6.7
0.7
(9.73
(..75.3.1)
srourry
(22)
65
129120.3.72 3.2.2.3
0.5
5.5...
1001:0.30:1111 10.000000.1100 1.3
(92)
1201:30.20
20.
0.3
(04)10017,33
0.5
(94.3 26,
53, 19,193
1o0 1002
(90)550
14,33,33333
200 400 1000
(23.24,89 54.75,44.0
(20 10
Figure 2: Left: The label of each line denotes receptive field size and the kernel configuration of each ID-CNN. For example, (9):5 means the ID-CNN has five layers and the receptive field size is 9, and from the first layer to the last layer, kernel sizes of each layer are 5, 5, 1, 1, and 1. Lines of similar color are ID-CNNs with the same receptive field size, and they are also of similar performance. Right: Lines with similar colors are models which have the same best receptive field size. For example, all (red/green/blue) lines have the receptive field size (9/39/99), and their performances are similar to the bright (red/green/blue) line which denotes the model only has the receptive field size (9/39/99).
the accuracy of the multiple- Cialtiple-RFsiieeee RF-ssize-modee will be similar the highest value of A. An example given in the left image of Figure Specifically, when testing single-RF Beele-RF--izzemoons models on the Google Speechcommands dataset, the model's performance is positive correlation with the model's RF size. For example, the light blue line whose set of RF size is {99} outperforms the light green line {39} and light red [9]. For those multiple-RF-siee Ro-siizs-smooellls which has more than one element in their set of RF sizes, their performance are determined by the best (also the largest because of the positive correlation) RF size it has. Having more worse (smaller) RF sizes will not have much influence the performance.
The second phenomenon means that, instead of scarching for the best time scales, if the model covers all RF sizes, its performance will be similar that of model with the best RF size. However, there are many designs that can cover all RF sizes. Which one should be preferred? Based on the first phenomenon, from the performance perspective, However, as we will show in Section 3.3, those candidate designs are not of the same characteristics such the model size the expandability for long TS data. Therefore, the design of the OS-block that we propose aims at covering all RF sizes in an efficient manner.
could choose any design that want.
3 METHOD
The section organized as follows: Firstly, we give the problem definition Section 3.1 will explain how to construct the Omni-scale block (OS-bll (SS-block which covers all receptive field sizes in Section 3.2 Section 3.3 will explain the reason why OS-block can cover RF all sizes in an efficient manner. In Section 3.4 will introduce how apply the OS-block TSC tasks.
Then,
3.1 PROBLEM DEFINITION
TS data is denoted as X = [1 ..., Rm., where m is the number of variates. For uni- and for m tor of length I, A TS dataset, which has n data and label pairs, can be denoted as: D = {(X) ,y"), (X2,, y²),.., X",y"), where (X*,, xx,y") denotes the TS data X belongs to the class y". The task of TSC is to predict the class label y² when given TS X*.
variate TS data,
the TS are multivariate. Each variate is
3.2 ARCHITECTURE OF OS- -SOBLOCK
The architecture of the OS-block shown in Figure 3 is a three-layer multi-kernel structure, and each kernel does the same padding convolution with input. For the kernel size configuration, we use p(i) to denote the kernel size set of the i-th layer:
{1, 2, 3, 5, ... pk [1,2]
E (1,2) , ==
pp())
(1)
3

---

Published as a conference paper at ICLR 2022
Mathematical phanomenon
1D-CNN block design
1D-CNN classiffer
O5-block
Prime numberits
Sum
Layer
Concatenation
Bitch-oom-RRLU
O6-CNN
Layer oulput
05 biock
36 Jqquiu
Layor
Gobal amerage posling
BRcchoom-RcL
Pully
Lepar ocipon
Ouegot result
Concateration
Green: prime number Red number
Sonchnonm RaUU
Figure 3: The left image shows that every even number from 2 to 38 can be composed via two prime numbers from to 19. This phenomenon can be extended to all even numbers. Based on this phenomenon, with the OS-block structure in the middle image, sizes. Specifically, the first two layers have prime-sized kernels from to Pk- Thus, the two layers cover all even number receptive field sizes. With kernels of sizes and 2 in the third layer, could cover all integer receptive field sizes a range via selecting the value Pk- The OS-block is easy to be applied on time series classification tasks. A simple classifier with the OS-block, namely OS-CNN, is given in the right image, which achieves a series of SOTA performances.
could cover all receptive field
Where (1,2,3,5,7,...,,2 is set of prime numbers from 1 to Pk- The value of ps is the smallest prime number that can cover all sizes of RF in a range. Here, range that we mentioned is all meaningful scales. For example, since the TS length is we don't need to cover RFs that are larger than or smaller than Therefore, the py is the smallest prime number that can cover the RF size from to 1. If have prior knowledge, such as that we know there cycles in the TS, know the length range of the hidden representative pattern. We could change the RF size range of the OS- bloklock by simply changing the prime number list. An example given in the left image in Figure 3. which uses the prime number list in the blue block to cover the RF size range from 10 to 26.
RF sizes of the OS-block: The RF is defined as the size of the region in the input that produces the feature. Because each layer of the OS-block has more than one convolution kernel, there will be several different paths from the input signal to the final output feature (Araujo et al.) [019 Luo al. 2016), and each path will have a RF size. For the 3-layer OS-block, which has no pooling layer and the stride size is 1, the set of RF sizes S is the set of RF size of all paths, and it can be described as:
$ = (1(0) p(2) + p() p E POO, i (1,2,3)}
(2)
For the reasons that (P( are prime number list when E {1,2), the set (p()) + pp1) E pp()) {1,2)) 2 is the set of all even numbers E Thus, we have
S = {e + (3) - (3) E p(3) e E}.
(3)
With Equation 3 and Equation I we have
(ele (()
1
(4)
Where N+ is the set of all integer numbers in the range. Specifically, the 3 = N+ is because a real number must an odd number or an even number, while E the even number set, {e 1le E E} is
"This according to Goldbach's 's conjecture. Specifically, the conjecture states that any positive even num- ber can be composed of two prime numbers. For example, 8 = 5 - 3. 12 7 5, and more examples can be found in the left image of Figure 3 Despite that the conjecture is unproven in theory, but its correctness has been validated up 10 (Richstein) 2001), which is larger than the length of all available TS data.
4

---

Published as a conference paper at ICLR 2022
each variate time series data
Os block with residual connection
05-block with ensemble learning
Input data
Input
Input data
Input data variate
0S-block
05-block
OS
Input
05-alock
05-block
05-block
Global poeling
OS block
Fully conne
Global average pooling
Concatination
OS-block
Fully conne
Global average pooling
Fully connected
05-block
average pooling
Giobal average pooling
connected
Fully connected
Gutppu reaut
Voting
Figure 4: Examples of using OS-block with other deep learning structures.
the odd number set. Therefore, with the proper selection of pk, we could cover any integer RF size in range. It should be noticed that, there might be many options cover all RF sizes, the Godlach's conjecture to make sure that we could all scales.
3.3 OS-BLOCK COVER ALL SCALES AN EFFICIENT MANNER
From the model size perspective, using prime numbers is more efficient than using even numbers or odd numbers. To be specific, to cover receptive fields up to size r, the model size complexity of number pairs
using prime size kernels is O(r²/ (log(r)) On the other hand, no matter or odd number pairs, the model size complexity is (r²²). We also empirically show the advantage of our model efficiency in the following table and this table has been added Appendix A.7
AA.7
3.4 HOw TO APPLY OS-BLOCK ON TSC TASKS
Firstly, the OS-block could take both univariate and multivariate data by adjusting the input channel the same the variate number of input TS data. A simple example classifier with OS- block, namely OS-CNN, is given in Figure 3 The OS-CNN is composed of an OS-block with one global average pooling layer the dimensional reduction module and one fully connected layer as the classification module. Other than OS-CNN, the OS-block is flexible and easy to extend. Specifically, convolution layers of OS-block can be calculated parallelly. Thus, each layer can be viewed as one convolutional layer with zero masks. Therefore, both the multi-kernel layers or the OS-block itself are easy to extend with more complicated structures (such dilation (Oord al. 2016), attention or transformer (Shen et al.] (2188)), and bottleneck) that are normally used in ID- CNN for performance gain. Figure 4 we give three examples which uses OS-block with other structures.
EXPERIMENT
4.1 BENCIMARKS
We evaluate OS-block on TSC benchmarks which include, in total, 159 datasets. The details of each benchmark is as follows:
Magneetoencephalorrapyy recording for Temporal Lobe Epilepsy diagnosis (MEG- TLE) dataset (Gu et al. 2020): The Magneetoencephalograpyy dataset was recorded from epilepsy patients and was introduced
classify two subtypes (simple and complex) of temporal Lobe Epilepsy. The dataset contains 2877 recordings which were obtained at the sampling frequency 1200 Hz. Each recording approximately sec. Therefore the length is about 2400.
University of East Anglia (UEA) 30 archive (Bagnall et al. 2018): This formulation of the archive was a collaborative effort between researchers at the University of East Anglia and the University of Califomia, Riverside. It is an archive of 30 multivariate TS datasets
5

---

Published as a conference paper at ICLR 2022
Individual dataset benchmark
Daraset
Method
Accuracy(55)
FI-score
parameters
CNN [Gi
2020
83.2
82.3
3.8M
PF [Gu et al
A020
82.6
68.2
MEG-TLE
SVM
55.2
Gu et al. 2020
MSAM eu al 2020
83.6
83.4
2.3M
Rocket (Dempster
87.7
OS-CNN (Ours)
91.3
91.6
235k
Multivariate dataset archive benchmark
Archive
Methol
Baseline
OS-CNN (Ours)
lic
Average Rank
DTW-INNDK
Zhang et al 2020
23
0
3.68
INN
Zhatig
ED. (NN(m
Zhangg ef al 100
25
0
7.45
DTW-INND
Zhang al
AM0
5.28
UFA 30 archive
DTW-INN.1
Zhang Mr
4044
22
1
6.07
Bagnall 2018
INN Thiigg arai 2020
7.12
WEASEL+MUSE
Ssciaie Leser
2017
10
19
1
4.15
MLSTM-FOC FCN
KKarm dia 2019
0
5.62
TapNet (Thing
3.80
OS-CNN (Ours)
3.13
Univariate dataset archives benchmarks
Archive
Method
Baseline wins
OS-CNN (Ours) wins
Tic
Avernge rank
PF Lucas et 2019
6.57
ResNet Wain
201
19
61
5
5.41
STC [Hamuuiiann et 2017
56
5.05
UCR archive
InceptionTime
2019
42
4.05
Chen et al. 2015
ROCKET (Deppster et a.
2020
33
44
8
3.64
HIVE-COTE
TS CHIEF (Shilaz el al 2120
42
39
3.68
OS-CNN
3,59
ResNet Wang a.. 2017
19
83
26
3,21
UCR 128 archive Dau al. 2018
Inception Time
[smal Fawaz alal.
2019
39
2,41
ROCKET
[Dempsser
62
23
2.36
OS-CNN (Uurs)
2.02
Table 1: Performance comparison on 4 time series classification benchmarks
from various domains such as motion detection, physiological data, audio spectra classi- fication. Besides domains, those datasets also have various characteristics. For instance, among those datasets, the class number various from 2 to 39, the length of each dataset various from 8 to 17,894, and the number of variates various from to 963.
University of California, Riverside (UCR) 85 archive (Chen et al. 2015): This is an archive of 85 univariate TS datasets from various domains such as speech reorganizations, health monitoring, and spectrum analysis. What's 's more, those datasets also have different characteristics. For instance, among those datasets, the class number varies from 2 to 60, the length of each dataset varies from 24 to 2709. The number of training data varies from 16 8.926.
University of California, Riverside (UCR) 128 archive (Dau al. 2018): This is an archive of 128 univariate TS datasets. It is the updated version of the UCR 85 archive. However, the new archive cannot viewed as replacement for the former because they have different characteristics. For example, for the UCR 85 archive, all TS data within single dataset are of the same length, but that not the same for the UCR 128 archive. Besides that, in general, the added data in the UCR 128 archive, their default test set is bigger than the train set to reflect real-world scenarios.
4.2 EVALUATION CRITERIA
For all benchmarks, we follow the standard settings from previous literature. Specifically, for the MEG-TLE dataset, following Multi-Head Self-Autention Model (MSAM) (Gu et al. 2020), models are evaluated by test accuracy and fl score. Besides using recommended metrics of each benchmark. we also compare the model size of OS-block with other deep learning methods. For UEA 30, UCR 85 archives, and UCR 128 archives. following the evaluation advice from the archive (Dau al. 2018 Bagnall et al. 2018), count wins, and critical difference diagrams (cd-diagram) (Dau et al, were selected the evaluation method. Due to the page limitation, list the average rank
2018

---

Published as a conference paper at ICLR 2022
Accurscy 05-CW sunving
ATOS-CMV AIYAERTY AT SErruSS Carlee
jang
Daranet tamee
Figure 5:
Classification accuracies for OS-CNN vs, accuracies from 20 ID-CNNs with receptive field size. As we can see, for most of the dataset, the orange points (accuracy of OS-CNN) are near the top of blue points. More analysis for this comparison can be found in Appendix A.I
in the result table because it is the main criteria of the cd-diagram. The full cd-diagram results are listed in Appendix A.3
4.3 EXPERIMENT SETUP
For the MEG-TLE dataset, they were normalized by z-normalization (Chen et al. 2015). For the other archives, we take the raw dataset without processing for datasets in those archives already normalized with 7-normalization.
Following the setup of (Wang et al. 2017). we use the learning rate of 0.001, batch size of 16, and Adam (Kingma & Ba (014) optimizer. The baselines are chosen from the top seven methods from the leaderboard of each benchmark. For UCR archives, we ensemble five OS-CNNs, which stacks two OS-blocks with residential connections followed by the baseline IncpetionTime (Ismail Fawaz al. 2019). We use PyTorch 6 to implement our method and run our experiments on Nvidia Titan XP.
4.4 STATE-OF-THE-ANE ART PERFORMANCE ON BENCHMARKS
We show consistent state-of-the-art performance on four benchmarks as in Table As we can see, for the MIT-TLE dataset, OS-CNN outperforms baselines in times smaller model size. For all dataset archives, the OS-block achieves the best average rank, which means that, in general, the OS block design can achieve better performance.
4.5 OS- BLOCK CAN CAPTURE THE BEST TIME SCALE
To demonstrate that the OS-block can capture the best time scale, we build 20 FCN models with different RF sizes (from 10 to 200 with step 10), and compare their performance with OS-CNN on the UCR 85 archive. Specifically, the FCN (Wang et al. 2017) is selected as the backbone model for it has similar structure as OS-CNN.
To obtain FCN with various RF sizes, we change the kernel size of each layer proportionally. To be specific, the kernel sizes of the original three layer FCN 8, 5, and 3, and the RF size is 14. To obtain the RF size 30, we will set kernel sizes of each layer as 16, 10, and 6. To control variables, when the kernel size increases, we will reduce the channel number keep the model size constant. This will not influence the conclusion. To check that, in Appendix A.2] we also provide the static result comparison between OS-CNN and FCNs with the fixed channel number.
http://wwwwwwoeeeesssssssscs/t// hhopss//ptorrch.org/

---

Published as a conference paper at ICLR 2022
Screen doamin
Issettingheetnnee Frequency doamin
FCN2201
PCNI2021
OS:CMN
50
100
153
203
25n
100
350
50
100 150
Broquanyy
1rcouencyy
Figure 6: The class activation map of OS-CNN is similar that of the model which has better performance. For the ScreenType dataset, FCN(10) outperforms FCN(200), and the class activation map of OS-CNN (green) similar to FCN (b))uee) For the Insec(WingbeatSound dataset, the class activation map is similar to FCN(200) for FCN(200) outperforms FCN(10).
Due to the page limitation, the full result can found the supplementary material. And in Figure a simple result comparison is given, and we could see that for most of the datasets, OS- CNN can achieve similar result as models with the best time scale.
4.6 DISCUSSION ABOUT BEST TIME SCALE CAPTURE ABILITY
The result in Figure 5 empirically verifies two phenomena that we mentioned Section 2 with multiple datasets from multiple domains. Firstly, the OS-block covers all scales. Therefore, besides the important size, also covers many redundant sizes, but those redundancies will not pull down the performance. Secondly, OS-block composes the RF size via the prime design while the FCN uses different design. It means that the performances of ID-CNNs are determined mainly by the RF size instead of the kernel configuration to compose that.
4.7 CASE STUDY FOR THE BEST TIME SCALE CAPTURE ABILITY
To further demonstrate the RF size capture ability, we will give a case study that compares the class activation map (Zhou et al. 2016) of OS-CNN with that of models with the best RF size. We select SyreenType and Insect Wingeeetnnee datasets for the case study. They were selected because they are of the largest and the smallest accuracy difference calculated by the accuracy of FCN with RF size 10 (FCN(10)) minus accuracy of FCN with RF size 200 (FCN(200). Specifically, it can be as, among UCR 85 datasets, the ScreenType is the dataset which the FCN(10) outperform FCN((00) most, and Insect Wingbettnnn the dataset which the FCN(200) outperforms FCN(10) most. We visualize the class activation map of the first instance in the datasets, and the results are shown in Figure As we can see in Figure 6 the class activation map of the OS-block is similar that of the model with the best RF size.
RELATED WORKS
A TS data is a series of data points. TSC aims at labeling unseen TS data via model trained by labeled data (Dau al. 2018 Chen et al. 2015). One well-known challenge for TSC telling the model in what time scale to extract features (Hills et al., 2014 Schater 2013) Berndt & Clifford] 19941. This is because TS data is naturally composed of multiple signals on different scales (Hills et al. 2014; Schääfer 2015 Dau et al. 2018) but, without prior knowledge, it is hard to find those scales directly.
The success of deep learning encourages researchers to explore its application on TS data (1 (Längkvist) al. 2014 Fawaz al.] 2019 Dong et al. (02T). The Recurrent Neural Network (RNN) is de- signed for temporal sequence. In general, it does not need extra paper--prameterss to identify infor- mation extraction scales. However, RNN is rarely applied on TS classification (Fawaz et al. 2019).
There many reasons for this situation. One widely accepted reason is that when faced with long TS data, RNN models suffer from vanishing gradient and exploding gradient (Pascanu et al. 2013
Fawaz al. 2019 Bengio et al. 1994).
Nowadays, the most popular deep-learnig method for TSC is ID-CNN. However, for ID-CNNs, the feature extraction scale is still problem. For example, there is unresolved challenge with
8

---

Published as a conference paper at ICLR 2022
kernel size selection where there exists different approaches but non consensus which is best. To date, the selection of feature extraction scales for ID-CNN is regarded as a pprammer selection problem e.g.. (Cui et al. 2 uses grid search to find kernel sizes, while the following methods tune it empirically (Zheng et al. 2014; Wang et al. 2017 Rajpurkar et al. 2017: Serra
et al. 2018 Ismail Fawaz al. 2019 Kashiparekh et al. 2019
Dilated convolution (Oord et al. (016) is widely adopted in ID-CNN to improve generalization ability for TS tasks (Oord al. [016;] Zhang al., 2020: Li et al. 2021). It takes a lower sampling frequency than the raw signal input thus can be viewed as a structure-based low bandpass filter. Compared with the OS- OS-block, the dilated convolution also needs prior knowledge or searching work to set the dilation size which will determine the threshold to filter out redundant information from TS data.
Inception structure (Szegedy et al. 2015) is widely used in ID-CNN for TSC tasks (Ismail Fawaz jt al., 2019 Kashiparekh et al.J 2019 Chen & Shij 202T] Dong et al., 202T). The design of the multi- structure of OS-block is inspired from the inception structure (Szegedy et 2015). Compared with existing works, the OS-block has two differences. Firstly, the OS-block does not need to assign weight to important scales via complicated methods such as pre-train (Kashiparekh cet al. 2019), attention (Chen & Shi 2021 Shen et al., 2018b), or a series of modifications such as bias removal and bottleneck for convolutions (Ismail Fawaz et al. 2019). Secondly, OS-block does not need to search for candidate scales. Specifically, those methods can only assign weight to a limited number of scales. Thus, they still need searching works to answer series of questions. For example, Which sequence, such as geometric or arithmetic, should be preferred? What's the largest length to stop? How do they select the depth of the neural network? And how do they select the common difference or ratio for their sequence?
Adaptive receptive field (Han et al.] 2018; Tabernik et al. 2020 Xiong et al. 2020 Pintea et al. 2021 Liu et al. [202 Tomen et al. [ et al. 2021] D. has been proposed to learn the optimal kernel sizes during the training stage. Generally, can viewed as learning weight mask on kernels to control the receptive field size. The weight of the mask can be learned during the training step. On the other hand, OS-block learns the linkage between kernels and uses kernels of different sizes to compose different receptive field sizes. In principle, the adaptive receptive field can be used on the time series classification tasks. It improves the performance by enabling ID-CNNs to have the best receptive field size. But the OS-block targets at covering all sizes of receptive filed sizes and assign large weight on important sizes.
Mathematically, OS-block is very general technique and can be extended to time series vision tasks by using the prime size design on the time dimension. This is because the video classification task and time series classification task share the same challenge (Xie et al., 2018 Bian al. 2017 Tan
jet al. 2021 Liu et al.) 2020 Li et al. 2020), which is the same region of interest might of different time scales for different data. Thus, using the kernel of various sizes will increase the probability to catch proper scales. However, in this paper, we mainly target the classic ID time series classification.
which is an active research area with many open problems (Fawaz et al. 2019 Zhang et al. 2020 Dempster et al.) (220)) unsolven.
CONCLUSION
The paper presents a simple ID-CNN block, namely OS-block. It does not need any feature extrac- tion scale tuning and can achieve similar performance models with the best feature extraction scales. The key idea is using prime number design to cover all RF sizes in an efficient manner. We conduct experiments
demonstrate that the OS-block can robustly capture the best time scale on datasets from multiple domains. Due to the strong scale capture ability, it achieves series SOTA performance on multiple TSC benchmarks. Besides that, the OS-CNN results reveal two charac- teristics of ID-CNN models, which will benefit the development of the domain. In the future, we could extend our work in the following aspects. Firstly, other than the prime kernel size design, there might be more efficient design to cover all RF sizes. Secondly, the OS-block can work with existing deep neural structures to achieve better performance, but there might be unique structures or variants of those existing structures that are more suitable for the Olockock. Besides that, char- acteristics of OS- block are empirically analyzed via the way there must be a theoretical explanation of the characteristics.

---

Published as a conference paper at ICLR 2022
REFERENCES
André Araujo, Wade Norris, and Jack Sim. Computing receptive fields of convolutional neural networks. Distill, 4(11):e21, 2019.
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint arXiv: /8/ .00075, 2018.
Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157- 1994.
Donald Berndt and James Clifford. Using dynamic time warping to find patterns in time series. In KDD workshop, volume 10, pp. 359-370. Seattle, WA, 1994.
Yunlong Bian, Chuang Gan, Xiao Liu, Fu Li, Xiang Long, Yandong Li, Heng Qi. Jie Zhou, Shilei Wen, and Yuanqing Lin. Revisiting the effectiveness of off-the-shelf temporal modeling ap- proaches for large-scale video classification. arXiv preprint arXiv: ntt.:770.800033 2017.
Wei Chen and Ke Shi. Multi-scale attention convolutional neural network for time series classifica- tion. Neural Networks, 136: 140-100, 2021.
Yanping Chen, Eamonn Keogh, Bing Hu, Nurjahan Begum, Anthony Bagnall, Abdullah Mueen, and Gustavo Batista. The ucr time series classification archive, July 2015. www. cs. ucr edd/ -eamonn/t. htte:/ttmmm___eeee leeeeeeereee_eeeeeeeeeeeeeeeeeee data/
Zhicheng Cui, Wenlin Chen, and Yixin Chen. Multi-scale convolutional neural networks for time 2016.
series classification.
Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, and et.al. The ucr time series archive. arXiv:1810.07758. 2018.
Angus Dempster, François Petitjean, and Geoffrey I Webb. Rocket: Exceptionally fast and aceu- rate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, pp. 1-42, 2020.
Xuanyi Dong, David Kedziora, Katarzyna Musial, and Bogdan Gabrys. Automated deep learning: Neural architecture search is not the end. arXiv preprint arXiv:21/2.092455 2021.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. Deep learning for time series classification: review. Data Mining and Knowledge Discovery, 33(4):9177963, 2019.
Peipei Gu, Ting Wu, Mingyang Zou, Yijie Pan, Jiayang Guo, Jianbing Xiahou, Xueping Peng, Hailong Li, Junxia Ma, and Ling Zhang. Multi-head self self-attention model for classification of temporal lobe epilepsy subtypes. Frontiers in Physiology, 1 11:447,, 2020.
A. Hameurlain, Kčung, R. Wagner, S. Madria, and T. Hara. Transactions on Large-Scale Systems XXXII: Special Issue Big Data Analytics and Knowl- Data- and Transactions on Large-Scale Data- and Wwwlledge-eenteree Systems. Springer edge Discovery. Berlin Heidelberg, 2017. ISBN 9783662556092. btRttttss:/ htttpp:://boos.... wtwpoooooo..oollleoooooooooooooe com. au/ books?id-RR SHZQEACAAJ
Shizhong Han, Zibo Meng, Zhiyuan Li, James O'Reilly, Jie Cai, Xiaofeng Wang, and Yan Tong. Optimizing filter size in convolutional neural networks for facial action unit recognition. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 5070-5078, 2018.
Jon Hills, Jason Lines, Edgaras Baranauskas, James Mapp. and Anthony Bagnall. Classification of time series by shapelet transformation. Data Mining and Knowledge Discovery, 28(4):851-881, 2014.
Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F. Schmidt, Jonathan Weber, Geoffrey Webb, and et.al. InceptionTime: Finding AlexNet for Time Series Classification. arXiv e-prints, art. arXiv: 190901999 Sep 2019.
10

---

Published as a conference paper at ICLR 2022
Fazle Karim, Somshubra Majumdar, Houshang Darabi, and Samuel Harford. Multivariate Istm-fens for time series classification. Neural Networks, 116:237-245, 2019.
Kathan Kashiparekh, Jyoti Narwariya, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. Convtimenet: pre-trained deep convolutional neural network for time series classification. arXiv:/904. /2546, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Martin Längkvist, Lars Karlsson, and Amy Loutfi A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognition Letters, 42:11-24 2014.
Guozhong Li, Byron Choi, Jianliang Xu, Sourav S Bhowmick, Kwok-Pan Chun, and Grace LH Wong. Shapenet: shapelet-neural network approach for multivariate time series classification. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8375-8383, 2021.
Xianhang Li, Yali Wang, Zhipeng Zhou, and Yu Qiao. Smallbignet: Integrating core and contextual views for video classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1092-1101, 2020.
Jason Lines, Luke M Davis, Jon Hills, and Anthony Bagnall. A shapelet transform for time series classification. In ACM SIGKDD, pp. 289-297. ACM, 2012.
Jason Lines, Sarah Taylor, and Anthony Bagnall. Hive-cote: The hierarchical vote collective of transformation-bast based ensembles for time series classification. In 2016 IEEE 16th international conference on data mining (ICDM), pp. 1041-1046. IEEE, 2016.
Lu Liu, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. Attribute propagation network for graph zero-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 4868-4875, 2020.
Lu Liu, Tianyi Zhou, Guodong Long, Jing Jiang, Xuanyi Dong, and Chengqi Zhang. Isometric propagation network for generalized zero-shot learning. arXiv preprint 210.::2:22220038 2021.
Benjamin Lucas, Ahmed Shifaz, Charlotte Pelletier, Lachlan O'Neill, Nayyar Zaidi, Bart Goethals, François Petitjean, and Geoffrey I Webb. Proximity forest: an effective and scalable distance- based classifier for time series. Data Mining and Knowledge Discovery, 33(3))607-6335 2019.
Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 4905-4913, 2016.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv: 1609.639999..... 2016.
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International conference on machine learning, pp. 1310-1318, 2013.
Silvia L Pintea, Nergis Tomen, Stanley F Goes, Marco Loog, and Jan C van Gemert. Res- olution learning in deep convolutional networks using scale-space theory. arXiv preprint arXiv:2106.032 022 2021.
Pranav Rajpurkar, Awni Y Hannun, Masoumeh Haghpanahi, Codie Bour, and Andrew Y Ng- Cardiologist-level arrhythmia detection with convolutional neural networks. 2017.
Jörg Richstein. Verifying the goldbach conjecture up to 4/cdot10*. Mathematicc of computation, 701236): 2001.
Patrick Schäfer. The boss is concerned with time series classification in the presence of noise. Data Mining and Knowledge Discovery, 105--1500 2015.

---

Published as a conference paper at ICLR 2022
Patrick Schäfer and Ulf Leser. Multivariate time series classification with weasel+ muse. arXiv preprint arXiv:1711.1134 2017.
Joan Serrà, Santiago Pascual, and Alexandros Karatzoglou. Towards a universal neural network encoder for time series. In CCIA, pp. 120-129, 2018
Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. Disan: Di- rectional self-attention network for mn/cnn-free language understanding. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018a.
Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. Bi-directional block self- attention for fast and memory-efficiee sequence modeling. arXiv preprint arXiv:/804.00 1804.00857, 2018b.
Ahmed Shifaz. Charlotte Pelletier, François Petitjean, and Geoffrey I Webb. Ts-chief: A scalable and accurate forest algorithm for time series classification. Data Mining and Knowledge Discovery, pp. 1-34, 2020.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE CVPR, pp. 1-9, 2015.
Domen Tabernik, Matej Kristan, and Aleš Leonardis. Spatially-adaptive filter units for compact and efficient deep neural networks. International Journal of Computer Vision, 1280))2009,2 2020.
Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning over heterogeneous devices. arXiv preprint 105.v:2105.000443 2021.
Nergis Tomen, Lilvia-Laura Pintea, and Jan Van Gemert. Deep continuous networks. Interna- tional Conference on Machine Learning, pp. 10324-10335. PMLR, 2021.
Zhiguang Wang, Weizhong Yan, and Tim Oates. Time series classification from seratch with deep neural networks: A strong baseline. In 2017 international joint conference on neural networks, pp. 1578-1585. IEBE, 2017.
Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy. Rethinking spatiotem- poral feature learning: Speed-accuracy trade-offs in video classification. In Proceedings of the European conference on computer vision (ECCV), pp. 305-321, 2018.
Zhitong Xiong, Yuan Yuan, Nianhui Guo, and Qi Wang. Variational context-deformable convnets for indoor scene parsing. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2020.
Xuchao Zhang, Yifeng Gao, Jessica Lin, and Chang- Tien Lu. Tapnet: Multivariate time series classification with attentional prototypical network. In AAAI, pp. 6845-6852, 2020.
Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. Time series classification using multi-channels deep convolutional neural networks. In International Conference on Web-Age Information Management, pp. 298-310. Springer, 2014.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE CVPR, pp. 2921-2929, 2016.
12

---

Published as a conference paper at ICLR 2022
A
APPENDIX
A.1 STATISTIC OF THE COMPARISON (FIX MODEL SIZE)
More statistic results of the result in Figure is shown in Figure 7 Figure g and Figure
Count of dalasets the +F turings percentile orrg tf resut sebongs to
Coupe owweee
(47 .690
313.3361 2.2,359
15.08991
2.2,35%)
3135391
(10.5991
33.553
5,0,835
+19.4190
<6.5
0.55
2.75 Parconcle Farge
2.85
6.95
Figure 7: The histogram statics the count of datasets by which percentile range of the blue line that the orange point belongs to. Specifically, we could see that for more than 56% datasets (8+40 out of 85 datasets), the result of OS-block is larger than 0.95 percentile. This means that, for an unknown dataset, using OS-block will have more than 56% chance to achieve a better result than grid search from 20 candidate scales. When seeing the count of the number larger than 0.5 percentile, we could see that, for an unknown dataset, using -S-bkock will have more than 96% chance to achieve a better result than selecting a random scale.
Dabwet ccrted by mastare aagee are crOS-CAN
3.0
Al Alceneaann
Figure 8: The red line is the accuracy range obtained via subtracting the accuracy of OS-CNN from the accuracy range of FCN with various kernels. We sorted those datasets in ascending order. We could see that for most of the datasets, the highest value of the FCN accuracy range is lower than the accuracy of OS-CNN. Which supports the OS-block has the ability to capture the best scales.
Dataact soreed by Naxtacurucy rangei- accuracy OS-CWN dataset type
3.2
Votion Suruzr
Server, Nuading
Syrthili:
pas pag aappapattaa
Figure 9: Sort datasets by dataset type and max accuracy range- accuracy of the OS-CNN. We could see that the best scale capture ability keeps the consistency cross different dataset types.
13

---

Published as a conference paper at ICLR 2022
A.2 STATISTIC OF THE COMPARISON (FIX CHANNEL NUMBER)
When we keep the number of channels constant in FCN, the statistic result will be this. The OS-CNN still achieves similar performance as the model with the best scales.
Accuncy of OS-CMA Accurvy rasse sie running
Accunscy modil vertozu alsa
us
safe
Datasst same
Figure 10: Same static metric as that of Figure 5 and Figure
Dakaset ted Manroccurcyy accurecy O5-OWN
Satanet nime
Figure 11: Same static metric that of Figure g
prage Duciie
Motte, Sonnare
Unbi intirgeee www.eeeee wo0.
Dataset name
Figure 12: Same static metric as that of Figure
14

---

Published as a conference paper at ICLR 2022
A.3 THE DD-DIAGRAM RESULT
The critical difference diagram shows the average rank of each method with Wilcoxon-Holm post- hoc analysis between each series.
10
9
8
7
6
5
4
ED-1NN(norm)
7.45
3.13
OS-CNN
ED-1NN
7.12
TapNet
DTW-1NN-I
3.84
WEASEL+ MUSE
DTW-1NN-1
DTW-1NND
MLSTM-FCN
DTW-INND(norm)
Figure 13: SOTA for UEA 30 multivariate dataset archive
8
6
3
1
PF
6.58
3.59
OS-CNN
ResNet 5.41
3.64
ROCKET
STC 5.05
3.68
TS-CHIEF
InceptionTime
4.05
HIVE-COTE
Figure 14: SOTA the UCR 85 datasets
4
3
2
1
ResNet 3.21
2.02
OS-CNN ROCKET
InceptionTime 2.41
2.36
Figure 15: SOTA the UCR 128 datasets
A.4 EXAMPLES OF THE TWO PHENOMENA
In the Figure the Google speechcommands dataset is selected as the dataset to show the example. This is because, for this dataset, the relationship between performance and receptive field size is proportional (As it is shown in Figure 16). Thus, it is casy to control variables. What's more, in Figure and Figure 17 we show those two phenomena with more train and test split.
A.5 EXTEND OS- OS-BLOCK WITH OTHER STRUCTURES
Layers in the OS-block and the OS-block itself are easy extend with other complicated structures. Figure 19 gives an explanation about the how to view the multi-kernel layers in OS-block as a single layer, and gives example that how to combine the layer with dilation. The Figure 4 shows that how to view the OS-block as a layer, and gives another two examples rather than OS-CNN.
15

---

Published as a conference paper at ICLR 2022
Accuracy and receptive field size on Google Speach commands datasets with different trainttest split
0.8
Trainitest
0.7
0.8/0.2
Accura.y
0.6
125
Receptive
Figure 16: The relationship between performance and receptive field size are proportional
CDreachC beat asc
Speechho 5/0.5
SpeechCommantuu 8.02)
0.8
3.8
0.5
(15,5)
[12)
05 c1o-13
[ec, soub))
(42,25,30
(00-3
[120. 01300130.000
[150, 81.00.75.70
0.3
3.3
0.3
[150, 00 60)
1500
(1:0)
05:300-13
Figure 17: Lines in the figure are models with different sets of receptive field sizes. Lines with similar colors are models which have the same best receptive field size. We could see that the best receptive field size mainly dominates the performance in the set of receptive fieldsizes.
SpeechhCom mondodst
SpeechhComma
Sp ShoechhCoomeen do:0
Set of receptivve Field size
(100)
0.75
5.75
(300)
(300,100)
0.70
0.70
5,73
0.00
0.35
0.55
3.59
0.30
0.50
2.50
Figure 18: The label of each line denotes the kernel configuration of each ID-CNN. For example, 5_5.1_1_1 means the ID-CNN has five layers, and from the first layer to the last layer, kerell sizes of each layer are 5, 5, 1, 1, and 1. Lines of similar color are ID-CNNs with the same receptive field size, and they also of similar performance.
The layer with multibe prime size kemes :0
layer prime
20
10 46
70
20
120
260
Figure 19: Purple color in those images are the zero mask and yellow denotes the location where has the ability to hold weight. Left: Convolution layers in of OS-block can be calculated parallelly, thus, each layer can be viewed as one convolutional layer with zero (ssks.()) Right: layers in the OS-block can work with the dilation design
16

---

Published as a conference paper at ICLR 2022
A.6 EXPERIMENT RESULT OF OS- ES-BLOCK WITH OTHER STRUCTURES
The Figure 20 and Figure 21 show that applied OS-block with residual connection, ensemble, and multi-channel architectures (individually together) could further improve the performance. The evaluation was on both UCR 85 and UEA 30 archives which contain datasets from different domains such electrical devices analysis. Spectrum analysis, traffic analysis. EEG analysis.
4
A55
S.8.
connection
2.4
6.05
TS-CHIEF
05-olock
1.85
ROCKET
STC
A.4
5.0
05-block ensemble
OS-block residual connection iccptionTime
476
1.2.
HIVE-CO1E
5:42
Figure 20: Using the OS-block with residual connection and ensemble (individually together) could increase the performance
11
10
9
7
6
4
3
2
1
ED-1NN(norm)
8.35
3.22.
OS-block multi-channel
ED-1NN
3.02
3.65
OS-block
DTW-1NN- ((nom)))mm))
WEASEL+MUSE
DTW- DTW-1NN-I INNI
6.81
4.52 TapNet
MLSTM-FCN
5.42
6.08 DTW-1NND
DTW-1NND(narm)
6.33
Figure 21: Using the multi-channel architecture with OS-block could improve the performance
A.7 COMPARE THE DS-BLOCK WITH OTHER DESIGNS
Mathematically, finding the optimal kernel configuration is challenging, for it is a constrained com- binatorial optimization searching for the best configuration among exponential number of didates. Our contribution is simple and effective model design that does not need to solve the complex optimization problems and achieves state-of-the-art performance on several benchmarks.
From the model size perspective, using prime numbers more efficient than using even numbers or odd numbers. To be specific, to cover RF of range r, the model size complexity of using prime size kernels is (rr ((r2/log /lo((r)) On the other hand, no matter we use even number pairs or odd number pairs, kernel sizes each layer, the model size complexity of using the sequence Or²²). As Table 2 shows, compared with using odd number pairs or even numbers pairs prime numbers can achieve similar performance in smaller model size.
Number of parameters
Channel number
RF range
Prime numbers (Ours)
odd numbers
even numbers
16
45
304k
507k
491k
32
to 45
1.203 k
2,009k
1,948k
Accuracy
Channel number
RF range
Prime numbers (Ours)
odd numbers
even numbers
16
to 45
0.7524
0.7687
0.7561
32
to 45
0.7845
0.7783
0.7725
Table 2: Model size and performance comparison on Google SpeechCommands dataset
17


Reviews:
[{"review_id": "UksWavC6op3", "reviewer": "Reviewer_aFAf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\n+ Elegant solution to design an architecture to cover all RFs\n+ Good results\n\nWeaknesses\n\n- No related work on adaptive RFs is mentioned at all. See my detailed comments below.\n- The paper is sometimes difficult to understand. See my detailed comments below.\n\n\nAll these following citations about adaptive receptive fields in CNNs are missed:\n\n- Han et al. Optimizing filter size in convolutional neural networks for facial action unit recognition. In\nProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018\n\n- Tabernik et al. Spatially-adaptive filter units for compact and efficient deep neural networks. International Journal of Computer Vision (IJCV), 2020.\n\n- Zhitong Xiong et al. Variational context-deformable convnets for indoor scene parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.\n\n- Pintea et al. Resolution learning in deep convolutional networks using scale-space theory in Transactions on Image Processing (TIP), 2021\n\n- Tomen et al. Deep Continuous Networks in International Conference on Machine Learning (ICML), 2021.\n\nAdaptive receptive fields are quite related to this paper; and I expect them to be included and their relation to this ICLR submission explained and motivated. In addition I would like to see the paper state if these aproaches are, or are not, applicable here.\n\n\nDetailed comments:\n\nFig 1: I like the figure. Yet, a \"win\" is based on the very best score, and thus quite sensitive to small changes; it could be that a method is only slightly worse in which case it would not \"win\". It would be a stronger plot to also show the average accuracy (or the average rank) for each Receptive field size on all datasets.\n\nPage 2: \"Firsty, we find that\".. I do not understand what is meant here.\n\nPage 3: It would be good to remind the reader what the term \"OS block\" means, to make the method section more modular and better readable as a stand-alone text.\n\nPage 3: Section 3.2: At this point in the text I'm confused how these kernel sizes are used in a \"multi-kernel structure\" as this is not explained.\n\nFig 3: \"medium\" = \"middle\"\n\nPage 4: Section 3.2: I'm still confused here about \"because each layer of the OS-block has more than one convolution kernel\"; how does that work? Fig 3 seems to hint as something, but is not sufficiently exlained.\n\nEq 2: I do not understand why a specific RF size is now coupled to it's specific position in the layers (ie: I do not understand why Godlbach's conjecture is needed to uniquely define unique RF size paths through the layers).\n\nSection 3.3. I do not understand what is meant by \"For example, suppose we use 2 as the common ratio to cover RF size 15\". \n\nFig 5: There is no need to sort the datasets alphabetically. Please sort the datasets on OS-CNN accuracy: it will probably make the graph much easier to understand. (indeed, in appendix A there seem to be such plots, but I do not understand the diffences between all those plotted variants; I would just sort fig 5 directly on the accuracy of OS-CNN).\n\nFig 6: the graphs are small, and difficult to see the differences.\n\n", "summary_of_the_paper": "\nThe paper is on the receptive field of CNNs for 1D time series classification. It proposes an elegant decomposition of receptive fields based on the Goldbach conjecture that any number can be represented by a sum of primes. The paper thus puts RFs with prime numbers in CNN layers, and by having multiple layers, these RFs will sum, and thus allowing the proposed network to cover all RF sizes. Experiments on several datasets shows that the method gives good results.", "summary_of_the_review": "I like the elegant decomposition to cover all RFs. Yet, much related work is missed, and clarity can be improved.\n\nThese issues seem fixable.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "QJS1R3MCmVV", "reviewer": "Reviewer_rAQQ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The strengths of the papers are as follow :\n\n1- The paper is well structured and easy to follow\n\n2- Tables and figures are very clear.\n\n3- The authors have conducted extensive experiments to support the claim. Moreover, the results are discussed and compared in a consistent manner.\n\n4- The originality of the paper is related to the introduction of a concept from the theory of numbers, namely Goldbach conjecture allowing to cover the receptive fields at different scales\n\n5- The claim is validated on univariate and multivariate time series considering  different tasks.\n\nThe weaknesses of the paper :\n\n1- The proposed receptive field module could have been studied for time series vision tasks, namely video classification for instance\n\n2- The work lacks of theoretical study. The latter is very important to study the stability of the representations. Moreover, Goldbach conjecture is not well motivated and analyzed.\n\n3-  The proposed module could be studied under different neural network architectures to assess its genericity, including different tasks and domain.\n", "summary_of_the_paper": "The paper proposes an original trick to tune the receptive field kernels for 1D ConvNets to tackle time series problems. ", "summary_of_the_review": "Following the aforementioned consideration, l recommend to accept the paper \"6: marginally above the acceptance threshold\"", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-A9524bbjzz", "reviewer": "Reviewer_1qMR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While the proposed method yields a simple and universal rule for kernel size tuning, and has shown to perform as best as the state-of-the-art methods in the literature, there are some major points that are neglected in the current version of the manuscript:\n\n1) The motivation and justification of using the Goldbach’s conjecture is missing. According to the conjecture, any even number such as \"e\" can be written as the sum of two prime numbers such as \"p1\" and \"p2\". Authors have used this decomposition and prime numbers p1 and p2 for the kernel size of their OS-block. However, any even number \"e\" can also be written as the sum of many other pairs of even numbers such as \"e1\" and ''e2\" or odd numbers such as \"o1\" and ''o2\". Why not using these (e1, e2) pairs or (o1, o2) pairs for the kernel size of the OS-block? What advantage does (p1, p2) have over (e1, e2) or (o1, o2)?\n\n2) As shown in Figure 2, model's performance has a positive correlation with the single RF size. Furthermore, authors have claimed in the beginning of Section 2 that the performance of 1D-CNN is not sensitive to the specific kernel size configuration that leads to a specific RF size. If that is the case, then using the Goldbach’s conjecture makes even less sense since any other decomposition of the even number \"e\" could perform the same according to this claim.", "summary_of_the_paper": "This paper presents an Omni-Scale block (OS-block) to efficiently determine the kernel sizes for 1D-CNNs. The proposed method achieves similar performance as models with optimized RF sizes.", "summary_of_the_review": "The main novelty and contribution of the paper lies in using the Goldback's conjecture to design the OS-block for an efficient kernel size design. However, this key factors suffer from the above two points and the manuscript needs to be revised to address these issue. Authors can either theoretically show the benefit of prime decomposition or experimentally verify the efficacy of the prime decomposition of the even number \"e\" over other trivial decomposition of \"e\" such as (e1, e2) or (o1, o2). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a simple and effective solution to tune the receptive field of CNNs for 1D time series classification. The reviewers think the idea is original and elegant but would appreciate more theoretical insights into the solution.
Decision: Accept (Poster)