dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2015). In addition to the huge training time, HIVE-COTE's classification time is bounded by a linear scan of the training set due to employing a nearest neighbor classifier, whereas the trivial GPU parallelization of DNNs provides instant classification. Finally we should note that unlike HIVE-COTE, ResNet's hyperparameters were not tuned for each dataset but rather the same architecture was used for the whole benchmark suggesting further investigation of these hyperparameters should improve DNNs' accuracy for TSC. These results should give an insight of deep learning for TSC therefore encouraging researchers to consider the DNNs as robust real time classifiers for time series data.
Results for multivariate time series We provide on our companion repository 2 the detailed performance of the nine deep learning classifiers for 10 different random initializations over the 12 MTS classification datasets  (Baydogan, 2015). Although Time-CNN and MCDCNN are the only architectures originally proposed for MTS data, they were outperformed by the three deep CNNs (ResNet, FCN and Encoder), which shows the superiority of these approaches on the MTS classification task. The corresponding critical difference diagram is depicted in Figure  9 , where the statistical test failed to find any significant difference between the nine classifiers which is mainly due to the small number of datasets compared to their univariate counterpart. Therefore, we illustrated in Figure  10  the critical difference diagram when both archives are combined (evaluation on 97 datasets in total).
At first glance, we can notice that when adding the MTS datasets to the evaluation, the critical difference diagram in Figure  10  is not significantly different than the one in Figure  7  (where only the univariate UCR/UEA archive was taken into consideration). This is probably due to the fact that the algorithms' performance over the 12 MTS datasets is negligible to a certain degree when compared to the performance over the 85 univariate datasets. These observations reinforces the need to have an equally large MTS classification archive in order to evaluate hybrid univariate/multivariate time series classifiers. The rest of the analysis is dedicated to studying the effect of the datasets' characteristics on the algorithms' performance.
What can the dataset's characteristics tell us about the best architecture? The first dataset characteristic we have investigated is the problem's domain. Table  4  shows the algorithms' performance with respect to the dataset's theme. These themes were first defined in  Bagnall et al. (2017). Again, we can clearly see the dominance of ResNet as the best performing approach across different domains. One exception is the electrocardiography (ECG) datasets (7 in total) where ResNet was drastically beaten by the FCN model in 71. 4% of ECG datasets. However, given the small sample size (only 7 datasets), we cannot conclude that FCN will almost always outperform the ResNet model for ECG datasets  (Bagnall et al. , 2017). The second characteristic which we have studied is the time series length.
Similar to the findings for non deep learning models in  Bagnall et al. (2017) , the time series length does not give information on deep learning approaches' performance. Table  5  shows the average rank of each DNN over the univariate datasets grouped by the datasets' lengths. One might expect that the relatively short filters (3) might affect the performance of ResNet and FCN since longer patterns cannot be captured by short filters. However, since increasing the number of convolutional layers will increase the path length viewed by the CNN model  (Vaswani et al. , 2017) , ResNet and FCN managed to outperform other approaches whose filter length is longer (21) such as Encoder. For the recurrent TWIESN algorithm, we were expecting a poor accuracy for very long time series since a recurrent model may "forget" a useful information present in the early elements of a long time series.
However, A third important characteristic is the training size of datasets and how it affects a DNN's performance. Table  6  shows the average rank for each classifier grouped by the train set's size. Again, ResNet and FCN still dominate with not much of a difference. However we found one very interesting dataset: DiatomSizeReduction. ResNet and FCN achieved the worst accuracy (30%) on this dataset while Time-CNN reached the best accuracy (95%). Interestingly, DiatomSizeReduction is the smallest datasets in the UCR/UEA archive (with 16 training instances), which suggests that ResNet and FCN are easily overfitting this dataset. This suggestion is also supported by the fact that Time-CNN is the smallest model: it contains a very small number of parameters by design with only 18 filters compared to the 512 filters of FCN.
This simple architecture of Time-CNN renders overfitting the dataset much harder. Therefore, we conclude that the small number of filters in Time-CNN is the main reason behind its success on small datasets, however this shallow architecture is unable to capture the variability in larger time series datasets which is modeled efficiently by the FCN and ResNet architectures. One final observation that is in agreement with the deep learning literature is that in order to achieve high accuracies while training a DNN, a large training set is needed. Figure  11  shows the effect of the training size on ResNet's accuracy for the TwoPatterns dataset: the accuracy increases significantly when adding more training instances until it reaches 100% for 75% of the training data. Finally, we should note that the number of classes in a dataset -although it yielded some variability in the results for the recent TSC experimental study conducted by  Bagnall et al.
(2017)  -did not show any significance when comparing the classifiers based on this characteristic. In fact, most DNNs architectures, with the categorical cross-entropy as their cost function, employ mainly the same classifier: softmax which is basically designed for multi-class classification. Overall, our results show that, on average, ResNet is the best architecture with FCN and Encoder following as second and third respectively. ResNet performed very well in general except for the ECG datasets where it was outperformed by FCN. MCNN and t-LeNet, where time series were cropped into subsequences, were the worst on average. We found small variance between the approaches that replace the GAP layer with an FC dense layer (MCDCNN, CNN) which also showed similar performance to TWIESN and MLP.
Effect of random initializations The initialization of deep neural networks has received a significant amount of interest from many researchers in the field  (LeCun et al. , 2015). These advancement have contributed to a better understanding and initialization of deep learning models in order to maximize the quality of nonoptimal solutions found by the gradient descent algorithm  (Glorot and Bengio, 2010). Nevertheless, we observed in our experiments, that DNNs for TSC suffer from a significant decrease (increase) in accuracy when initialized with bad (good) random weights. Therefore, we study in this section, how random initializations can affect the performance of ResNet and FCN on the whole benchmark in a best and worst case scenario. Figure  12  shows the accuracy plot of ResNet versus FCN on the 85 univariate time series datasets when aggregated over the 10 random initializations using three different functions: the minimum, median and maximum.
When first observing Figure  12  one can easily conclude that ResNet has a better performance than FCN across most of the datasets regardless of the aggregation method. This is in agreement with the critical difference diagram as well as the analysis conducted in the previous subsections, where ResNet was shown to achieve higher performance on most datasets with different characteristics. A deeper look into the minimum aggregation (red points in Figure  12 ) shows that FCN's performance is less stable compared to ResNet's. In other words, the weight's initial value can easily decrease the accuracy of FCN whereas ResNet maintained a relatively high accuracy when taking the worst initial weight values. This is also in agreement with the average standard deviation of ResNet (1. 48) which is less than  FCN's (1.
Class Activation Map We investigate the use of Class Activation Map (CAM) which was first introduced by  Zhou et al. (2016)  to highlight the parts of an image that contributed the most for a given class identification. Wang et al. (2017b)  later introduced a one-dimensional CAM with an application to TSC. This method explains the classification of a certain deep learning model by highlighting the subsequences that contributed the most to a certain classification. Figure  13  and 14 show the results of applying CAM respectively on GunPoint and Meat datasets. Note that employing the CAM is only possible for the approaches with a GAP layer preceding the softmax classifier  (Zhou et al. , 2016). Therefore, we only considered in this section the ResNet and FCN models, who also achieved the best accuracies overall.
Note that  Wang et al. (2017b)  was the only paper to propose an interpretable analysis of TSC with a DNN. We should emphasize that this is a very important research area which is usually neglected for the sake of improving accuracy: only 2 out of the 9 approaches provided a method that explains the decision taken by a deep learning model. In this section, we start by presenting the CAM method from a mathematical point of view and follow it with two interesting case studies on Meat and GunPoint datasets. By employing a Global Average Pooling (GAP) layer, ResNet and FCN benefit from the CAM method  (Zhou et al. , 2016) , which makes it possible to identify which regions of an input time series constitute the reason for a certain classification. Formally, let A(t) be the result of the last convolutional layer which is an MTS with M variables.
GunPoint dataset The GunPoint dataset was first introduced by  Ratanamahatana and Keogh (2005)  as a TSC problem. This dataset involves one male and one female actor performing two actions (Gun-Draw and Point) which makes it a binary classification problem. For Gun-Draw (Class-1 in Figure  13 ), the actors have first their hands by their sides, then draw a replicate gun from hip-mounted holster, point it towards the target for one second, then finally place the gun in the holster and their hands to their initial position. Similarly to Gun-Draw, for Point (Class-2 in Figure  13 ) the actors follow the same steps but instead of pointing a gun they point their index finger. For each task, the  centroid of the actor's right hands on both X and Y axes were tracked and seemed to be very correlated, therefore the dataset contains only one univariate time series: the X-axis.
We chose to start by visualizing the CAM for GunPoint for three main reasons. First, it is easy to visualize unlike other noisy datasets. Second, both FCN and ResNet models achieved almost 100% accuracy on this dataset which will help us to verify if both models are reaching the same decision for the same reasons. Finally, it contains only two classes which allow us to analyze the data much more easily. Figure  13  shows the CAM's result when applied on each time series from both classes in the training set while classifying using the FCN model (Figure  13a  and 13b ) and the ResNet model (Figure  13c  and 13d ). At first glance, we can clearly see how both DNNs are neglecting the plateau non-discriminative regions of the time series when taking the classification decision. It is depicted by the blue flat parts of the time series which indicates no contribution to the classifier's decision.
We should note that this observation cannot confirm which classifier extracted the most discriminative subsequences especially because all algorithms achieved similar accuracy on GunPoint dataset. Perhaps a bigger dataset might provide a deeper insight into the interpretability of these machine learning models. Finally, we stress that the shapelet transformation classifier  (Hills et al. , 2014 ) is an ensemble approach, which makes unclear how the shapelets affect the decision taken by the individual classifiers whereas for an end-to-end deep learning model we can directly explain the classification by using the Class Activation Map.
Meat dataset Although the previous case study on GunPoint yielded interesting results in terms of showing that both models are localizing meaningful features, it failed to show the difference between the two most accurate deep learning classifiers: ResNet and FCN. Therefore we decided to further analyze the CAM's result for the two models on the Meat dataset. Meat is a food spectrograph dataset which are usually used in chemometrics to classify food types, a task that has obvious applications in food safety and quality assurance. There are three classes in this dataset: Chicken, Pork and Turkey corresponding respectively to classes 1, 2 and 3 in Figure  14. Al-Jowder et al. (1997)  described how the data is acquired from 60 independent samples using Fourier transform infrared (FTIR) spectroscopy with attenuated total reflectance (ATR) sampling.
Similarly to GunPoint, this dataset is easy to visualize and does not contain very noisy time series. In addition, with only three classes, the visualization is possible to understand and analyze. Finally, unlike for the GunPoint dataset, the two approaches ResNet and FCN reached significantly different results on Meat with respectively 97% and 83% accuracy. Figure  14  enables the comparison between FCN's CAM (left) and ResNet's CAM (right). We first observe that ResNet is much more firm when it comes to highlighting the regions. In other words, FCN's CAM contains much more smoother regions with cyan, green and yellow regions,  whereas ResNet's CAM contains more dark red and blue subsequences showing that ResNet can filter out non-discriminative and discriminative regions with a higher confidence than FCN, which probably explains why FCN is less accurate than ResNet on this dataset.
Multi-Dimensional Scaling We propose the use of Multi-Dimensional Scaling (MDS)  (Kruskal and Wish, 1978)  with the objective to gain some insights on the spatial distribution of the input time series belonging to different classes in the dataset. MDS uses a pairwise distance matrix as input and aims at placing each object in a N-dimensional space such as the between-object distances are preserved as well as possible. Using the Euclidean Distance (ED) on a set of input time series belonging to the test set, it is then possible to create a similarity matrix and apply MDS to display the set into a two dimensional space. This straightforward approach supposes that the ED is able to strongly separate the raw time series, which is usually not the case evident by the low accuracy of the nearest neighbor when coupled with ED  (Bagnall et al.
, 2017). On the other hand, we propose to apply this MDS method to visualize the set of time series with its latent representation learned by the network. Usually in a deep neural network, we have several hidden layers and one can find several latent representation of the dataset. But since we are aiming at visualizing the class specific latent space, we chose to use the last latent representation of a DNN (the one directly before the softmax classifier), which is known to be a class specific layer  (Yosinski et al. , 2014). We decided to apply this method only on ResNet and FCN for two reasons: (1) when evaluated on the UCR/UEA archive they reached the highest ranks; (2) they both employ a GAP layer before the softmax layer making the number of latent features invariant to the time series length.
To better explain this process, for each input time series, the last convolution (for ResNet and FCN) outputs a multivariate time series whose dimensions are equal to the number of filters (128) in the last convolution, then the GAP layer averages the latter 128-dimensional multivariate time series over the time dimension resulting in a vector of 128 real values over which the ED is computed. As we worked with the ED, we used metric MDS  (Kruskal and Wish, 1978)  that minimizes a cost function called Stress which is a residual sum of squares: Stress D (X 1 ,. , X N ) = i,j (d ij -x i -x j ) 2 i,j d 2 ij 1/2 (13) where d ij is the ED between the GAP vectors of time series X i and X j. Obviously, one has to be careful about the interpretation of MDS output, as the data space is highly simplified (each time series X i is represented as a single data point x i ).
Although the visualization of MDS on GunPoint yielded some interesting results, it failed to pinpoint the difference between the two deep learning models FCN and ResNet. Therefore we decided to analyze another dataset where the accuracy of both models differed by almost 15%. Figure 16 shows three MDS plots for the Wine dataset using: (1) the raw input time series (Figure  16a ); (2) the learned latent features from the GAP layer for FCN (Figure  16b ); and (3) the learned latent features from the GAP layer for ResNet (Figure  16c ). At first glimpse of Figure  16 , the reader can conclude that all projections, even when using the learned representation, are not linearly separable which is evident by the relatively low accuracy of both models FCN and ResNet which is equal respectively to 58.
7% and 74. 4%. A thorough observation shows us that the learned hidden representation of ResNet (Figure  16c ) separates the data from both classes in a much clearer way than the FCN (Figure  16b ). In other words, FCN's learned representation has too many data points close to the decision boundary whereas ResNet's hidden features enables projecting data points further away from the decision boundary. This observation could explain why ResNet achieves a better performance than FCN on the Wine dataset.
Although we have conducted an extensive experimental evaluation, deep learning for time series classification, unlike for computer vision and NLP tasks, still lacks a thorough study of data augmentation  (Ismail Fawaz et al. , 2018a; Forestier et al. , 2017)  and transfer learning  (Ismail Fawaz et al. , 2018c; Serrà et al. , 2018). In addition, the time series community would benefit from an extension of this empirical study that compares in addition to accuracy, the training and testing time of these deep learning models. Furthermore, we think that the effect of z-normalization (and other normalization methods) on the learning capabilities of DNNs should also be thoroughly explored. In our future work, we aim to investigate and answer the aforementioned limitations by conducting more extensive experiments especially on multivariate time series datasets.
In order to achieve all of these goals, one important challenge for the TSC community is to provide one large generic labeled dataset similar to the large images database in computer vision such as ImageNet  (Russakovsky et al. , 2015)  that contains 1000 classes. In conclusion, with data mining repositories becoming more frequent, leveraging deeper architectures that can learn automatically from annotated data in an end-to-end fashion, makes deep learning a very enticing approach.
Acknowledgements The authors would like to thank the creators and providers of the datasets:  Hoang Anh Dau ,  Anthony Bagnall ,  Kaveh Kamgar ,  Chin-Chia Michael Yeh ,  Yan Zhu ,  Shaghayegh Gharghabi ,  Chotirat Ann Ratanamahatana ,  Eamonn Keogh  and  Mustafa Baydogan. The authors would also like to thank  NVIDIA Corporation  for the  GPU Grant  and the  Mésocentre of Strasbourg  for providing access to the cluster. The authors would also like to thank  François Petitjean  and  Charlotte Pelletier  for the fruitful discussions, their feedback and comments while writing this paper. This work was supported by the  ANR   TIMES  project (grant  ANR-17-CE23-0015 ) of the  French Agence Nationale de la Recherche.
Context. HD113337 is a Main-Sequence F6V field star more massive than the Sun. It hosts one confirmed giant planet and possibly a second, candidate one, detected by radial velocities. It also hosts a cold debris disk detected through the presence of an infrared excess, making it an interesting system to explore. Aims. We aim at bringing new constraints on the star's fundamental parameters, the debris disk properties and the planetary companion(s) by combining complementary techniques. Methods. We used the VEGA interferometer on the CHARA array to measure HD113337 angular diameter. We derived its linear radius using the parallax from Gaia Second Data Release. We computed the bolometric flux to derive its effective temperature and luminosity, and we estimated its mass and age using evolutionary tracks.
Furthermore and if possible, using two different calibrators with well-defined angular diameters (C 1 -S-C 2 sequence) instead of one (C-S-C sequence) reinforces the robustness of the target V 2 computation by reducing its dependency to the calibrator diameter value and uncertainty. We used the SearchCal software  (Bonneau et al. , 2006)  to select adequate calibrators at the different observation epochs. We acquired ten observation points on five different nights from May 2013 to June 2015, using VEGA red spectral channel in the ∼700 to ∼750 nm range. We computed the V 2 values in either two or three 20 nm-wide spectral bands (depending on the observing conditions) with the standard VEGA reduction pipeline vegadrs  (Mourard et al. , 2012). Due to the small angular diameters of both our target and calibrators, we had to discard a signif- icant part of the data that revealed themselves not robust enough.
Herschel observations of the cold outer disk The Herschel data of HD113337 were obtained using the Photodetector Array Camera and Spectrograph (PACS) instrument  (Poglitsch et al. , 2010) , in the mini scanmap mode with simultaneous observations at 70 and 160 µm under the open time program OT 2 ksu 3 (Fig. 3 ). The data were obtained on March 3, 2012 with OBSID 1342243344 and 1342243345. Herschel PACS data reduction were performed following the procedure published by  Balog et al. (2014)  for calibration stars. We fitted a 2-D Gaussian function to estimate the source's Full Width at Half Maximum (FWHM). At 70 µm, the measured FWHM is 7. 44×7. 15, ∼1. 3 times larger than the FWHM of typical point sources (5. 76×5. 58). The measured FWHM at 160 µm is slightly larger than the typical value for point sources, but less significant.
2 ± 0. 2 pc, i. e. in good agreement with the HD113337 distance of 36. 18 ± 0. 06 pc derived by Bailer-Jones et al. (  2018 ) based on a Bayesian analysis of Gaia DR2 parallaxes. We used our limb-darkened angular diameter θ LD in combination with the above Gaia parallax to derive the stellar radius and its error through a Monte-Carlo simulation R ± δR = θ LD + δθ LD 9. 305 × (π P + δπ P ). (1) We obtained R = 1. 50 ± 0. 04 R (precision better than 2. 7%). Bolometric flux -We collected spectroscopic and photometric data to compute the bolometric flux f bol. We determined HD113337 SED, and then computed the area under the curve of that distribution (Fig. 4 ). The flux distribution of HD113337 was determined by concatenating several flux distributions: for the ultraviolet region, the rebinned spectrum from the Sky Survey Telescope obtained by the IUE 'Newly Extracted Spectra'(INES) data archive 3.