dataesr/scientific-paragraphs-categorization huggingface dataset:
5-upper bound of 95% credibility interval (Mg ha -1 ) • AGB_Feldpausch -AGB (Mg ha -1 ) using equation 4 in Chave  30  with wood density, DBH and H derived from Feldpausch  31  height-diameter relationship. Fig. 3  The database structure of the plot information. • Cred_2. 5 -lower bound of 95% credibility interval (Mg ha -1 ) • Cred_97. 5 -upper bound of 95% credibility interval (Mg ha -1 ) • AGB_Chave -aboveground biomass (in Mg ha -1 ) estimated using equation 7 in Chave  30  with wood density, DBH and H implicitly taken into consideration through the use of the bioclimatic predictor E • Cred_2. 5 -lower bound of 95% credibility interval (Mg ha -1 ) • Cred_97. 5 -upper bound of 95% credibility interval (Mg ha -1 ) • Wood_density -mean wood density of the trees (g cm -3 ) • GSV -growing stock volume (m 3 ha -1 ) • BA -basal area (m 2 ha -1 ) • Ndens -number of trees per hectare Note that we have merged the Plot and Sub-plot tables in the data package associated with this paper  58  for the user's convenience.
e. , allometric equations, R packages, etc. ), and by promoting shared publications. (6) Cross-network collaboration. In the FOS, by applying a uniform R script for data aggregation and AGB estimation, potential biases from using different height-diameter, wood density and allometric relations are strongly reduced. The distribution of FOS plots by continent is presented in Table  1. Africa, Europe and South America are represented by similar numbers of locations (i. e. , 62-80 plots) and contribute more than 80% of the plots at the time of publication, but in terms of coverage, South America alone comprises 49% of the forest area covered. The IIASA network provides the highest number of plot locations to FOS (Table  2 ), while the TmFO network contributes the most in terms of areal coverage.
The range of values of major forest parameters represented in the FOS database is shown in Table  3. The maximum AGB value (918 Mg ha -1 ) and canopy height (41. 7 m) at a 0. 25 ha sub-plot were recorded in Lopé, Gabon. Some savannah sub-plots (e. g. , in Gabon) have a few or no trees >5 cm dbh, which leads to low or no biomass estimation. The tallest trees (60. 1 m) was found in Costa Rica and the maximum basal area (85. 6 m 2 ha -1 ) was found in the Caucasus, Russia. Table  4  contains information about the AGB for different biomes and globally. As expected, the average AGB increases from boreal to temperate and then from temperate to tropical forests.
Usage Notes This data set will be essential for validating and calibrating satellite observations and forest biometric models. The focus is to provide ground support for current and planned space-borne missions, such as NASA GEDI (https:// gedi. umd. edu/), NASA-ISRO NISAR (https://nisar. jpl. nasa. gov/), JAXA ALOS PALSAR (http://global. jaxa. jp/ projects/sat/alos/) and ESA BIOMASS (https://earth. esa. int/web/guest/missions/esa-future-missions/biomass), which are aimed at retrieving forest structure parameters such as forest height and biomass. At this stage, we are making no claims regarding the statistical robustness of the FOS data set for global or regional biomass estimations. Instead our aim is to present uniformly processed data on forest biomass from available locations (see Table  1 ).
One of the main goals of the FOS is to highlight gaps in the observations. Using sub-plot data for validation of RS data might lead to spatial autocorrelation problems so possible solutions would be to use a plot average, use only values from the plot or test for the presence of spatial autocorrelation. This data package contains geographical coordinates rounded to 2 digits after decimal point (up to 1 km at equator). The most up-to-date extended data set with accurate geolocation is available in the FOS portal: https:// forest-observation-system. net/ The FOS initiative depends on the contributions of high-quality forest plot data from participating networks. The fair use of the data presented here requires respecting the efforts and rights of the partners and supporting the long-term future of these observational efforts.
acknowledgements This study has been partly supported by the  IFBN  ( 4000114425/15/NL/FF/gp ) and  CCI Biomass  ( 4000123662/18/I-NB ) projects funded by  ESA ; the  Austrian Federal Ministry of Science and Research  ( BMWF-4. 409/30-II/4/2009 ); the  Austrian Academy of Sciences  ( ÖAW2007-11 ); the  Research  Project  AGL2009-08562 ,  Ministry of Science's Research and Development, Spain ; the  Project LIFE+  " ForBioSensing PL Comprehensive monitoring of stand dynamics in Białowieża Forest  supported with remote sensing techniques" cofounded by  Life+ UE program  (contract number  LIFE13 ENV/PL/000048 ) and  The National Fund for Environmental Protection and Water Management in Poland  (contract number  485/2014/WN10/OP-NM-LF/D ); the  Brazilian National Council of Science and Technology  ( PVE  project # 401279/2014-6  and  PELD (LTER)  project # 441244/2016-5 );  USAID  ( 1993-2006 );  Brazilian National Council of Science and Technology-CNPq  ( Processes 481097/2008-2 ,  201138/2012-3 );  Foundation for Research Support of the State of Sao Paulo-FAPESP  ( Processes 2013/16262-4 ,  2013/50718-5 ).
By highlighting which features are retained by filters, heatmaps come in handy when a security evaluator tries to interpret and understand the efficiency of CNN. We propose a methodology for building efficient CNN architectures in terms of attack efficiency and network complexity, even in the presence of desynchronization. We evaluate our methodology using public datasets with and without desynchronization. In each case, our methodology outperforms the previous state-of-the-art CNN models while significantly reducing network complexity. Our networks are up to 25 times more efficient than previous state-of-the-art while their complexity is up to 31810 times smaller. Our results show that CNN networks do not need to be very complex to perform well in the side-channel context.
Here, we propose two new visualization techniques widely used in the deep learning field i. e. , Weight Visualization and Heatmaps. These tools help us to evaluate the impact of each model hyperparameter including the number of filters, the length of each filter and the number of convolutional blocks in order to identify a suitable architecture, minimizing its complexity while increasing its attack efficiency. Then, we propose a method to build efficient CNN architectures. We confirm the relevance of our methodology by applying it to the main public datasets and comparing our state-of-the-art results with those of previous studies. This new methodology also enables a better tradeoff between the complexity of the model and its performance, thereby allowing us to reduce learning time. Paper Organization.
The paper is organized as follows. Section 2 presents the convolutional neural network in particular the convolutional part. After describing standard feature selection techniques in Section 3, Section 3. 2 and Section 3. 3 present two new visualization tools applied to the SCA context. In Section 4, we highlight the impact of each model hyperparameter on the convolutional part. We then propose a methodology to build an efficient architecture depending on the countermeasures implemented. The new methodology is applied to public datasets and the state-of-the-art results are detailed in Section 5.
Once the leakage model is generated, adversaries estimate which intermediate value is processed thanks to a predicting function F (•) they designed themselves. By predicting this sensitive variable and knowing the input used during the encryption, adversaries can compute a score vector, based on F (t i ), i ∈ [0, |N a | -1], for each trace included in a dataset of N a attack traces. Then, log-likelihood scores are computed to make predictions for each key hypothesis. The candidate with the highest value will be defined as the recovered key. Metric in Side-Channel Attacks. During the building phase, adversaries want to find a model F that optimizes recovery of the secret key by minimizing the size of N a. In SCA, a common metric used is called guessing entropy (GE)  [SMY09] , which defines the average rank of k * through all the key hypotheses.
Settings To illustrate and explain the effect of each hyperparameters as far as possible, let us consider an unmasked implementation of AES-128 on the Chipwhisperer a (8-bit XMEGA a https://newae. com/tools/chipwhisperer/ Target). Our experience is conducted with 45, 000 power traces of 3000 points. The targeted operation is Z = Sbox[P 1 ⊕ k * ] where P 1 denotes the first byte of the plaintext and k * denotes the secret key. The measured SNR equals 2. 28 (see Appendix A). The CNN is implemented in Python using the keras library [C + 15] and is run on a workstation equipped with 8GB RAM and a NVIDIA GTX1060 with 8GB memory. We use the Categorical Cross-Entropy as loss function because minimizing the cross entropy is equivalent to maximizing the lower bound of the mutual information between the traces and the secret variable  [MDP19a].
Optimization is done using the Adam  [GBC16]  approach on batch size 50 and the learning rate is set to 10 -3. As explained in Section 2. 4, we use the SeLU activation function to avoid vanishing and exploding gradient problems  [KUMH17]. For improved weight initialization, we use the He Uniform initialization  [HZRS15]. The loss function, the optimizer, the activation function and the weight initialization follow this setup in Section 5. These values are selected following a grid search optimization  [GBC16]. For each optimizer hyperparameter, we select a finite set of values (see Table  1 ). Our choices are motivated by the fact that they provide good results, in terms of classification, on our datasets. Indeed, using these hyperparameters lead to a faster convergence towards a constant guessing entropy of 1.
Length of filters In this section, we demonstrate that increasing the length of the filter causes entanglement and reduces the weight related to a single information and therefore, the network confidence. The network confidence is defined as its ability to detect accurately the PoIs in the convolutional part. To this end, we need an assumption such that there exist only a small dataset E = {x 0 , x 1 ,. , x l } of time samples such that P r [Z|T] = P r [Z|T[x 0 ],. , T[x l ]] where l << dim(T). Let us denote W ∈ R n a filter of length n such that W = {α 0 , α 1 ,. , α n } is optimized for maximizing the detection of l PoIs. Convolution operation between a trace t and a filter W follows the equation: where ≈ 0 and α j denotes the weight related to the index j. Denote t a trace such that (t W )[i] covers the l relevant points and (t W )[i + m] covers less than l relevant points.
We illustrate these properties by applying different architectures to our Chipwhisperer dataset to efficiently evaluate the impact of the length of the filter. We select a range of lengths (see Appendix B) and visualize the weights to experimentally verify this claim. The shorter the length, the less the relevant information is spread. Figure  14  in Appendix B shows that the weight visualization succeeds in recovering the location of the leakage. Indeed, the PoIs correspond to the relevant information detected by the SNR (see Appendix A). The number of samples is divided by 2 because of the pooling stride value (i. e. , 2) that we used in the first convolutional block. As explained above, small filters extract information better because the PoIs are not shared with a lot of convoluted samples, while using larger filters causes entanglement and provide the detection of "global features".
Indeed, in most cases, dim [h] t = dim [h-1] t pooling_stride [h-1]. Then, we reduce by a factor pooling_stride [h-1] the distance between the relevant points. Hence, the impact of desynchronization is reduced by the same factor. It will be much easier, for the network, to characterize the desynchronization. By adding more convolutional blocks, we can drastically reduce the impact of desynchronization by choosing an appropriate pooling_stride [h]. These properties show that the pooling functions reduce the trace dimension while preserving the most relevant samples. To illustrate these properties, we apply an average pooling or a max pooling (see Appendix D) to the Chipwhisperer dataset. Each convolutional block is configured with one convolutional layer composed by two filters of size 1 and one pooling layer with a pooling_stride of 2.
Experimental results In the following section, we apply our methodology on different publicly available datasets. The methodology is applied on both unprotected and protected implementations. We compare the performance of our architectures b with state-of-the-art results. For a good estimation of N t GE , the attack traces are randomly shuffled and 100 N t GE are computed to give the average value for N t GE. We denote N t GE this average over the 100 tests. Because the attack efficiency is not the only metric, we also compare the complexity of our architectures with the state-of-the-art. Furthermore, when considering profiled SCA, the template attack (TA) is the best technique from a theoretical point of view. Therefore, we will compare our result with the TA that are performed on the same datasets.
Datasets We used four different datasets for our experiments. All the datasets correspond to implementations of Advanced Encryption Standard (AES)  [DR02]. The datasets offer a wide range of use cases: high-SNR unprotected implementation on a smart card, low-SNR unprotected implementation on a FPGA, low-SNR protected with a random delay desynchronization, low-SNR protected implementation with first-order masking  [SPQ05]  and desynchronization with a random delay. • DPA contest v-4 is an AES software implementation with a first-order masking [BBD + 14] c. Knowing the mask value, we can consider this implementation as unprotected and recover the secret directly. In this experiment, we attack the first round S-box operation. We identify each trace with the sensitive variable Y (i) (k * ) = Sbox[P (i) 0 ⊕ k * ] ⊕ M where M denotes the known mask and P (i) 0 the first byte of the i-th plaintext.
0073 (see Appendix E Figure  19 ). • ASCAD is introduced in [PSB + 18] and is the first open database f that has been specified to serve as a common basis for further works on the application of deep learning techniques in the side-channel context. The target platform is an 8-bit AVR microcontroller (ATmega8515) where a masked AES-128 is implemented. In addition, a random delay is applied to the set of traces in order to make it more robust against side-channel attacks. The leakage model is the first round S-box operation such that Y (i) (k * ) = Sbox[P (i) 3 ⊕ k * ]. As explained in [PSB + 18], the third byte is exploited. The measured SNR equals 0. 007 (see Appendix E Figure  20 ). Remark 1: the learning rate. During the training phase, for each architecture, we use a technique called One Cycle Policy [ST17, Smi17, Smi18] that helps choose the right learning rate.
Results on synchronized traces As explained in Section 5. 2, only 1 convolutional block is set for each network. Using our methodology, we want to locate local perturbations that generate confusion during the classification of the network. Thus, the length of the filters is set to 1. Finally, the number of filters and the dense layers that compose the classification part should be managed depending on the device under test. To accelerate the learning phase, we also pre-processed each dataset such that all trace samples are standardized and normalized between 0 and 1  [LBOM12].
DPA-contest v4 DPA-contest v4 is the easiest dataset because we consider it without countermeasures, consequently the relevant information leak a lot. Thus, it is straightforward for the network to extract sensitive variables. To generate our CNN, we divide the dataset into three subsets such that 4, 000 traces are used for the training, 500 for the validation set and 500 for attacking the device. Only two filters are used and the classification part is composed of one dense layer of 2 nodes. Finally, we set our learning rate to 10 -3. The network is trained for 50 epochs with a batch size of 50. Visualizations help evaluate the training process (see Figure  6 ). As explained in section Section 3. 2, visualization of the weights shows that the convolutional part is accurately set because the PoIs are extracted.
However, by visualizing the gradient, the global learning achieved by our network does not enable it to significantly recognize the relevant points. Thanks to these visualization techniques, we can identify which part of the network needs optimizing. Currently, no tool is available that allows interpretation of the classification part to improve its efficiency. However, even if the classification part is not powerful, our network still performs much better than the stateof-the-art. We compare our results with the architecture proposed in [KPH + 19] where they achieved the best performance on this dataset. On average, N t GE reaches 25, 000 traces. By applying our methodology, we dramatically improve this result. First, the new architecture has 31, 810 times fewer parameters. Now, only 31 seconds are necessary to train the network.
ASCAD with N [0] = 0 Following the previous studies, we can find a suitable architecture for the ASCAD database. To generate our network, we divide the dataset of ASCAD into three subsets: 45, 000 We initialize the number of filters to 4. Then, two dense layers composed of 10 neurons complete the network. Thanks to the one cycle policy, we are able to configure our learning rate in the range 5 × 10 -4 ; 5 × 10 -3. The network is trained for 50 epochs with a batch size of 50. Next, we compare our result with the original paper [PSB + 18]. Surprisingly, we can notice that the network detects more PoIs than the mask and the masked value. By visualizing the weights, we can evaluate that the network recognizes relevant information related to the mask and the masked values, although a sensitive area (between samples 380 and 450) is detected by the network whereas nothing appears on the SNR.
AES_RD The dataset consists of 50, 000 traces of 3, 500 features each. For this experiment, we use 20, 000 traces for the training set, 5, 000 for the validation set and 25, 000 to attack the device. Because desynchronization is generated, we have to use more than one convolutional block. According to Section 4. 4. 2, we recommend using three convolutional blocks. Concentrating the attention of the network on the PoIs is helpful in order to reduce the desynchronization effect. Our new architecture can be set as shown in Figure  5. Each hyperparameter, defining the convolutional part, is set as mentioned in Section 4. 4. 2. In the second convolutional block, the length of the filters and the pooling stride are configured following the value of the desynchronization effect (i. e. , N [1] = N [0]   pooling_stride  [0]  ).
ASCAD Finally, we test our methodology on a system that implements random delay and 1 st order-masked AES. We evaluate our methodology when N [0] = 50 and N [0] = 100. Like in Section 5. 2. 3, we split our dataset into three subsets: 45, 000 traces for the training process, 5, 000 for the validation set and 10, 000 traces for the attack. Random delay: N [0] = 50. By applying our new methodology, we want to generate a suitable architecture according to Section 4. 4. 2. Thanks to Figure  8 , we set L = 3 because three global leakage areas appear. As the previous section, the hyperparameters has configured following Section 4. 4. 2. In this dataset, N [0] = 50, the length of the filters and the pooling stride defining in the second convolutional block are configured as N [1] = N [0]   pooling_stride  [0].
Applied on this dataset, our methodology outperforms the state of the art remarkably. In the original paper, Prouff et al. did not reach a constant guessing entropy of 1 with 5, 000 traces. Applying our methodology, we converge towards a constant guessing entropy of 1 with 244 traces while the complexity of our networks is divided by 763. As a reminder, the performance related to the network trained with synchronized traces converges towards a guessing entropy of 1 with around 200 traces. Thus, we succeed in dramatically reducing  Random delay: N [0] = 100. Finally, we apply our methodology on an even more complex system implementing a random delay with N [0] = 100. As the previous sections, we set our network with the following hyperparameters. In the second convolutional block, we set the length of the filters and the pooling stride to N [1] = 50 to focus the interest of the network on the desynchronization effect.
If an evaluator wants to generate a network for synchronized traces, we recommend using only one convolutional block and minimizing the length of the filters to enable accurate identifying the relevant information. However, when desynchronization is applied, we propose an architecture that enables the network to focus its feature selection on the PoIs themselves while the dimensionality of the traces is reduced. This dramatically reduce the complexity of the CNN. We test our methodology on four datasets with a wide range of use-cases. In all cases, our methodology outperforms the state-of-the-art. Indeed, our attacks are performed much faster and each new architecture is less complex than the state of-the-art. This paper thus shows that the networks needed to perform side channel attacks do not have to be complex.
This methodology is not focused on optimizer hyperparameters and could benefit from techniques such as data augmentation  [CDP17]  or adding noise [KPH + 19]. In some cases, we show that the aggregation of information through the classification part could be difficult. Future works could study this issue to generate suitable fullyconnected layers. More complex systems with dynamic desynchronization or in presence of higher order masking scheme could be analyzed. Finally, other masking scheme should be investigated in order to validate our methodology on other masking implementation. A Signal to Noise Ratio (Chipwhisperer dataset).
• AES_HD is an unprotected AES-128 implemented on FPGA. Introduced in [PHJ + 19] d , the authors decided to attack the register writing in the last round b https://github. com/gabzai/Methodology-for-efficient-CNN-architectures-in-SCA c http://www. dpacontest. org/v4/42_traces. php d https://github. com/AESHD/AES_HD_Dataset such that the label of the i-th trace is Y.
Participants Eighty-three male undergraduate students from the University of Western Australia participated in the study in exchange for course credit. Seven participants were excluded from the main statistical analyses due to either exceedingly high error rates or errors in data recording (see section 2. 4). Participants were between the ages of 17 and 37 (M = 22. 22, SD = 4. 52) with an objective mean body mass index (BMI = kg/m 2 ) of 23. 34 (SD = 3. 43). 2 Ethics approval for this study was granted in accordance with the requirements of the National 2 These demographic characteristics are consistent with the 76 participants included in the statistical analyses.
Statistical Analysis Regarding the SRC task, RTs until first response were used in the statistical analyses. In line with the criteria used by  Dondzilo et al. (2018) , RTs were based on correct trials only and all RTs less than 300 ms and greater than 2. 5 standard deviations above each individual's mean RT were excluded. Data from five participants were removed for performing at nearchance level (>40% error rate; Booth, Spronk, Grol, & Fox, 2018) and from a further two participants due to an error in data recording. The remaining 76 participants demonstrated high overall accuracy (92. 62%). In accordance with previous SRC task studies (e. g. ,  Dondzilo et al. , 2018; Schoenmakers, Wiers, & Field, 2008; Woud et al. , 2011) , median RTs were computed per body type and response type (e. g.