dataesr/scientific-paragraphs-categorization huggingface dataset:
The weather through experiences in projects such as DuraTB and Woodlife are translatable into microclimate in the wood and its response. A relationship connecting climate parameters to the material climate makes it possible to analyse a wide range of regional climates through use of open meteorological databases such as ECMWF www. ecmwf. int. Dosimeter approaches developed to describe the interrelationship between material-climatic parameters (exposure dose), material-specific resistance (resistance dose) and wood degradation (response) can then be used to study the regional-specific performance. The same methodology is also be employed for predicting degradation by termites and other insects. For that purpose, input variables need to be different and will be sought from existing field test data and short-term resistance tests performed within CLICKdesign.
Acknowledgement Project CLICKdesign is supported under the umbrella of  ERA-NET Cofund ForestValue  by the  Ministry of Education, Science and Sport  (MIZS) -Slovenia; The  Ministry of the Environment (YM) -Finland ; The  Forestry Commissioners (FC) -UK ;  Research Council of Norway (RCN) -Norway ; The  French Environment & Energy Management Agency (ADEME)  and  The French National Research Agency (ANR) -France ; The  Swedish Research Council for Environment, Agricultural Sciences and Spatial Planning (FORMAS) ,  Swedish Energy Agency (SWEA) ,  Swedish Governmental Agency for Innovation Systems (Vinnova) -Sweden ;  Federal Ministry of Food and Agriculture (BMEL)  and  Agency for Renewable Resources (FNR) -Germany. ForestValue has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under grant agreement  N° 773324.
DFTB+ is a versatile community developed open source software package offering fast and efficient methods for carrying out atomistic quantum mechanical simulations. By implementing various methods approximating density functional theory (DFT), such as the density functional based tight binding (DFTB) and the extended tight binding method, it enables simulations of large systems and long timescales with reasonable accuracy while being considerably faster for typical simulations than the respective ab initio methods. Based on the DFTB framework, it additionally offers approximated versions of various DFT extensions including hybrid functionals, time dependent formalism for treating excited systems, electron transport using non-equilibrium Green's functions, and many more. DFTB+ can be used as a userfriendly standalone application in addition to being embedded into other software packages as a library or acting as a calculation-server accessed by socket communication.
The database at www. dftb. org currently hosts the ob2 set  45  for the elements O, N, C, and H with ω = 0. 3 a -1 0. LC-DFTB calculations can also be performed for spin-polarized systems, enabling evaluation of triplet excited states and their corresponding relaxed geometries. It also paves the way for a rational determination and tuning  31  of the range-separation parameter ω, which amounts to total energy evaluations for neutral and singly ionized species. Note that the required atomic spin constants are functional specific. The spin parameters for the ob2 Slater-Koster set are available in the manual. b. Spin restricted ensemble references. Instead of single determinants, the spin-restricted ensemble-referenced Kohn-Sham (REKS) method and its state-interaction state-averaged variant (SI-SA-REKS or SSR)  [46] [47] [48] [49] [50] [51].
The compressed densities and wave-functions come from solving a Kohn-Sham-problem for a single atom with an additional confinement potential (usually a power function), as shown in Eq. (  4 ). One may use different compression radii (and make separate calculations) to obtain the compressed density and the compressed atomic wave-functions for a given atom. The atomic calculations are currently carried out with a code implementing the Hartree-Fock theory based atomic problem  140, 141  extended with the possibility of including DFT exchange-correlation potentials via the libxc library  142  and scalar relativistic effects via the zero-order relativistic approximation (ZORA). 143  The resulting densities and atomic wave-functions are stored on a grid. The two-center integration tool reads those grid-based quantities and calculates the hamiltonian and overlap two-center integrals for various distances using the Becke-method.
The ELSI interface and supported solvers ELSI  157  features a unified software interface that simplifies the use of various high-performance eigensolvers (ELPA  158  EigenExa,  159  SLEPc,  160  and MAGMA  161  ) and density matrix solvers (libOMM,  162  PEXSI,  163  and NTPoly  164  ). We convert the sparse DFTB+ H and S structures 8 into either standard 2D block-cyclic distributed dense matrices or sparse 1D block distributed matrices compatible with the ELSI interface. All k-points and spin channels are then solved in parallel. 6] [167]  Figure  5  demonstrates two examples: non-self-consistent-charge, spin-non-polarized, Γ-point calculations for a C 64000 nanotube (CNT) and a Si 6750 supercell, with 25 600 and 27 000 basis functions, respectively. Figure  5(c)  shows the time to build the density matrix for the CNT model with three solvers, the pdsyevr eigensolver in the MKL implementation of ScaLAPACK, the ELPA2 eigensolver, and the PEXSI density matrix solver.
C. GPU computing Graphics processing unit (GPU) acceleration is implemented in DFTB+. Given the nature of the underlying theory, the timelimiting step in routine calculations corresponds to the diagonalization of the hamiltonian matrix, taking in the order of 90%-95% of FIG. 6. CPU time for the density matrix construction for different varying sizes of water box systems. Regular diagonalization (black curve) was compared to the SP2 method (red curve). A numerical threshold of 10 -5 was used in the sparse matrix-matrix multiplications of the SP2 algorithm. the total running time. The hybrid CPU-GPU implementation in DFTB+ replaces the LAPACK-based eigensolver with a GPU eigensolver based on the divide-and-conquer algorithm as implemented in MAGMA. 178 enchmarking of the code shows that at least 5000 basis functions are necessary to exploit the power of the GPUs and to produce an observable speedup with respect to the CPU-only code.
A. File based communication When using file based communication, the external driver creates necessary input files and starts an individual DFTB+ program for each of the inputs. After DFTB+ has finished, the driver analyses the created output files and extracts the necessary information from those. DFTB+ had been interfaced using file based communication to, among others, the phonopy 179 code for finite difference harmonic and anharmonic phonon calculations and the Atomic Simulation Environment (ASE) package  180  (a set of tools and Python modules for setting up, manipulating, running, visualizing, and analyzing atomistic simulations).
B. Socket interface The i-PI  181  interface for communication with external driving codes is supported by DFTB+. DFTB+ can then be driven directly instead of using file I/O. The initial input to DFTB+ specifies the boundary conditions, type of calculation, and chemical information for atoms, and the code then waits to be externally contacted. This kind of communication with DFTB+ can be used by, among others, the i-PI universal force engine package  181  and ASE. 180  DFTB+ library, QM/MM simulations 1. Gromacs integration DFTB quantum-chemical models may be utilized as a QM engine in hybrid quantum-mechanical/molecular mechanical (QM/MM) approaches. This allows, for example, efficient simulations of chemical processes taking place in bio-molecular complexes. The DFTB+ library interface has been connected to the Gromacs 182 MM-simulation software package.
E. DFTB+ in Materials Studio DFTB+ is included as a module in the commercial modeling and simulation software package, BIOVIA Materials Studio (MS). 193 FTB+ runs as an in-process energy server, supplying energies, forces, and stresses to drive the MS in-house simulations tools. Supported tasks include energy calculation, geometry optimization, molecular dynamics, electron transport calculation, mechanical properties, and parameterization. The module also supports calculation and visualization of standard electronic properties, such as band structure, density of states, orbitals, and so on. The DFTB+ module integrates closely with the data model and the Materials Studio Visualizer, allowing the user to construct structures and start calculations quickly, with fully automated creation of the DFTB+ input file.
The DFTB+ module is also supported in the MS Mate-rialsScript interface and the Materials Studio Collection for Pipeline Pilot, 194 allowing creation of more complicated workflows. 195, 196 he DFTB+ parameterization workflow in MS supports fitting of both electronic parameters and repulsive pair potentials using DFT calculations with the DMol3 module  197, 198  as a reference. The DFTB+ module includes scripts for validation of parameters in terms of band structure, bond length, bond angles, and so on, as well as visualization for the hamiltonian, overlap matrix elements, and the repulsive pair potentials. The parameterization tools allow extension of existing parameters or incremental development of a parameter set. Parameters developed using the DFTB+ module can, after conversion, be used outside MS.
A. Modern Fortran wrappers for MPI and ScaLAPACK functions Modern scientific modeling packages must be able to run on massive parallel architectures to utilize high performance computing, often using the Message Passing Interface (MPI) framework. While the MPI offers a versatile parallelization framework, its application interface was designed to support C and Fortran 77-like interfaces. This requires the programmer to explicitly pass arguments to the MPI-routines, which should be automatically deduced by the compiler for languages with higher abstraction levels (C++ or Fortran 95 and newer versions). In order to eliminate developer need to pass redundant information (and to reduce associated programming bugs), we have developed modern Fortran wrappers around the MPI-routines. These have been collected in the MPIFX-library,  200  which is an independent software project outside of the DFTB+ software suite, being licensed under the more permissive BSD-license.
B. Fortran meta-programming using Fypp Although the latest Fortran standard (Fortran 2018) offers many constructs to support modern programming paradigms, it does not allow for generic template based programming. This would avoid substantial code duplication and offer useful metaprogramming capabilities for Fortran programmers. We have developed the Python based pre-processor, Fypp, 202 which offers a workaround for the missing features. Fypp is used during the build process to turn the meta-programming constructs into the standard Fortran code. The Fypp project is independent of the DFTB+ software package and is licensed under the BSD-license, being also used by other scientific software packages, for example, by the CP2K code 203 and both the MPIFX and the SCALAPACKFX libraries.
ACKNOWLEDGMENTS The authors, especially  B. Hourahine  and  B. Aradi , are thankful to  Gotthard Seifert  for his suggestions and insights into density functional tight binding throughout the development of the DFTB+ code. B. Hourahine acknowledges the  EPSRC  (Grant No. EP/P015719/1 ) for financial support. B. Aradi  and  T. Frauenheim  acknowledge the research training group  DFG-RTG 2247 (QM3 ). A. Buccheri  acknowledges the  EPSRC  (Grant No. EP/P022308/1 ). C. Camacho  acknowledges financial support from the  Vice-Rectory for research of the University of Costa Rica  (Grant No. 115-B9-461 ) and the  Oak Ridge Leadership Computing Facility  at the  Oak Ridge National Laboratory (ORNL) , which is managed by  UT-Battelle, LLC , for DOE under Contract No. DE-AC05-00OR22725. S. Irle  acknowledges support from the  U.
17-SC-20-SC ), a collaborative effort of the  U. S. Department of Energy, Office of Science  and the  National Nuclear Security Administration. T. A. Niehaus would like to thank the  Laboratoire d'Excellence iMUST  for financial support. M. Stöhr  acknowledges financial support from the  Fonds National de la Recherche, Luxembourg (AFR Ph. D. Grant No. CNDTEC). A. Tkatchenko was supported by the  European Research Council (ERC-CoG BeStMo). V. W. -z. Yu was supported by the  National Science Foundation (NSF)  under Grant No. 1450280  and a fellowship from the  Molecular Sciences Software Institute  under  NSF  Grant No. 1547580.
201 See https://github. com/dftbplus/scalapackfx for SCALAPACKFX library; accessed 15 December 2019. 202 See https://github. com/aradi/fypp for Fypp preprocessor; accessed 15 December 2019. 203 See https://github. com/cp2k/cp2k for CP2K software package; accessed 15 December 2019.
Recent breakthroughs in representation learning of unseen classes and examples have been made in deep metric learning by training at the same time the image representations and a corresponding metric with deep networks. Recent contributions mostly address the training part (loss functions, sampling strategies, etc. ), while a few works focus on improving the discriminative power of the image representation. In this paper, we propose DIABLO, a dictionary-based attention method for image embedding. DIABLO produces richer representations by aggregating only visually-related features together while being easier to train than other attention-based methods in deep metric learning. This is experimentally confirmed on four deep metric learning datasets (Cub-200-2011,  Cars-196, Stanford Online Products, and In-Shop  Clothes Retrieval)  for which DIABLO shows stateof-the-art performances.
However, the feature-wise selection does not contribute to feature denoising. In this paper, we introduce the method DIABLO, a DIctionary-based Attention BLOck, that produces robust image representations by taking advantage of both ABE and NetVLAD benefits. We evaluate attention strategies named pre-attention and postattention in the DML setup, together with two selection strategies named feature-wise and dimensionwise selection. We show in practice that DIA-BLO consistently improves the state-of-the-art on four DML datasets  (Cub-200-2011, Cars-196, Stanford Online Products and In-Shop Clothes Retrieval). The remaining of the paper is organized as follows: in the next section, we present the related work on deep metric learning and support our proposition. In section 3, we give an overview of the proposed architecture, then we detail the attention strategies and the feature-wise and dimension-wise selection methods.
Feature-wise selection ensures that only visuallyrelated features are pooled together. This selection strategy does not have the denoising property of the dimension-wise selection. Also, in the case of NetVLAD, the pre-processing is a centering and an intra-projection, that can be improved using non-linear transformation learned by a CNN. In DIABLO, we show the benefits of dimensionwise selection versus feature-wise selection with preattention or post-attention strategies. With this aim, we propose to replace the divergence loss in ABE by a structural constraint, which is based on a dictionary strategy. This dictionary is trained without direct supervision as it is the case with NetVLAD. We show that it leads to better results than ABE on four DML datasets.
8, 1. 8] times the crop size as well as random 256×256 crop and horizontal flip during the training. During testing, we re-scale the images to 256 × 256. We use Adam  [Kingma and Ba, 2015]  with the default parameters and a learning rate of 10 -5. For all models, the function ψ is shared across the attention maps to reduce the large number of parameters induced. In practice, it still leads to strong experimental results and avoids over-fitting on small datasets, for instance: Cub-200-2011 and Cars-196.
As it the case with pre-attention, each branch is pooled using a global average pooling, an embedding layer of size 512/N and a 2 -norm are added before the concatenation of all branches in order to produce the full 512d representation. These choices directly follow ABE  [Kim et al. , 2018]  and we refer the reader to the related paper for more ablations on the architecture, including the multi-head approach, the attention module and the sharing of parameters across the attention modules. All additional parameters are included in the computation of the function φ. By using the five Inception blocks from the fourth scale of GoogleNet, this leads to 3. 5M additional parameters. Note that these parameters are shared across the dictionary entries and this drastically reduces the number of parameters.
Model selection protocol In this section, we detail the evaluation protocol for all ablation studies on the Cub-200-2011 dataset that are performed to select the best model. We perform 10 random train-val splits on the training set of Cub-200-2011 for deep metric learning: We randomly choose 50 classes for the training set and we keep the rest of the classes for the validation set. Then, we train each model on the training set and we select the model that gives the best performances on the validation set. We then compute Recall@K on the testing set of Cub-200-2011 for each train-val split. All reported results in Table  4  and Table  3  are the average and the standard-deviation of Recall@K on the testing set for the ten runs.
Feature selection and attention strategy First, we evaluate the benefit of the pre and postattention strategies with respect to the assignment strategy. To that end, we fix the dictionary size to 8 and we use the training procedure from subsection 3. 5. Results are reported in Table  3  for the dataset Cub-200-2011 with binomial loss. We remark that all strategies with the exception of the featurewise selection with pre-attention improve over the baseline and this confirms the benefit of attention maps. This experiment also shows that feature-wise attention and dimension-wise attention impact differently the model. In Feature-wise attention, the post-attention leads to stronger representations with +1. 7% on Recall@1 over the pre-attention. We argue that selecting features make better sense with post-attention than preattention: only related features are aggregated together with post-attention whereas in pre-attention, the refinement part mostly processes sparse feature maps.
Comparison to ABE In a second time, we want to evaluate the impact of the structural constraints imposed by the dictionary (Equation  7 and Equation  11 ) by comparing our method to ABE  [Kim et al. , 2018]. In ABE, the authors show that a M-head approach already provides strong results on the Cars-196 dataset with 76. 1% (+8. 9%) Recall@1 for M = 8. However, this architecture tends to overfit due to the large number of parameters. Then, they propose an enhanced version named ABE that takes advantage of attention maps. The divergence loss increases the performance from 69. 7% without the divergence loss to 85. 2% (+15. 5%) in Recall@1 (see Table  2  in  [Kim et al. , 2018] ). Among the drawback of the divergence loss, despite of the additional hyper-parameter, figure that optimizing the loss generates gradients which are in opposition with the metric learning loss ones.
Indeed, it is designed to reduce the similarity between different branches even when the images are similar. We solve this issue in DIABLO where this optimization constraint is replaced by a structural constraint (softmax in Equation  7 and Equation  11 ). The orthogonality is ensured by the design of DIABLO, which allows to simply remove the divergence loss at the price of a reduced expressiveness: feature map entries can be chosen independently in ABE whereas in DIABLO they are constrained to only one dictionary entry. In Table  1  and Table 2 , DIABLO performs well compared to ABE with similar results on the Cars-196 dataset (-0. 2% compared to ABE) and higher ones on other datasets such as Cub-200-2011 (+2. 2%), Stanford Online Products (+1. 5%) and Inshop Clothes Retrieval (+3. 4%) for this set of parameters.
Dictionary size In this section, we evaluate the impact of the dictionary size on DIABLO. We evaluate both the featurewise and the dimension-wise with dictionary sizes in {2, 4, 8, 16}. Recall@1 on the Cub-200-2011 dataset are reported in Table  4. In post-attention with feature-wise selection, extreme combinations (e. g. , with 16 branches with 32 dimensions) lead to results under the baseline. Thus, to increase the performances of such attention strategy, there is a compromise between the representation size of each branch and the number of branches (+1. 1% over the baseline). In pre-attention with dimension-wise selection, all parameter combinations for this approach lead to better results than the baseline (+4. 9% to +5. 4%). The number of branches increases the performance with the log of the dictionary size on the contrary to the previous strategy.
Comparison to the state-ofthe-art In this section, we compare DIABLO to the stateof-the-art on 4 DML datasets, named Cub-200-2011  [Wah et al. , 2011 ], Cars-196 [Krause et al. , 2013] , Stanford Online Products  [Oh Song et al. , 2016]  and In-Shop Clothes Retrieval  [Liu et al. , 2016]. For  Cub-200-2011, Cars-196   3 : Impact of the pre-attention or post-attention on the performances for the three proposed attention strategies. Reported Recall@1 (R@1) is on the Cub-200-2011 dataset in percent.
Feature Dimension Baseline Pre-att Post-att Pre-att Post-att R@1 52. 9 ± 0. 2 52. 3 ± 0. 4 54. 0 ± 0. 4 58. 3 ± 0. 2 57. 8 ± 0. 1 Table  4 : Impact of the dictionary size for the featurewise post-attention mapping and for the dimensionwise pre-attention mapping. Reported results are Recall@1 (R@1) on the  Cub-200-2011   2  for Stanford Online Products and In-Shop Clothes Retrieval. DIABLO consistently improves the already strong baseline on the four datasets and for three different loss functions. E. g. , using the binomial loss, the baseline is improved from 59. 6% to 63. 9% (+4. 3%) on Cub-200-2011 and from 78. 8% to 85. 4% (+6. 6%) on the Cars-196 dataset. The same observation is made for both the other loss functions on these datasets. Moreover, DIABLO leads to better results when compared to the similar approach ABE.
Their best reported approach, ABE-8, is outperformed by DIA-BLO with N = 8 and c = 64 (total dimension 512) by 2. 2% in R@1 on Cub-200-2011, by 1. 5% on Stanford Online Products and by 4% on In-Shop Clothes Retrieval. We also report results with N = 16 and c = 32 (total dimension 512) which are further improved on these datasets: 1. 1% in Recall@1 on the Cub-200-2011 and by 0. 4% on the Cars-196 dataset, leading to the state-of-the-art on the four deep metric learning datasets.
Conclusion In this paper, we have presented a dictionary-based attention method named DIABLO which consistently improves DML models. An ablation study is undertaken to evaluate the benefits of the featurewise and the dimension-wise selections for two attention strategies named pre-attention and postattention. We show that DIABLO consistently outperforms the baseline for three different loss functions on four datasets  (Cub-200-2011, Cars-196, Stanford Online Products, and Inshop Clothes Retrieval). Moreover, it outperforms the current state-of-the-art methods on the four datasets.