dataesr/scientific-paragraphs-categorization huggingface dataset:
While resulting in SSH fields that are dynamically constrained, this method suffers from a drawback that it requires additional observations to constrain other essential model variables like the subsurface flow and/or the density field. Also, data assimilation for complex ocean models at eddy-resolving scales is often underdetermined and is computationally demanding. A recent study by  (Ubelmann et al. , 2015)  demonstrated that representing SSH propagation with a single equivalent barotropic mode (that mimics the first baroclinic mode) in a quasigeostrophic (QG) model results in significant improvements in the spatiotemporal interpolation of sparse SSH observations. In particular,  Ubelmann et al. (2015)  considered a fundamental problem of reconstructing the SSH distribution that occurred in between two observed SSH fields separated by about 20 days, a characteristic timescale required by a set of altimeters to reconstruct a spatial SSH field.
They found that integrating the earlier SSH observation forward in time (following the assumed dynamics of an equivalent barotropic mode) and averaging it with the later observed SSH anomalies that were integrated backward in time resulted in an improvement compared to conventional linear interpolation methods. In follow-up work,  Ubelmann et al. (2016)  generalized this temporal interpolation method to the spatiotemporal interpolation of alongtrack SSH observations by essentially performing data assimilation on the one-layer QG model. The advantage of the dynamical interpolation method is that it relies on the advection of potential vorticity-a nonlinear process that is inherently present in ocean dynamics and cannot be represented by linear or objective interpolation techniques. A drawback of the dynamical interpolation is that it assumes that SSH evolves independently of deep ocean flows, considering the so-called equivalent barotropic mode dynamics  (Berloff & Meacham, 1997).
We thus hypothesize that deep learning techniques could outperform conventional interpolation methods including linear and dynamical interpolation. In this manuscript, we use synthetic model observations related to the QG turbulence in the Gulf Stream, typically forced by the baroclinic instability of a westerly sheared flow  (Tulloch et al. , 2011). These model observations only involve mesoscale eddies and no submesoscales (resulting for example from mixed-layer instabilities) and do not include other processes, such as Ekman pumping or incoherent internal tides, known to impact SSH. These model observations are used to present a first proof of concept for using deep learning to shortcut the formal process of data assimilation and reconstruct not only the interpolated SSH field but also the corresponding unobserved deep ocean currents, thus providing a complete state estimate of the baroclinic ocean turbulence.
Methods We implement a range of deep NN architectures to address a basic question of interpolating SSH fields in baroclinic ocean turbulence. To exclude potential limitations of real-world data, our study is entirely based upon synthetic data that we generate using the QG model of baroclinic ocean turbulence. We find the QG model to be optimal for our goals as it is pertinent to many energetic regions in the ocean while being relatively simple such that a large volume of data can be generated for training and testing; furthermore, the model allows us to directly benchmark deep learning against the dynamical interpolation technique that also utilizes QG dynamics. Below we describe our NN architectures, the QG model used for the generation of training and testing data sets, and the details of the dynamical interpolation that we implemented for direct skill comparisons with deep learning and linear interpolation.
Deep Learning Framework: Residual Convolutional Neural Networks Artificial NNs are based on the idea of approximating the "output" by taking the "input" variable and performing a large number of matrix additions and multiplications, applying nonlinearity functions, and either condensing or expanding the variable dimension as it passes from layer to layer. The resulting network contains a large number of free parameters that are later adjusted to optimize a given loss function, commonly taken as a measure of difference between the prediction and the truth. Because we are trying to extract information from the eddy patterns expressed in SSH fields, the choice of CNNs is rationalized. In passing the data from layer to layer, CNNs use a set of convolutional filters (kernel matrices with prescribed dimensions) to produce increasingly more abstract levels of data representation that are passed on to the next layer.
Here we implement the ResNet architecture-a CNN with Residual Learning blocks  (He et al. , 2016). The Residual Learning is a process by which the data are not only transferred sequentially from one layer to another but is also transferred by skipping several layers via the so-called skip connections (blue arrows in Figure  1 ). The presence of skip connections can result in better performances for a wide range of computer vision problems  (Targ et al. , 2016). A brief description of the ResNet architecture as shown schematically in Figure  1  follows. The input consists of two SSH snapshots represented by a (32,32,2) matrix. The very first convolutional layer takes the input and applies a set of 32 convolutional filters of size (5,5) with a stride of (1,1), followed by the batch normalization, the nonlinearity function taken to be the Leaky Rectified Linear Unit (Leaky ReLU), and the maximum 2D pooling of size (2,2) with a stride of (2,2).
VGG_Deep: same as VGG but repeating each convolutional layer three times before proceeding to the next one. ResNet_Small, ResNet, and ResNet_Large are residual NNs with architectures as depicted in Figure  1  but with a total of 31,52, and 161 convolutional layers correspondingly; _Dropout denotes the use of 20% dropout rate in the last layer. We have implemented the architectures in Tensorflow/Keras and provided the Python scripts along with the training data in the Zenodo data repository  (Manucharyan, 2020). As a performance metric, we define the model skill that is proportional to the loss function and normalized by the standard deviation of the SSH signal in the following way:              1 2 2 2 | | 1. | | predicted true true SSH SSH Skill SSH (4) For reference, the maximum skill = 1 is achieved when the predicted and true images are identical; the skill = 0 corresponds to a prediction that makes the same error as assuming a spatially homogeneous SSH field, and negative skill implies an even worst fit.
The parameter optimization procedure requires evaluating NN predictions for a large volume of training data and hence the final optimized state of a particular NN depends only on the training data itself. To ensure that no overfitting has occurred, the NN skill is evaluated for a group of three independent data sets: training, validation, and testing. The training data are used only for the training purposes, the validation data are used to evaluate the skill of the NN and to identify a stoppage criterion for the training, while the testing data are used at the very last step to define the skill of a trained NN. All three data sets are generated from different numerical simulations to ensure that overfitting did not occur.
Given such low similarity skills, the existence of analog states in our training/testing data sets can be ruled out. The lack of analogs between the different QG simulations is expected as considering just a handful of the its principle component modes (out of about 30 modes that have significant energy) as independent degrees of freedom would already amount to astronomically low probabilities of encountering true analogs (Van den Dool, 1994). To evaluate the efficacy of the NN approach, we consider the tasks of (i) temporal interpolation where the input consists of two SSH snapshots separated by 20 days, (ii) spatiotemporal interpolation with the same input as for the temporal interpolation but with SSH images having missing data, and (iii) the state estimation of unobserved deep ocean flows from SSH snapshots.
Discussion Our study explored the efficacy of deep learning in reconstructing the unobserved state variables of the chaotic ocean turbulence. The motivation for addressing the specific problem of SSH interpolation came from the present-day use of relatively rudimentary techniques of reconstructing continuous fields from sparse satellite data. Using synthetic data from an idealized model of baroclinic ocean turbulence, we presented the proof of concept for using deep neural networks as an efficient technique to extract nontrivial information from sparse SSH observations. Specifically, we demonstrated that residual CNNs can reconstruct SSH snapshots at the intermediate time between the 20 days separated observations with an average skill of 0. 75, significantly outperforming the commonly used linear interpolation (skill = 0.
After estimating the mode amplitudes based on individual SSH snapshots and learning from many synthetic examples of SSH evolution in time, the NNs are then capable to effectively integrate the QG equations forward or backward in time for a skillful temporal interpolation between the two SSH snapshots. While the complexity of deep learning algorithms makes it impossible to interpret them, our hypothetical two-step process of the mode decomposition followed by the forward and backward integration provides a plausible dynamical rationalization for the superiority of deep learning over methods that ignore the influence of deep ocean flows on SSH evolution. We chose to use the QG simulations of baroclinic turbulence as the synthetic training data set because it presents a hard test for the temporal SSH interpolation due to its chaotic nature and an a priori unknown impact of the dynamically active bottom layer on SSH evolution.
However, for the case of submesoscale turbulence (length scales smaller than about 100 km), the question remains open as to how SWOT's 2D high-resolution swath measurements could be used to enhance the resolution of SSH data. While we expect the deep learning framework to perform well in reconstructing both large and small mesoscale eddies, its limitations still need to be understood when considering mesoscale and submesoscale turbulence as a continuum. It is thus necessary to develop more general training data sets that are representative of the SSH dynamics for any given region or process of interest. Including satellite observations from Synthetic Aperture Radars or of sea surface temperatures in addition to the SSH observations could provide additional information for improved reconstruction of SSH.
The training data sets could be assembled ranging from more realistic submesoscale-resolving general circulation models to simplified stochastic models in various parameter regimes  (Samelson et al. , 2019). While diversifying the training data sets should increase the versatility of NN interpolation methods, the crucial constraint of their performance would likely come from the chaotic evolution of submesoscale eddies that occurs on substantially shorter timescales compared to mesoscale eddies. While we have demonstrated the efficacy of supervised deep learning using synthetic data, its practical utility in interpolating real-world SSH observations remains to be tested. The drawback of deep learning is that it requires a large volume of training data, although there are continuously improving methods aimed at addressing this practical issue, for example, transfer learning  (Pan & Yang, 2009) , data augmentation  (Perez & Wang, 2017) , one-shot learning  (Fei-Fei et al.
Developing deep learning SSH interpolation techniques that would steer away from solely relying on dynamical models to provide training data is a necessary next step toward practical implementation with real satellite observations. Nonetheless, our work presents an important proof of concept that SSH observations do contain dynamically relevant information about subsurface flows, and hence with deep learning it should be possible to build a skillful model of SSH evolution and as a consequence improve the existing SSH estimates. Finally, we note another potentially important application of deep learning for state estimation at eddy-resolving scales. Since mesoscale-resolving data assimilation methods require large computations, providing an accurate initial guess would substantially reduce the number of iterations necessary for optimization.
Georgy E. Manucharyan was funded by the ONR Grant N00014-19-1-2421 at the University of Washington and thanks Charles Trimble for providing support at the California Institute of Technology. Lia Siegelman was a NASA-JVSRP affiliate and supported by a Caltech-JPL postdoctoral fellowship. Patrice Klein was supported by the NASA NPP Senior Fellowship and the NASA SWOT project. Discussions with Dimitris Giannakis, Andrew Thompson, Clement Ubelmann, Laure Zanna, and participants of the 2019 SWOT Science Team Meeting are highly appreciated. The authors thank Glenn Flierl for providing the Matlab code for simulating the two-layer QG turbulence and Michael Dietz for the open-source implementation of ResNet, modifications of which were used in this study. The authors thank the two anonymous reviewers and the associate editor, Annalisa Bracco, for insightful comments that led to improvements of the manuscript.
AutoMeKin2021 is an updated version of tsscds2018, a program for the automated discovery of reaction mechanisms (J. Comput. Chem. 2018, 39, 1922-1930). This release features a number of new capabilities: rare-event molecular dynamics simulations to enhance reaction discovery, extension of the original search algorithm to study van der Waals complexes, use of chemical knowledge, a new search algorithm based on bond-order time series analysis, statistics of the chemical reaction networks, a web application to submit jobs, and other features. The source code, manual, installation.
7 The first version of the computer program implementing our approach was released three years ago under the name tsscds2018. 46 This approach, along with the algorithms described in this paper, has now been combined and implemented in the software package named AutoMeKin,  78  which stands for Automated Mechanisms and Kinetics. AutoMeKin2021 includes the following new features: (a) the rare-event acceleration method BXDE;  76, 79  (b) a generalization of a Graph Theory-based algorithm to locate TS structures for the study of non-covalent interactions;  80  (c) a chemical knowledge-based method for reaction discovery; (d) a new TS search algorithm based on a bond-order time series analysis;  81  (e) a statistical analysis of the chemical reaction networks using the Python library NetworkX;  82  (f) a web application for online job submission; as well as other features.
Rare-event acceleration method BXDE Standard MD simulations are typically biased towards the entropically-favored reaction pathways. AutoMeKin's standard MD module employs initial conditions with substantial amounts of vibrational energy to accelerate the incidence of reactive events. An alternative way of accelerating reactive events, called Boxed Molecular Dynamics in Energy space (BXDE)  79  has recently been proposed. BXDE belongs to the family of BXD methods,  [86] [87] [88] [89]  which introduce reflective barriers in the phase space of an MD trajectory along a particular (collective) variable. The boundaries are employed to push the dynamics along the collective variable into regions of phase space which would rarely be sampled in an unbiased trajectory. In BXDE, the bias is introduced into the potential energy, rather than in any particular collective variable of the system.
The different chemical reaction channels are sampled by gradually scanning through potential energy "boxes" or energetic "windows. " The BXDE simulation module in AutoMeKin utilizes the Atomistic Simulation Environment (ASE) package, and MOPAC2016 or Entos Qcore are interfaced via the ASE calculator class. 90 Figure  1  shows the section of an input file where a BXDE calculation is requested. Each line in the input file consists of a keyword value pair. BXDE is one of the different sampling methods employed in AutoMeKin for finding stationary states in a potential energy surface, with other sampling alternatives such as: MD, MD-micro, external, ChemKnow, association, and vdW. Although some of the options are described in this work, the reader is referred to the documentation for more details.
5 Å). The keyword Nassoc is used to select the number of initial structures generated. The randomly generated structures are subjected to optimization using xtb. The global minimum from the set of optimized structures is then used as the starting point for the MD simulations. where llcalcs. sh is AutoMeKin's script to run all low-level calculations,  46  vdW. dat is the input file (summarized in Figure  3 ), 100 is the number of BXDE trajectories per iteration, 3 is the number of iterations, and 48 is the number of concurrent simulations in a multithreading CPU. This calculation results in a total of 112 minima and 115 TSs for the NO2 + pyrene system. Ignoring the unconnected minima, the reaction network and its corresponding minimum energy structures are displayed in Figure  4. In sum, the more general definition of the adjacency matrix of Eq.
As a second check, its energy must be lower than emax to retain the newly generated geometry. 4) A path connecting the initial and final geometries is constructed using the Nudged Elastic Band (NEB) method, and the highest point along the path is subjected to TS optimization. 5) Successive iterations of AutoMeKin start from a new (connected) graph, and steps 1-4 are repeated until no new graphs are found. To avoid sampling equivalent paths multiple times in step 2, a descriptor for each unique TS is book-kept in a Python dictionary. The chosen descriptor is the list of eigenvalues of a TS adjacency matrix: where 7 and 9 are the reactant and product adjacency matrices, respectively, with the atomic numbers filling the diagonals. This descriptor has the property of being invariant with respect to permutations of like atoms, thus avoiding sampling equivalent paths more than once.
The CO2:CO branching ratio obtained at 150 kcal/mol of excitation energy is 3:97. In comparison, an MD sampling with 200 trajectories leads to 11 TSs and 7 minima and a CO2:CO ratio of 2:98. A test was also done on vinyl cyanide (VC), to compare the performance of ChemKnow vs an MD-based sampling. Chemknow's constraints are similar to those employed for FA, including now (2,2) combinations of (nf,nb) for the graph transformations, and min_val and max_val for N are 1 and 4, respectively. In this example, ChemKnow needs to sample 850 paths to obtain 59 TSs and 31 minima vs 2000 trajectories employed by the MD module, which affords 64 TSs and 27 minima. Although the efficiency of ChemKnow is superior to MD for VC, the former failed to optimize a significant TS connecting VC to vinyl iso-cyanide.
Properties of the reaction network In AutoMeKin2021, properties of the reaction networks are analyzed using the NetworkX Python library. 101 Here, a node of the graph is either an intermediate or a product, while an edge represents a pathway connecting two nodes. Two types of networks are constructed in the example shown here. In the "all-states" network, every single intermediate constitutes a node, while in the "coarse-grained" one a family of conformers is lumped together to form each node. Additionally, in both networks, edges have weights representing the number of pathways connecting a pair of nodes. Finally, self-loops are avoided by removing paths connecting permutation-inversion isomers of the same node. Common properties of a network can be studied in this new version of the program: the average shortest path length, the average clustering coefficient, the transitivity, and the assortativity.
The number of jobs is not limited, but users should try to limit the number of jobs they submit at once so that other users can also use the server at the same time. The status of the jobs is shown on a different page (Figure  8c ). Depending on the size of the system and workload, the execution time can vary substantially, and users can log out. Upon job completion, they will receive a notification in their email account. Finished jobs appear with a "Completed" status in Figure  8c  and users will be able to download a brief summary of the results in PDF format ("Report"), as well as a tarball file with detailed data ("Data"). The web interface was built following HTML5 recommendations. 109 The Bootstrap framework 110 is included to develop a responsive design. The Apache HTTP server, the MariaDB server, and PHP are used to build the backend.
The figure displays only a reduced part of the pathways (those involving different fragments) with the positive charges assigned to the corresponding fragments. Since the reaction detection algorithms focus on bond formation/breakage, there is an additional tool to scan dihedral angles and find transition states for interconverting conformers. Torsions around bonds with bond orders greater than 1. 5 and/or those belonging to rings are excluded. Finally, an auto-installer script is now available, which eases the burden of installing third-party packages. The script installs singularity  112  and downloads the latest container image from sylabs (https://sylabs. io/). An instance of the container is started using a sandbox image deployed under $(TMPDIR-/tmp) folder. The container comes with all AutoMeKin's tools installed in $AMK.
Conclusions Presented here is the open-source software package AutoMeKin, for automated reaction discovery. AutoMeKin is an updated version of tsscds2018 featuring several new tools: rareevent MD simulations, a search algorithm to study van der Waals complexes, a chemicalknowledge based search procedure, a reactive-event detection method based on bond orders, statistics of the chemical reaction networks, and a web application to submit online jobs. AutoMeKin is actively developed, and the most relevant functionalities that will be incorporated in the future include (but are not limited to): a) An interface with M3C.
699840 -3. 368670 vdW H Energy 150 0. 000000 0. 000000 -4. 589720 H --Screening--0. 000000 Input files H 0. 000000 imagmin 200 3. 380390 3. 380390 1. 229240 -1. 229240 eigLmax 0. 1 NO2. xyz BAPEmax 2. 5 vdW. dat MAPEmax 0. 008 uracil. xyz --General--molecule pyrene-NO2 --Kinetics--3 13 fragmentA pyrene fragmentB NO2 LowLevel qcore xtb N 3. 073876 0. 000123 Energy 150 N 0. 000000 1. 016495 0. 000000 -0. 318014 O 3. 181202 -1. 096194 C 1. 112839 0. 313955 0. 000000 0. 133758 O 3. 181114 1. 096449 FA. xyz N 1. 126388 -1. 043152 0. 000000 0. 133758 C -0. 049649 -1. 744872 0. 000000 --Method--sampling vdW rotate 5 C -1. 235176 -1. 071681 0. 000000 Results: http://doi. org/10. 5281/zenodo. 4700262 C -1. 161437 0. 348965 0. 000000 com com 4. 0 1. 5 Nassoc C 0. 000000 0. 000000 0. 000000 O 2. 301950 0.
000000 0. 000000 -2. 68680 C 0. 000000 H -1. 40050 imagmin 200 C 0. 000000 MAPEmax 0. 008 C 0. 000000 BAPEmax 2. 5 C 0. 000000 eigLmax 0. 1 Results: http://doi. org/10. 5281/zenodo. 4700257 0. 000000 -0. 704710 -1. 223960 0. 08560 -0. 00010 -1. 409630 0. 000000 1. 44380 0. 00080 0. 704710 -1. 223960 1. 409630 -2. 435000 0. 699840 -2. 435000 -0. 699840 C H --Kinetics--0. 000000 0. 000000 C 0. 000000 Energy 150 1. 223960 -3. 380390 -1. 214210 1. 409630 -1. 229240 -2. 812580 C C FA_md. dat 0. 000000 0. 000000 0. 000000 -1. 214210 -3. 507360 2. 812580 H 0. 000000 -3. 380390 1. 229240 C --General--0. 000000 1. 214210 2. 812580 C molecule FA 0. 000000 0. 000000 3. 507360 C LowLevel mopac pm7 0. 000000 2. 435000 0. 699840 Results: http://doi. org/10. 5281/zenodo. 4711089.