dataesr/scientific-paragraphs-categorization huggingface dataset:
The latter point is of great importance for the operando   2. (c) Experimental ex situ pXRD diffractograms of the corresponding "pure" compounds when isolated under the milling conditions reported in Table  2  (in black), and comparisons with the simulated pXRD data for these Zn-BDC phases (colored lines; pXRD simulated from available CIF files in the CCDC database, as reported in Table  1 ). Raman measurements, justifying the use of an excess amount of water in this particular example (≥16 equiv). The system undergoes several structural transformations during the milling process, as illustrated by the shifts of specific vibration modes in the time-resolved experiment (Figure  1a ). 7] [88]  The signal from the starting H 2 BDC molecule quickly shifts from 832 to 878 cm -1 , and after ∼8 min of milling, it evolves to 862 cm -1 , and finally moves to 865 cm -1 in the final compound.
Algorithm & implementation We now describe the decomposition method from Sect. 3 from an algorithmic point of view and discuss some crucial details of our implementation in Julia  [50]. Given an LTI system of the form (2)-(3), we first apply a suitable approximation model from Sect. 2. 4. Then we execute the corresponding decomposed recurrence from Sect. 3. 3 to compute the reach tube or to check a safety property.
Sparse and dense matrices We use a different implementation for models with dense and sparse matrices , respectively. For instance, the loop around line 14 only has to be executed if the submatrix k [b i , b j ] is non-zero. As the linear algebra back-end we use either a BLAS-compatible library  [56]  or a native Julia implementation for sparse matrices following Gustavson  [57]. These specializations have a major impact on the runtime (around one order of magnitude).
Evaluation We evaluate our implementation called JuliaReach  [59]  on a set of SLICOT benchmarks  [60, 61, 49]  which reflect "real world" applications. Some of the original models are differential algebraic equations (DAEs), in which case we only kept the ODE part, i. e. , the coefficient matrices A and B, which is consistent with related literature. The basic model statistics are outlined in Table  2. We use a machine with an Intel i5 3. 50 GHz CPU and 16 GB RAM, running Linux. For matrix functions of exponential type, ExponentialUtilitiles. jl is used  [62].
Reach tube computation We compare JuliaReach to the state-of-the-art support function algorithm LGG implemented in SpaceEx. We consider two cases: one dimension, where we only compute the reach tube in one variable, and full dimensions, where we compute the whole reach tube. For our approach we use one-dimensional blocks in both cases. We note that we do not use parallelization or coordinate transformations in the evaluation. The results are given in Table  3. We compare the precision using the bounds on a single variable (column "Var. " in the table) at the last time step. The SpaceEx bounds are taken as the baseline, and we report the relative deviation. We expect a lower precision than SpaceEx for two reasons: First, we decompose X (0) into a Cartesian product, which induces an error that is inherent to the decomposition method, as explained in Sect.
Improving precision In both the discrete-time and dense-time cases, JuliaReach fails to verify the same instances with one-dimensional block decomposition, namely ISS and FOM, because it is not precise enough. For both models, the property involves a linear combination of state variables from different blocks. It is not surprising that this leads to a high approximation error. We thus increase the block size for a more precise analysis: For the ISS model, we wrap the output variables x 136 to x 270 in one 135-dimensional block. For FOM we use uniform six-dimensional blocks (plus one 4D block in the end). Additionally, we represent the inputs lazily (see Section 5. 5) in order to get rid of the wrapping effect in the inputs. Fig. 11  shows the output function corresponding to the safety property for the ISS model.
Sparsity Each LTI system has its own structural properties, describing how each state influences the system dynamics. The SLICOT models have different sparsity patterns, which we effectively exploited in JuliaReach. We measure the sparsity of as the number of low-dimensional blocks with at least one non-zero element, divided by the total number of blocks. Observe that line 14 of Algorithm 1 simplifies to a no-op for each zero block, and thus, as a rule of thumb, for a given row-block the cost increases linearly in the number of occupied blocks. Consider the case of 1× 1 blocks. For models such as Heat and Beam the sparsity is 0%, meaning that the matrix is completely dense, while for models such as ISS and FOM the sparsity is 99. 3% and 99. 9%, respectively. We note that the matrix power operation does not necessarily preserve the sparsity pattern, although it does in some particular cases, e.
Acknowledgments M. F. acknowledges stimulating discussions with  Alexandre Rocca  and  Cesare Molinari. We are thankful to  Daniel Freire  and  Jorge Pérez  for discussions with the scalable discretization approach. This work was partially supported by the  European Commission  under grant no. 643921  (UnCoVerCPS), by the  Metro Grenoble  through the project  NANO2017 , by the  Air Force Office of Scientific Research  under award no. FA2386-17-1-4065 , by the  ARC  project  DP140104219  ( Robust AI Planning for Hybrid Systems ), by the  Austrian Science Fund (FWF)  under grants  S11402-N23  ( RiSE/SHiNE ) and  Z211-N23  ( Wittgenstein Award ), by the  European Union 's  Horizon 2020 research and innovation programme  under the  Marie Skłodowska-Curie  grant agreement No  754411 , and by a grant from  Toyota Motors North America R&D.
Post processing of the measurements was conducted in MATLAB 2018a using ''VideoReader'' built-in function which converts a video to a set of images. These images contained pixels of the size (0. 32 mm × 0. 24 mm), which is different from the size of the finite elements (0. 4 mm × 0. 4 mm) that were used in this work. The experimental data was linearly interpolated to pixels having the same dimensions as the finite elements for comparison purposes.
FFF process simulation methodology We implemented the workflow presented in Fig. 1  to simulate the FFF process. The G-code containing process parameters was decoded using a custom python code. The decoded output was processed with an in-house python procedure to compute the activation times of each finite element of the voxelized CAD geometry, as discussed in Section 2. 1 and reported in Appendix B. The obtained activation times were programmed into ABAQUS's UEPACTIVATIONVOL subroutine and input into FEM thermal and mechanical simulations to implement progressive element activation. We used the full activation of the elements. The boundary conditions for the thermal and mechanical problems were specified on inactivate elements and were not applied until the element was activated.
Comparison between the distortion computed numerically and that experimentally measured The CAD model of the specimen was set as a reference with zero distortions and referred as the nominal. This model was uploaded to the GOM Inspection Software which resulted in the mesh containing the elements referred as nominal. The file with digitized surfaces of the specimen was also (Section 3) uploaded to the same software which generated the surface mesh consisting of the elements referred as actual. Due to the difference in spatial orientations, we aligned the reference and the reconstructed surfaces. The GOM Inspection Software associated actual and nominal elements to compute distortions. We (i) Layer 20, points 25-27. Note that, since the first layer deposition speed was almost half as compared to the rest of layers (see Section 3.
B. 2. The workflow inspired from the work of Favaloro et al. [12]  implementing the progressive element activation in ABAQUS. ABAQUS' input file containing the coordinates of each finite element's center along with the G-code data containing coordinates of the toolpath and the size of the filament were input to the custom toolpath-mesh intersection module that resulted in the text files with activation times and local orientations. These text files were read at  = 0 by ABAQUS' subroutine UEXTERNALDB and the data was forwarded to the subroutines ORIENT that assigns elements' local orientations at the beginning of the analysis and to the UEPACTIVATIONVOL that activates elements with respect to their activation times during the analysis. [12]  implementing the progressive element activation in ABAQUS.
ABAQUS' input file containing the coordinates of each finite element's center along with the G-code data containing coordinates of the toolpath and the filament dimensions were input to the custom toolpath-mesh intersection module that resulted in the text files with activation times and local orientations. These text files were read at  = 0 by ABAQUS' subroutine UEXTERNALDB and the data was forwarded to the subroutines ORIENT that assigns elements' local orientations at the beginning of the analysis and to the UEPACTIVATIONVOL that activates elements with respect to their activation times during the analysis. , where  is the total number of elements in the system), the elements stacking direction (   ), the filament's width (  ) and thickness (ℎ), the coordinates of toolpath th segment's start and end points (   and    ,  = 1, 2.
Acknowledgments We acknowledge the support of  Safran S. A. , and the  Natural Sciences and Engineering Research Council of Canada (NSERC)  ( CRDPJ514761-17 ) We acknowledge  Mr. Paul Gregorio  a member of the  Laboratory for Multiscale Mechanics  ( LM 2  ) who helped with the distortion measurements. We acknowledge  Mr. Olivier Duchesne  a member of the LM 2 who performed the tensile tests.
( 2 ) 2 Fig. B. 2 presents the workflow inspired from the work of Favaloro et al. [12]  implementing the progressive element activation in ABAQUS. ABAQUS' input file containing the coordinates of each finite element's center along with the G-code data containing coordinates of the toolpath and the filament dimensions were input to the custom toolpath-mesh intersection module that resulted in the text files with activation times and local orientations. These text files were read at  = 0 by ABAQUS' subroutine UEXTERNALDB and the data was forwarded to the subroutines ORIENT that assigns elements' local orientations at the beginning of the analysis and to the UEPACTIVATIONVOL that activates elements with respect to their activation times during the analysis. Fig. B. 3(a)illustrates the details of the toolpath-mesh intersection.
Scanning transmission electron microscopy A field-emission Cs-corrected (S)TEM (Grand ARM300F, JEOL) operated at 300 kV was used for in situ heating experiments. The in situ heating were performed using a heating holder by DENSsolutions. Atomic-scale characterization during in situ heating experiment was recorded in annular dark field (ABF) and high-angle annular dark field (HAADF) imaging modes. Detector angle ranges of 7. 5 to 17 and 70 to 175 mrad were set for these imaging modes, respectively. The convergence semi-angle for forming the focused probe was 23 mrad. The statistical noise floor in all STEM images was removed using local 2-D Wiener filtering implemented using a commercial software (HRTEM Filter Pro, HREM Research Ltd. ). Energy-dispersive X-ray spectroscopy (EDS) was acquired under 36 mrad convergence angle with spectrometer of a detectable area of 100 mm 2 in STEM imaging mode.
1 eV and 0. 5 s, respectively. The loss energy of the core-loss EELS data was calibrated by tracking the energy drift of the zero-loss peak, which was simultaneously recorded with the core-loss data. The electron dose rate is about 10 6 e - nm -2 s -1. There is no noticeable e -beam damage during acquisition. STEM image simulations were performed using the multislice method in the QSTEM software package  38  using the microscope parameters that closely represent the experimental conditions.
Segregation energy calculations Our first-principles study of APBs electronic and structural properties of LSTO perovskite was carried out in the framework of the density functional theory (DFT) as it is implemented in the VASP (Vienna ab initio Simulation Package) code  39, 40. The Kohn-Sham equations  41  are solved within a supercell approach considering the periodic boundary conditions and a plane-wave basis set. Expansion of the plane-wave basis set was limited by a cut-off energy of 400 eV to describe the electronic states in the system. The electron exchange and correlation effects have been taken into account using the generalized gradient approximation (GGA) in the framework of Perdew-Burke-Ernzerhof (PBE) treatment  42. For total energy and force calculations, we employed an allelectron projector augmented wave (PAW) technique  43.
Migration energy calculations A 164-atom supercell was constructed to model APB-containing La 0. 5 0. 5 TiO 3. Initial lattice parameters of the supercell are a = b = 7:89 Å and c = 31:56 Å with nine TiO 2 layers. Two APBs per supercell were considered. Density functional theory (DFT) calculations were performed to calculate the Ni migration barrier in APBcontaining La 0. 5 Sr 0. 5 TiO 3 with Ni Ti and V Ti using the Vienna Ab Initio Simulation Package (VASP)  40. The projected augmented wave (PAW) method  43  was used to approximate the electron-ion potential. Exchange-correlation effects were treated within the Perdew-Burke-Ernzerhof (PBE)  42  functional form of the generalized gradient approximation (GGA). A 450 eV cut-off energy for the plane wave basis set and 2 × 2 × 1 Γ-centered k-point meshes were used.
Acknowledgements This work was supported by  National Research Foundation of Korea (NRF)  grant funded by the  Korea government (MSIT)  (No. NRF-2020R1A2C2101735 ),  Creative Materials Discovery Program  ( NRF-2019M3D1A1078299 ), the  Samsung Research Funding & Incubation Center of Samsung Electronics  under Project Number  SRFC-MA1702-01 , and the  KENTECH Research Grant  ( KRG2022-01-019 ). D. I. B. acknowledges the financial support from  Russian Foundation for Basic Research  under Grant No. 19-29-03051MK. The first-principle calculations were performed using the facilities of the  Joint Supercomputer Center of the Russian Academy of Sciences (JSCC RAS ). J. L. acknowledges the support of an  NRF grant  funded by the  Korean government  ( NRF-2018R1A2B6004394 ). J. T. S. I. thanks the  EPSRC  for support on emergent nanomaterials through Grant  EP/R023522/1.
Learning-based program repair has achieved good results in a recent series of papers. Yet, we observe that the related work fails to repair some bugs because of a lack of knowledge about 1) the application domain of the program being repaired, and 2) the fault type being repaired. In this paper, we solve both problems by changing the learning paradigm from supervised training to self-supervised training in an approach called SelfAPR. First, SelfAPR generates training samples on disk by perturbing a previous version of the program being repaired, enforcing the neural model to capture projectspecific knowledge. This is different from the previous work based on mined past commits. Second, SelfAPR executes all training samples and extracts and encodes test execution diagnostics into the input representation, steering the neural model to fix the kind of fault.
When those past works are compared one against the others on the same benchmark  [83] , those variations explain the performance differences. Despite those differences, the previous work on neural program repair is dominantly founded on the same machine learning paradigm: supervised learning  [26]. In that context, the supervised training samples come from mining real-world commits made by human developers. For example, Recoder's  [85]  training samples were downloaded from GitHub, totaling 1 083 185 commits done between March 2011 and March 2018. In this paper, we claim and provide evidence that this supervised paradigm for neural program repair has two fundamental limitations as follows. Problem 1: Lack of project-specific knowledge. Past commits used for training are typically collected from open-source projects.
To address the above two problems, we devise, implement and evaluate SelfAPR. The radical novelty of SelfAPR is to be based on self-supervised learning instead of supervised learning. SelfAPR consists of an original training loop on a past version of the project under repair. The core intuition is that the past version contains a wealth of useful project-specific knowledge. The self-supervised learning loop means generating training samples ourselves (instead of mining commits), by perturbing an old version of the project under repair in a careful and systematic manner. This is a potent solution to solve our two problems. Self-supervised training enables us to learn projectspecific knowledge. SelfAPR leans from a past version of the program to be repaired. From a single past version, SelfAPR generates thousands of synthetic training samples, which all contain project-specific tokens by construction.
Consequently, SelfAPR proposes a novel input representation that includes the execution information, called "diagnostic" for short in the rest of this paper. Put simply, SelfAPR's self-supervised paradigm augments the training samples from <bug, fix> to <bug, execution diagnostic, fix>. Overall, SelfAPR works as follows. It first generates thousands of training samples based on perturbation. Next, each perturbed program is executed to see whether it is buggy (does not compile or does not pass the test). For each buggy program, we extract the diagnostic and incorporate it into the input representation. Next, we train SelfAPR with a state-of-the-art transformed-based neural model  [70]  with this input, using the correct program before perturbation as the expected output. At inference time, we use the error of the failing test case of the program under repair as input diagnostic.
In other terms, this performance breakthrough is due to the paradigm shift from supervised neural networks to self-supervised neural networks. To sum up, we make the following contributions: • We devise, SelfAPR, an original self-supervised neural model for program repair based on execution. To our knowledge, this is the first learning architecture that encodes and leverages test execution diagnostics for repair. Notably, Self-APR is able to learn project-specific repair knowledge in an effective way.
BACKGROUND 2. 1 Automated Program Repair (APR) Search-based Repair. Search-based approaches such as GenProg  [36] , SPR  [40]  and others  [37, 43, 58, 73, 81, 84] , typically generate a large number of tentative patches according to different edit patterns, such as inserting null checks  [16] , mutating operators  [66]  or copying statement  [36]. In the patch search space, they then employ heuristic search algorithms (e. g. , genetic programming) to quickly find patches that can pass the given test specification. Search-based approaches suffer from both the immense scale of search space and the effectiveness of search algorithms  [55]. Semantics-based Repair. Semantics-based approaches, such as Angelix  [47] , S3  [35]  and others  [21, 45, 46, 50, 60, 61, 75, 76] , first construct a constraint problem that should be satisfied to fix a bug, and then they use some kind of program synthesis to synthesize patches that satisfy the repair constraints.
Semantics-based approaches effectively narrow down search spaces, yet they mostly limit the edit patterns to small-scale expressions in order to make program synthesis tractable  [73]. Learning-based Repair. Learning-based approaches based on supervised learning  [6, 10, 11, 13, 19, 31, 39, 42, 48, 68, 69, 83, 85]  are data-driven, employing pairs of the buggy and fixed code crawled from open-source projects and learning the edit patterns from a large scale training dataset. Learning-based approaches typically rank the patches based on the maximum likelihood estimation of tokens, which potentially narrows down the search space to likely patches. Nevertheless, learning-based approaches treat source code as natural language translation, suffering a lack of knowledge in programming languages.
Overview Figure  2  gives an overview of SelfAPR, where the upper part shows the training phase and the bottom part presents the inference phase. The core idea of SelfAPR is to generate training samples in a fully project-specific and self-supervised manner. In order to be capable of doing project-specific training, we take one project's past version to generate training samples. The learned model is used to repair future bugs from the same project. For self-supervised training, the training buggy samples are generated by perturbing correct programs given as input. This is contrary to the related work on neural program repair which is based on supervised learning with mined past commits  [13, 31, 42, 68, 69, 83, 85]. Notably, the perturbation-based programs can be compiled and executed with project-specific test suites while past commits do not.
Training Phase. SelfAPR's training phase consists of two main components: a training sample generator based on perturbing the source code (left part) and a neural transformer architecture fed with an input representation embedding test execution information (right part). Given a correct program P which can be compiled and executed (i. e. , pass test suite specification), SelfAPR generates variants of them, called here "perturbation-based programs" (denoted as P ′ ), according to a perturbation model (Section 3. 2). For one single program P, a number of P ′ can be generated in a configurable way, depending on the required size of the training dataset. All the generated P ′ are compiled and executed by invoking the test suite included in the correct version. Compilation and test execution determine whether the perturbation model introduces a bug, and yields the error type and the error diagnostic D.
Finally, SelfAPR's learning model takes input as P ′ and its accompanying diagnostic D. The goal of the machine learning model is to predict the expected output P, the original source code before perturbation. They are denoted as follows: 𝑡𝑟𝑎𝑖𝑛𝑖𝑛𝑔_𝑠𝑎𝑚𝑝𝑙𝑒𝑠 = {𝑖𝑛𝑝𝑢𝑡 :< P ′ , D > 𝑜𝑢𝑡𝑝𝑢𝑡 :< P >} Perturbation Model. The goal of the perturbation model is to generate training samples with bugs. Perturbation-based programs are valuable for training a neural repair network if they can be considered as buggy in some sense. To determine this, SelfAPR employs a compiler and the available test suite to identify their correctness. If a perturbation-based program P ′ produces either a compiler or a test execution error, it is deemed as buggy, and consequently, used as a training sample later. The perturbation model will be described in Section 3.
2. Diagnostic Extraction. Not only SelfAPR uses the compiler and test execution to select buggy training samples, it also uses them to extract diagnostics D about the bug. The D is then a first-class part of the neural network input. Our intuition is that error diagnostics could be useful to guide patch generation, for example, the fact that a NullPointerException is thrown provides explicit information regarding the repair action to be used. We note that this radically departs from the related work which only uses source code as input  [31, 68, 69, 83, 85].