dataesr/scientific-paragraphs-categorization huggingface dataset:
FT3D (FlyingThings3D)  (Mayer et al. , 2016 ) is a large optical flow data set, providing ground truth optical flow, RGB images, camera motion and depth. It is a synthetic data set showing random objects like chairs, tables, etc. , flying in a 3D world along random trajectories. The data set is split into test and training sets. Sintel  (Butler et al. , 2012; Wulff et al. , 2012)  is the de facto benchmark for optical flow algorithms, containing 23 video sequences with 20 to 50 frames each. These short video sequences are taken from an animated movie. The scenes are realistically simulated. Synthetic videos are available with ground truth optical flow, depth, camera motion and material segmentation.
MoCA: Optical flow only Data sets like MoCA focus in particular on the segmentation of objects that can only be robustly recognized based on their unique motion. Where as most data sets for moving object segmentation combine several cues (motion and appearance) that are helpful for recognizing moving objects, this data set highlights the relevance of motion. Thus MoCA allows to evaluate the strengths of motion models in isolation. It is not surprising that appearance cues are rather weak in cases of camouflage, therefore methods based on RGB frames only (e. g. COSNet  (Lu et al. , 2019b) ) show a weak performance in these settings (see Table  2 ). On more general data set like the DAVIS, those methods show a superior performance among all other methods and achieve a similar segmentation quality as MATNet  (Zhou et al.
, 2020)  (Table  3 ). Our approach taking a single optical flow frame (compensated for camera rotation) as input, performs comparable to other supervised approaches. A simple post-processing step -convolution with a 3D Gaussian filter and frame-wise application of a dense CRF, eliminates temporal instabilities (Ours+Temp in Table  2 ). Among all methods SegI  (Lamdouar et al. , 2021)  shows best results on MoCA, on DAVIS their performance falls rather short due to their lack of a strong appearance model. SegI combines multiple ConvNets where each of them encode a flow frame together with a transformer network without taking RGB frames into consideration. The model is trained on synthetically generated data, thus can be considered as unsupervised. In contrast, our approach was trained using rotation compensated flow frames estimated from the synthetic dataset FlyingThings3D.
Rotation estimation via likelihood maximization We show results on the Sintel data set (Table  6 ), and compare our new likelihood optimization procedure with  (Bideau and Learned-Miller, 2016b). The ground truth optical flow and focal length is provided, so an accurate estimate of the camera's rotation is possible. Our camera rotation estimation based on maximizing the flow likelihood shows consistently better results on the Sintel data set. More importantly, the performance gap gets significant when using estimated flow as input for camera motion estimation. Since our proposed optimization approach incorporates an explicit noise model, it is significantly more robust to noisy flow data. Table  6  Ablation study: Camera rotation estimation. Avg. yaw/pitch/roll error in degrees between two consecutive frames.
MDTOMO allows obtaining an atomic-scale model of conformational variability and the corresponding free-energy landscape, from a given set of cryo-ET subtomograms. The article presents the performance of MDTOMO on a synthetic ABC exporter dataset and an in situ SARS-CoV-2 spike dataset. MDTOMO allows analyzing dynamic properties of molecular complexes to understand their biological functions, which could also be useful for structure-based drug discovery.
STA can be combined with discrete 3D classification to tackle the conformational heterogeneity of the complexes  [3] [4] [5] [6]. The goal of the classification is to make the groups (classes) of subtomograms that are as homogeneous as possible regarding the conformation of the complex. To obtain a subtomogram average of good quality from a class, the class should be highly populated. However, considering a relatively small number of available subtomograms, discrete classification methods are usually used with a small number of classes, which makes them ill-suited in the context of highly heterogeneous datasets composed of a continuum of conformational states. This heterogeneity issue, known as continuous conformational variability, is well-known in single particle (SPA) cryo electron microscopy (cryo-EM) and is currently an active field of research  [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] 17.
The number of methods for analyzing continuous conformational variability in SPA datasets has been growing over the last few years, but only a few such methods have been proposed for cryo-ET so far  18, 19. This is likely due to the challenges of analyzing cryo-ET data that were mentioned earlier, namely low SNR, missing-wedge-induced deformations, and small number of subtomograms. Also, volumetric data processing induces additional computational complexity. Recently, a method named Tomoflow has been proposed for analyzing continuous conformational variability in cryo-ET subtomograms, which uses a computer vision technique of dense 3D optical flow to estimate the displacement of voxels in each subtomogram with respect to the voxels in a global subtomogram average  18. This subtomogram average (reference volume) is calculated at each iteration from all subtomograms that were elastically and rigid-body aligned with respect to the subtomogram average obtained in the previous iteration  18.
In this article, we present MDTOMO, a novel approach for analyzing continuous conformational variability in subtomograms, which is based on extracting atomic models from subtomograms by flexible fitting of a reference atomic structure using a combination of normal modes and MD We tested MDTOMO on a synthetic dataset of an ABC exporter  21  and in situ dataset of a SARS-CoV-2 spike protein (S)  22. The synthetic dataset provides a controlled environment where the results of the method can be compared with the ground-truth data, which allows assessing the method performance and robustness to noise and missing-wedge-induced deformations. Although MDTOMO does not currently correct for the missing wedge, it was able to produce meaningful results even for high amounts of noise. MDTOMO applied to in situ data indicates its ability to decipher the conformational landscape as a continuum of intermediate states.
Coarse-grained simulations To further reduce the computational cost of the analysis, we used a coarse-grained model based on off-lattice Cα Gō model  26  , which drastically reduces the computational time compared to all-atom simulations. The Gō-like models are a class of models that share similar concepts as the model proposed by Gō (e. g. , defining the long-range interactions based on the native contacts of the initial model, which tends to bias the simulation towards the reference). Cα Gō model represents the protein with Cα atoms of the amino acid chain and captures better the native dynamics than the original Gō model  27. Here, the coarse-grained models were obtained by the SMOG2 software  28.
Low-dimensional conformational space and free-energy calculation As MDTOMO results in one atomic model per subtomogram, a further analysis must be performed to visualize the conformational landscape. In this context, the set of atomic models obtained by MDTOMO can be projected onto a low-dimensional (e. g. , 2D or 3D) space. In this study, two methods were used to perform the dimension reduction of the atomic coordinates, namely Principal Component Analysis (PCA) and Uniform Manifold Approximation and Projection (UMAP)  29. PCA is the well-established technique for dimension reduction, offering a fast, deterministic and linear reduction method, which is useful to describe the overall variability in most datasets. UMAP is a recent method that allows capturing the non-linear motions, which can, in some cases, achieve a better separation of the different populations compared to PCA.
Rigid-body pre-alignment of subtomograms MDTOMO refines the rigid-body pre-alignment of the subtomograms with the initial model. The prealignment prevents the simulation to get trapped into local minima. For the synthetic ABC exporter experiments, the pre-alignment was obtained by STA based on the fast rotational matching algorithm (FRM)  30  implemented in Scipion  31. Concerning the in situ dataset of SARS-CoV-2 spikes, we used the STA alignment that had been obtained in the original work  22. The rigid-body pre-alignment is refined in MDTOMO by estimating the rigid-body rearrangements that occurred during the MD simulation, and using them to correct the initial alignment. The estimation of the rigid-body alignment parameters is obtained by optimizing the rotation and the translations that minimize the root mean square deviation (RMSD) between the initial conformation and the fitted conformation, using an optimization algorithm based on singular value decomposition that is available in BioPython  32.
Software implementation MDTOMO is implemented as part of ContinuousFlex 33 , a software package for continuous conformational heterogeneity analysis in SPA and cryo-ET data, which is also available as a plugin of Scipion  31. The Scipion back-end allows a user-friendly graphical interface for MDTOMO and integrating the data analysis results from most of the recent cryo-ET methods through the new ScipionTomo suite  34. The MD and NMMD simulation back-ends are handled by GENESIS  35.
Synthetic dataset of ABC exporter To test the performance of MDTOMO, we synthesized a dataset as close as possible to experimental cryo-ET conditions. The dataset simulates a highly heterogeneous cryo-ET dataset of a heterodimeric ABC exporter, TmrAB, along the substrate translocation cycle. During the translocation cycle, the ABC exporter changes its conformation from outward-facing to inward-facing, allowing the substrate to enter the intracellular cavity. ATP-binding first induces closing of the intracellular gate (an occluded conformation of the exporter, where both intracellular and extracellular gates are closed) and, then, opening of the extracellular gate (an outward-facing conformation, where the extracellular gate opens while the intracellular gate remains closed), which allows a release of the substrate in the extracellular environment.
The structure of TmrAB in inward-facing, outward-facing, and occluded conformations were derived from cryo-EM maps  21  and are available in the Protein Data Bank (PDB) under the accession codes 6RAF, 6RAH, and 6RAK, respectively. Figure  1a  shows the PDB: 6RAK structure of TmrAB in the occluded conformation while Figure  1b  shows a sketch of the translocation cycle, with all three mentioned conformational states of TmrAB during the cycle. The conformational heterogeneity was generated by standard MD-based flexible fitting (without normal mode acceleration) to obtain a dataset with realistic conformational variability, composed of a number of more populated conformational states (representing more stable states), close to the three mentioned states of TmrAB during the translocation cycle, and a number of less populated, intermediate states between these more stable states.
Nevertheless, the dataset consisting of a continuum of conformational states visited during this MD simulation is rich and representative of the translocation cycle enough for performing the tests of the method that is proposed here. To simulate the sub-tomograms, we followed a previously published method  18, 19. We started by collecting 3000 atomic models from the simulated trajectory and we converted each model to a volume of size 100 3 voxels and a voxel size of (2 Å) 3 using a method based on scattering factors  36. The obtained volumes were low-pass filtered at a cutoff frequency of 1/(6 Å) to take into account the impact of various imperfections (e. g. , average radiation damage per image and tilt-series alignment errors). Then, the volumes were rotated and shifted randomly, and projected with a tilt angle from -60° to +60° using an angular step of 2°.
Recovery of the ground-truth conformational space We applied MDTOMO to each of the synthetic datasets (one for each SNR value) composed of 3000 subtomograms using 100-ps MD simulations starting from the PDB:6RAK model (occluded conformation). Figure  1c  shows the free-energy landscape along the first two PCA components for the synthetic MD trajectory (ground truth) and the free-energy landscapes obtained with MDTOMO for each of the three datasets (subtomograms obtained from the images with the SNR of 0. 05, 0. 03, and 0. 01). We observe that MDTOMO was able to recover the conformational landscape very accurately for the datasets with the SNR of 0. 05 and 0. 03. Indeed, Figure  1c  shows that the most dense regions of the conformational space (i. e. , the regions of the lowest free energy) are those that are the closest to the three given PDB structures (6RAF, 6RAH, and 6RAK) that were used to target the MD simulations during the data synthesis.
Recovery of the rigid-body alignments The final rigid-body alignment, obtained by MDTOMO, was compared with the ground truth rigidbody displacement (the rotations and shifts used to synthesize the data). The initial rigid-body alignment, done by STA, was also compared with the ground truth rigid-body displacement. The initial and final alignment errors are compared in Figure  1d  and Figure  1e , which show respectively the rotational and translational errors for the three datasets (three SNR values). We observe that, for both rotational and translational parameters and each value of the SNR, MDTOMO allows a refinement of the rigid-body alignment (the final errors are smaller than the initial errors).
Experiment with a dataset of SARS-CoV-2 spike protein in situ In situ dataset of SARS-CoV-2 spike protein The SARS-CoV-2 spike (S) protein is a trimer of three identical glycoproteins (Figure  2a ). It is involved in the cell infection as it mediates the viral entry by binding to the angiotensin-converting enzyme 2 (ACE2) receptor on the cell surface  39, 40. The S-protein head includes three receptor binding domains (RBDs) that are located at the top of the spike and exists in different conformations on the viral capsid. We used MDTOMO to analyze a set of experimental subtomograms of the S protein in situ that had previously been obtained and analyzed using STA and 3D classification (the corresponding tilt-series data are available in the EMPIAR database; accession code: EMPIAR-10453)  22.
Discussion In this article, we presented MDTOMO, a novel approach to explore continuous conformational then projected onto a low-dimensional space, which can be considered as an essential conformational space. In this space, closer points represent more similar conformations, and denser regions of points represent the regions containing more likely or more stable (low-energy) conformations in the given dataset. The density of points in this space can be interpreted in terms of the free energy of the complex or by computing and analyzing the averages of the subtomograms and their associated atomic models in the regions of the space characterized by the lowest free energy of the complex. In this article, we showed that MDTOMO produces expected results using synthetic data (ABC exporter). Also, we showed that the results produced using experimental SARS-CoV-2 spike protein data are coherent with previously published results regarding the existence of conformations with all three RBDs closed and those with a single RBD open  22.
1 and Eq. 4 in Vuillemot et al (2023)  13  ). The weight given to the potential defined by the correlation coefficient, with respect to the classical MD potential, is a free parameter (the so-called force constant; k in Eq. 1 in Vuillemot et al (2023)  13 )  whose value is set so as to drive the fitting toward the conformation in the subtomogram by preserving structural constraints, which reduces the risk of overfitting the noise and the missing-wedge-induced deformations in the subtomograms. Compared to the majority of other methods for analyzing conformational flexibility in single particle cryo-EM datasets (e. g. , 3DVA 10 in cryoSPARC  43  or cryoDRGN  11  ) or cryo-ET datasets (e. g. , Tomoflow  18  ), MDTOMO uses an available atomic structure, whose coordinates are displaced (by performing NMMD simulation) to match the conformations in the subtomograms.
Such atomic structure is not available for all datasets, but those datasets for which an atomic structure is available can efficiently be analyzed using MDTOMO. Indeed, MDTOMO incorporates structural constraints into the analysis via MD simulation, which is particularly important for analyzing cryo-ET subtomograms that suffer from noise and missing-wedge-induced deformations, as these structural constraints limit the motions to those that are feasible (underlying biochemical processes of the complex) and, therefore, they reduce the risk of overfitting mentioned previously. Also, the use of an available atomic structure in MDTOMO allows obtaining a conformational landscape in which the conformational transitions can be visualized directly on the atomic structure, contrary to the majority of other methods that visualize the conformational transitions on density volumes.
Another method that can use an available atomic structure to analyze conformational flexibility in cryo-ET datasets is HEMNMA-3D  19. More precisely, HEMNMA-3D can use either an available atomic structure or an available density volume (e. g. , subtomogram average or cryo-EM map)  19  , which is flexibly fitted into subtomograms by solely using normal mode analysis. Normal modes are useful for their simplicity and computational efficiency. However, they tend to induce distortions of the structure in case of large amplitudes of conformational changes. Also, they restrict the conformational exploration to a set of collective motions that must be defined in advance and that may be insufficient to describe the full range of conformational changes. MDTOMO overcomes these limitations by employing coarse-grained MD simulations to allow a full exploration of the conformational landscape without inducing structural distortions, and by empowering these MD simulations using normal modes to make simulations fast.
The MDTOMO approach requires running one MD simulation per subtomogram, which can be computationally expensive. The MDTOMO software is parallelized so that multiple simulations can run on multiple CPU cores simultaneously, with one simulation running only on one CPU core. Furthermore, the use of coarse-grained simulation coupled with normal-mode-based acceleration allows performing MDTOMO in a reasonable amount of time. The total execution time on 8 nodes of 2x20 Intel 6248 2. 50 GHz for the two presented datasets is shown in Table  1. However, it is important to note that the computational cost scales with the molecular weight of the complex (the higher the number of atoms, the higher the cost of the MD simulation).
Acknowledgments We acknowledge the support of the  French National Research Agency -ANR  ( ANR-19-CE11-0008  to SJ and IR and  ANR-20-CE11-0020-03  to SJ); the cooperation between the  CNRS  and the  University of Melbourne  (The Melbourne-CNRS Network,  PRC 2889  to SJ and IR,  CNRS 80 Prime  to SJ, joint PhD scholarship to RV);  The University of Melbourne  start-up fund (to IR); and access to HPC resources of CINES and IDRIS granted by  GENCI  ( A0100710998R ,  A0100710998 ,  A0070710998 ,  AP010712190 ,  AD011012188  to SJ). We thank  Beata Turoňová  ( Max Planck Institute of Biophysics, Frankfurt am Main, Germany ) for providing the subtomograms and the initial rigid-body alignment parameters that were used in the SARS-CoV-2 S protein experiment described in this article (from the dataset available in the EMPIAR database under the code EMPIAR-10453) as well as for critical reading of our manuscript and discussing the results that are presented here.
simulation. MDTOMO analyzes each individual subtomogram by performing NMMD-based flexible fitting to extract the atomic model that matches the conformational state in the subtomogram. MDTOMO allows interpreting datasets in the form of atomic conformational landscapes and their further analysis in terms of free-energy landscapes. Moreover, MDTOMO allows obtaining subtomogram averages from localized regions in the energy landscape. While other MD-based flexible fitting methods are used for fitting cryo-EM maps or SPA images, this is the first method that uses MDbased flexible fitting in the context of analyzing subtomograms.
landscapes of biomolecular complexes by analyzing subtomogram datasets. MDTOMO uses efficient, and normal-mode-empowered MD simulations to extract the conformational states from subtomograms, which are typically averaged out during classical STA, and allows reconstructing low-dimensional (usually 2D or 3D) energy profiles of the structures from the experimental subtomogram data. MDTOMO is the first method for analyzing subtomograms using MD-based flexible fitting. MDTOMO starts by extracting an atomic model from each individual subtomogram, by NMMD flexible fitting of a given reference atomic model into the subtomogram. The extracted atomic models are.
simulation MDTOMO on Intel 6248 2. 50 GHz one CPU core -one particle 320 CPU cores -full dataset ABC exporter 150. 85 kDa 372. 3 seconds 0. 9 hours (3000 particles) SARS-CoV2 spike 438. 26 kDa 987. 5 seconds 17. 8 hours (20830 particles).
Standard scores are either calculated for the whole recording or inside each time window. Then, inside each window, the baseline median absolute deviation (1) F F 0 = signal -fitted isosbestic fitted isosbestic. (2) Z = signal -mean signal SD signal - isosbestic -mean isosbestic SD isosbestic. Table 1. Example data format that behavioral data must adhere to in order to be integrated into Pyfiber. A complete example of the necessary structure, based on the data from the bsaRat12_28032022c20_01. dat file, can be found in the git repository. Dim1_on , Dim1_off , Dim2 timestamp#1 , timestamp#1 , timestamp#1 timestamp#2 , timestamp#2 , timestamp#2 … , … , … (MAD, or bMAD) is calculated, and a first threshold is set (most commonly 2 to 2. 5 times the MAD, default of Pyfiber = 2. 5). Signal data below this first threshold is removed, and a new threshold is based on the MAD of the remaining data (pMAD, most commonly equal to 3.
Perievent analysis A major aim of the combination of operant behavior and fiber photometry is to determine whether the neural activity before or after a particular event (for example a light switch or a nose poke triggering a cocaine injection) is significantly different than at baseline, as well as comparing between subjects. In order to evaluate this, additional data processing surrounding events (perievent analysis) is performed. Z-scores. To do so, Z-scores (Eq. 3) are calculated by taking the time preceding the event as the baseline. Depending on which method was initially used for signal processing, perievent Z-scores can be generated from either the ΔF/F or Z-score. Robust Z-scores. For dispersed data, outliers can have a detrimental effect on both the mean and the standard deviation.
Summary of Pyfiber Pyfiber was created to fill the gap in flexible, open source analytical tools merging operant behavior and fiber photometry (Fig. 2 ). Pyfiber requires inputs of behavioral files (Imetronic or formatted data from other behavioral systems) and fiber photometry files (Doric Lenses,. csv or. doric). Pyfiber dissects and stores the data within the behavioral file by identifying and categorizing events and intervals. Additionally, Pyfiber dissects and stores the data in the photometry file by processing the photometry signals using previously established and reputable photometry analysis methods. Then, it uses the aforementioned stored data (both behavior and fiber photometry) to fuse the stored data, resulting in the output of processed fiber photometry data surrounding the events of interest that were extracted from the behavioral file.