dataesr/scientific-paragraphs-categorization huggingface dataset:
The diagrams were created using the matplotlib python library. 74 For EGSCyP, the energy landscape was projected as a function of the dihedral angles φ and ψ for each residue (on each line). The color code corresponds to the minimal potential energy in kcal/mol of the whole peptide associated to each combination of φψ angles. Energy minima are represented in black/red, while areas with no conformation are in white, and areas of high energy are in blue. For REMD, the normalized frequency (nf req i = 100 -100 max f req -f req i max f req ) of the conformations found during the simulations at the lowest temperature replica is projected as a function of φ and ψ. The color code corresponds to this normalized frequency, with the maximal frequency in black and a null frequency in white. The pink stars represent the values of the dihedral angles for the crystallized Cilengitide (sequence: RGDfV', PDB ID: 1L5G) in complex with integrin.
(c) R I G II D III f IV V' V R I G II D III f IV V' V R I G II D III f IV V' V R I G II Comparison with crystallographic structure: Results provided by the two conformational exploration methods were compared to the X-Ray structure of Cilengitide bound to integrin (PDB code: 1L5G). The dihedral angles values of this X-Ray structure were projected on all the RGDfV' Ramachandran diagrams. They are indicated with a pink star in the third and forth columns of Figure  3. We can observe that this experimental structure is close to energy minima (in black). The RMSD on dihedral angles was computed between this structure and the clustered minima structures from the two methods (see section above). (1) EGSCyP explores a larger landscape than REMD, due the algorithm itself and as measured by the percentage of coverage, so that it probably finds one conformation closer to the crystallographic one with a lower energy.
The diagrams were created using the matplotlib python library. 74 For EGSCyP, the color codes correspond to the minimal potential energy of the whole peptide associated to this combination of χ 1 -χ 2. For the REMD simulations, the color code corresponds to the normalized frequency (nf req i = 100 -100 max f req -f req i max f req ) of the conformations found during the simulations at the lowest temperature replica as a function of χ 1 and χ 2. high frequency of the REMD simulations. On the contrary, the BH/MC sampling shows a good correspondence between its energy minima basins and the frequency maxima from the REMD simulations for the three residues of Figure  6. The percentage coverage for the REMD side chain placement is rather constant among the different residues (around 58%) and the part of the coverage in common between the BH/MC and REMD is about 50% of the diagrams.
Moreover, we have demonstrated the effectiveness of the alternation of BH/local minimization for the sampling of the side chains conformations, whereas SCWRL4 fails to correctly position the side chains of small cyclic peptides. Finally, this approach clearly shows the effect on the conformational landscape of the D-residue that constrains the landscape and the N-methylation that also modulates it. In the short future, we will apply EGSCyP to a larger dataset, including all the available short head to tail cyclic peptide experimental structures, in order to more robustly evaluate the efficiency and accuracy of our approach. Other force fields could be used within our approach. Indeed, the accuracy of the energy computation for EGSCyP as well as REMD simulations depend on the force field used.
Figure6: Comparison of the side chain conformational landscapes obtained for the RGDFV cyclic peptide with EGSCyP using SCWRL4 50 (left column), with EGSCyP and the alternation of BH and local minimization (middle column), and with the REMD simulations (right column). The conformational landscape was projected as a function of the χ 1 -χ 2 dihedral angles of the side chains for ARG (first line), ASP (second line) and PHE (last line). The diagrams were created using the matplotlib python library. 74 For EGSCyP, the color codes correspond to the minimal potential energy of the whole peptide associated to this combination of χ 1 -χ 2. For the REMD simulations, the color code corresponds to the normalized frequency (nf req i = 100 -100.
Acknowledgement This work was performed using HPC resources from  GENCI-CINES  (Grant  2016-c2016077641 ). CALMIP is gratefully acknowledged for the access to the super-computer EOS (allocation  2016-P16032 ) for developing the EGSCyP method. We also thank  Dr Michele Lazzeri  for letting us using his cluster, and  Dr Guillaume Postic  for valuable discussions and advices. Finally, we express our sincere thanks to  Pr Horst Kessler  for giving us NMR structure of Cilengitide.
TempWeb 2018 Chairs' Welcome Time is a key dimension to understand the Web. It is fair to say that it has not received yet all the attention it deserves and TempWeb is an attempt to help remedy this situation by putting time as the center of its reflection. Studying time in this context actually covers a large spectrum, from the extraction of temporal information and knowledge, to diachronic studies for the design of infrastructural and experimental settings enabling a proper observation of this dimension. TempWeb focuses on investigating infrastructures, scalable methods, and innovative software for aggregating, querying, and analyzing heterogeneous data at Internet scale. Particular emphasis will be given to temporal Web data analysis that has been collected over extended time periods. A major challenge in this regard is the sheer size of the data it exposes and the ability to make sense of it in a useful and meaningful manner for its users.
It is worth noting that this trend of using big data to make inferences is not specific to Web content analytics. Web scale data analytics therefore needs to develop infrastructures and extended analytical tools to make sense of these. For its eight edition, TempWeb accepted seven very positively reviewed submissions for oral presentation. We interpret the high quality of the submissions and the frequent contributors to TempWeb, as indicators of an evolving community. It shows a clear sign of a positive dynamic in the study of time in the scope of the Web and evidence of the relevance of this effort. The workshop proceedings are published by ACM DL as part of the WWW 2018 Companion Publication. We hope you will find in these papers as well as the keynote of Pierre Senellart (École Normale Supérieure Paris, France) on "Timely crawling of the structured Web", and the discussion and exchanges of this edition of TempWeb, some motivations to look more into this important aspect of the temporal Web.
Cloning and in silico analyses Total RNAs were extracted from 2-week old H. cylindrosporum axenic cultures grown in N6 medium by tissue grinding in liquid nitrogen, followed by a TRIzol® Reagent purification (Invitrogen, Carlsbad, CA, USA). RNAs were treated with DNAse I (Invitrogen), checked for integrity by electrophoresis in 1 % agarose gel, and retrotranscribed to cDNA using the Superscript III enzyme (Invitrogen). Primers with flanking SpeI restriction sites were designed to amplify the sequences coding for HcTOK1 (Protein ID 31571), HcTOK2. 1 (Protein ID 129509) and HcTOK2. 2 (Protein ID 127201) (H. cylindrosporum h7 genome database v2. 0; http://genome. jgi-psf. org/Hebcy2/Hebcy2. home. html; Table  S1 ). A poly-A tail was added to the product of the previous reaction and the construct was inserted into a TOPO-TA cloning vector (Invitrogen).
The HcTOK coding sequences were amplified by PCR with primers with flanking HindIII (for HcTOK1) or EcoRI (for HcTOK2. 1 and HcTOK2. 2) restriction sites and cloned in the Zero Blunt® vector (Invitrogen) (Table  S1 ). For the generation of HcTOK1, HcTOK2. 1 and HcTOK2. 2 overexpressing (OE) fungal lines, three plasmids named pPZP-OE-TOK1 or -2. 1 or -2. 2 were constructed by digestion of the TOPO-TA-TOK1, -2. 1 or -2. 2 vectors using SpeI and ligation into the pPZP-Pgpd vector  (Ngari et al. , 2009; Garcia et al. , 2013). This article is protected by copyright. All rights reserved. In silico analysis of the secondary structure was performed with the online TMHMM software  (Krogh et al. , 2001)  and the predicted structures were represented with the Protter website tool  (Omasits et al. , 2014).
In situ hybridization Two PCR reactions on cDNA were performed to obtain 350 bp sense and antisense probes for the in situ detection of HcTOK1, -2. 1 and -2. 2 transcripts, as described by Garcia et al. (2014), using specific sets of primers (Table  S2 ). PCR products were purified by ethanol precipitation and in vitro transcription was made using the MAXIscript® T7 Transcription Kit (Invitrogen) to synthetize digoxigenin-UTP-labelled RNA probes. The hybridization and detection of the probes were performed on 8 µm-thick sections of mycorrhizal roots fixed in paraffin as described by  Garcia et al. (2014). Images were taken on a wide-field microscope Leica DM6000 (Leica Microsystems, Wetzler GmbH, Germany) and analyzed with the Volocity Acquisition 5. 1. 0 software (Perkin Elmer, Boston, MA, USA) and FiJi  (Schindelin et al.
5 (the HEPES buffer present in the other bath solutions being then substituted by MES buffer), 6. 5 and 7. 5. The TEVC protocol started at a holding potential of -60 mV and then imposed voltage pulses to the oocyte membrane from hyperpolarizing potentials (-100 mV or -60 mV) to depolarizing potentials (+80 mV) in steps of +10 mV. Voltage-clamp protocols, measurements and data analyses were performed using the pClamp10 software (Axon Instruments, Foster City, CA, USA).
Acknowledgments We thank the  French Ministry of Higher Education and Research  for financing the fellowships of C. G. G. , K. G. and G. H. , and the  ANR Project  ' TRANSMUT '  2010 BLAN 1604 03  for the financing of the contract of A. D. K. G. was also supported by the  U. S. Department of Agriculture  ( USDA NIFA 2017-67014-26530 ). We also thank  Baptiste Monterroso  for technical support in microscopy experiments,  Sylvain Kumansky  for participating in this project during an internship and  Cécile Fizames  for advice in phylogenetic analyses. Kind gift of K + -transport deficient yeast mutants by  Adam Bertl  is thanked. We are grateful to  F. Martin  and the  Mycorrhizal Genomics Initiative consortium  for access to unpublished genome data. The genome sequence data were produced by the  US Department of Energy Joint Genome Institute  in collaboration with the user community.
Results are visualized and some informal qualitative evaluations are discussed. The autoencoder was able to reproduce the steady state of the synthesizer with and without regularization, although reproduction error increased, expectedly, in the presence of regularization. Some parameter estimation problems were identified with the dataset and sampling method used, and we conclude with some lessons learned in the art of "synth cloning". A real-time system, Sounderfeit, built on a generative neural network is presented, allowing to construct engaging sound synthesizers based purely on recorded data and optional prior knowledge of parameters.
A parameter space quantized (also according to a perceptual model) was used. Similarly, Gabrielli (2017) used a multilayered convolutional neural network (deep CNN) to determine parameters of an organ physical model that supports up to 58 parameters (organ stops) per key. Short-time Fourier spectra were used as input calculated from a dataset of 2220 samples, and the network was trained to minimize the mean squared error on the parameter reconstruction. A notion of spectral irregularity was used to judge the similarity of resulting synthesized sounds. Pfalz and Berdahl (2017)  explored the use of a long short-term memory recursive neural network (LSTM-RNN) to estimate the continuous control signals from the output of a physical model. Mean squared error of the reconstructed parameters to the original parameters is used as the loss.
, 2016) , which used multi-scaled deep RNNs to capture long-term dependencies as a stacked autoregressive model, i. e. , it encodes the conditional probability distribution of the next sample based on previous samples and encodings produced by other layers. The multiple scales allow this sample-at-a-time model to also take into account frame-level information, and can therefore use higher levels to encode longer-term dependencies. It was compared to WaveNet and a standard RNN with Gaussian mixture model in terms of reconstruction mean squared error, and with human listening preference experiments on encodings of voice, human non-vocal sounds, and piano music, and performed favourably. Interestingly, rather than purely real-valued output, all of the above-mentioned sample-at-a-time methods made use of a quantized one-hot categorical softmax over an 8-bit µ-law encoding for estimating the real value of the audio signal.
The intuition is that such an encoding allows to remove any prior assumptions about distribution and simply take the most probable discrete value. As for frame-at-a-time methods, the baseline model from  Engel (2017)  is applicable, as it consists of a deep CNN trained on spectrograms. As mentioned, they used a large instrument dataset and a large latent space of approximately 2000 dimensions to encode both the time and frequency domains. Indeed, this model can be thought of more as a spectrogram-at-a-time rather than frame-at-a-time, since the encoding takes multiple frames into account; the latent dimensions per frame were on the order of 16 and 32. The authors reported poor performance for encoding phase or complex representations, and thus used only spectral magnitude as input, and applied a phase reconstruction technique to synthesize the final audio.
This is of course qualitatively open to interpretation, but does imply some kind of structure imposed on the latent space that appears to differ significantly from the use of non-learned descriptors. In this work, we found similar patterns in unregularized latent spaces, e. g. Figures  6a  and 8a. A distinguishing factor of this work compared to NSynth is that rather than attempt to model a large set of instruments, which requires a large model, large dataset, and large-dimensional latent space (16 or 32) with unknown meaning, we focus on representing the sound with a comparatively small set of parameters (2 to 3) and attempt to learn a minimal encoding based on previous knowledge of the model parameters, adding latent dimensions only as necessary. This stems from a different motivation, which, instead of being to determine multi-instrument embedding spaces as in the case of NSynth, is to better understand the inverse data-parameter relationship, as well as to provide a small, salient set of "knobs" for real-time synthesis of a single family of timbres.
Datasets Given a network with sufficient capacity we can encode any functional relationship, but for the experiments described herein a periodic signal specified by a small number of parameters was sought that nonetheless features some complexity and is related to sound synthesis. Thus, a physical modeling synthesizer proved a good choice. We used the bowed string model from the STK Synthesis Toolkit in C++ (COOK; SCAVONE, 1999), which uses digital waveguide synthesis and is controlled by 4 parameters: bow pressure, the force of the bow on the string; bow velocity, the velocity of the bow across the string; bow position, the distance of the string-bow intersection from the bridge; and frequency, which controls the length of the delay lines and filter parameters, and thus the tuning of the instrument.
The parameters are represented in STK as values from 0 to 128, and thus we do not worry about physical units in this paper; all parameters were linearly scaled to a range [-1, 1] for input to the neural network. The data was similarly scaled for input, and a linear descaling of the output is performed for the diagrams in this paper. Additionally, the per-element mean and standard deviations across the entire dataset were subtracted and divided respectively in order to ensure similar variance for each discrete step of the waveform period. To extract the data, a program was written to evaluate the bowed string model at 48000 Hz for 1 second for each combination of bow position and bow pressure for integers 0 to 128. The 1-second interval was used to ensure the sound reached a steady state with a constant period size.
The recordings were phase-aligned using a cross-correlation analysis with a representative random sample, then differentiated by first-order difference, and 200 sample-to-sample differences were thus used as the training data, normalized as stated above. This dataset we refer to as bowed1. Although it may be beneficial to use a log-spectrum representation rather than "raw" (pulse-coded) audio (ENGEL, 2017), we found that learning the time-domain oscillation cycle was no problem. In this manner we avoided the need to perform phase reconstruction. The use of a differentiated representation also helped to suppress noise. As will be discussed below, parameter estimation on new data was not successful based on this dataset due to the lack of representation of the synthesizer's dynamic regimes. To resolve this, a second extended dataset, bowed2, was created in a similar manner, however instead of recording only the steady state portion, the synthesizer was executed continuously while changing the parameters randomly at random intervals.
100,000 samples uniformly covering the parameter range were captured for bowed2. Finally, in order to test the idea on a completely independent albeit simple dataset, a human voice was recorded uttering constant vowel sounds. The voice (the author's own voice) was held steady in frequency for a period of 3 seconds for vowels a, e, i, o, and u. The beginning and end of each utterance was clipped and periods were extracted and globally phase-aligned by aligning peaks. The voice was recorded at 44100 Hz and had a frequency between 114 and 117 Hz, thus slightly long cycles were extracted to have exactly 400. 5 samples per period, so that two periods were 801 samples, or 800 samples in the differential representation used here. This created a final fundamental frequency of 110 Hz in the synthesized sound.
(Due to overlap-add, artifacts in 2 or 3 samples at the beginning and end of a cycle are mostly surpressed. ) Integers 0 through 4 were assigned to each vowel and used as the single conditional parameter. This resulted in 996 two-period samples, or approximately 200 samples per vowel. As will be shown, since the voice was held quite steady, most periods for the same vowel were quite similar, however a low-quality microphone and natural vocal variation contributed to differences between samples. This dataset is referred to in this text as vowels. In all cases, reproduction consists of de-normalizing, concatenating using an overlapadd method, and first-order integrating the final signal. A 50% overlap-add with a Hanning window was used, which features a constant overlap summation thereby avoiding modula-tion artifacts (SMITH;  SERRA, 1987).
The principal dataset, described below, was composed of 200-wide 1-D vectors, and we had acceptable results using hidden layers of half that size, so w 1 ∈ R 200×100 , w 2 ∈ R 100×(n+m) and w 3 , w 5 ∈ R (n+m)×100 , w 4 , w 6 ∈ R 100×1 , where (n + m), the total size of the hidden code, was 2 or 3, depending on the experiment. The bias vectors b 1. b 6 had corresponding sizes accordingly.
Experiments Six conditions were tested in order to explore the role of conditional and latent parameters. The number of known parameters in the dataset was 2. We tried training the bowed1 dataset with and without an extra latent parameter. We label these conditions D1 Z 2 Y and D0 Z 2 Y respectively. The third condition, N 1 Z 2 Y , was like the D1 Z 2 Y condition but without adversarial regularization on q(z|x). Thus the D label is to indicate the use of training on the discriminator, while N indicates No discriminator. To compare conditioning with the "natural" distribution of the data among latent parameters and the effects of adversarial regularization thereupon, two configurations with no conditional parameters, with and without the discriminator, were explored, named D2 Z 0 Y and N 2 Z 0 Y respectively.
Results Figure  2  demonstrates the results of D1 Z 2 Y. Comparing the middle and bottom curves, we can see that while it has some trouble with low values of bow pressure and the extremes of bow position, the autoencoder is able to more or less encode the distribution in our dataset. The top curve (red) was generated by explicitly specifying the y (conditional) parameters instead of letting the autoencoder infer them, with z 0 = 0, and demonstrates the output for parameter-driven reconstruction if z 0 is held constant. Although not a perfect reproduction, particularly at extremes of the bow position range where there is more variance, this demonstrates that the trained network is able to approximate the data-parameter relationship present in the dataset. The role of z is now considered in Figure  3 , by holding bow position constant (y 1 = 100) and examining how the signal changes with z 0.
(2) underrepresented variance in the dataset; the new testing data varies continuously in the parameters, but the dataset was constructed based on the per-parameter steady state. To investigate this, the network was trained on a "half dataset", consisting only of samples of bowed1 where bow position < 64. Furthermore, as mentioned, an extended dataset, bowed2, was constructed based on random parameter variations. Results in Figure  4(b)-(d)  show that training on the half-bowed1 dataset changed the character of errors, but did not improve overall, however the extended bowed2 dataset gave improved parameter estimation, and much improved in the bow position < 64 case. Thus it can be concluded that both sources contributed to parameter estimation difficulties. Figure  6  shows the resulting parameter space if both parameters are left to be absorbed by the unsupervised latent space.
The adversarial regularization regime can be seen in the generator and discriminator losses of Fig. 6(b) , which encourages the autoencoder to make the distribution of these variables similar to U(-1, 1), i. e. , a rectangle. This facilitates user interaction with the generator, since limited-range control knobs can be mapped to this rectangle, thus having a strong chance to access the full range of variance present in the dataset; conversely, the chance of synthesizing a sound that does not correspond with the training data is minimized. Without regularization, Fig. 6(a)  (N 2 Z 0 Y ), we see some relationship between the two inferred variables z 0 and z 1 (Fig. 6 )- although it appears more complex than could be captured by a Pearson's correlation-while this is completely gone for the regularized version (D2 Z 0 y ).
In this sense the network architecture itself can be understood as contributing to regularization by enforcing hard constraints, rather than the soft constraints of the cost function. In Fig. 8 , this tanh architecture is demonstrated on the bowed1 dataset, and it can be seen that the adversarial regularization nonetheless is still useful for ensuring that the domain is used effectively, i. e. , despite some visual clusters still being apparent, they are much more spread out, better approximating a uniform distribution and, to a large degree, breaking up the piecewise correlations between the parameters that can be seen by inspection when regularization is not used. In general, we found the ReLU approach more stable and better at producing uniform coverage. One will undoubtedly notice that the reconstruction error as reported in Fig.