dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgments We thank the  MPPE , MGF and SERENA instrument teams of the  BepiColombo mission  for providing excellent calibrated data. We thank the  BepiColombo ground operation teams at   ESAC, Madrid , and  ISAS, Tokyo , for providing spacecraft housekeeping data. We also thank  Lubos Brieda  for providing the Starfish PIC code. This work was support by grant  50QW2101  of German space agency DLR. French co-authors acknowledge the support of  CNES  for the BepiColombo mission.
SOFT-IO model The SOFT-IO (SOft attribution using FlexparT and carbon monoxide emission inventories for In-situ Observation database) tool  (Sauvage et al. , 2017 ; http://dx. doi. org/10. 25326/2) has been developed to investigate the origin of the observed IAGOS-CO, by coupling FLEXPART 20-days backward transport simulations with emission inventories. For each point of IAGOS trajectory, SOFT-IO estimates the CO contribution coming from 14 different geographical regions (see Fig. 1 ), for AN and BB origin separately. We use Community Emissions Data System (CEDS2) AN emissions  (McDuffie et al. , 2020)  and the Global Fire Assimilation System (GFAS) BB emissions  (Kaiser et al. , 2012)  which include fire injection heights, to discriminate sources of CO anomalies over different regions of interest.
Acknowledgements. We acknowledge the strong support of the  European Commission, Airbus  and the airlines ( Deutsche Lufthansa ,  Air France ,  Austrian ,  Air Namimbia ,  Cathay Pacific ,  Iberia ,  China Airlines  and  Hawaiian Airlines ) that have carried the MOZAIC or IAGOS equipment and performed the maintenance since 1994. IAGOS has been funded by the  European Union  projects  IAGOS-DS and IA-GOS-ERI. Additionally, IAGOS has been funded by  INSU-CNRS (France) ,  Météo-France ,  Université Paul Sabatier (Toulouse, France)  and  Research Center Jülich (FZJ, Jülich, Germany ). The IAGOS database is supported in France by  AERIS  (https://www. aeris-data. fr). IASI is a joint mission of  EUMETSAT  and the  Centre National d'Etudes Spatiales (CNES, France). The authors acknowledge the  CNES  for financial support for the IASI activities.
Data availability. The IAGOS data are available on the IAGOS data portal (https://doi. org/10. 25326/20). The SOFT-IO v1. 0 products are part of the ancillary products of IAGOS central database (https://doi. org/10. 25326/2; https://doi. org/10. 25326/3). The SOFRID-O3 data are freely available on the IASI-SOFRID website (http://thredds. sedoo. fr/iasi-sofrid-o3-co/, last access: 8 June 2022; SEDOO, 2014).
Complementary to finding good general word embeddings, an important question for representation learning is to find dynamic word embeddings, for example, across time or domain. Current methods do not offer a way to use or predict information on structure between sub-corpora, time or domain and dynamic embeddings can only be compared after postalignment. We propose novel word embedding methods that provide general word representations for the whole corpus, domainspecific representations for each sub-corpus, sub-corpus structure, and embedding alignment simultaneously. We present an empirical evaluation on New York Times articles and two English Wikipedia datasets with articles on science and philosophy. Our method, called Word2Vec with Structure Prediction (W2VPred), provides better performance than baselines in terms of the general analogy tests, domain-specific analogy tests, and multiple specific word embedding evaluations as well as structure prediction performance when no structure is given a priori.
It is important that those embeddings can be trained on small datasets. We therefore propose two generalizations of Dynamic Word2Vec. Our first method is called Word2Vec with Structure Constraint (W2VConstr), where domain-specific embeddings are learned under regularization with any kind of structure. This method performs well when a respective graph structure is given a priori. For more general cases where no structure information is given, we propose our second method, called Word2Vec with Structure Prediction (W2VPred), where domain-specific embeddings and sub-corpora structure are learned at the same time. W2VPred simultaneously solves three central problems that arise with word embedding representations: 1. Words in the sub-corpora are embedded in the same vector space, and are therefore directly comparable without post-alignment.
As we learn one representation per slice such as year or author, thus considering a much broader context than contextualized embeddings, we are able to find a meaningful structure-between corresponding slices. Another main advantage comes from the fact that our methods do not require any pre-training and can be run on a single GPU. We test our methods on 4 different datasets with different structures (sequences, trees, and general graphs), domains (news, wikipedia, high literature), and languages (English and German). We show on numerous established evaluation methods that W2VConstr and W2VPred significantly outperform baseline methods with regard to general as well as domain-specific embedding quality. We also show that W2VPred is able to predict the structure of a given corpus, outperforming all baselines.
, 2013b)  to learn dynamic word embeddings that evolve over time. Rudolph and Blei (2018)  analyzed dynamic changes in word embeddings based on exponential family embeddings, a probabilistic framework that generalizes the concept of word embeddings to other types of data  (Rudolph et al. , 2016). Yao et al. (2018)  proposed Dynamic Word2Vec (DW2V) to learn individual word embeddings for each year of the New York Times dataset  while simultaneously aligning the embeddings in the same vector space. Specifically, they solve the following problem for each timepoint t = 1,. , T sequentially: min U t L F + τ L R + λL D , where (1) L F = Y t -U t U ⊤ t 2 F , L R = U t 2 F , L D = U t-1 -U t 2 F + U t -U t+1 2 F (2) represent the losses for data fidelity, regularization, and diachronic constraint, respectively.
Word2Vec with Denoised Structure Constraint We propose a third method that combines W2VConstr and W2VPred for the scenario where W2VConstr results in poor word embeddings because the a priori structure is not optimal. In this case, we suggest applying W2VPred and consider the resulting structure as an input for W2VConstr. This procedure needs prior knowledge of the dataset and a human-in-the-loop to interpret the predicted structure by W2VPred in order to add or remove specific edges in the new ground truth structure. In the experiment section, we will condense the predicted structure by W2VPred into a sparse, denoised ground truth structure that is meaningful. We call this method Word2Vec with Denoised Structure Constraint (W2VDen).
Experiments on Benchmark Data We conducted four experiments starting with well-known settings and datasets and incrementally moving to new datasets with different structures. The first experiment focuses on the general embedding quality, the second one presents results on domain-specific embeddings, the third one evaluates the method's ability to predict structure and the fourth one shows the method's performance on various word similarity tasks. In the following subsections, we will first describe the data, preprocessing, and then the results. Further details on implementation and hyperparameters can be found in Appendix A.
Datasets We evaluated our methods on the following three benchmark datasets. New York Times (NYT): The New York Times dataset 1  (NYT) contains headlines, lead texts, and paragraphs of English news articles published online and offline between January 1990 and June 2016 with a total of 100,945 documents. We grouped the dataset by years with 1990-1998 as the train set and 1999-2016 as the test set. 1 ). We published the data set at https://huggingface. co/datasets/millawell/wikipedia field of science. The articles belonging to multiple categories 3  were randomly assigned to a single category in order to avoid similarity because of overlapping texts instead of structural similarity. In each category, we randomly chose 1/3 of the articles for the train set, and the remaining 2/3 were used as the test set.
Preprocessing We lemmatized all tokens, that is, assigned their base forms with spaCy 4  and grouped the data by years (for NYT) or categories (for WikiPhil and WikiFoS). For each dataset, we defined one individual vocabulary where we considered the 20,000 most frequent (lemmatized) words of the entire dataset that are also within the 20,000 most frequent words in at least 3 independent slices, that is, years or categories. This way, we filtered out ''trend'' words that are of significance only within a very short time period/only a few categories. The 100 most frequent words were filtered out as stop words. We set the symmetric context window (the number of words before and after a specific word considered as context for the PPMI matrix) to 5.
Ex1: General Embedding Performance In our first experiment, we compare the quality of the word embeddings trained by W2VConstr and W2VPred with the embeddings trained by baseline methods, GloVe, Skip-Gram, CBOW and DW2V. For GloVe, Skip-Gram and CBOW, we computed one set of embeddings on the entire dataset. For DW2V, W2VConstr, and W2VPred, domain-specific embeddings {U t } were averaged over all domains. We use the same vocabulary for all methods. For W2VConstr, we set the affinity matrix W as shown in the upper row of Figure  1 , based on the a priori known structure, that is, diachronic structure for NYT, and the category structure in Tables  1  and 2  for WikiFoS and WikiPhil. The lower row of Figure  1  shows the learned structure by W2VPred. Specifically, we set the ground-truth affinity   1  and 2  for the category structure of WikiFoS and WikiPhil, respectively, and the top row of Figure  1  for the visualization of the ground-truth affinity matrices).
28-3. 17). These different behaviors can be explained by comparing the estimated (lower row) and the a priori given (upper row) affinity matrices shown in Figure  1. In NYT, the estimated affinity decays smoothly as the time difference between two slices increases. This implies that the a priori given diachronic structure is good enough to enhance the word embedding quality (by W2VConstr and DW2V), and estimating the affinity matrix (by W2VPred) slightly degrades the performance due to the increased number of unknown parameters to Table  4  shows temporal analogy test accuracies on the NYT dataset. As expected, GloVe, Skip-Gram, and CBOW perform poorly. We assume this is because the individual slices are too small to train reliable embeddings. The embeddings trained with DW2V and W2VConstr are learned collaboratively between slices due to the diachronic and structure terms and significantly improve the performance.
Ex3: Structure Prediction This subsection discusses the structure prediction performance by W2VPred. We first evaluate the prediction performance by using the a priori affinity structure as the ground-truth structure. The results of this experiment should be interpreted with care, because we have already seen in Section 4. 3 that the given a priori affinity does not necessarily reflect the similarity structure of the slices in the corpus, in particular for WikiPhil. We then analyze the correlation between the embedding quality and the structure Nat   5 : Five nearest neighbors to the word ''power'' in the domain-specific embedding space, learned by W2VPred, of four main categories of WikiFoS (left four columns), and in the general embedding space learned by GloVe and Skip-Gram on the entire dataset (right-most columns, respectively).
This implies that the structure that we previously assumed for WikiFoS already works well. This shows that applying W2VDen is in fact a general purpose method that can be applied on any of the data sets but it is especially useful when there is a mismatch between the assumed structure and the structure predicted by W2VPred. In WikiPhil, we see that W2VDen further improves the performance by W2VPred, which already outperforms all other methods with a large margin. The correlation between the embedding quality and the structure prediction performance-with the denoised estimated affinity matrix as the ground truth-is shown in Table  7. The Pearson correlation is still negative, -0. 14, but no longer statistically significant (p = 0. 11). NYT WikiFos WikiPhil G lo V e S k ip -G. C B O W D W 2 V W 2 V C W 2 V P G lo V e S k ip -G.
Ex4: Evaluation in Word Similarity Tasks We further evaluate word embeddings on various word similarity tasks where human-annotated similarity between words is compared with the cosine similarity in the embedding space, as proposed in  Faruqui and Dyer (2014). Table  8  shows the correlation coefficients between the human-annotated similarity and the embedding cosine similarity, where, again, the best method and the runner-ups (if not significantly outperformed) are highlighted. 8 We observe that W2VPred outperforms the other methods in 7 out of 12 datasets for NYT, and W2VConstr in 8 out of 12 for WikiFoS. For WikiPhil, since we already know that W2VConstr with the given affinity matrix does not improve the embedding performance, we instead evaluated W2VDen, which outperforms 9 out of 12 datasets in WikiPhil.
In addition, W2VPred gives comparable performance to the best method over all experiments. We also apply QVEC, which measures component-wise correlation between distributed word embeddings, as we use them throughout the paper, and linguistic word vectors based on WordNet  (Fellbaum, 1998). High correlation values indicate high saliency of linguistic properties and thus serve as an intrinsic evaluation method that has been shown to highly correlate with downstream task performance  (Tsvetkov et al. , 2015). Results are shown in Table  9 , where we observe that W2VConstr (as well as W2VDen for WikiPhil) outperforms all baseline methods, except CBOW in NYT, on all datasets, and W2VPred performs comparably with the best method.
Summarizing Discussion In this section, we have shown a good performance of W2VConstr and W2VPred in terms of global and domain-specific embedding quality on news articles (NYT) and articles from Wikipedia (WikiFoS, WikiPhil). We have also shown that W2VPred is able to extract the underlying sub-corpora structure from NYT and WikiFoS. On the WikiPhil dataset, the following observations implied that the prior sub-corpora structure, based on the Wikipedia's definition, was not suitable for analyzing semantic relations: • Poor general analogy test performance by W2VConstr (Table  3 ), • Low structure prediction performance by all methods (Table  6 ) • Negative correlation between embedding accuracy and structure score (Table  7 ). Accordingly, we analyzed the learned structure by W2VPred, and further refined it by denoising with human intervention.
If W2VPred performs well, while W2VConstr performs poorly, we doubt that the given prior structure would be suitable, and update the learned structure by W2VPred. When no prior strucuture is given, we simply apply W2VPred to learn the structure. We can furthermore refine the learned structure with side information, which results in a clean and human interpretable structure. Here W2VDen is used to validate the new structure, and to provide enhanced word embeddings. In our experiment on the WikiPhil dataset, the embeddings obtained this way significantly outperformed all other methods. The improved performance from W2VPred is probably due to the fewer degrees of freedom of W2VConstr, that is, once we know a reasonable structure, the embeddings can be more accurately trained with the fixed affinity matrix.
Reference Dimensions In this visualization we want to compare the pairwise distance matrix that our method predicted with the distance matrices that can be obtained by meta data available in the DTA corpus-the reference dimensions: 1. Temporal difference between authors. We collect the publication year for each title in the corpus and compute the average publication year for each author. The temporal distance between one author A t1 and another author A t2 is computed by |A t1 -A t2 |, the absolute difference of the average publication year. 2. Spatial difference between authors. We query the German Integrated Authority File for the authors' different work places and extract them as longitude and latitude coordinates on the earths surface. We compute the average coordinates for each author by converting the coordinates into the Cartesian system and take the average on each dimension.
Acknowledgments We thank  Gilles Blanchard  for valuable comments on the manuscript. We further thank  Felix Herron  for his support in the data collection process. DL and SN are supported by the  German Ministry for Education and Research (BMBF)  as  BIFOLD -Berlin Institute  for the Foundations of Learning and Data under grants  01IS18025A  and  01IS18037A. SB was partially funded by the  Platform Intelligence in News project , which is supported by  Innovation Fund Denmark  via the  Grand Solutions program  and by the  European Union  under the Grant Agreement no. 10106555 ,  FairER. Views  and opinions expressed are those of the author(s) only and do not necessarily reflect those of the  European Union or European Research Executive Agency (REA). Neither the European Union nor REA can be held responsible for them.
and other baseline methods based on the GloVe, Skip-Gram, and CBOW embeddings. Burrows' Delta is a commonly used method in stylometrics to analyze the similarity between Dataset NYT WikiFos WikiPhil Method GloVe 67. 22 51. 66 36. 67 Skip-Gram 71. 11 54. 59 26. 67 CBOW 65. 28 45. 00 23. 33 W2VPred (our) 81. 67 62. 50 23. 33 Burrows'delta 55. 56 22. 92 6. 67.
5-D Sonnet ® package or Keysight's Advanced Design System ® (ADS), to fully describe the physical layout of the unit cell. This approach is extremely flexible for obtaining γ to feed into the framework above for calculating the gain-bandwidth product of the entire KITWPA, since one could include almost any possible structures within the modelling environment. In our case, we focus on the use of Ansys software suites (Ansys ® Electronics Desktop) which make use of the full Maxwell equations and finite element method to calculate the electromagnetic properties of a 3D structure, and we use the example of a periodically loaded KITWPA for explaining the simulation procedure below. Due to the complexity of Ansys ® Electronics Desktop, there are no freely available alternative open-source software that can replace the full functionality of this em software to perform the same simulation (to our knowledge).
Substrate and topology considerations The choice of the superconducting thin film forms only part of the design consideration for developing an optimal KITWPA. The planar circuit amplifier can in principle be formed using any type of transmission line topology fabricated on different varieties of supporting substrate. In this section, we explore how these transmission line topology and substrate choices would affect the performance of the KITWPA. For this exercise, we too make use of em simulators Ansys HFSS and the Mattis-Bardeen equations. The source code for calculating the superconducting surface impedance via the Mattis-Bardeen equation, namely the SuperMix software libraries, were previously developed by several contributors at the California Institute of Technology (CalTech), and not by the authors of this article.
Source code of SuperMix is available at: here In particular, we focus on the conduction and dielectric losses, the compactness of the device and the feasibility of fabrication in terms of circuit dimensions. Figure  9  shows the four different types of STL topologies we investigate here i. e. , coplanar waveguide (CPW), inverted CPW, microstrip, and inverted microstrip line. Among the chosen four topologies, the inverted lines have the advantage of protecting the long transmission line via the top ground layer, especially the central conductor or the wiring layer, from potential physical damage during handling of the device such as subsequent chip dicing and wire bonding process after fabrication. This also helps to avoid potential unwanted scratches, which could cause open circuit or debris of conducting materials that cause short circuit to ground.
Designs of KITWPA In this section, we perform the gain-bandwidth product prediction using the method presented in Section 2 for both KITWPAs identified earlier. Please note that all data underlying the graphs and the scripts for plotting the graphs are available in Underlying data and Extended data  51. To achieve exponential gain, the power dependent phase difference between all the propagating tones needs to be minimised. As explained previously, this can be done by altering the dispersion relation of the transmission line through periodically varying the impedance along the line i. e. , the periodic loading scheme. Here, we aim to achieve more than 20 dB gain centred at 8 GHz with minimal length for compactness. For an optimal amplifier design, we take into account considerations that are needed to mitigate risk of physical damage, promoting uniformity of the transmission line, and target for minimum pump power requirement.
1. Section 2, Methods. The sequence of TPA simulation is explained in Figure  1. It is not difficult to understand how the ABCD matrix of each unit cell is obtained by using EM simulation. But it is not clear how these linear matrixes are combined with the coupled equations to calculate the gain-bandwidth profile of the TPA. It is obvious that the explanation of solving the nonlinear equations is missing. Readers will be unable to reproduce this calculation without knowing the method of solving the equations. Response:As mentioned in the article, the unit cells are cascaded using transfer matrix method, which is a commonly used method. Basically, it's just a dot-matrix product of the unit cells. This can be easily done using commercial or standard circuit emulators, or by writing a simple python code to do the matrix product.
As pointed out in the text and in Fig. 1 , one can easily extract the propagation constant (\gamma) using the cascaded ABCD matrix, which are available in many electromagnetisms' textbook e. g. , Pozar's Microwave Engineering. The CME noted in the article can then be solved numerically using python, Mathematica, Matlab or any other numerical software one prefers. As the article is focusing on the main message of the finding, these trivial step-by-step instructions would distract the focus of the reader on the scientific content of paper, hence not included. For these instructions, I invite you to look out for Joseph's thesis which should be published shortly after he completed his 2. PhD. Section 2, All R s in the text and the figures are in unit of kΩ / sq. They are unreasonably large sheet resistance and, therefore, the unit must be mistaken.
• Dataset for Figure 2 ( 2 Raw datasets in. dat for each panel in Figure 2) • Dataset for Figure 3 (Raw datasets in. txt for each panel in Figure 3) • Dataset for Figure 4 (Raw datasets in. txt for each panel in Figure 4) • Dataset for Figure 5 (Raw datasets in. txt for each panel in Figure 5) • Dataset for Figure 6 (Raw datasets in. txt for each panel in Figure 6) • Dataset for Figure 7 (Raw datasets in. txt for each panel in Figure 7) • Dataset for Figure 8 (Raw datasets in. txt for each panel in Figure 8) • Dataset for Figure 10 (Raw datasets in. dat for each panel in Figure 10) • Dataset for Figure 11 (Raw datasets in. dat for each panel in Figure 11) • Dataset for Figure 12 (Raw datasets in. dat for each panel in Figure 12) • Dataset for Figure 13 (Raw datasets in. dat and. csv for each panel in Figure 13) • Dataset for Figure 14 (Raw datasets in.