nils-herrmann/clean_scirepeval huggingface dataset:
Changing the tune: mixtures of network models that vary in time

Many of the complex systems we study in their representation as networks are growing objects, evolving by the addition of nodes and links over time. The rules governing this growth are attributed to mechanisms such as preferential attachment and triangle closure. We demonstrate a method for estimating the relative roles of these mechanisms, and further, investigating how they change as the network evolves. We show that a rich class of network evolution models can be built from a weighted mixture of these model mechanisms. Using a likelihood based formulation we show how to calculate the optimal mixture for a given set of observations of network data, and show that this framework can be used to distinguish competing models that are indistinguishable by their summary statistics. Using real data from Facebook user interactions, we show that we can improve the ability of a model to reproduce network statistics using tuned model mixtures. We further investigate the idea that the underlying model of a network can change in time, for example, a technology based network might respond to changes in the underlying technology or a financial network might respond to economic shocks. Using artificial data we show that we can recapture the time at which a known change occurred. We use the Enron email dataset to show that we can estimate how mixtures of models change over time.
Active Anomaly Detection for time-domain discoveries

We present the first application of adaptive machine learning to the identification of anomalies in a data set of non-periodic astronomical light curves. The method follows an active learning strategy where highly informative objects are selected to be labelled. This new information is subsequently used to improve the machine learning model, allowing its accuracy to evolve with the addition of every new classification. For the case of anomaly detection, the algorithm aims to maximize the number of real anomalies presented to the expert by slightly modifying the decision boundary of a traditional isolation forest in each iteration. As a proof of concept, we apply the Active Anomaly Discovery (AAD) algorithm to light curves from the Open Supernova Catalog and compare its results to those of a static Isolation Forest (IF). For both methods, we visually inspected objects within 2% highest anomaly scores. We show that AAD was able to identify 80% more true anomalies than IF. This result is the first evidence that AAD algorithms can play a central role in the search for new physics in the era of large scale sky surveys.
Model-Aided Deep Neural Network for Source Number Detection

Source number detection is a critical problem in array signal processing. Conventional model-driven methods e.g., Akaikes information criterion and minimum description length, suffer from severe performance degradation when the number of samples is small or the signal-to-noise ratio is low. In this letter, we exploit the model-aided based deep neural network to estimate the source number. Specifically, we propose two eigenvalue based networks, i.e., a regression network (ERNet) and a classification network (ECNet), for source number detection, where the eigenvalues of the received signal covariance matrix and the source number are used as the input and the label of the networks, respectively. Furthermore, ERNet and ECNet can be easily generalized to handle coherent sources by adopting, e.g., the forward-backward spatial smoothing technique. Numerical results are included to showcase the remarkable improvements of ERNet and ECNet over the existing methods.
High-Throughput Detection and Manipulation of Single Nitrogen-Vacancy Center's Charge in Nanodiamonds.

The fluorescent nitrogen-vacancy (NV) defect in diamond has remarkable photophysical properties, including high photostability, which allows stable fluorescence emission for hours; as a result, there has been much interest in using nanodiamonds (NDs) for quantum optics and biological imaging. Such applications have been limited by the complexity and heterogeneity of ND photophysics. Photophysics of the NV center in NDs have been studied before, but the lack of a sensitive and high-throughput method has limited the characterization of NDs. Here is reported a systematic analysis of NDs using two-color wide-field epifluorescence imaging coupled to high-throughput single-particle detection of single NVs in NDs with sizes down to 5-10 nm. By using fluorescence intensity ratios, the charge conversion of single NV center (NV- or NV0) is observed and the lifetimes of different NV charge states in NDs is measured. The discovery of reversible manipulation of NV charge states by pH is also presented. In addition to provide another mechanism to control the NV charge state in nanodiamonds, our findings open the possibility to perform pH nanosensing with a non-photobleachable probe.
Lifelong Neural Topic Learning in Contextualized Autoregressive Topic Models of Language via Informative Transfers

Topic models such as LDA, DocNADE, iDocNADEe have been popular in document analysis. However, the traditional topic models have several limitations including: (1) Bag-of-words (BoW) assumption, where they ignore word ordering, (2) Data sparsity, where the application of topic models is challenging due to limited word co-occurrences, leading to incoherent topics and (3) No Continuous Learning framework for topic learning in lifelong fashion, exploiting historical knowledge (or latent topics) and minimizing catastrophic forgetting. This thesis focuses on addressing the above challenges within neural topic modeling framework. We propose: (1) Contextualized topic model that combines a topic and a language model and introduces linguistic structures (such as word ordering, syntactic and semantic features, etc.) in topic modeling, (2) A novel lifelong learning mechanism into neural topic modeling framework to demonstrate continuous learning in sequential document collections and minimizing catastrophic forgetting. Additionally, we perform a selective data augmentation to alleviate the need for complete historical corpora during data hallucination or replay.
Language-Agnostic Syllabification with Neural Sequence Labeling

The identification of syllables within phonetic sequences is known as syllabification. This task is thought to play an important role in natural language understanding, speech production, and the development of speech recognition systems. The concept of the syllable is cross-linguistic, though formal definitions are rarely agreed upon, even within a language. In response, data-driven syllabification methods have been developed to learn from syllabified examples. These methods often employ classical machine learning sequence labeling models. In recent years, recurrence-based neural networks have been shown to perform increasingly well for sequence labeling tasks such as named entity recognition (NER), part of speech (POS) tagging, and chunking. We present a novel approach to the syllabification problem which leverages modern neural network techniques. Our network is constructed with long short-term memory (LSTM) cells, a convolutional component, and a conditional random field (CRF) output layer. Existing syllabification approaches are rarely evaluated across multiple language families. To demonstrate cross-linguistic generalizability, we show that the network is competitive with state of the art systems in syllabifying English, Dutch, Italian, French, Manipuri, and Basque datasets.
Not so Particular about Calibration: Smile Problem Resolved

We present a novel Monte Carlo based LSV calibration algorithm that applies to all stochastic volatility models, including the non-Markovian rough volatility family. Our framework overcomes the limitations of the particle method proposed by Guyon and Henry-Labordere (2012) and theoretically guarantees a variance reduction without additional computational complexity. Specifically, we obtain a closed-form and exact calibration method that allows us to remove the dependency on both the kernel function and bandwidth parameter. This makes the algorithm more robust and less prone to errors or instabilities in a production environment. We test the efficiency of our algorithm on various hybrid (rough) local stochastic volatility models.
Gradient Descent: The Ultimate Optimizer

Working with any gradient-based machine learning algorithm involves the tedious task of tuning the optimizer's hyperparameters, such as the learning rate. There exist many techniques for automated hyperparameter optimization, but they typically introduce even more hyperparameters to control the hyperparameter optimization process. We propose to instead learn the hyperparameters themselves by gradient descent, and furthermore to learn the hyper-hyperparameters by gradient descent as well, and so on ad infinitum. As these towers of gradient-based optimizers grow, they become significantly less sensitive to the choice of top-level hyperparameters, hence decreasing the burden on the user to search for optimal values.
Strong Baseline Defenses Against Clean-Label Poisoning Attacks

Targeted clean-label poisoning is a type of adversarial attack on machine learning systems where the adversary injects a few correctly-labeled, minimally-perturbed samples into the training data thus causing the deployed model to misclassify a particular test sample during inference. Although defenses have been proposed for general poisoning attacks (those which aim to reduce overall test accuracy), no reliable defense for clean-label attacks has been demonstrated, despite the attacks' effectiveness and their realistic use cases. In this work, we propose a set of simple, yet highly-effective defenses against these attacks. We test our proposed approach against two recently published clean-label poisoning attacks, both of which use the CIFAR-10 dataset. After reproducing their experiments, we demonstrate that our defenses are able to detect over 99% of poisoning examples in both attacks and remove them without any compromise on model performance. Our simple defenses show that current clean-label poisoning attack strategies can be annulled, and serve as strong but simple-to-implement baseline defense for which to test future clean-label poisoning attacks.
A Simple and Effective Model for Answering Multi-span Questions

With models reaching human performance on many popular reading comprehension datasets in recent years, a new dataset, DROP, introduced questions that were expected to present a harder challenge for reading comprehension models. Among these new types of questions were "multi-span questions", questions whose answers consist of several spans from either the paragraph or the question itself. Until now, only one model attempted to tackle multi-span questions as a part of its design. In this work, we suggest a new approach for tackling multi-span questions, based on sequence tagging, which differs from previous approaches for answering span questions. We show that our approach leads to an absolute improvement of 29.7 EM and 15.1 F1 compared to existing state-of-the-art results, while not hurting performance on other question types. Furthermore, we show that our model slightly eclipses the current state-of-the-art results on the entire DROP dataset.
Lane-Attention: Predicting Vehicles’ Moving Trajectories by Learning Their Attention Over Lanes

Accurately forecasting the future movements of surrounding vehicles is essential for safe and efficient operations of autonomous driving cars. This task is difficult because a vehicle’s moving trajectory is greatly determined by its driver’s intention, which is often hard to estimate. By leveraging attention mechanisms along with long short-term memory (LSTM) networks, this work learns the relation between a driver’s intention and the vehicle’s changing positions relative to road infrastructures, and uses it to guide the prediction. Different from other state-of-the-art solutions, our work treats the on-road lanes as non-Euclidean structures, unfolds the vehicle’s moving history to form a spatio-temporal graph, and uses methods from Graph Neural Networks to solve the problem. Not only is our approach a pioneering attempt in using non-Euclidean methods to process static environmental features around a predicted object, our model also outperforms other state-of-the-art models in several metrics. The practicability and interpretability analysis of the model shows great potential for large-scale deployment in various autonomous driving systems in addition to our own.
REQ-YOLO: A Resource-Aware, Efficient Quantization Framework for Object Detection on FPGAs

Deep neural networks (DNNs), as the basis of object detection, will play a key role in the development of future autonomous systems with full autonomy. The autonomous systems have special requirements of real-time, energy-e cient implementations of DNNs on a power-budgeted system. Two research thrusts are dedicated to per- formance and energy e ciency enhancement of the inference phase of DNNs. The first one is model compression techniques while the second is e cient hardware implementations. Recent researches on extremely-low-bit CNNs such as binary neural network (BNN) and XNOR-Net replace the traditional oating point operations with bi- nary bit operations, signi cantly reducing memory bandwidth and storage requirement, whereas suffering non-negligible accuracy loss and waste of digital signal processing (DSP) blocks on FPGAs. To overcome these limitations, this paper proposes REQ-YOLO, a resource aware, systematic weight quantization framework for object detection, considering both algorithm and hardware resource aspects in object detection. We adopt the block-circulant matrix method and propose a heterogeneous weight quantization using Alternative Direction Method of Multipliers (ADMM), an e ective optimization technique for general, non-convex optimization problems. To achieve real-time, highly efficient implementations on FPGA, we present the detailed hardware implementation of block circulant matrices on CONV layers and de- velop an e cient processing element (PE) structure supporting the heterogeneous weight quantization, CONV data ow and pipelining techniques, design optimization, and a template-based automatic synthesis framework to optimally exploit hardware resource. Experimental results show that our proposed REQ-YOLO framework can signi cantly compress the YOLO model while introducing very small accuracy degradation. The related codes are here: https://github.com/Anonymous788/heterogeneous_ADMM_YOLO.
SymmetricNet: A mesoscale eddy detection method based on multivariate fusion data

Mesoscale eddies play a significant role in marine energy transport, marine biological environment and marine climate. Due to their huge impact on the ocean, mesoscale eddy detection has become a hot research area in recent years. Therefore, more and more people are entering the field of mesoscale eddy detection. However, the existing detection methods mainly based on traditional detection methods typically only use Sea Surface Height (SSH) as a variable to detect, resulting in inaccurate performance. In this paper, we propose a mesoscale eddy detection method based on multivariate fusion data to solve this problem. We not only use the SSH variable, but also add the two variables: Sea Surface Temperature (SST) and velocity of flow, achieving a multivariate information fusion input. We design a novel symmetric network, which merges low-level feature maps from the downsampling pathway and high-level feature maps from the upsampling pathway by lateral connection. In addition, we apply dilated convolutions to network structure to increase the receptive field and obtain more contextual information in the case of constant parameter. In the end, we demonstrate the effectiveness of our method on dataset provided by us, achieving the test set performance of 97.06% , greatly improved the performance of previous methods of mesoscale eddy detection.
Random Bias Initialization Improves Quantized Training

Binary neural networks improve computationally efficiency of deep models with a large margin. However, there is still a performance gap between a successful full-precision training and binary training. We bring some insights about why this accuracy drop exists and call for a better understanding of binary network geometry. We start with analyzing full-precision neural networks with ReLU activation and compare it with its binarized version. This comparison suggests to initialize networks with random bias, a counter-intuitive remedy.
Reconstructing star formation histories of recently formed ultra-diffuse galaxies.

Observational studies of ultra-diffuse galaxies (UDGs) represent a significant challenge because of their very low surface brightnesses. A feasible approach is to identify "future" UDGs when their stars are still young. Using data mining, we found 12 such low-mass spatially extended quiescent galaxies in the Coma and Abell 2147 clusters in the SDSS legacy galaxy sample and followed them up using a new high-throughput Binospec spectrograph at the 6.5m MMT. Several of them exhibit signs of the recently finished ram pressure stripping. Here we describe our data analysis approach that uses spectroscopic and photometric measurements with a dedicated set of stellar population models, which include realistic chemical enrichment and star formation histories. From our analysis we can precisely estimate stellar mass-to-light ratios and dark matter content of UDGs.
Regressing Word and Sentence Embeddings for Regularization of Neural Machine Translation

In recent years, neural machine translation (NMT) has become the dominant approach in automated translation. However, like many other deep learning approaches, NMT suffers from overfitting when the amount of training data is limited. This is a serious issue for low-resource language pairs and many specialized translation domains that are inherently limited in the amount of available supervised data. For this reason, in this paper we propose regressing word (ReWE) and sentence (ReSE) embeddings at training time as a way to regularize NMT models and improve their generalization. During training, our models are trained to jointly predict categorical (words in the vocabulary) and continuous (word and sentence embeddings) outputs. An extensive set of experiments over four language pairs of variable training set size has showed that ReWE and ReSE can outperform strong state-of-the-art baseline models, with an improvement that is larger for smaller training sets (e.g., up to +5:15 BLEU points in Basque-English translation). Visualizations of the decoder's output space show that the proposed regularizers improve the clustering of unique words, facilitating correct predictions. In a final experiment on unsupervised NMT, we show that ReWE and ReSE are also able to improve the quality of machine translation when no parallel data are available.
On Incorporating Semantic Prior Knowlegde in Deep Learning Through Embedding-Space Constraints

The knowledge that humans hold about a problem often extends far beyond a set of training data and output labels. While the success of deep learning mostly relies on supervised training, important properties cannot be inferred efficiently from end-to-end annotations alone, for example causal relations or domain-specific invariances. We present a general technique to supplement supervised training with prior knowledge expressed as relations between training instances. We illustrate the method on the task of visual question answering to exploit various auxiliary annotations, including relations of equivalence and of logical entailment between questions. Existing methods to use these annotations, including auxiliary losses and data augmentation, cannot guarantee the strict inclusion of these relations into the model since they require a careful balancing against the end-to-end objective. Our method uses these relations to shape the embedding space of the model, and treats them as strict constraints on its learned representations. In the context of VQA, this approach brings significant improvements in accuracy and robustness, in particular over the common practice of incorporating the constraints as a soft regularizer. We also show that incorporating this type of prior knowledge with our method brings consistent improvements, independently from the amount of supervised data used. It demonstrates the value of an additional training signal that is otherwise difficult to extract from end-to-end annotations alone.
Promotion of Cooperation in Coevolutionary Public Goods Game on Complex Networks with and without Topology Change

The evolution of cooperation among unrelated individuals in human and animal societies remains a challenging issue across disciplines. It is an important subject also in the evolutionary game theory to understand how cooperation arises. The subject has been extensively studied especially in Prisoners' dilemma game(PD) but the emergence of cooperation is also an important subject in public goods game(PGG). In this article, we consider coevolutionary PGG on complex networks where both the topology of the networks and strategies that players adopt under the influence of game dynamics varies. Though cooperators can contribute a fixed amount per game in PGG on networks in the previous studies, the cooperators contribute a fixed amount per member of the group in PGG of this article. The latter is seemed to be more natural than the former model. These models lead to great differences in both PGGs. We study what effects on the evolution of player's strategies, defection and cooperation and the average payoff does the interaction between the game dynamics and the network topology bring. Moreover by comparing the models that do not depend on game dynamics to the models without topology changing, we intend to uncover the effect of the topology chnage and the game dynamics on the promotion of the cooperation. As result we intend to clear in what situations cooperation strategy is promoted or preserved by investigating them by making computer simulations. We also investigate the relation between the ratio of the cooperator and the average payoff over all players. Furthermore we study how do the topology of initial networks have an influence on the cooperation and the average payoff.
Exoplanet characterisation in the longest known resonant chain: the K2-138 system seen by HARPS

The detection of low-mass transiting exoplanets in multiple systems brings new constraints to planetary formation and evolution processes and challenges the current planet formation theories. Nevertheless, only a mere fraction of the small planets detected by Kepler and K2 have precise mass measurements, which are mandatory to constrain their composition. We aim to characterise the planets that orbit the relatively bright star K2-138. This system is dynamically particular as it presents the longest chain known to date of planets close to the 3:2 resonance. We obtained 215 HARPS spectra from which we derived the radial-velocity variations of K2-138. Via a joint Bayesian analysis of both the K2 photometry and HARPS radial-velocities (RVs), we constrained the parameters of the six planets in orbit. The masses of the four inner planets, from b to e, are 3.1, 6.3, 7.9, and 13.0 M⊕ with a precision of 34, 20, 18, and 15%, respectively. The bulk densities are 4.9, 2.8, 3.2, and 1.8 g cm−3, ranging from Earth to Neptune-like values. For planets f and g, we report upper limits. Finally, we predict transit timing variations of the order two to six minutes from the masses derived. Given its peculiar dynamics, K2-138 is an ideal target for transit timing variation (TTV) measurements from space with the upcoming CHaracterizing ExOPlanet Satellite (CHEOPS) to study this highly-packed system and compare TTV and RV masses.