nils-herrmann/clean_scirepeval huggingface dataset:
Adversarial Robustness through Local Linearization

Adversarial training is an effective methodology for training deep neural networks that are robust against adversarial, norm-bounded perturbations. However, the computational cost of adversarial training grows prohibitively as the size of the model and number of input dimensions increase. Further, training against less expensive and therefore weaker adversaries produces models that are robust against weak attacks but break down under attacks that are stronger. This is often attributed to the phenomenon of gradient obfuscation; such models have a highly non-linear loss surface in the vicinity of training examples, making it hard for gradient-based attacks to succeed even though adversarial examples still exist. In this work, we introduce a novel regularizer that encourages the loss to behave linearly in the vicinity of the training data, thereby penalizing gradient obfuscation while encouraging robustness. We show via extensive experiments on CIFAR-10 and ImageNet, that models trained with our regularizer avoid gradient obfuscation and can be trained significantly faster than adversarial training. Using this regularizer, we exceed current state of the art and achieve 47% adversarial accuracy for ImageNet with l-infinity adversarial perturbations of radius 4/255 under an untargeted, strong, white-box attack. Additionally, we match state of the art results for CIFAR-10 at 8/255.
On the Convergence of Model Free Learning in Mean Field Games

Learning by experience in Multi-Agent Systems (MAS) is a difficult and exciting task, due to the lack of stationarity of the environment, whose dynamics evolves as the population learns. In order to design scalable algorithms for systems with a large population of interacting agents (e.g., swarms), this paper focuses on Mean Field MAS, where the number of agents is asymptotically infinite. Recently, a very active burgeoning field studies the effects of diverse reinforcement learning algorithms for agents with no prior information on a stationary Mean Field Game (MFG) and learn their policy through repeated experience. We adopt a high perspective on this problem and analyze in full generality the convergence of a fictitious iterative scheme using any single agent learning algorithm at each step. We quantify the quality of the computed approximate Nash equilibrium, in terms of the accumulated errors arising at each learning iteration step. Notably, we show for the first time convergence of model free learning algorithms towards non-stationary MFG equilibria, relying only on classical assumptions on the MFG dynamics. We illustrate our theoretical results with a numerical experiment in a continuous action-space environment, where the approximate best response of the iterative fictitious play scheme is computed with a deep RL algorithm.
Neural Drum Machine : An Interactive System for Real-time Synthesis of Drum Sounds

In this work, we introduce a system for real-time generation of drum sounds. This system is composed of two parts: a generative model for drum sounds together with a Max4Live plugin providing intuitive controls on the generative process. The generative model consists of a Conditional Wasserstein autoencoder (CWAE), which learns to generate Mel-scaled magnitude spectrograms of short percussion samples, coupled with a Multi-Head Convolutional Neural Network (MCNN) which estimates the corresponding audio signal from the magnitude spectrogram. The design of this model makes it lightweight, so that it allows one to perform real-time generation of novel drum sounds on an average CPU, removing the need for the users to possess dedicated hardware in order to use this system. We then present our Max4Live interface designed to interact with this generative model. With this setup, the system can be easily integrated into a studio-production environment and enhance the creative process. Finally, we discuss the advantages of our system and how the interaction of music producers with such tools could change the way drum tracks are composed.
Pathology GAN: Learning deep representations of cancer tissue

We apply Generative Adversarial Networks (GANs) to the domain of digital pathology. Current machine learning research for digital pathology focuses on diagnosis, but we suggest a different approach and advocate that generative models could drive forward the understanding of morphological characteristics of cancer tissue. In this paper, we develop a framework which allows GANs to capture key tissue features and uses these characteristics to give structure to its latent space. To this end, we trained our model on 249K H&E breast cancer tissue images, extracted from 576 TMA images of patients from the Netherlands Cancer Institute (NKI) and Vancouver General Hospital (VGH) cohorts. We show that our model generates high quality images, with a Frechet Inception Distance (FID) of 16.65. We further assess the quality of the images with cancer tissue characteristics (e.g. count of cancer, lymphocytes, or stromal cells), using quantitative information to calculate the FID and showing consistent performance of 9.86. Additionally, the latent space of our model shows an interpretable structure and allows semantic vector operations that translate into tissue feature transformations. Furthermore, ratings from two expert pathologists found no significant difference between our generated tissue images from real ones. The code, generated images, and pretrained model are available at this https URL
What is the Benefit of Code-domain NOMA in Massive MIMO?

In overloaded Massive MIMO systems, wherein the number K of user equipments (UEs) exceeds the number of base station antennas M, it has recently been shown that non-orthogonal multiple access (NOMA) can increase performance. This paper aims at identifying cases of the classical operating regime K < M, where code-domain NOMA can also improve the spectral efficiency of Massive MIMO. Particular attention is given to use cases in which poor favorable propagation conditions are experienced. Numerical results show that Massive MIMO with planar antenna arrays can benefit from NOMA in practical scenarios where the UEs are spatially close to each other.
Evaluating the distribution learning capabilities of GANs

We evaluate the distribution learning capabilities of generative adversarial networks by testing them on synthetic datasets. The datasets include common distributions of points in $R^n$ space and images containing polygons of various shapes and sizes. We find that by and large GANs fail to faithfully recreate point datasets which contain discontinous support or sharp bends with noise. Additionally, on image datasets, we find that GANs do not seem to learn to count the number of objects of the same kind in an image. We also highlight the apparent tension between generalization and learning in GANs.
Forecasting security's volatility using low-frequency historical data, high-frequency historical data and option-implied volatility

Low-frequency historical data, high-frequency historical data and option data are three major sources, which can be used to forecast the underlying security's volatility. In this paper, we propose two econometric models, which integrate three information sources. In GARCH-It\^{o}-OI model, we assume that the option-implied volatility can influence the security's future volatility, and the option-implied volatility is treated as an observable exogenous variable. In GARCH-It\^{o}-IV model, we assume that the option-implied volatility can not influence the security's volatility directly, and the relationship between the option-implied volatility and the security's volatility is constructed to extract useful information of the underlying security. After providing the quasi-maximum likelihood estimators for the parameters and establishing their asymptotic properties, we also conduct a series of simulation analysis and empirical analysis to compare the proposed models with other popular models in the literature. We find that when the sampling interval of the high-frequency data is 5 minutes, the GARCH-It\^{o}-OI model and GARCH-It\^{o}-IV model has better forecasting performance than other models.
Zero-shot Learning for Audio-based Music Classification and Tagging

Audio-based music classification and tagging is typically based on categorical supervised learning with a fixed set of labels. This intrinsically cannot handle unseen labels such as newly added music genres or semantic words that users arbitrarily choose for music retrieval. Zero-shot learning can address this problem by leveraging an additional semantic space of labels where side information about the labels is used to unveil the relationship between each other. In this work, we investigate the zero-shot learning in the music domain and organize two different setups of side information. One is using human-labeled attribute information based on Free Music Archive and OpenMIC-2018 datasets. The other is using general word semantic information based on Million Song Dataset and this http URL tag annotations. Considering a music track is usually multi-labeled in music classification and tagging datasets, we also propose a data split scheme and associated evaluation settings for the multi-label zero-shot learning. Finally, we report experimental results and discuss the effectiveness and new possibilities of zero-shot learning in the music domain.
Networkmetrics unraveled: MBDA in Action

We propose networkmetrics, a new data-driven approach for monitoring, troubleshooting and understanding communication networks using multivariate analysis. Networkmetric models are powerful machine-learning tools to interpret and interact with data collected from a network. In this paper, we illustrate the application of Multivariate Big Data Analysis (MBDA), a recently proposed networkmetric method with application to Big Data sets. We use MBDA for the detection and troubleshooting of network problems in a campus-wide Wi-Fi network. Data includes a seven-year trace (from 2012 to 2018) of the network's most recent activity, with approximately 3,000 distinct access points, 40,000 authenticated users, and 600,000 distinct Wi-Fi stations. This is the longest and largest Wi-Fi trace known to date. To analyze this data, we propose learning and visualization procedures that extend MBDA. These procedures result in a methodology that allows network analysts to identify problems and diagnose and troubleshoot them, optimizing the network performance. In the paper, we go through the entire workflow of the approach, illustrating its application in detail and discussing processing times for parallel hardware.
Head-Driven Phrase Structure Grammar Parsing on Penn Treebank

Head-driven phrase structure grammar (HPSG) enjoys a uniform formalism representing rich contextual syntactic and even semantic meanings. This paper makes the first attempt to formulate a simplified HPSG by integrating constituent and dependency formal representations into head-driven phrase structure. Then two parsing algorithms are respectively proposed for two converted tree representations, division span and joint span. As HPSG encodes both constituent and dependency structure information, the proposed HPSG parsers may be regarded as a sort of joint decoder for both types of structures and thus are evaluated in terms of extracted or converted constituent and dependency parsing trees. Our parser achieves new state-of-the-art performance for both parsing tasks on Penn Treebank (PTB) and Chinese Penn Treebank, verifying the effectiveness of joint learning constituent and dependency structures. In details, we report 95.84 F1 of constituent parsing and 97.00% UAS of dependency parsing on PTB.
Twin Auxiliary Classifiers GAN

Conditional generative models enjoy remarkable progress over the past few years. One of the popular conditional models is Auxiliary Classifier GAN (AC-GAN), which generates highly discriminative images by extending the loss function of GAN with an auxiliary classifier. However, the diversity of the generated samples by AC-GAN tends to decrease as the number of classes increases, hence limiting its power on large-scale data. In this paper, we identify the source of the low diversity issue theoretically and propose a practical solution to solve the problem. We show that the auxiliary classifier in AC-GAN imposes perfect separability, which is disadvantageous when the supports of the class distributions have significant overlap. To address the issue, we propose Twin Auxiliary Classifiers Generative Adversarial Net (TAC-GAN) that further benefits from a new player that interacts with other players (the generator and the discriminator) in GAN. Theoretically, we demonstrate that TAC-GAN can effectively minimize the divergence between the generated and real-data distributions. Extensive experimental results show that our TAC-GAN can successfully replicate the true data distributions on simulated data, and significantly improves the diversity of class-conditional image generation on real datasets.
Intelligent social bots uncover the link between user preference and diversity of news consumption

The boom of online social media and microblogging platforms has rapidly alter the way we consume news and exchange opinions. Even though considerable efforts try to recommend various contents to users, loss of information diversity and the polarization of interest groups are still an enormous challenge for industry and academia. Here, we take advantage of benign social bots to design a controlled experiment on Weibo (the largest microblogging platform in China). These software bots can exhibit human-like behavior (e.g., preferring particular content) and simulate the formation of personal social networks and news consumption under two well-accepted sociological hypotheses (i.e., homophily and triadic closure). We deployed 68 bots to Weibo, and each bot ran for at least 2 months and followed 100 to 120 accounts. In total, we observed 5,318 users and recorded about 630,000 messages exposed to these bots. Our results show, even with the same selection behaviors, bots preferring entertainment content are more likely to form polarized communities with their peers, in which about 80\% of the information they consume is of the same type, which is a significant difference for bots preferring sci-tech content. The result suggests that users preference played a more crucial role in limiting themselves access to diverse content by compared with the two well-known drivers (self-selection and pre-selection). Furthermore, our results reveal an ingenious connection between specific content and its propagating sub-structures in the same social network. In the Weibo network, entertainment news favors a unidirectional star-like sub-structure, while sci-tech news spreads on a bidirectional clustering sub-structure. This connection can amplify the diversity effect of user preference. The discovery may have important implications for diffusion dynamics study and recommendation system design.
Extraction and Analysis of Fictional Character Networks

A character network is a graph extracted from a narrative in which vertices represent characters and edges correspond to interactions between them. A number of narrative-related problems can be addressed automatically through the analysis of character networks, such as summarization, classification, or role detection. Character networks are particularly relevant when considering works of fiction (e.g., novels, plays, movies, TV series), as their exploitation allows developing information retrieval and recommendation systems. However, works of fiction possess specific properties that make these tasks harder. This survey aims at presenting and organizing the scientific literature related to the extraction of character networks from works of fiction, as well as their analysis. We first describe the extraction process in a generic way and explain how its constituting steps are implemented in practice, depending on the medium of the narrative, the goal of the network analysis, and other factors. We then review the descriptive tools used to characterize character networks, with a focus on the way they are interpreted in this context. We illustrate the relevance of character networks by also providing a review of applications derived from their analysis. Finally, we identify the limitations of the existing approaches and the most promising perspectives.
Prior Activation Distribution (PAD): A Versatile Representation to Utilize DNN Hidden Units

In this paper, we introduce the concept of Prior Activation Distribution (PAD) as a versatile and general technique to capture the typical activation patterns of hidden layer units of a Deep Neural Network used for classification tasks. We show that the combined neural activations of such a hidden layer have class-specific distributional properties, and then define multiple statistical measures to compute how far a test sample's activations deviate from such distributions. Using a variety of benchmark datasets (including MNIST, CIFAR10, Fashion-MNIST & notMNIST), we show how such PAD-based measures can be used, independent of any training technique, to (a) derive fine-grained uncertainty estimates for inferences; (b) provide inferencing accuracy competitive with alternatives that require execution of the full pipeline, and (c) reliably isolate out-of-distribution test samples.
Lepton specific two-Higgs-doublet model based on a 
U(1)X
 gauge symmetry with dark matter

We discuss a two Higgs doublet model with extra $U(1)_X$ gauge symmetry where lepton specific (type-X) structure for Yukawa interactions is realized by charge assignment of fields under the $U(1)_X$. Extra charged leptons are introduced to cancel gauge anomaly associated with extra gauge symmetry. In addition, we introduce scalar fields as dark matter candidates to which we assign $Z_2$ odd parity for guaranteeing stability of them. We then analyze phenomenology of the model such as scalar potential, muon anomalous magnetic dipole moment, collider physics associated with $Z'$ boson from $U(1)_X$, and dark matter physics. Carrying out numerical analysis we search for phenomenologically viable parameter region.
Connecting ANITA anomalous events to a nonthermal dark matter scenario

The ANtarctic Impulsive Transient Antenna (ANITA) collaboration has observed two EeV-energy, upward going events originating from below the horizon. As no standard model (SM) particles, propagating through the earth at such energy and exit angles, can give rise to the required survival probability for the observed events, several beyond standard model (BSM) proposals have come up. We propose a scenario where a $Z_2$ odd sector is responsible for such anomalous events. The next to lightest $Z_2$ odd particle or the next to lightest stable particle (NLSP), created from ultra high energy neutrino interactions with nucleons, can pass through the earth and then decay into the lightest $Z_2$ odd particle or the lightest stable particle (LSP) and a tau lepton. The long-lived nature of the NLSP requires its coupling with the LSP to be very small, keeping the LSP out of thermal equilibrium in the early universe. The LSP can then be a non-thermal dark matter while the tau leptons produced from NLSP decay after passing through the earth can explain the ANITA events. We first show that a purely non-thermal dark matter scenario can not give rise to the required event rates while a hybrid scenario where dark matter can have both thermal as a well non-thermal contribution to its relic abundance, serves the purpose.
Search for heavy neutral Higgs bosons produced in association with 
b
-quarks and decaying into 
b
-quarks at 
s=13  TeV
 with the ATLAS detector

A search for heavy neutral Higgs bosons produced in association with one or two b -quarks and decaying to b -quark pairs is presented using 27 . 8 fb − 1 of ﬃﬃﬃ s p ¼ 13 TeV proton-proton collision data recorded by the ATLAS detector at the Large Hadron Collider during 2015 and 2016. No evidence of a signal is found. Upper limits on the heavy neutral Higgs boson production cross section times its branching ratio to b ¯ b are set, ranging from 4.0 to 0.6 pb at 95% confidence level over a Higgs boson mass range of 450 to 1400 GeV. Results are interpreted within the two-Higgs-doublet model and the minimal supersymmetric Standard Model.
A Methodology for Controlling the Emotional Expressiveness in Synthetic Speech - a Deep Learning approach

In this project, we aim to build a Text-to-Speech system able to produce speech with a controllable emotional expressiveness. We propose a methodology for solving this problem in three main steps. The first is the collection of emotional speech data. We discuss the various formats of existing datasets and their usability in speech generation. The second step is the development of a system to automatically annotate data with emotion/expressiveness features. We compare several techniques using transfer learning to extract such a representation through other tasks and propose a method to visualize and interpret the correlation between vocal and emotional features. The third step is the development of a deep learning-based system taking text and emotion/expressiveness as input and producing speech as output. We study the impact of fine tuning from a neutral TTS towards an emotional TTS in terms of intelligibility and perception of the emotion.
Predicting e-commerce customer conversion from minimal temporal patterns on symbolized clickstream trajectories

Knowing if a user is a buyer or window shopper solely based on clickstream data is of crucial importance for e-commerce platforms seeking to implement real-time accurate NBA (next best action) policies. However, due to the low frequency of conversion events and the noisiness of browsing data, classifying user sessions is very challenging. In this paper, we address the clickstream classification problem in the eCommerce industry and present three major contributions to the burgeoning field of AI-for-retail: first, we collected, normalized and prepared a novel dataset of live shopping sessions from a major European e-commerce website; second, we use the dataset to test in a controlled environment strong baselines and SOTA models from the literature; finally, we propose a new discriminative neural model that outperforms neural architectures recently proposed at Rakuten labs.
AI-based evaluation of the SDGs: The case of crop detection with earth observation data

The framework of the seventeen sustainable development goals is a challenge for developers and researchers applying artificial intelligence (AI). AI and earth observations (EO) can provide reliable and disaggregated data for better monitoring of the sustainable development goals (SDGs). In this paper, we present an overview of SDG targets, which can be effectively measured with AI tools. We identify indicators with the most significant contribution from the AI and EO and describe an application of state-of-the-art machine learning models to one of the indicators. We describe an application of U-net with SE blocks for efficient segmentation of satellite imagery for crop detection. Finally, we demonstrate how AI can be more effectively applied in solutions directly contributing towards specific SDGs and propose further research on an AI-based evaluative infrastructure for SDGs.