nils-herrmann/clean_scirepeval huggingface dataset:
Gradual DropIn of Layers to Train Very Deep Neural Networks

We introduce the concept of dynamically growing a neural network during training. In particular, an untrainable deep network starts as a trainable shallow network and newly added layers are slowly, organically added during training, thereby increasing the network's depth. This is accomplished by a new layer, which we call DropIn. The DropIn layer starts by passing the output from a previous layer (effectively skipping over the newly added layers), then increasingly including units from the new layers for both feedforward and backpropagation. We show that deep networks, which are untrainable with conventional methods, will converge with DropIn layers interspersed in the architecture. In addition, we demonstrate that DropIn provides regularization during training in an analogous way as dropout. Experiments are described with the MNIST dataset and various expanded LeNet architectures, CIFAR-10 dataset with its architecture expanded from 3 to 11 layers, and on the ImageNet dataset with the AlexNet architecture expanded to 13 layers and the VGG 16-layer architecture.
Real-Time EEG-Based Human Emotion Recognition and Visualization

Emotions accompany everyone in the daily life, playing a key role in non-verbal communication, and they are essential to the understanding of human behavior. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of “inner” emotions from electroencephalogram (EEG) signals as humans could control their facial expressions or vocal intonation. The need and importance of the automatic emotion recognition from EEG signals has grown with increasing role of brain computer interface applications and development of new forms of human-centric and human-driven interaction with digital media. We propose fractal dimension based algorithm of quantification of basic emotions and describe its implementation as a feedback in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called “emotion dimension” to human computer interfaces.
FALKON: An Optimal Large Scale Kernel Method

Kernel methods provide a principled way to perform non linear, nonparametric learning. They rely on solid functional analytic foundations and enjoy optimal statistical properties. However, at least in their basic form, they have limited applicability in large scale scenarios because of stringent computational requirements in terms of time and especially memory. In this paper, we take a substantial step in scaling up kernel methods, proposing FALKON, a novel algorithm that allows to efficiently process millions of points. FALKON is derived combining several algorithmic principles, namely stochastic subsampling, iterative solvers and preconditioning. Our theoretical analysis shows that optimal statistical accuracy is achieved requiring essentially O(n) memory and O(n √ n) time. An extensive experimental analysis on large scale datasets shows that, even with a single machine, FALKON outperforms previous state of the art solutions, which exploit parallel/distributed architectures.
Combinatorial Attacks on Binarized Neural Networks

q Adversarial Attack: a small perturbation to an input that fools ML model into predicting incorrect output (STOP→YIELD) q Attack-to-Protect: designing better algorithms for generating adversarial perturbations helps train robust ML models q ML model ⎯ Binarized Neural Network (BNN): •  Activation: sign & ∈ {−1,+1} q Why care about BNNs? • Fast inference / Small size • Great for low-power devices, smartphones
GLoMo: Unsupervised Learning of Transferable Relational Graphs

Modern deep transfer learning approaches have mainly focused on learning generic feature vectors from one task that are transferable to other tasks, such as word embeddings in language and pretrained convolutional features in vision. However, these approaches usually transfer unary features and largely ignore more structured graphical representations. This work explores the possibility of learning generic latent relational graphs that capture dependencies between pairs of data units (e.g., words or pixels) from large-scale unlabeled data and transferring the graphs to downstream tasks. Our proposed transfer learning framework improves performance on various tasks including question answering, natural language inference, sentiment analysis, and image classification. We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden units), or embedding-free units such as image pixels.
Wearable Wireless Health Monitoring: Current Developments, Challenges, and Future Trends

Wearable health-monitoring systems are becoming very popular, especially in enabling the noninvasive diagnosis of vital functions of the human body. Besides typical singular heartbeat or perspiration sensors, which have been commercially available in recent years, the deployment of a series of body-worn sensors can enable an effective health-monitoring mechanism. The combined information obtained from such systems can either be relayed directly to any health-monitoring personnel in the case of emergencies or can be logged and analyzed as a part of preventive health measures. However, the deployment of on-body nodes for humans must be performed with care, as they may interfere with the patients' regular movements. This is especially challenging because the relationship between the electromagnetic waves is influenced by the patient's movements, distance from the nearest base station, operating environment, etc. Additional challenges to the deployment of such mechanisms are also faced in situations where the nodes require additional on-body space, impose additional weight, or are not conformal enough to the patient's body. On the hardware design aspect, the sensory and communication functions on the electronic node have to be designed using special materials to avoid reliability issues or damage due to repeated or intense movements. Finally, and perhaps the most important aspect that needs to be addressed concerning such systems, is their electromagnetic safety level, which is defined by their specific absorption rates (SARs). This article aims to review the latest developments in body-worn wireless health-monitoring systems and their current challenges and limitations and to discuss future trends for such worn devices for these applications.
The influence of implicit and explicit biofeedback in first-person shooter games

To understand how implicit and explicit biofeedback work in games, we developed a first-person shooter (FPS) game to experiment with different biofeedback techniques. While this area has seen plenty of discussion, there is little rigorous experimentation addressing how biofeedback can enhance human-computer interaction. In our two-part study, (N=36) subjects first played eight different game stages with two implicit biofeedback conditions, with two simulation-based comparison and repetition rounds, then repeated the two biofeedback stages when given explicit information on the biofeedback. The biofeedback conditions were respiration and skin-conductance (EDA) adaptations. Adaptation targets were four balanced player avatar attributes. We collected data with psycho¬physiological measures (electromyography, respiration, and EDA), a game experience questionnaire, and game-play measures.
 According to our experiment, implicit biofeedback does not produce significant effects in player experience in an FPS game. In the explicit biofeedback conditions, players were more immersed and positively affected, and they were able to manipulate the game play with the biosignal interface. We recommend exploring the possibilities of using explicit biofeedback interaction in commercial games.
A Capacity Scaling Law for Artificial Neural Networks

In this article, we derive the calculation of two critical numbers that quantify the capabilities of artificial neural networks with gating functions, such as sign, sigmoid, or rectified linear units. First, we derive the calculation of the Vapnik–Chervonenkis (VC) dimension of a network with binary output layer, which is the theoretical limit for perfect fitting of the training data. Second, we derive what we call the MacKay dimension of the network. This is a theoretical limit indicating necessary catastrophic forgetting i.e., the upper limit for most uses of the network. Our derivation of the capacity is embedded into Shannon’s communication model, which allows measuring the capacities of neural networks in bits. We then compare our theoretical derivations with experiments using different network configurations, diverse neural network implementations, varying activation functions, and several learning algorithms to confirm our upper bound. The result is that the capacity of a fully connected perceptron network scales strictly linear with the number of weights.
Weakly-Supervised Deep Embedding for Product Review Sentiment Analysis

Product reviews are valuable for upcoming buyers in helping them make decisions. To this end, different opinion mining techniques have been proposed, where judging a review sentence's orientation (e.g., positive or negative) is one of their key challenges. Recently, deep learning has emerged as an effective means for solving sentiment classification problems. A neural network intrinsically learns a useful representation automatically without human efforts. However, the success of deep learning highly relies on the availability of large-scale training data. We propose a novel deep learning framework for product review sentiment classification which employs prevalently available ratings as weak supervision signals. The framework consists of two steps: (1) learning a high level representation (an embedding space) which captures the general sentiment distribution of sentences through rating information; and (2) adding a classification layer on top of the embedding layer and use labeled sentences for supervised fine-tuning. We explore two kinds of low level network structure for modeling review sentences, namely, convolutional feature extractors and long short-term memory. To evaluate the proposed framework, we construct a dataset containing 1.1M weakly labeled review sentences and 11,754 labeled review sentences from Amazon. Experimental results show the efficacy of the proposed framework and its superiority over baselines.
A Robust and Scalable Peer-to-Peer Gossiping Protocol

The newscast model is a general approach for communication in large agent-based distributed systems. The two basic services— membership management and information dissemination—are implemented by the same epidemic-style protocol. In this paper we present the newscast model and report on experiments using a Java implementation. The experiments involve communication in a large, wide-area cluster computer. By analysis of the outcome of the experiments we demonstrate that the system indeed shows the scalability and dependability properties predicted by our previous theoretical and simulation results.
Singing Voice Separation with Deep U-Net Convolutional Networks

The decomposition of a music audio signal into its vocal and backing track components is analogous to image-toimage translation, where a mixed spectrogram is transformed into its constituent sources. We propose a novel application of the U-Net architecture — initially developed for medical imaging — for the task of source separation, given its proven capacity for recreating the fine, low-level detail required for high-quality audio reproduction. Through both quantitative evaluation and subjective assessment, experiments demonstrate that the proposed algorithm achieves state-of-the-art performance.
Modifiable risk factors in the first 1000 days for subsequent risk of childhood overweight in an Asian cohort: significance of parental overweight status

Many studies have identified early-life risk factors for subsequent childhood overweight/obesity, but few have evaluated how they combine to influence risk of childhood overweight/obesity. We examined associations, individually and in combination, of potentially modifiable risk factors in the first 1000 days after conception with childhood adiposity and risk of overweight/obesity in an Asian cohort.Six risk factors were examined: maternal pre-pregnancy overweight/obesity (body mass index (BMI) ⩾25 kg m−2), paternal overweight/obesity at 24 months post delivery, maternal excessive gestational weight gain, raised maternal fasting glucose during pregnancy (⩾5.1 mmol l−1), breastfeeding duration <4 months and early introduction of solid foods (<4 months). Associations between number of risk factors and adiposity measures (BMI, waist-to-height ratio (WHtR), sum of skinfolds (SSFs), fat mass index (FMI) and overweight/obesity) at 48 months were assessed using multivariable regression models.Of 858 children followed up at 48 months, 172 (19%) had none, 274 (32%) had 1, 244 (29%) had 2, 126 (15%) had 3 and 42 (5%) had ⩾4 risk factors. Adjusting for confounders, significant graded positive associations were observed between number of risk factors and adiposity outcomes at 48 months. Compared with children with no risk factors, those with four or more risk factors had s.d. unit increases of 0.78 (95% confidence interval 0.41–1.15) for BMI, 0.79 (0.41–1.16) for WHtR, 0.46 (0.06–0.83) for SSF and 0.67 (0.07–1.27) for FMI. The adjusted relative risk of overweight/obesity in children with four or more risk factors was 11.1(2.5–49.1) compared with children with no risk factors. Children exposed to maternal pre-pregnancy (11.8(9.8–13.8)%) or paternal overweight status (10.6(9.6-11.6)%) had the largest individual predicted probability of child overweight/obesity.Early-life risk factors added cumulatively to increase childhood adiposity and risk of overweight/obesity. Early-life and preconception intervention programmes may be more effective in preventing overweight/obesity if they concurrently address these multiple modifiable risk factors.
An End-to-End TextSpotter with Explicit Alignment and Attention

Text detection and recognition in natural images have long been considered as two separate tasks that are processed sequentially. Jointly training two tasks is non-trivial due to significant differences in learning difficulties and convergence rates. In this work, we present a conceptually simple yet efficient framework that simultaneously processes the two tasks in a united framework. Our main contributions are three-fold: (1) we propose a novel text-alignment layer that allows it to precisely compute convolutional features of a text instance in arbitrary orientation, which is the key to boost the performance; (2) a character attention mechanism is introduced by using character spatial information as explicit supervision, leading to large improvements in recognition; (3) two technologies, together with a new RNN branch for word recognition, are integrated seamlessly into a single model which is end-to-end trainable. This allows the two tasks to work collaboratively by sharing convolutional features, which is critical to identify challenging text instances. Our model obtains impressive results in end-to-end recognition on the ICDAR 2015 , significantly advancing the most recent results , with improvements of F-measure from (0.54, 0.51, 0.47) to (0.82, 0.77, 0.63), by using a strong, weak and generic lexicon respectively. Thanks to joint training, our method can also serve as a good detector by achieving a new state-of-the-art detection performance on related benchmarks. Code is available at https://github.com/tonghe90/textspotter.
A Method for Analyzing a Linear Series-Fed Rectangular Microstrip Antenna Array

A novel method of analysis for a linear series-fed microstrip antenna array is developed, which is based on a set of canonical coefficients defined for the elements of the array. This method accounts for the mutual coupling between the elements, and allows for a design that has an arbitrary amplitude and phase of the radiating patch currents. This method has the simplicity of a CAD approach while maintaining an accuracy close to that of a full-wave method. The coefficients involved in the formulation are determined by full-wave simulation on either a single patch element or on two patch elements (for mutual coupling calculations).
The gender similarities hypothesis.

The differences model, which argues that males and females are vastly different psychologically, dominates the popular media. Here, the author advances a very different view, the gender similarities hypothesis, which holds that males and females are similar on most, but not all, psychological variables. Results from a review of 46 meta-analyses support the gender similarities hypothesis. Gender differences can vary substantially in magnitude at different ages and depend on the context in which measurement occurs. Overinflated claims of gender differences carry substantial costs in areas such as the workplace and relationships.
Distance-based Confidence Score for Neural Network Classifiers

The reliable measurement of confidence in classifiers’ predictions is very important for many applications, and is therefore an important part of classifier design. Yet, although deep learning has received tremendous attention in recent years, not much progress has been made in quantifying the prediction confidence of neural network classifiers. Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with prohibitive computational costs. In this paper we propose a simple, scalable method to achieve a reliable confidence score, based on the data embedding derived from the penultimate layer of the network. We investigate two ways to achieve desirable embeddings, by using either a distance-based loss or Adversarial Training. We then test the benefits of our method when used for classification error prediction, weighting an ensemble of classifiers, and novelty detection. In all tasks we show significant improvement over traditional, commonly used confidence scores.
Dynamic Heterogeneous Learning Games for Opportunistic Access in LTE-Based Macro/Femtocell Deployments

Interference is one of the most limiting factors when trying to achieve high spectral efficiency in the deployment of heterogeneous networks (HNs). In this paper, the HN is modeled as a layer of closed-access LTE femtocells (FCs) overlaid upon an LTE radio access network. Within the context of dynamic learning games, this work proposes a novel heterogeneous multiobjective fully distributed strategy based on a reinforcement learning (RL) model (CODIPAS-HRL) for FC self-configuration/optimization. The self-organization capability enables the FCs to autonomously and opportunistically sense the radio environment using different learning strategies and tune their parameters accordingly, in order to operate under restrictions of avoiding interference to both network tiers and satisfy certain quality-of-service requirements. The proposed model reduces the learning cost associated with each learning strategy. We also study the convergence behavior under different learning rates and derive a new accuracy metric in order to provide comparisons between the different learning strategies. The simulation results show the convergence of the learning model to a solution concept based on satisfaction equilibrium, under the uncertainty of the HN environment. We show that intra/inter-tier interference can be significantly reduced, thus resulting in higher cell throughputs.
PointConv: Deep Convolutional Networks on 3D Point Clouds

Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the dynamic filter to a new convolution operation, named PointConv. PointConv can be applied on point clouds to build deep convolutional networks. We treat convolution kernels as nonlinear functions of the local coordinates of 3D points comprised of weight and density functions. With respect to a given point, the weight functions are learned with multi-layer perceptron networks and the density functions through kernel density estimation. A novel reformulation is proposed for efficiently computing the weight functions, which allowed us to dramatically scale up the network and significantly improve its performance. The learned convolution kernel can be used to compute translation-invariant and permutation-invariant convolution on any point set in the 3D space. Besides, PointConv can also be used as deconvolution operators to propagate features from a subsampled point cloud back to its original resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep convolutional neural networks built on PointConv are able to achieve state-ofthe-art on challenging semantic segmentation benchmarks on 3D point clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed that networks built on PointConv can match the performance of convolutional networks in 2D images of a similar structure.
Dual Transfer Learning for Neural Machine Translation with Marginal Distribution Regularization

Neural machine translation (NMT) heavily relies on parallel bilingual data for training. Since large-scale, high-quality parallel corpora are usually costly to collect, it is appealing to exploit monolingual corpora to improve NMT. Inspired by the law of total probability, which connects the probability of a given target-side monolingual sentence to the conditional probability of translating from a source sentence to the target one, we propose to explicitly exploit this connection to learn from and regularize the training of NMT models using monolingual data. The key technical challenge of this approach is that there are exponentially many source sentences for a target monolingual sentence while computing the sum of the conditional probability given each possible source sentence. We address this challenge by leveraging the dual translation model (target-to-source translation) to sample several mostly likely source-side sentences and avoid enumerating all possible candidate source sentences. That is, we transfer the knowledge contained in the dual model to boost the training of the primal model (source-to-target translation), and we call such an approach dual transfer learning. Experiment results on English→French and German→English tasks demonstrate that dual transfer learning achieves significant improvement over several strong baselines and obtains new state-of-the-art results.
Multi-instance multi-label learning

In this paper, we propose the MIML (Multi-Instance Multi-Label learning) framework where an example is described by multiple instances and associated with multiple class labels. Compared to traditional learning frameworks, the MIML framework is more convenient and natural for representing complicated objects which have multiple semantic meanings. To learn from MIML examples, we propose the MimlBoost and MimlSvm algorithms based on a simple degeneration strategy, and experiments show that solving problems involving complicated objects with multiple semantic meanings in the MIML framework can lead to good performance. Considering that the degeneration process may lose information, we propose the D-MimlSvm algorithm which tackles MIML problems directly in a regularization framework. Moreover, we show that even when we do not have access to the real objects and thus cannot capture more information from real objects by using the MIML representation, MIML is still useful. We propose the InsDif and SubCod algorithms. InsDif works by transforming single-instances into the MIML representation for learning, while SubCod works by transforming single-label examples into the MIML representation for learning. Experiments show that in some tasks they are able to achieve better performance than learning the single-instances or single-label examples directly.