nils-herrmann/clean_scirepeval huggingface dataset:
Training Faster by Separating Modes of Variation in Batch-normalized Models

Batch Normalization (BN) is essential to effectively train state-of-the-art deep Convolutional Neural Networks (CNN). It normalizes the layer outputs during training using the statistics of each mini-batch. BN accelerates training procedure by allowing to safely utilize large learning rates and alleviates the need for careful initialization of the parameters. In this work, we study BN from the viewpoint of Fisher kernels that arise from generative probability models. We show that assuming samples within a mini-batch are from the same probability density function, then BN is identical to the Fisher vector of a Gaussian distribution. That means batch normalizing transform can be explained in terms of kernels that naturally emerge from the probability density function that models the generative process of the underlying data distribution. Consequently, it promises higher discrimination power for the batch-normalized mini-batch. However, given the rectifying non-linearities employed in CNN architectures, distribution of the layer outputs show an asymmetric characteristic. Therefore, in order for BN to fully benefit from the aforementioned properties, we propose approximating underlying data distribution not with one, but a mixture of Gaussian densities. Deriving Fisher vector for a Gaussian Mixture Model (GMM), reveals that batch normalization can be improved by independently normalizing with respect to the statistics of disentangled sub-populations. We refer to our proposed soft piecewise version of batch normalization as Mixture Normalization (MN). Through extensive set of experiments on CIFAR-10 and CIFAR-100, using both a 5-layers deep CNN and modern Inception-V3 architecture, we show that mixture normalization reduces required number of gradient updates to reach the maximum test accuracy of the batch normalized model by ∼31%-47% across a variety of training scenarios. Replacing even a few BN modules with MN in the 48-layers deep Inception-V3 architecture is sufficient to not only obtain considerable training acceleration but also better final test accuracy. We show that similar observations are valid for 40 and 100-layers deep DenseNet architectures as well. We complement our study by evaluating the application of mixture normalization to the Generative Adversarial Networks (GANs), where "mode collapse" hinders the training process. We solely replace a few batch normalization layers in the generator with our proposed mixture normalization. Our experiments using Deep Convolutional GAN (DCGAN) on CIFAR-10 show that mixture normalized DCGAN not only provides an acceleration of ∼58% but also reaches lower (better) "Fréchet Inception Distance" (FID) of 33.35 compared to 37.56 of its batch normalized counterpart.
Active learning for online bayesian matrix factorization

The problem of large-scale online matrix completion is addressed via a Bayesian approach. The proposed method learns a factor analysis (FA) model for large matrices, based on a small number of observed matrix elements, and leverages the statistical model to actively select which new matrix entries/observations would be most informative if they could be acquired, to improve the model; the model inference and active learning are performed in an online setting. In the context of online learning, a greedy, fast and provably near-optimal algorithm is employed to sequentially maximize the mutual information between past and future observations, taking advantage of submodularity properties. Additionally, a simpler procedure, which directly uses the posterior parameters learned by the Bayesian approach, is shown to achieve slightly lower estimation quality, with far less computational effort. Inference is performed using a computationally efficient online variational Bayes (VB) procedure. Competitive results are obtained in a very large collaborative filtering problem, namely the Yahoo! Music ratings dataset.
The base-rate fallacy and the difficulty of intrusion detection

Many different demands can be made of intrusion detection systems. An important requirement is that an intrusion detection system be effective; that is, it should detect a substantial percentage of intrusions into the supervised system, while still keeping the false alarm rate at an acceptable level. This article demonstrates that, for a reasonable set of assumptions, the false alarm rate is the limiting factor for the performance of an intrusion detection system. This is due to the base-rate fallacy phenomenon, that in order to achieve substantial values of the Bayesian detection rate P(Intrusion***Alarm), we have to achieve a (perhaps in some cases unattainably) low false alarm rate. A selection of reports of intrusion detection performance are reviewed, and the conclusion is reached that there are indications that at least some types of intrusion detection have far to go before they can attain such low false alarm rates.
A Taxonomy of Workflow Management Systems for Grid Computing

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.
Design of a new broadband EMC double ridged guide horn antenna

A new design for broadband EMC double ridged guide horn (DRGH) antennas is presented. A conventional 1-18 GHz double ridged guide horn has been investigated rigorously. Then some modifications have been performed in the structure of the antenna. Elimination of radiation pattern deficiencies especially at higher frequencies accompanying with better EM characteristics of the antenna have been the main purposes for these modifications. The main modifications are imposed on the profile of ridges, H-plane flares and E-plane flares. The resulting antenna not only has considerably better performance but also has smaller physical dimensions and less weight in comparison with the conventional one.
Robotic grasp detection using deep convolutional neural networks

Deep learning has significantly advanced computer vision and natural language processing. While there have been some successes in robotics using deep learning, it has not been widely adopted. In this paper, we present a novel robotic grasp detection system that predicts the best grasping pose of a parallel-plate robotic gripper for novel objects using the RGB-D image of the scene. The proposed model uses a deep convolutional neural network to extract features from the scene and then uses a shallow convolutional neural network to predict the grasp configuration for the object of interest. Our multi-modal model achieved an accuracy of 89.21% on the standard Cornell Grasp Dataset and runs at real-time speeds. This redefines the state-of-the-art for robotic grasp detection.
Comparison of Sentinel-1 and TerraSAR-X for ship detection

Sentinel-1A (S1-A) is the first of a pair of satellites operating a C-band Synthetic Aperture Radar (SAR) developed to give continuity to the European SAR programme. After 6 months commissioning phase from its lunch on April 3rd 2014, S1-A started to systematically deliver data to Copernicus Ocean, Land and Emergency services. Interferometric Wide Swath (IWS) mode in dual-polarization VV+VH is the standard acquisition mode used for the observation of the European marine environment. The objectives of this study are the comparison and synergetic use of the S1-A imagery in IWS mode with TerraSAR-X (TS-X) imagery in StripMap mode for SAR ship detection application. Exemplary dataset acquired over the English Channel during a controlled experiment is used to pursue the objective. Automatic Identification System ships messages received by terrestrial stations in the area are used as ground truth and to identify the SAR detections.
Content-based book recommending using learning for text categorization

Recommender systems improve access to relevant products and information by making personalized suggestions based on previous examples of a user's likes and dislikes. Most existing recommender systems use collaborative filtering methods that base recommendations on other users' preferences. By contrast,content-based methods use information about an item itself to make suggestions.This approach has the advantage of being able to recommend previously unrated items to users with unique interests and to provide explanations for its recommendations. We describe a content-based book recommending system that utilizes information extraction and a machine-learning algorithm for text categorization. Initial experimental results demonstrate that this approach can produce accurate recommendations.
Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time

Human Affectiveness, i.e., the emotional state of a person, plays a crucial role in many domains where it can make or break a team's ability to produce successful products. Software development is a collaborative activity as well, yet there is little information on how affectiveness impacts software productivity. As a first measure of this impact, this paper analyzes the relation between sentiment, emotions and politeness of developers in more than 560K Jira comments with the time to fix a Jira issue. We found that the happier developers are (expressing emotions such as JOY and LOVE in their comments), the shorter the issue fixing time is likely to be. In contrast, negative emotions such as SADNESS, are linked with longer issue fixing time. Politeness plays a more complex role and we empirically analyze its impact on developers' productivity.
The organization of intrinsic computation: complexity-entropy diagrams and the diversity of natural information processing.

Intrinsic computation refers to how dynamical systems store, structure, and transform historical and spatial information. By graphing a measure of structural complexity against a measure of randomness, complexity-entropy diagrams display the different kinds of intrinsic computation across an entire class of systems. Here, we use complexity-entropy diagrams to analyze intrinsic computation in a broad array of deterministic nonlinear and linear stochastic processes, including maps of the interval, cellular automata, and Ising spin systems in one and two dimensions, Markov chains, and probabilistic minimal finite-state machines. Since complexity-entropy diagrams are a function only of observed configurations, they can be used to compare systems without reference to system coordinates or parameters. It has been known for some time that in special cases complexity-entropy diagrams reveal that high degrees of information processing are associated with phase transitions in the underlying process space, the so-called "edge of chaos." Generally, though, complexity-entropy diagrams differ substantially in character, demonstrating a genuine diversity of distinct kinds of intrinsic computation.
Proving Theorems about LISP Functions

We d e s c r i b e some s i m p l e h e u r i s t i c s comb in ing e v a l u a t i o n and ma themat i ca l i n d u c t i o n wh ich we have implemented in a program t h a t a u t o m a t i c a l l y p roves a wide v a r i e t y o f theorems about r e c u r s i v e LISP f u n c t i o n s . The method the program uses t o gene ra te i n d u c t i o n f o r m u l a s i s d e s c r i b e d a t l e n g t h . The theorems proved by t h e p r o ­ gram i n c l u d e t h a t REVERSE i s i t s own i n v e r s e and t h a t a p a r t i c u l a r SORT program is c o r r e c t . Append ix B c o n t a i n s a l i s t o f the theorems p roved by t h e p r o g r a m .
The Structure and Function of Complex Networks

Inspired by empirical studies of networked systems such as the Internet, social networks, and biological networks, researchers have in recent years developed a variety of techniques and models to help us understand or predict the behavior of these systems. Here we review developments in this field, including such concepts as the small-world effect, degree distributions, clustering, network correlations, random graph models, models of network growth and preferential attachment, and dynamical processes taking place on networks.
Marine Litter Distribution and Density in European Seas, from the Shelves to Deep Basins

Anthropogenic litter is present in all marine habitats, from beaches to the most remote points in the oceans. On the seafloor, marine litter, particularly plastic, can accumulate in high densities with deleterious consequences for its inhabitants. Yet, because of the high cost involved with sampling the seafloor, no large-scale assessment of distribution patterns was available to date. Here, we present data on litter distribution and density collected during 588 video and trawl surveys across 32 sites in European waters. We found litter to be present in the deepest areas and at locations as remote from land as the Charlie-Gibbs Fracture Zone across the Mid-Atlantic Ridge. The highest litter density occurs in submarine canyons, whilst the lowest density can be found on continental shelves and on ocean ridges. Plastic was the most prevalent litter item found on the seafloor. Litter from fishing activities (derelict fishing lines and nets) was particularly common on seamounts, banks, mounds and ocean ridges. Our results highlight the extent of the problem and the need for action to prevent increasing accumulation of litter in marine environments.
Routing in multi-radio, multi-hop wireless mesh networks

We present a new metric for routing in multi-radio, multi-hop wireless networks. We focus on wireless networks with stationary nodes, such as community wireless networks.The goal of the metric is to choose a high-throughput path between a source and a destination. Our metric assigns weights to individual links based on the Expected Transmission Time (ETT) of a packet over the link. The ETT is a function of the loss rate and the bandwidth of the link. The individual link weights are combined into a path metric called Weighted Cumulative ETT (WCETT) that explicitly accounts for the interference among links that use the same channel. The WCETT metric is incorporated into a routing protocol that we call Multi-Radio Link-Quality Source Routing.We studied the performance of our metric by implementing it in a wireless testbed consisting of 23 nodes, each equipped with two 802.11 wireless cards. We find that in a multi-radio environment, our metric significantly outperforms previously-proposed routing metrics by making judicious use of the second radio.
On partial least squares in head pose estimation: How to simultaneously deal with misalignment

Head pose estimation is a critical problem in many computer vision applications. These include human computer interaction, video surveillance, face and expression recognition. In most prior work on heads pose estimation, the positions of the faces on which the pose is to be estimated are specified manually. Therefore, the results are reported without studying the effect of misalignment. We propose a method based on partial least squares (PLS) regression to estimate pose and solve the alignment problem simultaneously. The contributions of this paper are two-fold: 1) we show that the kernel version of PLS (kPLS) achieves better than state-of-the-art results on the estimation problem and 2) we develop a technique to reduce misalignment based on the learned PLS factors.
Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization

In the paper we present new Alternating Least Squares (ALS) algorithms for Nonnegative Matrix Factorization (NMF) and their extensions to 3D Nonnegative Tensor Factorization (NTF) that are robust in the presence of noise and have many potential applications, including multi-way Blind Source Separation (BSS), multi-sensory or multi-dimensional data analysis, and nonnegative neural sparse coding. We propose to use local cost functions whose simultaneous or sequential (one by one) minimization leads to a very simple ALS algorithm which works under some sparsity constraints both for an under-determined (a system which has less sensors than sources) and over-determined model. The extensive experimental results confirm the validity and high performance of the developed algorithms, especially with usage of the multi-layer hierarchical NMF. Extension of the proposed algorithm to multidimensional Sparse Component Analysis and Smooth Component Analysis is also proposed.
Learning methods for generic object recognition with invariance to pose and lighting

We assess the applicability of several popular learning methods for the problem of recognizing generic visual categories with invariance to pose, lighting, and surrounding clutter. A large dataset comprising stereo image pairs of 50 uniform-colored toys under 36 azimuths, 9 elevations, and 6 lighting conditions was collected (for a total of 194,400 individual images). The objects were 10 instances of 5 generic categories: four-legged animals, human figures, airplanes, trucks, and cars. Five instances of each category were used for training, and the other five for testing. Low-resolution grayscale images of the objects with various amounts of variability and surrounding clutter were used for training and testing. Nearest neighbor methods, support vector machines, and convolutional networks, operating on raw pixels or on PCA-derived features were tested. Test error rates for unseen object instances placed on uniform backgrounds were around 13% for SVM and 7% for convolutional nets. On a segmentation/recognition task with highly cluttered images, SVM proved impractical, while convolutional nets yielded 16/7% error. A real-time version of the system was implemented that can detect and classify objects in natural scenes at around 10 frames per second.
Standardization of the in-car gesture interaction space

Driven by technological advancements, gesture interfaces have recently found their way into vehicular prototypes of various kind. Unfortunately, their application is less than perfect and detailed information about preferred gesture execution regions, spatial extent, and time behavior are not available yet. Providing car (interior) manufacturer with gesture characteristics would allow them to design future in-vehicle concepts in a way to not interfere with gestural interaction. To tackle the problem, this research aims as preliminary work for a later standardization of the diverse properties of gestures and gesture classes similarly to what is already standardized in norms such as ISO 3958/4040 for placement and reachability of traditional controls and indicators. We have set up a real driving experiment recording trajectories and time behavior of gestures related to car and media control tasks. Data evaluation reveals that most of the subjects perform gestures in the same region (bounded by a "triangle" steering wheel, rear mirror, and gearshift) and with similar spatial extent (on average below 2 sec.). The generated density plots can be further used for an initial discussion about gesture execution in the passenger compartment. The final aim is to propose a new standard on permitted gesture properties (time, space) in the car.
Navigating nets: simple algorithms for proximity search

We present a simple deterministic data structure for maintaining a set S of points in a general metric space, while supporting proximity search (nearest neighbor and range queries) and updates to S (insertions and deletions). Our data structure consists of a sequence of progressively finer ε-nets of S, with pointers that allow us to navigate easily from one scale to the next.We analyze the worst-case complexity of this data structure in terms of the "abstract dimensionality" of the metric S. Our data structure is extremely efficient for metrics of bounded dimension and is essentially optimal in a certain model of distance computation. Finally, as a special case, our approach improves over one recently devised by Karger and Ruhl [KR02].
Document Image Quality Assessment: A Brief Survey

To maintain, control and enhance the quality of document images and minimize the negative impact of degradations on various analysis and processing systems, it is critical to understand the types and sources of degradations and develop reliable methods for estimating the levels of degradations. This paper provides a brief survey of research on the topic of document image quality assessment. We first present a detailed analysis of the types and sources of document degradations. We then review techniques for document image degradation modeling. Finally, we discuss objective measures and subjective experiments that are used to characterize document image quality.
A Path-constrained Framework for Discriminating Substitutable and Complementary Products in E-commerce

In personalized recommendation, candidate generation plays an infrastructural role by retrieving candidates out of billions of items. During this process, substitutes and complements constitute two main classes of retrieved candidates: substitutable products are interchangeable, whereas complementary products might be purchased together by users. Discriminating substitutable and complementary products is playing an increasingly important role in e-commerce portals by affecting the performance of candidate generation, e.g., when a user has browsed a t-shirt, it is reasonable to retrieve similar t-shirts, i.e., substitutes; whereas if the user has already purchased one, it would be better to retrieve trousers, hats or shoes, as complements of t-shirts. In this paper, we propose a path-constrained framework (PMSC) for discriminating substitutes and complements. Specifically, for each product, we first learn its embedding representations in a general semantic space. Thereafter, we project the embedding vectors into two separate spaces via a novel mapping function. In the end, we incorporate each embedding with path-constraints to further boost the discriminative ability of the model. Extensive experiments conducted on two e-commerce datasets show the effectiveness of our proposed method.
Supervised Dictionary Learning by a Variational Bayesian Group Sparse Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) with group sparsity constraints is formulated as a probabilistic graphical model and, assuming some observed data have been generated by the model, a feasible variational Bayesian algorithm is derived for learning model parameters. When used in a supervised learning scenario, NMF is most often utilized as an unsupervised feature extractor followed by classification in the obtained feature subspace. Having mapped the class labels to a more general concept of groups which underlie sparsity of the coefficients, what the proposed group sparse NMF model allows is incorporating class label information to find low dimensional label-driven dictionaries which not only aim to represent the data faithfully, but are also suitable for class discrimination. Experiments performed in face recognition and facial expression recognition domains point to advantages of classification in such label-driven feature subspaces over classification in feature subspaces obtained in an unsupervised manner.