nils-herrmann/clean_scirepeval huggingface dataset:
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks

Intermediate features at different layers of a deep neural network are known to be discriminative for visual patterns of different complexities. However, most existing works ignore such cross-layer heterogeneities when classifying samples of different complexities. For example, if a training sample has already been correctly classified at a specific layer with high confidence, we argue that it is unnecessary to enforce rest layers to classify this sample correctly and a better strategy is to encourage those layers to focus on other samples. In this paper, we propose a layer-wise discriminative learning method to enhance the discriminative capability of a deep network by allowing its layers to work collaboratively for classification. Towards this target, we introduce multiple classifiers on top of multiple layers. Each classifier not only tries to correctly classify the features from its input layer, but also coordinates with other classifiers to jointly maximize the final classification performance. Guided by the other companion classifiers, each classifier learns to concentrate on certain training examples and boosts the overall performance. Allowing for end-to-end training, our method can be conveniently embedded into state-of-the-art deep networks. Experiments with multiple popular deep networks, including Network in Network, GoogLeNet and VGGNet, on scale-various object classification benchmarks, including CIFAR100, MNIST and ImageNet, and scene classification benchmarks, including MIT67, SUN397 and Places205, demonstrate the effectiveness of our method. In addition, we also analyze the relationship between the proposed method and classical conditional random fields models.
Estimating average growth trajectories in shape-space using kernel smoothing

In this paper, we show how a dense surface point distribution model of the human face can be computed and demonstrate the usefulness of the high-dimensional shape-space for expressing the shape changes associated with growth and aging. We show how average growth trajectories for the human face can be computed in the absence of longitudinal data by using kernel smoothing across a population. A training set of three-dimensional surface scans of 199 male and 201 female subjects of between 0 and 50 years of age is used to build the model.
Node Representation Learning for Multiple Networks: The Case of Graph Alignment

Recent advances in representation learning produce node embeddings that may be used successfully in many downstream tasks (e.g., link prediction), but do not generally extend beyond a single network. Motivated by the prevalence of multi-network problems, such as graph alignment, similarity, and transfer learning, we introduce an elegant and principled node embedding formulation, Cross-network Matrix Factorization or xNetMF, to effectively and efficiently learn embeddings from similarities based on the nodes’ structural and attribute identities (if available), which are comparable across networks. Leveraging the Nyström method for low-rank approximation, we achieve significant reduction in space and time compared to the naïve approach of directly factorizing the node similarity matrix, with xNetMF running up to 30× faster than existing representation learning methods. We study its effectiveness on the task of network alignment, and introduce REGAL (REpresentationbased Graph ALignment), the first method to use node representation learning for this task. Extensive experiments on a variety of challenging real-world networks with and without attributes show that REGAL outperforms existing network alignment methods by 20 to 30% in accuracy on average, uses attributes up to 40× faster, and scales to networks with millions of nodes each.
The role of Kaizen (continuous improvement) in improving companies' performance: A case study

Kaizen refers to the philosophy or practices that focus upon continuous improvement of processes in manufacturing, engineering and business management. The improvements are usually accomplished at little or no expense and without sophisticated techniques. The purpose of this paper is to present the results of implementing the Kaizen principles at Port Installed Options Center in Toyota Saudi Arabia. Using relevant Kaizen tools including Toyota Production System (TPS), 5S and the seven Muda enabled us to understand the system and identify the most critical problem areas. The Kaizen target was set after analyzing the problems and identifying the gap between the existing system and the TPS. The main results of the study can be summarized as: using the same facilities it was possible to be more efficient by reducing the manpower needed by 26.9% (from 349 to 275 installers) and more effective by increasing the annual output by 13% (from 188000 to 212400 vehicles); improve the Associates Engagement Index (Q12) by 6.4% (from 2.91 to 3.11); potential for inventory reduction due to the use of Kanban system with just-in-time production; avoid investing in new facilities needed to meet the increase in the demand in the market. All these resulted in improving the production line productivity which were appreciated and acknowledged by Toyota Motor Corporation.
Taste, olfactory, and food reward value processing in the brain

Complementary neuronal recordings in primates, and functional neuroimaging in humans, show that the primary taste cortex in the anterior insula provides separate and combined representations of the taste, temperature, and texture (including fat texture) of food in the mouth independently of hunger and thus of reward value and pleasantness. One synapse on, in a second tier of processing, in the orbitofrontal cortex, these sensory inputs are for some neurons combined by associative learning with olfactory and visual inputs, and these neurons encode food reward value on a continuous scale in that they only respond to food when hungry, and in that activations correlate linearly with subjective pleasantness. Cognitive factors, including word-level descriptions, and selective attention to affective value, modulate the representation of the reward value of taste and olfactory stimuli in the orbitofrontal cortex and a region to which it projects, the anterior cingulate cortex, a tertiary taste cortical area. The food reward representations formed in this way play an important role in the control of appetite, and food intake. Individual differences in these reward representations may contribute to obesity, and there are age-related differences in these value representations that shape the foods that people in different age groups find palatable. In a third tier of processing in medial prefrontal cortex area 10, decisions between stimuli of different reward value are taken, by attractor decision-making networks.
A survey of interestingness measures for knowledge discovery

It is a well known fact that the data mining process can generate many hundreds and often thousands of patterns from data. The task for the data miner then becomes one of determining the most useful patterns from those that are trivial or are already well known to the organization. It is therefore necessary to filter out those patterns through the use of some measure of the patterns actual worth. This article presents a review of the available literature on the various measures devised for evaluating and ranking the discovered patterns produced by the data mining process. These so called interestingness measures are generally divided into two categories: objective measures based on the statistical strengths or properties of the discovered patterns and subjective measures which are derived from the user’s beliefs or expectations of their particular problem domain. We evaluate the strengths and weaknesses of the various interestingness measures with respect to the level of user integration within the discovery process.
Diffusion Strategies for Distributed Kalman Filtering and Smoothing

We study the problem of distributed Kalman filtering and smoothing, where a set of nodes is required to estimate the state of a linear dynamic system from in a collaborative manner. Our focus is on diffusion strategies, where nodes communicate with their direct neighbors only, and the information is diffused across the network through a sequence of Kalman iterations and data-aggregation. We study the problems of Kalman filtering, fixed-lag smoothing and fixed-point smoothing, and propose diffusion algorithms to solve each one of these problems. We analyze the mean and mean-square performance of the proposed algorithms, provide expressions for their steady-state mean-square performance, and analyze the convergence of the diffusion Kalman filter recursions. Finally, we apply the proposed algorithms to the problem of estimating and tracking the position of a projectile. We compare our simulation results with the theoretical expressions, and note that the proposed approach outperforms existing techniques.
(DE)$^2$CO: Deep Depth Colorization

The ability to classify objects is fundamental for robots. Besides knowledge about their visual appearance, captured by the RGB channel, robots heavily need also depth information to make sense of the world. While the use of deep networks on RGB robot images has benefited from the plethora of results obtained on databases like ImageNet, using convnets on depth images requires mapping them into three-dimensional channels. This transfer learning procedure makes them processable by pretrained deep architectures. Current mappings are based on heuristic assumptions over preprocessing steps and on what depth properties should be most preserved, resulting often in cumbersome data visualizations, and in suboptimal performance in terms of generality and recognition results. Here, we take an alternative route and we attempt instead to learn an optimal colorization mapping for any given pretrained architecture, using as training data a reference RGB-D database. We propose a deep network architecture, exploiting the residual paradigm, that learns how to map depth data to three channel images. A qualitative analysis of the images obtained with this approach clearly indicates that learning the optimal mapping preserves the richness of depth information better than current hand-crafted approaches. Experiments on the Washington, JHUIT-50 and BigBIRD public benchmark databases, using CaffeNet, VGG-16, GoogleNet, and ResNet50 clearly showcase the power of our approach, with gains in performance of up to 16% compared to state of the art competitors on the depth channel only, leading to top performances when dealing with RGB-D data.
Executive functions and the frontal lobes: a conceptual view.

Several problems in understanding executive functions and their relationships to the frontal lobes are discussed. Data are then presented from several of our studies to support the following statements: (1) the examination of patients with focal frontal lobe lesions is a necessary first step in defining the relation of executive functions to the frontal lobes; (2) there is no unitary executive function. Rather, distinct processes related to the frontal lobes can be differentiated which converge on a general concept of control functions; (3) a simple control-automatic distinction is inadequate to explain the complexity of control-automatic processes; (4) the distinction between complex and simple tasks cannot explain the differences in functions between the frontal lobes and other brain regions; and (5) the most important role of the frontal lobes may be for affective responsiveness, social and personality development, and self-awareness and unconsciousness.
STRATOS: Using Visualization to Support Decisions in Strategic Software Release Planning

Software is typically developed incrementally and released in stages. Planning these releases involves deciding which features of the system should be implemented for each release. This is a complex planning process involving numerous trade-offs-constraints and factors that often make decisions difficult. Since the success of a product depends on this plan, it is important to understand the trade-offs between different release plans in order to make an informed choice. We present STRATOS, a tool that simultaneously visualizes several software release plans. The visualization shows several attributes about each plan that are important to planners. Multiple plans are shown in a single layout to help planners find and understand the trade-offs between alternative plans. We evaluated our tool via a qualitative study and found that STRATOS enables a range of decision-making processes, helping participants decide on which plan is most optimal.
Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replay

This paper introduces a novel method for learning how to play the most difficult Atari 2600 games from the Arcade Learning Environment using deep reinforcement learning. The proposed method, called human checkpoint replay, consists in using checkpoints sampled from human gameplay as starting points for the learning process. This is meant to compensate for the difficulties of current exploration strategies, such as ε-greedy, to find successful control policies in games with sparse rewards. Like other deep reinforcement learning architectures, our model uses a convolutional neural network that receives only raw pixel inputs to estimate the state value function. We tested our method on Montezuma’s Revenge and Private Eye, two of the most challenging games from the Atari platform. The results we obtained show a substantial improvement compared to previous learning approaches, as well as over a random player. We also propose a method for training deep reinforcement learning agents using human gameplay experience, which we call human experience replay.
Novel Design of Microstrip Bandpass Filters with a Controllable Dual-Passband Response: Description and Implementation

Novel microstrip dual-band bandpass filters with controllable fractional bandwidths and good in-between isolation are presented and implemented. A half-wavelength stepped-impedance resonator is firstly characterized, aiming at producing the two resonant frequencies at 2.4 and 5.2 GHz. Two types of coupled microstrip lines in the parallel and anti-parallel formats are then investigated in terms of unified equivalent J-inverter network. Extensive results are derived to quantitatively show their distinctive frequency-distributed coupling performances under different coupling lengths. The coupling degrees of these two coupled lines at the two resonances are properly adjusted to achieve the dual-passband response with varied or tunable bandwidths. In addition, the parallel coupled line is modeled to bring out a transmission zero between the two resonances so as to achieve the good in-between isolation. The three two-stage bandpass filters are initially designed to exhibit their dual-band response with changeable dual-band bandwidths. A three-stage dual-band filter is in final optimally designed and its predicted performance is confirmed in experiment. key words: controllable dual-passband, dual-band filter, equivalent Jinverter network, parallel-coupled microstrip line, stepped-impedance resonator
Meta-awareness, perceptual decoupling and the wandering mind

Mind wandering (i.e. engaging in cognitions unrelated to the current demands of the external environment) reflects the cyclic activity of two core processes: the capacity to disengage attention from perception (known as perceptual decoupling) and the ability to take explicit note of the current contents of consciousness (known as meta-awareness). Research on perceptual decoupling demonstrates that mental events that arise without any external precedent (known as stimulus independent thoughts) often interfere with the online processing of sensory information. Findings regarding meta-awareness reveal that the mind is only intermittently aware of engaging in mind wandering. These basic aspects of mind wandering are considered with respect to the activity of the default network, the role of executive processes, the contributions of meta-awareness and the functionality of mind wandering.
Reinforcement Learning with Perturbed Rewards

Recent studies have shown the vulnerability of reinforcement learning (RL) models in noisy settings. The sources of noises differ across scenarios. For instance, in practice, the observed reward channel is often subject to noise (e.g., when observed rewards are collected through sensors), and thus observed rewards may not be credible as a result. Also, in applications such as robotics, a deep reinforcement learning (DRL) algorithm can be manipulated to produce arbitrary errors. In this paper, we consider noisy RL problems where observed rewards by RL agents are generated with a reward confusion matrix. We call such observed rewards as perturbed rewards. We develop an unbiased reward estimator aided robust RL framework that enables RL agents to learn in noisy environments while observing only perturbed rewards. Our framework draws upon approaches for supervised learning with noisy data. The core ideas of our solution include estimating a reward confusion matrix and defining a set of unbiased surrogate rewards. We prove the convergence and sample complexity of our approach. Extensive experiments on different DRL platforms show that policies based on our estimated surrogate reward can achieve higher expected rewards, and converge faster than existing baselines. For instance, the state-of-the-art PPO algorithm is able to obtain 67.5% and 46.7% improvements in average on five Atari games, when the error rates are 10% and 30% respectively.
Artificial Intelligence and Economic Theory: Skynet in the Market

This chapter introduces this book, Artificial Intelligence and Economic Theory: Skynet in the market, and in the process studies some of the big ideas that have concerned economics and finance in the last 300 years. These ideas include Marxist thinking, the theory of invisible hand, the theory of equilibrium and the theory of comparative advantage. It, furthermore, describes methods in artificial intelligence such as learning, optimization and swarm intelligence. It sets a scene on how these theories can be better understood by using artificial intelligence techniques, thereby, setting a scene for the rest of the book.
Multilevel Converters as a Utility Interface for Renewable Energy Systems

Electric power production in the 21 Century will see dramatic changes in both the physical infrastructure and the control and information infrastructure. A shift will take place from a relatively few large, concentrated generation centers and the transmission of electricity over mostly a high voltage ac grid  to a more diverse and dispersed generation infrastructure that also has a higher percentage of dc transmission lines  . In the United States, generation capacity has not kept up with power demands, as over the last decade the reserve margins declined from 22% in 1990 to 16% in 1997. This declining margin trend is expected to continue over the next decade partly because of the uncertainty of how the future deregulated electrical environment will function. Less reserve margins will lead to less peaking capacity on highdemand days and more volatile energy prices . This change in a physical infrastructure combined with a more deregulated electric power industry will result in more parties generating power – or distributed generation. Some of the distributed generation power sources that are expected to increase greatly their market share of the total power produced in the United States and abroad include renewable energy sources such as photovoltaics, wind, low-head hydro, and geothermal . Fuel cell technology is also nearing the development point where it could start to supply a significant share of the power needs [4]. The advent of high power electronic modules has also encouraged the use of more dc transmission and made the prospects of interfacing dc power sources such as fuel cells and photovoltaics more easily attainable. A modular, scalable power electronics technology that is ideal for these types of utility applications is the transformerless multilevel converter .
A continuous-time sigma-delta modulator with 88-dB dynamic range and 1.1-MHz signal bandwidth

This paper presents the design and experimental results of a continuous-time /spl Sigma//spl Delta/ modulator for ADSL applications. Multibit nonreturn-to-zero (NRZ) DAC pulse shaping is used to reduce clock jitter sensitivity. The nonzero excess loop delay problem in conventional continuous-time /spl Sigma//spl Delta/ modulators is solved by our proposed architecture. A prototype third-order continuous-time /spl Sigma//spl Delta/ modulator with 5-bit internal quantization was realized in a 0.5-/spl mu/m double-poly triple-metal CMOS technology, with a chip area of 2.4 /spl times/ 2.4 mm/sup 2/. Experimental results show that the modulator achieves 88-dB dynamic range, 84-dB SNR, and 83-dB SNDR over a 1.1-MHz signal bandwidth with an oversampling ratio of 16, while dissipating 62 mW from a 3.3-V supply.
Brain storm-based Whale Optimization Algorithm for privacy-protected data publishing in cloud computing

Cloud computing serves as a major boost for the digital era since it handles data from a large number of users simultaneously. Besides the several useful characteristics, providing security to the data stored in the cloud platform is a major challenge for the service providers. Privacy preservation schemes introduced in the literature trying to enhance the privacy and utility of the data structures by modifying the database with the secret key. In this paper, an optimization scheme, Brain Storm based Whale Optimization Algorithm (BS-WOA), is introduced for identifying the secret key. The database from the data owner is modified with the optimal secret key for constructing the retrievable perturbation data for preserving the privacy and utility. The proposed BS-WOA is designed through the hybridization of Brain Storm Optimization and Whale Optimization Algorithm. Simulation of the proposed technique with the BS-WOA is done in the three standard databases, such as chess T10I4D100 K, and the retail databases. When evaluated for the key size of 256, the proposed BS-WOA achieved privacy value of 0.186 and utility value of 0.8777 for the chess database, and thus, has improved performance.
Expert Finding in Heterogeneous Bibliographic Networks with Locally-trained Embeddings

Expert finding is an important task in both industry and academia. It is challenging to rank candidates with appropriate expertise for various queries. In addition, different types of objects interact with one another, which naturally forms heterogeneous information networks. We study the task of expert finding in heterogeneous bibliographical networks based on two aspects: textual content analysis and authority ranking. Regarding the textual content analysis, we propose a new method for query expansion via locally-trained embedding learning with concept hierarchy as guidance, which is particularly tailored for specific queries with narrow semantic meanings. Compared with global embedding learning, locallytrained embedding learning projects the terms into a latent semantic space constrained on relevant topics, therefore it preserves more precise and subtle information for specific queries. Considering the candidate ranking, the heterogeneous information network structure, while being largely ignored in the previous studies of expert finding, provides additional information. Specifically, different types of interactions among objects play different roles. We propose a ranking algorithm to estimate the authority of objects in the network, treating each strongly-typed edge type individually. To demonstrate the effectiveness of the proposed framework, we apply the proposed method to a large-scale bibliographical dataset with over two million entries and one million researcher candidates. The experiment results show that the proposed framework outperforms existing methods for both general and specific queries.