OpenRaiser/Intern-Atlas huggingface dataset:
Privacy-Preserving Collaborative Prediction using Random Forests
Venue: arXiv cs.LG 2018
Year: 2018

We study the problem of privacy-preserving machine learning (PPML) for ensemble methods, focusing our effort on random forests. In collaborative analysis, PPML attempts to solve the conflict between the need for data sharing and privacy. This is especially important in privacy sensitive applications such as learning predictive models for clinical decision support from EHR data from different clinics, where each clinic has a responsibility for its patients' privacy. We propose a new approach for ensemble methods: each entity learns a model, from its own data, and then when a client asks the prediction for a new private instance, the answers from all the locally trained models are used to compute the prediction in such a way that no extra information is revealed. We implement this approach for random forests and we demonstrate its high efficiency and potential accuracy benefit via experiments on real-world datasets, including actual EHR data.
The contribution of star scientists to overall sex differences in research productivity
Venue: arXiv cs.DL 2018
Year: 2018

The state of the art on the issue of sex differences in research efficiency agrees in recognizing higher performances for males, however there are divergences in explaining the possible causes. One of the causes advanced is that there are sex differences in the availability of aptitude at the "high end". By comparing sex differences in concentration and performance of Italian academic star scientists to the case in the population complement, this work aims to verify if star, or "high-end", scientists play a preponderant role in determining higher performance among males. The study reveals the existence of a greater relative concentration of males among star scientists, as well as a performance gap between male and female star scientists that is greater than for the rest of the population. In the latter subpopulation the performance gap between the two sexs is seen as truly marginal.
Improving PSO Global Method for Feature Selection According to Iterations Global Search and Chaotic Theory
Venue: arXiv cs.LG 2018
Year: 2018

Making a simple model by choosing a limited number of features with the purpose of reducing the computational complexity of the algorithms involved in classification is one of the main issues in machine learning and data mining. The aim of Feature Selection (FS) is to reduce the number of redundant and irrelevant features and improve the accuracy of classification in a data set. We propose an efficient ISPSO-GLOBAL (Improved Seeding Particle Swarm Optimization GLOBAL) method which investigates the specified iterations to produce prominent features and store them in storage list. The goal is to find informative features based on its iteration frequency with favorable fitness for the next generation and high exploration. Our method exploits of a new initialization strategy in PSO which improves space search and utilizes chaos theory to enhance the population initialization, then we offer a new formula to determine the features size used in proposed method. Our experiments with real-world data sets show that the performance of the ISPSO-GLOBAL is superior comparing with state-of-the-art methods in most of the data sets.
Media-Based Modulation for Future Wireless Systems: A Tutorial
Venue: arXiv cs.IT 2018
Year: 2018

The wireless revolution has already started with the specified vision, overall objectives, and the first official 3GPP release of 5th generation (5G) wireless networks. Despite the development of several modern communication technologies, since the beginning of the modern era of digital communications, we have been mostly conveying information by altering the amplitude, the phase, or the frequency of sinusoidal carrier signals, which has inherent drawbacks. On the other hand, index modulation (IM) provides an alternative dimension to transmit digital information: the indices of the corresponding communication systems' building blocks. Media-based modulation (MBM), which is one of the newest and the most prominent members of the IM family, performs the transmission of information by altering the far-field radiation pattern of reconfigurable antennas (RAs) and provides a completely new dimension to convey information: wireless channel fade realizations themselves through the unique signature of received signals. The aim of this article is to shed light on this promising frontier from a broad communication engineering perspective by discussing the most recent advances as well as possible interesting research directions in MBM technologies.
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Venue: arXiv cs.LG 2018
Year: 2018

We study the problem of training deep neural networks with Rectified Linear Unit (ReLU) activation function using gradient descent and stochastic gradient descent. In particular, we study the binary classification problem and show that for a broad family of loss functions, with proper random weight initialization, both gradient descent and stochastic gradient descent can find the global minima of the training loss for an over-parameterized deep ReLU network, under mild assumption on the training data. The key idea of our proof is that Gaussian random initialization followed by (stochastic) gradient descent produces a sequence of iterates that stay inside a small perturbation region centering around the initial weights, in which the empirical loss function of deep ReLU networks enjoys nice local curvature properties that ensure the global convergence of (stochastic) gradient descent. Our theoretical results shed light on understanding the optimization for deep learning, and pave the way for studying the optimization dynamics of training modern deep neural networks.
MAC: Mining Activity Concepts for Language-based Temporal Localization
Venue: arXiv cs.CV 2018
Year: 2018

We address the problem of language-based temporal localization in untrimmed videos. Compared to temporal localization with fixed categories, this problem is more challenging as the language-based queries not only have no pre-defined activity list but also may contain complex descriptions. Previous methods address the problem by considering features from video sliding windows and language queries and learning a subspace to encode their correlation, which ignore rich semantic cues about activities in videos and queries. We propose to mine activity concepts from both video and language modalities by applying the actionness score enhanced Activity Concepts based Localizer (ACL). Specifically, the novel ACL encodes the semantic concepts from verb-obj pairs in language queries and leverages activity classifiers' prediction scores to encode visual concepts. Besides, ACL also has the capability to regress sliding windows as localization results. Experiments show that ACL significantly outperforms state-of-the-arts under the widely used metric, with more than 5% increase on both Charades-STA and TACoS datasets.
Integrating Task-Motion Planning with Reinforcement Learning for Robust Decision Making in Mobile Robots
Venue: arXiv cs.RO 2018
Year: 2018

Task-motion planning (TMP) addresses the problem of efficiently generating executable and low-cost task plans in a discrete space such that the (initially unknown) action costs are determined by motion plans in a corresponding continuous space. However, a task-motion plan can be sensitive to unexpected domain uncertainty and changes, leading to suboptimal behaviors or execution failures. In this paper, we propose a novel framework, TMP-RL, which is an integration of TMP and reinforcement learning (RL) from the execution experience, to solve the problem of robust task-motion planning in dynamic and uncertain domains. TMP-RL features two nested planning-learning loops. In the inner TMP loop, the robot generates a low-cost, feasible task-motion plan by iteratively planning in the discrete space and updating relevant action costs evaluated by the motion planner in continuous space. In the outer loop, the plan is executed, and the robot learns from the execution experience via model-free RL, to further improve its task-motion plans. RL in the outer loop is more accurate to the current domain but also more expensive, and using less costly task and motion planning leads to a jump-start for learning in the real world. Our approach is evaluated on a mobile service robot conducting navigation tasks in an office area. Results show that TMP-RL approach significantly improves adaptability and robustness (in comparison to TMP methods) and leads to rapid convergence (in comparison to task planning (TP)-RL methods). We also show that TMP-RL can reuse learned values to smoothly adapt to new scenarios during long-term deployments.
Risk Identification of Power Transmission System with Renewable Energy
Venue: arXiv cs.SY 2018
Year: 2018

This paper aims to investigate the risk identification problem of power transmission system that is integrated with renewable energy sources. In practice, the fluctuation of power generation from renewable energy sources can lead to severe consequences to power transmission network. By treating the fluctuation of power generation as the control input, the risk identification problem is formulated with the aid of optimal control theory. Thus, a control approach is developed to identify the fluctuation of power generation that results in the worst-case cascading failures of power systems. Theoretical analysis is also conducted to obtain the necessary condition for the worst fluctuations of power generation. Finally, numerical simulations are implemented on IEEE 9 Bus System to demonstrate the effectiveness of the proposed approach.
Automated Amortised Resource Analysis for Term Rewrite Systems
Venue: arXiv cs.LO 2018
Year: 2018

In this paper we establish an automated amortised resource analysis for term rewrite systems. The method is presented in an annotated type system and gives rise to polynomial bounds on the innermost runtime complexity of the analysed term rewrite system. Our analysis does not restrict the input rewrite system in any way so that rewrite systems may serve as abstractions of first-order, eagerly evaluated functional programs over user-defined inductive data-types. This facilitates integration in a general framework for resource analysis of programs. In particular, we have implemented the method and integrated it into our analysis tool TcT. Furthermore, we have coupled the established analysis with a complexity reflecting transformation from pure OCaml programs. This extends the provided analysis to a fully automated resource analysis of higher-order functional programs.
Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning
Venue: arXiv cs.LG 2018
Year: 2018

In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can execute them. A high-level policy can then direct the lower one by generating a sequence of continuous sub-goal vectors. We evaluate our model using Mazebase and Mujoco environments, including the challenging AntGather task. Visualizations of the sub-goal embeddings reveal a logical decomposition of tasks within the environment. Quantitatively, our approach obtains compelling performance gains over non-hierarchical approaches.
Towards energy efficient buildings: how ICTs can convert advances?
Venue: arXiv cs.NI 2018
Year: 2018

This work is a positioning research paper for energy efficient building based on ICT solutions. Through the literature about the solutions for energy control of buildings during operational phase, a 3-layers model is proposed to integrate these solutions: first level consists in communication technologies, second level is about data modelling and third level is related to decision-making tools. For each level, key research topics and remaining problems are identified in order to achieve a concrete step forward. 1. CONTEXT AND PROBLEMATICS Through studies on ICT solutions for energy control of buildings, a 3-layers model is proposed to integrate these solutions and position a new way for energy efficiency. The building sector is the largest user of energy and CO 2 emitter in the EU, estimated at approximately 40% of the total consumption (Sharples et al., 1999). According to the International Panel on Climate Change (European Union, 2010), 30% of energy used in buildings could be reduced with net economic benefits by 2030. Such a reduction, however, is meaningless unless "sustainability" is considered. Because of these factors, healthy, sustainable, and energy efficient buildings have become active topics in international research; there is an urgent need for a new kind of high-technology driven and integrative research that should lead to the massive development of smart buildings and, in the medium term, smart cities. From a building lifecycle perspective, most of the energy (~80%) is consumed during the operational stage of the building (European Union, 2010) (Bilsen et al., 2013). Reducing building energy consumption may be addressed by the physical modifications which can be operated on a building like upgrading windows, heating systems or modifying thermic characteristics by insulating. Another possible path to reduce the energy consumption of a building is to use Information and Communication Technologies (ICT). According to the International Panel on Climate Change, a reduction of energy even greater than the 30% can be targeted by 2030 by considering ICT solutions. In support of this claim, some specialists believe that ICT-based solutions have the potential to enable 50-80% greenhouse gas reduction globally. In this respect, ICT innovation opens prospects for the development of a new range of new services highly available, flexible, safe, easy to integrate, and user friendly (Bilsen et al., 2013). This, in turn, should foster a sophisticated, reliable and fast communication infrastructure for the connection of various distributed elements (sensors, generators, substations...) that enables to exchange real-time data, information and knowledge needed to improve efficiency (e.g., to monitor and control energy consumption), reliability (e.g., to facilitate maintenance operations), flexibility (e.g., to integrate new rules to meet new consumer expectations), and investment returns, but also to induce a shift in consumer behaviour.
Copy the Old or Paint Anew? An Adversarial Framework for (non-) Parametric Image Stylization
Venue: arXiv cs.CV 2018
Year: 2018

Parametric generative deep models are state-of-the-art for photo and non-photo realistic image stylization. However, learning complicated image representations requires compute-intense models parametrized by a huge number of weights, which in turn requires large datasets to make learning successful. Non-parametric exemplar-based generation is a technique that works well to reproduce style from small datasets, but is also compute-intensive. These aspects are a drawback for the practice of digital AI artists: typically one wants to use a small set of stylization images, and needs a fast flexible model in order to experiment with it. With this motivation, our work has these contributions: (i) a novel stylization method called Fully Adversarial Mosaics (FAMOS) that combines the strengths of both parametric and non-parametric approaches; (ii) multiple ablations and image examples that analyze the method and show its capabilities; (iii) source code that will empower artists and machine learning researchers to use and modify FAMOS.
Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack
Venue: arXiv cs.LG 2018
Year: 2018

Recent development in the field of Deep Learning have exposed the underlying vulnerability of Deep Neural Network (DNN) against adversarial examples. In image classification, an adversarial example is a carefully modified image that is visually imperceptible to the original image but can cause DNN model to misclassify it. Training the network with Gaussian noise is an effective technique to perform model regularization, thus improving model robustness against input variation. Inspired by this classical method, we explore to utilize the regularization characteristic of noise injection to improve DNN's robustness against adversarial attack. In this work, we propose Parametric-Noise-Injection (PNI) which involves trainable Gaussian noise injection at each layer on either activation or weights through solving the min-max optimization problem, embedded with adversarial training. These parameters are trained explicitly to achieve improved robustness. To the best of our knowledge, this is the first work that uses trainable noise injection to improve network robustness against adversarial attacks, rather than manually configuring the injected noise level through cross-validation. The extensive results show that our proposed PNI technique effectively improves the robustness against a variety of powerful white-box and black-box attacks such as PGD, C & W, FGSM, transferable attack and ZOO attack. Last but not the least, PNI method improves both clean- and perturbed-data accuracy in comparison to the state-of-the-art defense methods, which outperforms current unbroken PGD defense by 1.1 % and 6.8 % on clean test data and perturbed test data respectively using Resnet-20 architecture.
Improved Frequency Modulation Features for Multichannel Distant Speech Recognition
Venue: arXiv cs.SD 2018
Year: 2018

Frequency modulation features capture the fine structure of speech formants that constitute beneficial and supplementary to the traditional energy-based cepstral features. Improvements have been demonstrated mainly in GMM-HMM systems for small and large vocabulary tasks. Yet, they have limited applications in DNN-HMM systems and Distant Speech Recognition (DSR) tasks. Herein, we elaborate on their integration within state-of-the-art front-end schemes that include post-processing of MFCCs resulting in discriminant and speaker adapted features of large temporal contexts. We explore 1) multichannel demodulation schemes for multi-microphone setups, 2) richer descriptors of frequency modulations, and 3) feature transformation and combination via hierarchical deep networks. We present results for tandem and hybrid recognition with GMM and DNN acoustic models, respectively. The improved modulation features are combined efficiently with MFCCs yielding modest and consistent improvements in multichannel distant speech recognition tasks on reverberant and noisy environments, where recognition rates are far from human performance.
On the Importance of Strong Baselines in Bayesian Deep Learning
Venue: arXiv cs.LG 2018
Year: 2018

Like all sub-fields of machine learning Bayesian Deep Learning is driven by empirical validation of its theoretical proposals. Given the many aspects of an experiment it is always possible that minor or even major experimental flaws can slip by both authors and reviewers. One of the most popular experiments used to evaluate approximate inference techniques is the regression experiment on UCI datasets. However, in this experiment, models which have been trained to convergence have often been compared with baselines trained only for a fixed number of iterations. We find that a well-established baseline, Monte Carlo dropout, when evaluated under the same experimental settings shows significant improvements. In fact, the baseline outperforms or performs competitively with methods that claimed to be superior to the very same baseline method when they were introduced. Hence, by exposing this flaw in experimental procedure, we highlight the importance of using identical experimental setups to evaluate, compare, and benchmark methods in Bayesian Deep Learning.
Explicit Interaction Model towards Text Classification
Venue: arXiv cs.CL 2018
Year: 2018

Text classification is one of the fundamental tasks in natural language processing. Recently, deep neural networks have achieved promising performance in the text classification task compared to shallow models. Despite of the significance of deep models, they ignore the fine-grained (matching signals between words and classes) classification clues since their classifications mainly rely on the text-level representations. To address this problem, we introduce the interaction mechanism to incorporate word-level matching signals into the text classification task. In particular, we design a novel framework, EXplicit interAction Model (dubbed as EXAM), equipped with the interaction mechanism. We justified the proposed approach on several benchmark datasets including both multi-label and multi-class text classification tasks. Extensive experimental results demonstrate the superiority of the proposed method. As a byproduct, we have released the codes and parameter settings to facilitate other researches.
A Perspective on the Challenges and Opportunities for Privacy-Aware Big Transportation Data
Venue: arXiv cs.CY 2018
Year: 2018

In recent years, and especially since the development of the smartphone, enormous amounts of data relevant for transportation have become available. These data hold out the potential to redefine how transportation system (i.e. design, planning and operations) is done. While researchers in both academia and industry are making advances in using this data to transportation system ends (e.g. information inference from collected data), little attention has been paid to four larger scale challenges that will need to be overcome if the potential for Big Transportation Data is to be harnessed for transportation decision-making purposes. This paper aims to provide awareness of these large-scale challenges and provides insight into how we believe these challenges are likely to be met.
Individualized Time-Series Segmentation for Mining Mobile Phone User Behavior
Venue: arXiv cs.CY 2018
Year: 2018

Mobile phones can record individual's daily behavioral data as a time-series. In this paper, we present an effective time-series segmentation technique that extracts optimal time segments of individual's similar behavioral characteristics utilizing their mobile phone data. One of the determinants of an individual's behavior is the various activities undertaken at various times-of-the-day and days-of-the-week. In many cases, such behavior will follow temporal patterns. Currently, researchers use either equal or unequal interval-based segmentation of time for mining mobile phone users' behavior. Most of them take into account static temporal coverage of 24-h-a-day and few of them take into account the number of incidences in time-series data. However, such segmentations do not necessarily map to the patterns of individual user activity and subsequent behavior because of not taking into account the diverse behaviors of individuals over time-of-the-week. Therefore, we propose a behavior-oriented time segmentation (BOTS) technique that takes into account not only the temporal coverage of the week but also the number of incidences of diverse behaviors dynamically for producing similar behavioral time segments over the week utilizing time-series data. Experiments on the real mobile phone datasets show that our proposed segmentation technique better captures the user's dominant behavior at various times-of-the-day and days-of-the-week enabling the generation of high confidence temporal rules in order to mine individual mobile phone users' behavior.