OpenRaiser/Intern-Atlas huggingface dataset:
Feature Maps: A Comprehensible Software Representation for Design Pattern Detection
Venue: arXiv cs.SE 2019
Year: 2019

Design patterns are elegant and well-tested solutions to recurrent software development problems. They are the result of software developers dealing with problems that frequently occur, solving them in the same or a slightly adapted way. A pattern's semantics provide the intent, motivation, and applicability, describing what it does, why it is needed, and where it is useful. Consequently, design patterns encode a well of information. Developers weave this information into their systems whenever they use design patterns to solve problems. This work presents Feature Maps, a flexible human- and machine-comprehensible software representation based on micro-structures. Our algorithm, the Feature-Role Normalization, presses the high-dimensional, inhomogeneous vector space of micro-structures into a feature map. We apply these concepts to the problem of detecting instances of design patterns in source code. We evaluate our methodology on four design patterns, a wide range of balanced and imbalanced labeled training data, and compare classical machine learning (Random Forests) with modern deep learning approaches (Convolutional Neural Networks). Feature maps yield robust classifiers even under challenging settings of strongly imbalanced data distributions without sacrificing human comprehensibility. Results suggest that feature maps are an excellent addition in the software analysis toolbox that can reveal useful information hidden in the source code.
A 256kb 9T Near-Threshold SRAM With 1k Cells per Bit-Line and Enhanced Write and Read Operations
Venue: arXiv cs.AR 2019
Year: 2019

In this paper, we present a new 9T SRAM cell that has good write-ability and improves read stability at the same time. Simulation results show that the proposed design increases Read SNM (RSNM) and Ion/Ioff of read path by 219% and 113%, respectively at supply voltage of 300mV over conventional 6T SRAM cell in a 90nm CMOS technology. Proposed design lets us to reduce minimum operating voltage of SRAM (VDDmin) to 350mV, whereas conventional 6T SRAM cannot operate successfully with acceptable failure rate at supply voltages bellow 725mV. We also compared our design with three other SRAM cells from recent literature. To verify the proposed design, a 256kb SRAM is designed using new 9T and conventional 6T SRAM cells. Operating at their minimum possible VDDs, the proposed design decreases write and read power per operation by 92%, and 93%, respectively over the conventional rival. Area of proposed SRAM cell is increased by 83% over conventional 6T one. However, due to large Ion/Ioff of read path for 9T cell, we are able to put 1k cells in each column of 256kb SRAM block, resulting in the possibility for sharing write and read circuitries of each column between more cells compared to conventional 6T. Thus, area overhead of 256kb SRAM based on new 9T cell is reduced to 37% compared to 6T SRAM.
Trainable Projected Gradient Detector for Massive Overloaded MIMO Channels: Data-driven Tuning Approach
Venue: arXiv cs.IT 2019
Year: 2019

This paper presents a deep learning-aided iterative detection algorithm for massive overloaded multiple-input multiple-output (MIMO) systems where the number of transmit antennas $n$ is larger than that of receive antennas $m$. Since the proposed algorithm is based on the projected gradient descent method with trainable parameters, it is named the trainable projected gradient-detector (TPG-detector). The trainable internal parameters, such as the step-size parameter, can be optimized with standard deep learning techniques, i.e., the back propagation and stochastic gradient descent algorithms. This approach is referred to as data-driven tuning, and ensures fast convergence during parameter estimation in the proposed scheme. The TPG-detector mainly consists of matrix-vector product operations whose computational cost is proportional to $m n$ for each iteration. In addition, the number of trainable parameters in the TPG-detector is independent of the number of antennas. These features of the TPG-detector result in a fast and stable training process and reasonable scalability for large systems. Numerical simulations show that the proposed detector achieves a comparable detection performance to those of existing algorithms for massive overloaded MIMO channels, e.g., the state-of-the-art IW-SOAV detector, with a lower computation cost.
Privacy-Preserving Collaborative Deep Learning with Unreliable Participants
Venue: arXiv cs.CR 2019
Year: 2019

With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them in one single location may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $ε$-differential privacy. In particular, for the first time, we consider the existence of~\textit{unreliable participants}, i.e., the participants with low-quality data, and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world datasets for regression and classification tasks. The results demonstrate that the proposed system is robust against unreliable participants, and achieves high accuracy close to the model trained in a traditional centralized manner while ensuring rigorous privacy protection.
Profitable Bayesian implementation in one-shot mechanism settings
Venue: arXiv cs.GT 2019
Year: 2019

In the mechanism design theory, a designer would like to implement a desired social choice function which specifies her favorite outcome for each possible profile of all agents' types. Traditionally, the designer may be in a dilemma in the sense that even if she is not satisfied with some outcome with low profit, she has to announce it because she must obey the mechanism designed by herself. In this paper, we investigate a case where the designer can induce each agent to adjust his type in a one-shot mechanism. We propose that for a profitable Bayesian implementable social choice function, the designer may escape from the above-mentioned dilemma by spending the optimal adjustment cost and obtain a higher profit. Finally, we construct an example to show that the designer can breakthrough the limit of expected profit which she can obtain at most in the traditional optimal auction model.
Stochastic Trust Region Inexact Newton Method for Large-scale Machine Learning
Venue: arXiv cs.LG 2019
Year: 2019

Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this paper, we have proposed a novel Stochastic Trust RegiOn Inexact Newton method, called as STRON, to solve large-scale learning problems which uses conjugate gradient (CG) to inexactly solve trust region subproblem. The method uses progressive subsampling in the calculation of gradient and Hessian values to take the advantage of both, stochastic and full-batch regimes. We have extended STRON using existing variance reduction techniques to deal with the noisy gradients and using preconditioned conjugate gradient (PCG) as subproblem solver, and empirically proved that they do not work as expected, for the large-scale learning problems. Finally, our empirical results prove efficacy of the proposed method against existing methods with bench marked datasets.
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Venue: arXiv cs.CL 2019
Year: 2019

We introduce an architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts. Our system uses a single BiLSTM encoder with a shared BPE vocabulary for all languages, which is coupled with an auxiliary decoder and trained on publicly available parallel corpora. This enables us to learn a classifier on top of the resulting embeddings using English annotated data only, and transfer it to any of the 93 languages without any modification. Our experiments in cross-lingual natural language inference (XNLI dataset), cross-lingual document classification (MLDoc dataset) and parallel corpus mining (BUCC dataset) show the effectiveness of our approach. We also introduce a new test set of aligned sentences in 112 languages, and show that our sentence embeddings obtain strong results in multilingual similarity search even for low-resource languages. Our implementation, the pre-trained encoder and the multilingual test set are available at https://github.com/facebookresearch/LASER
Features and Machine Learning for Correlating and Classifying between Brain Areas and Dyslexia
Venue: arXiv cs.LG 2019
Year: 2019

We develop a method that is based on processing gathered Event Related Potentials (ERP) signals and the use of machine learning technique for multivariate analysis (i.e. classification) that we apply in order to analyze the differences between Dyslexic and Skilled readers. No human intervention is needed in the analysis process. This is the state of the art results for automatic identification of Dyslexic readers using a Lexical Decision Task. We use mathematical and machine learning based techniques to automatically discover novel complex features that (i) allow for reliable distinction between Dyslexic and Normal Control Skilled readers and (ii) to validate the assumption that the most of the differences between Dyslexic and Skilled readers located in the left hemisphere. Interestingly, these tools also pointed to the fact that High Pass signals (typically considered as "noise" during ERP/EEG analyses) in fact contains significant relevant information. Finally, the proposed scheme can be used for analysis of any ERP based studies.
Intent Detection and Slots Prompt in a Closed-Domain Chatbot
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we introduce a methodology for predicting intent and slots of a query for a chatbot that answers career-related queries. We take a multi-staged approach where both the processes (intent-classification and slot-tagging) inform each other's decision-making in different stages. The model breaks down the problem into stages, solving one problem at a time and passing on relevant results of the current stage to the next, thereby reducing search space for subsequent stages, and eventually making classification and tagging more viable after each stage. We also observe that relaxing rules for a fuzzy entity-matching in slot-tagging after each stage (by maintaining a separate Named Entity Tagger per stage) helps us improve performance, although at a slight cost of false-positives. Our model has achieved state-of-the-art performance with F1-score of 77.63% for intent-classification and 82.24% for slot-tagging on our dataset that we would publicly release along with the paper.
Exploring Weight Symmetry in Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

We propose to impose symmetry in neural network parameters to improve parameter usage and make use of dedicated convolution and matrix multiplication routines. Due to significant reduction in the number of parameters as a result of the symmetry constraints, one would expect a dramatic drop in accuracy. Surprisingly, we show that this is not the case, and, depending on network size, symmetry can have little or no negative effect on network accuracy, especially in deep overparameterized networks. We propose several ways to impose local symmetry in recurrent and convolutional neural networks, and show that our symmetry parameterizations satisfy universal approximation property for single hidden layer networks. We extensively evaluate these parameterizations on CIFAR, ImageNet and language modeling datasets, showing significant benefits from the use of symmetry. For instance, our ResNet-101 with channel-wise symmetry has almost 25% less parameters and only 0.2% accuracy loss on ImageNet. Code for our experiments is available at https://github.com/hushell/deep-symmetry
Evaluating Patient Readmission Risk: A Predictive Analytics Approach
Venue: arXiv cs.CY 2019
Year: 2019

With the emergence of the Hospital Readmission Reduction Program of the Center for Medicare and Medicaid Services on October 1, 2012, forecasting unplanned patient readmission risk became crucial to the healthcare domain. There are tangible works in the literature emphasizing on developing readmission risk prediction models; However, the models are not accurate enough to be deployed in an actual clinical setting. Our study considers patient readmission risk as the objective for optimization and develops a useful risk prediction model to address unplanned readmissions. Furthermore, Genetic Algorithm and Greedy Ensemble is used to optimize the developed model constraints.
State representation learning with recurrent capsule networks
Venue: arXiv cs.LG 2019
Year: 2019

Unsupervised learning of compact and relevant state representations has been proved very useful at solving complex reinforcement learning tasks. In this paper, we propose a recurrent capsule network that learns such representations by trying to predict the future observations in an agent's trajectory.
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Venue: arXiv cs.LG 2019
Year: 2019

Zeroth-order optimization is an important research topic in machine learning. In recent years, it has become a key tool in black-box adversarial attack to neural network based image classifiers. However, existing zeroth-order optimization algorithms rarely extract second-order information of the model function. In this paper, we utilize the second-order information of the objective function and propose a novel \textit{Hessian-aware zeroth-order algorithm} called \texttt{ZO-HessAware}. Our theoretical result shows that \texttt{ZO-HessAware} has an improved zeroth-order convergence rate and query complexity under structured Hessian approximation, where we propose a few approximation methods for estimating Hessian. Our empirical studies on the black-box adversarial attack problem validate that our algorithm can achieve improved success rates with a lower query complexity.
Talon: An Automated Framework for Cross-Device Tracking Detection
Venue: arXiv cs.CR 2019
Year: 2019

Although digital advertising fuels much of today's free Web, it typically does so at the cost of online users' privacy, due to the continuous tracking and leakage of users' personal data. In search for new ways to optimize the effectiveness of ads, advertisers have introduced new advanced paradigms such as cross-device tracking (CDT), to monitor users' browsing on multiple devices and screens, and deliver (re)targeted ads in the most appropriate screen.Unfortunately, this practice leads to greater privacy concerns for the end-user. Going beyond the state-of-the-art, we propose a novel methodology for detecting CDT and measuring the factors affecting its performance, in a repeatable and systematic way. This new methodology is based on emulating realistic browsing activity of end-users, from different devices, and thus triggering and detecting cross-device targeted ads. We design and build Talon a CDT measurement framework that implements our methodology and allows experimentation with multiple parallel devices, experimental setups and settings. By employing Talon, we perform several critical experiments, and we are able to not only detect and measure CDT with average AUC score of 0.78-0.96, but also to provide significant insights about the behavior of CDT entities and the impact on users' privacy. In the hands of privacy researchers, policy makers and end-users, Talon can be an invaluable tool for raising awareness and increasing transparency on tracking practices used by the ad-ecosystem.
A neural joint model for Vietnamese word segmentation, POS tagging and dependency parsing
Venue: arXiv cs.CL 2019
Year: 2019

We propose the first multi-task learning model for joint Vietnamese word segmentation, part-of-speech (POS) tagging and dependency parsing. In particular, our model extends the BIST graph-based dependency parser (Kiperwasser and Goldberg, 2016) with BiLSTM-CRF-based neural layers (Huang et al., 2015) for word segmentation and POS tagging. On Vietnamese benchmark datasets, experimental results show that our joint model obtains state-of-the-art or competitive performances.
Exact Guarantees on the Absence of Spurious Local Minima for Non-negative Rank-1 Robust Principal Component Analysis
Venue: arXiv cs.LG 2019
Year: 2019

This work is concerned with the non-negative rank-1 robust principal component analysis (RPCA), where the goal is to recover the dominant non-negative principal components of a data matrix precisely, where a number of measurements could be grossly corrupted with sparse and arbitrary large noise. Most of the known techniques for solving the RPCA rely on convex relaxation methods by lifting the problem to a higher dimension, which significantly increase the number of variables. As an alternative, the well-known Burer-Monteiro approach can be used to cast the RPCA as a non-convex and non-smooth $\ell_1$ optimization problem with a significantly smaller number of variables. In this work, we show that the low-dimensional formulation of the symmetric and asymmetric positive rank-1 RPCA based on the Burer-Monteiro approach has benign landscape, i.e., 1) it does not have any spurious local solution, 2) has a unique global solution, and 3) its unique global solution coincides with the true components. An implication of this result is that simple local search algorithms are guaranteed to achieve a zero global optimality gap when directly applied to the low-dimensional formulation. Furthermore, we provide strong deterministic and probabilistic guarantees for the exact recovery of the true principal components. In particular, it is shown that a constant fraction of the measurements could be grossly corrupted and yet they would not create any spurious local solution.
CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences
Venue: arXiv cs.CV 2019
Year: 2019

With a large amount of open satellite multispectral imagery (e.g., Sentinel-2 and Landsat-8), considerable attention has been paid to global multispectral land cover classification. However, its limited spectral information hinders further improving the classification performance. Hyperspectral imaging enables discrimination between spectrally similar classes but its swath width from space is narrow compared to multispectral ones. To achieve accurate land cover classification over a large coverage, we propose a cross-modality feature learning framework, called common subspace learning (CoSpace), by jointly considering subspace learning and supervised classification. By locally aligning the manifold structure of the two modalities, CoSpace linearly learns a shared latent subspace from hyperspectral-multispectral(HS-MS) correspondences. The multispectral out-of-samples can be then projected into the subspace, which are expected to take advantages of rich spectral information of the corresponding hyperspectral data used for learning, and thus leads to a better classification. Extensive experiments on two simulated HSMS datasets (University of Houston and Chikusei), where HS-MS data sets have trade-offs between coverage and spectral resolution, are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with previous state-of-the-art methods.
Solar Potential Analysis of Rooftops Using Satellite Imagery
Venue: arXiv cs.CV 2019
Year: 2019

Solar energy is one of the most important sources of renewable energy and the cleanest form of energy. In India, where solar energy could produce power around trillion kilowatt-hours in a year, our country is only able to produce power of around in gigawatts only. Many people are not aware of the solar potential of their rooftop, and hence they always think that installing solar panels is very much expensive. In this work, we introduce an approach through which we can generate a report remotely that provides the amount of solar potential of a building using only its latitude and longitude. We further evaluated various types of rooftops to make our solution more robust. We also provide an approximate area of rooftop that can be used for solar panels placement and a visual analysis of how solar panels can be placed to maximize the output of solar power at a location.
Stealing Neural Networks via Timing Side Channels
Venue: arXiv cs.CR 2019
Year: 2019

Deep learning is gaining importance in many applications. However, Neural Networks face several security and privacy threats. This is particularly significant in the scenario where Cloud infrastructures deploy a service with Neural Network model at the back end. Here, an adversary can extract the Neural Network parameters, infer the regularization hyperparameter, identify if a data point was part of the training data, and generate effective transferable adversarial examples to evade classifiers. This paper shows how a Neural Network model is susceptible to timing side channel attack. In this paper, a black box Neural Network extraction attack is proposed by exploiting the timing side channels to infer the depth of the network. Although, constructing an equivalent architecture is a complex search problem, it is shown how Reinforcement Learning with knowledge distillation can effectively reduce the search space to infer a target model. The proposed approach has been tested with VGG architectures on CIFAR10 data set. It is observed that it is possible to reconstruct substitute models with test accuracy close to the target models and the proposed approach is scalable and independent of type of Neural Network architectures.