OpenRaiser/Intern-Atlas huggingface dataset:
Phase Conductor on Multi-layered Attentions for Machine Comprehension
Venue: arXiv cs.CL 2017
Year: 2017

Attention models have been intensively studied to improve NLP tasks such as machine comprehension via both question-aware passage attention model and self-matching attention model. Our research proposes phase conductor (PhaseCond) for attention models in two meaningful ways. First, PhaseCond, an architecture of multi-layered attention models, consists of multiple phases each implementing a stack of attention layers producing passage representations and a stack of inner or outer fusion layers regulating the information flow. Second, we extend and improve the dot-product attention function for PhaseCond by simultaneously encoding multiple question and passage embedding layers from different perspectives. We demonstrate the effectiveness of our proposed model PhaseCond on the SQuAD dataset, showing that our model significantly outperforms both state-of-the-art single-layered and multiple-layered attention models. We deepen our results with new findings via both detailed qualitative analysis and visualized examples showing the dynamic changes through multi-layered attention models.
Interpretable Apprenticeship Learning with Temporal Logic Specifications
Venue: arXiv cs.SY 2017
Year: 2017

Recent work has addressed using formulas in linear temporal logic (LTL) as specifications for agents planning in Markov Decision Processes (MDPs). We consider the inverse problem: inferring an LTL specification from demonstrated behavior trajectories in MDPs. We formulate this as a multiobjective optimization problem, and describe state-based ("what actually happened") and action-based ("what the agent expected to happen") objective functions based on a notion of "violation cost". We demonstrate the efficacy of the approach by employing genetic programming to solve this problem in two simple domains.
A $o(d) \cdot \text{polylog}~n$ Monotonicity Tester for Boolean Functions over the Hypergrid $[n]^d$
Venue: arXiv cs.DM 2017
Year: 2017

We study monotonicity testing of Boolean functions over the hypergrid $[n]^d$ and design a non-adaptive tester with $1$-sided error whose query complexity is $\tilde{O}(d^{5/6})\cdot \text{poly}(\log n,1/ε)$. Previous to our work, the best known testers had query complexity linear in $d$ but independent of $n$. We improve upon these testers as long as $n = 2^{d^{o(1)}}$. To obtain our results, we work with what we call the augmented hypergrid, which adds extra edges to the hypergrid. Our main technical contribution is a Margulis-style isoperimetric result for the augmented hypergrid, and our tester, like previous testers for the hypercube domain, performs directed random walks on this structure.
Secrecy Rate Maximization with Outage Constraint in Multihop Relaying Networks
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, we study the secure transmission in multihop wireless networks with randomize-and-forward (RaF) relaying, in the presence of randomly distributed eavesdroppers. By considering adaptive encoder with on-off transmission (OFT) scheme, we investigate the optimal design of the wiretap code and routing strategies to maximize the secrecy rate while satisfying the secrecy outage probability (SOP) constraint. We derive the exact expressions for the optimal rate parameters of the wiretap code. Then the secure routing problem is solved by revising the classical Bellman-Ford algorithm. Simulation results are conducted to verify our analysis.
Social Welfare Maximization Auction in Edge Computing Resource Allocation for Mobile Blockchain
Venue: arXiv cs.GT 2017
Year: 2017

Blockchain, an emerging decentralized security system, has been applied in many applications, such as bitcoin, smart grid, and Internet-of-Things. However, running the mining process may cost too much energy consumption and computing resource usage on handheld devices, which restricts the use of blockchain in mobile environments. In this paper, we consider deploying edge computing service to support the mobile blockchain. We propose an auction-based edge computing resource market of the edge computing service provider. Since there is competition among miners, the allocative externalities (positive and negative) are taken into account in the model. In our auction mechanism, we maximize the social welfare while guaranteeing the truthfulness, individual rationality and computational efficiency. Based on blockchain mining experiment results, we define a hash power function that characterizes the probability of successfully mining a block. Through extensive simulations, we evaluate the performance of our auction mechanism which shows that our edge computing resources market model can efficiently solve the social welfare maximization problem for the edge computing service provider.
Design and Analysis of a Controller Using Quantitative Feedback Theory for a Vehicle Air Suspension System
Venue: arXiv cs.SY 2017
Year: 2017

This paper presents the design of a robust controller for a vehicle air suspension system using Quantitative Feedback Theory (QFT). This study is primarily focused on control of linearized active air suspension system. For the purpose of simplicity, the dynamics of the air suspension system is modeled using a simple 2-DOF quarter car model. Uncertain dynamic system with different working condition has been considered for the vehicle air suspension system.
Wideband Channel Estimation for Hybrid Beamforming Millimeter Wave Communication Systems with Low-Resolution ADCs
Venue: arXiv cs.IT 2017
Year: 2017

A potential tremendous spectrum resource makes millimeter wave (mmWave) communications a promising technology. High power consumption due to a large number of antennas and analog-to-digital converters (ADCs) for beamforming to overcome the large propagation losses is problematic in practice. As a hybrid beamforming architecture and low-resolution ADCs are considered to reduce power consumption, estimation of mmWave channels becomes challenging. We evaluate several channel estimation algorithms for wideband mmWave systems with hybrid beamforming and low-resolution ADCs. Through simulation, we show that 1) infinite bit ADCs with least-squares estimation have worse channel estimation performance than do one-bit ADCs with orthogonal matching pursuit (OMP) in an SNR range of interest, 2) three- and four-bit quantizers can achieve channel estimation performance close to the unquantized case when using OMP, 3) a receiver with a single RF chain can yield better estimates than that with four RF chains if enough frames are exploited, and 4) for one-bit ADCs, exploitation of higher transmit power and more frames for performance enhancement adversely affects estimation performance after a certain point.
Discovery Radiomics with CLEAR-DR: Interpretable Computer Aided Diagnosis of Diabetic Retinopathy
Venue: arXiv cs.AI 2017
Year: 2017

Objective: Radiomics-driven Computer Aided Diagnosis (CAD) has shown considerable promise in recent years as a potential tool for improving clinical decision support in medical oncology, particularly those based around the concept of Discovery Radiomics, where radiomic sequencers are discovered through the analysis of medical imaging data. One of the main limitations with current CAD approaches is that it is very difficult to gain insight or rationale as to how decisions are made, thus limiting their utility to clinicians. Methods: In this study, we propose CLEAR-DR, a novel interpretable CAD system based on the notion of CLass-Enhanced Attentive Response Discovery Radiomics for the purpose of clinical decision support for diabetic retinopathy. Results: In addition to disease grading via the discovered deep radiomic sequencer, the CLEAR-DR system also produces a visual interpretation of the decision-making process to provide better insight and understanding into the decision-making process of the system. Conclusion: We demonstrate the effectiveness and utility of the proposed CLEAR-DR system of enhancing the interpretability of diagnostic grading results for the application of diabetic retinopathy grading. Significance: CLEAR-DR can act as a potential powerful tool to address the uninterpretability issue of current CAD systems, thus improving their utility to clinicians.
On Pre-Trained Image Features and Synthetic Images for Deep Learning
Venue: arXiv cs.CV 2017
Year: 2017

Deep Learning methods usually require huge amounts of training data to perform at their full potential, and often require expensive manual labeling. Using synthetic images is therefore very attractive to train object detectors, as the labeling comes for free, and several approaches have been proposed to combine synthetic and real images for training. In this paper, we show that a simple trick is sufficient to train very effectively modern object detectors with synthetic images only: We freeze the layers responsible for feature extraction to generic layers pre-trained on real images, and train only the remaining layers with plain OpenGL rendering. Our experiments with very recent deep architectures for object recognition (Faster-RCNN, R-FCN, Mask-RCNN) and image feature extractors (InceptionResnet and Resnet) show this simple approach performs surprisingly well.
Stationarity Region of Mm-Wave Channel Based on Outdoor Microcellular Measurements at 28 GHz
Venue: arXiv cs.IT 2017
Year: 2017

The stationarity region, i.e., the area in which the statistics of a propagation channel remain constant, is an important measure of the propagation channel, and essential for efficient system design. This paper presents what is to our knowledge the first extensive measurement campaign for measuring the stationarity region of MIMO mm-wave channels. Using a novel 28 GHz phased-array sounder with high phase stability, we present results in an urban microcell LoS, and LOS to NLOS transition region scenario, for the stationarity region of shadowing, power delay profile, and the angular power spectrum. A comparison to results at cm-waves shows considerably reduced stationarity region size, which has an important impact on system design.
Transfer Learning to Learn with Multitask Neural Model Search
Venue: arXiv cs.AI 2017
Year: 2017

Deep learning models require extensive architecture design exploration and hyperparameter optimization to perform well on a given task. The exploration of the model design space is often made by a human expert, and optimized using a combination of grid search and search heuristics over a large space of possible choices. Neural Architecture Search (NAS) is a Reinforcement Learning approach that has been proposed to automate architecture design. NAS has been successfully applied to generate Neural Networks that rival the best human-designed architectures. However, NAS requires sampling, constructing, and training hundreds to thousands of models to achieve well-performing architectures. This procedure needs to be executed from scratch for each new task. The application of NAS to a wide set of tasks currently lacks a way to transfer generalizable knowledge across tasks. In this paper, we present the Multitask Neural Model Search (MNMS) controller. Our goal is to learn a generalizable framework that can condition model construction on successful model searches for previously seen tasks, thus significantly speeding up the search for new tasks. We demonstrate that MNMS can conduct an automated architecture search for multiple tasks simultaneously while still learning well-performing, specialized models for each task. We then show that pre-trained MNMS controllers can transfer learning to new tasks. By leveraging knowledge from previous searches, we find that pre-trained MNMS models start from a better location in the search space and reduce search time on unseen tasks, while still discovering models that outperform published human-designed models.
Understanding Hidden Memories of Recurrent Neural Networks
Venue: arXiv cs.CL 2017
Year: 2017

Recurrent neural networks (RNNs) have been successfully applied to various natural language processing (NLP) tasks and achieved better results than conventional methods. However, the lack of understanding of the mechanisms behind their effectiveness limits further improvements on their architectures. In this paper, we present a visual analytics method for understanding and comparing RNN models for NLP tasks. We propose a technique to explain the function of individual hidden state units based on their expected response to input texts. We then co-cluster hidden state units and words based on the expected response and visualize co-clustering results as memory chips and word clouds to provide more structured knowledge on RNNs' hidden states. We also propose a glyph-based sequence visualization based on aggregate information to analyze the behavior of an RNN's hidden state at the sentence-level. The usability and effectiveness of our method are demonstrated through case studies and reviews from domain experts.
Performance Analysis of Multi-Service Oriented Multiple Access Under General Channel Correlation
Venue: arXiv cs.IT 2017
Year: 2017

In this article, we provide both analytical and numerical performance analysis of multi-service oriented multiple access (MOMA), a recently proposed non-orthogonal multiple-access scheme for scenarios with a massive number of concurrent connections originating from separate service classes with diverse quality-of-service (QoS) profiles and running on both handheld terminals and Internet-of-Things (IoT) devices. MOMA is based on both class dependent hierarchical-spreading transmission scheme and per-class reception structure. The performance analysis presented in this article is based on realistic channel and signal models for scenarios where the base station is equipped with a large number of antennas. It provides asymptotically exact approximations of the ergodic rates achievable by MOMA.
Open Set Logo Detection and Retrieval
Venue: arXiv cs.CV 2017
Year: 2017

Current logo retrieval research focuses on closed set scenarios. We argue that the logo domain is too large for this strategy and requires an open set approach. To foster research in this direction, a large-scale logo dataset, called Logos in the Wild, is collected and released to the public. A typical open set logo retrieval application is, for example, assessing the effectiveness of advertisement in sports event broadcasts. Given a query sample in shape of a logo image, the task is to find all further occurrences of this logo in a set of images or videos. Currently, common logo retrieval approaches are unsuitable for this task because of their closed world assumption. Thus, an open set logo retrieval method is proposed in this work which allows searching for previously unseen logos by a single query sample. A two stage concept with separate logo detection and comparison is proposed where both modules are based on task specific CNNs. If trained with the Logos in the Wild data, significant performance improvements are observed, especially compared with state-of-the-art closed set approaches.
An FPTAS of Minimizing Total Weighted Completion Time on Single Machine with Position Constraint
Venue: arXiv cs.DS 2017
Year: 2017

In this paper we study the classical scheduling problem of minimizing the total weighted completion time on a single machine with the constraint that one specific job must be scheduled at a specified position. We give dynamic programs with pseudo-polynomial running time, and a fully polynomial-time approximation scheme (FPTAS).
Semantic Code Repair using Neuro-Symbolic Transformation Networks
Venue: arXiv cs.AI 2017
Year: 2017

We study the problem of semantic code repair, which can be broadly defined as automatically fixing non-syntactic bugs in source code. The majority of past work in semantic code repair assumed access to unit tests against which candidate repairs could be validated. In contrast, the goal here is to develop a strong statistical model to accurately predict both bug locations and exact fixes without access to information about the intended correct behavior of the program. Achieving such a goal requires a robust contextual repair model, which we train on a large corpus of real-world source code that has been augmented with synthetically injected bugs. Our framework adopts a two-stage approach where first a large set of repair candidates are generated by rule-based processors, and then these candidates are scored by a statistical model using a novel neural network architecture which we refer to as Share, Specialize, and Compete. Specifically, the architecture (1) generates a shared encoding of the source code using an RNN over the abstract syntax tree, (2) scores each candidate repair using specialized network modules, and (3) then normalizes these scores together so they can compete against one another in comparable probability space. We evaluate our model on a real-world test set gathered from GitHub containing four common categories of bugs. Our model is able to predict the exact correct repair 41\% of the time with a single guess, compared to 13\% accuracy for an attentional sequence-to-sequence model.
Techreport: Time-sensitive probabilistic inference for the edge
Venue: arXiv cs.DC 2017
Year: 2017

In recent years the two trends of edge computing and artificial intelligence became both crucial for information processing infrastructures. While the centralized analysis of massive amounts of data seems to be at odds with computation on the outer edge of distributed systems, we explore the properties of eventually consistent systems and statistics to identify sound formalisms for probabilistic inference on the edge. In particular we treat time itself as a random variable that we incorporate into statistical models through probabilistic programming.
Customized Routing Optimization Based on Gradient Boost Regressor Model
Venue: arXiv cs.OH 2017
Year: 2017

In this paper, we discussed limitation of current electronic-design-automoation (EDA) tool and proposed a machine learning framework to overcome the limitations and achieve better design quality. We explored how to efficiently extract relevant features and leverage gradient boost regressor (GBR) model to predict underestimated risky net (URN). Customized routing optimizations are applied to the URNs and results show clear timing improvement and trend to converge toward timing closure.
Creation of an Annotated Corpus of Spanish Radiology Reports
Venue: arXiv cs.CL 2017
Year: 2017

This paper presents a new annotated corpus of 513 anonymized radiology reports written in Spanish. Reports were manually annotated with entities, negation and uncertainty terms and relations. The corpus was conceived as an evaluation resource for named entity recognition and relation extraction algorithms, and as input for the use of supervised methods. Biomedical annotated resources are scarce due to confidentiality issues and associated costs. This work provides some guidelines that could help other researchers to undertake similar tasks.
Improve SAT-solving with Machine Learning
Venue: arXiv cs.AI 2017
Year: 2017

In this project, we aimed to improve the runtime of Minisat, a Conflict-Driven Clause Learning (CDCL) solver that solves the Propositional Boolean Satisfiability (SAT) problem. We first used a logistic regression model to predict the satisfiability of propositional boolean formulae after fixing the values of a certain fraction of the variables in each formula. We then applied the logistic model and added a preprocessing period to Minisat to determine the preferable initial value (either true or false) of each boolean variable using a Monte-Carlo approach. Concretely, for each Monte-Carlo trial, we fixed the values of a certain ratio of randomly selected variables, and calculated the confidence that the resulting sub-formula is satisfiable with our logistic regression model. The initial value of each variable was set based on the mean confidence scores of the trials that started from the literals of that variable. We were particularly interested in setting the initial values of the backbone variables correctly, which are variables that have the same value in all solutions of a SAT formula. Our Monte-Carlo method was able to set 78% of the backbones correctly. Excluding the preprocessing time, compared with the default setting of Minisat, the runtime of Minisat for satisfiable formulae decreased by 23%. However, our method did not outperform vanilla Minisat in runtime, as the decrease in the conflicts was outweighed by the long runtime of the preprocessing period.
Theoretical properties of the global optimizer of two layer neural network
Venue: arXiv cs.LG 2017
Year: 2017

In this paper, we study the problem of optimizing a two-layer artificial neural network that best fits a training dataset. We look at this problem in the setting where the number of parameters is greater than the number of sampled points. We show that for a wide class of differentiable activation functions (this class involves "almost" all functions which are not piecewise linear), we have that first-order optimal solutions satisfy global optimality provided the hidden layer is non-singular. Our results are easily extended to hidden layers given by a flat matrix from that of a square matrix. Results are applicable even if network has more than one hidden layer provided all hidden layers satisfy non-singularity, all activations are from the given "good" class of differentiable functions and optimization is only with respect to the last hidden layer. We also study the smoothness properties of the objective function and show that it is actually Lipschitz smooth, i.e., its gradients do not change sharply. We use smoothness properties to guarantee asymptotic convergence of O(1/number of iterations) to a first-order optimal solution. We also show that our algorithm will maintain non-singularity of hidden layer for any finite number of iterations.
Shallow Discourse Parsing with Maximum Entropy Model
Venue: arXiv cs.CL 2017
Year: 2017

In recent years, more research has been devoted to studying the subtask of the complete shallow discourse parsing, such as indentifying discourse connective and arguments of connective. There is a need to design a full discourse parser to pull these subtasks together. So we develop a discourse parser turning the free text into discourse relations. The parser includes connective identifier, arguments identifier, sense classifier and non-explicit identifier, which connects with each other in pipeline. Each component applies the maximum entropy model with abundant lexical and syntax features extracted from the Penn Discourse Tree-bank. The head-based representation of the PDTB is adopted in the arguments identifier, which turns the problem of indentifying the arguments of discourse connective into finding the head and end of the arguments. In the non-explicit identifier, the contextual type features like words which have high frequency and can reflect the discourse relation are introduced to improve the performance of non-explicit identifier. Compared with other methods, experimental results achieve the considerable performance.
Parametrizing filters of a CNN with a GAN
Venue: arXiv cs.LG 2017
Year: 2017

It is commonly agreed that the use of relevant invariances as a good statistical bias is important in machine-learning. However, most approaches that explicitly incorporate invariances into a model architecture only make use of very simple transformations, such as translations and rotations. Hence, there is a need for methods to model and extract richer transformations that capture much higher-level invariances. To that end, we introduce a tool allowing to parametrize the set of filters of a trained convolutional neural network with the latent space of a generative adversarial network. We then show that the method can capture highly non-linear invariances of the data by visualizing their effect in the data space.