OpenRaiser/Intern-Atlas huggingface dataset:
BioNLP-OST 2019 RDoC Tasks: Multi-grain Neural Relevance Ranking Using Topics and Attention Based Query-Document-Sentence Interactions
Venue: arXiv cs.LG 2019
Year: 2019

This paper presents our system details and results of participation in the RDoC Tasks of BioNLP-OST 2019. Research Domain Criteria (RDoC) construct is a multi-dimensional and broad framework to describe mental health disorders by combining knowledge from genomics to behaviour. Non-availability of RDoC labelled dataset and tedious labelling process hinders the use of RDoC framework to reach its full potential in Biomedical research community and Healthcare industry. Therefore, Task-1 aims at retrieval and ranking of PubMed abstracts relevant to a given RDoC construct and Task-2 aims at extraction of the most relevant sentence from a given PubMed abstract. We investigate (1) attention based supervised neural topic model and SVM for retrieval and ranking of PubMed abstracts and, further utilize BM25 and other relevance measures for re-ranking, (2) supervised and unsupervised sentence ranking models utilizing multi-view representations comprising of query-aware attention-based sentence representation (QAR), bag-of-words (BoW) and TF-IDF. Our best systems achieved 1st rank and scored 0.86 mean average precision (mAP) and 0.58 macro average accuracy (MAA) in Task-1 and Task-2 respectively.
Highway to HAL: Open-Sourcing the First Extendable Gate-Level Netlist Reverse Engineering Framework
Venue: arXiv cs.CR 2019
Year: 2019

Since hardware oftentimes serves as the root of trust in our modern interconnected world, malicious hardware manipulations constitute a ubiquitous threat in the context of the Internet of Things (IoT). Hardware reverse engineering is a prevalent technique to detect such manipulations. Over the last years, an active research community has significantly advanced the field of hardware reverse engineering. Notably, many open research questions regarding the extraction of functionally correct netlists from Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs) have been tackled. In order to facilitate further analysis of recovered netlists, a software framework is required, serving as the foundation for specialized algorithms. Currently, no such framework is publicly available. Therefore, we provide the first open-source gate-library agnostic framework for gate-level netlist analysis. In this positional paper, we demonstrate the workflow of our modular framework HAL on the basis of two case studies and provide profound insights on its technical foundations.
Proximal Policy Optimization for Improved Convergence in IRGAN
Venue: arXiv cs.IR 2019
Year: 2019

IRGAN is an information retrieval (IR) modeling approach that uses a theoretical minimax game between a generative and a discriminative model to iteratively optimize both of them, hence unifying the generative and discriminative approaches. Despite significant performance improvements in several information retrieval tasks, IRGAN training is an unstable process, and the solution varies largely with the random parameter initialization. In this work, we present an improved training objective based on proximal policy optimization objective and Gumbel-Softmax based sampling for the generator. We also propose a modified training algorithm which takes a single gradient update on both the generator as well as discriminator for each iteration step. We present empirical evidence of the improved convergence of the proposed model over the original IRGAN and a comparison on three different IR tasks on benchmark datasets is also discussed, emphasizing the proposed model's superior performance.
Leveraging Model Interpretability and Stability to increase Model Robustness
Venue: arXiv cs.LG 2019
Year: 2019

State of the art Deep Neural Networks (DNN) can now achieve above human level accuracy on image classification tasks. However their outstanding performances come along with a complex inference mechanism making them arduously interpretable models. In order to understand the underlying prediction rules of DNNs, Dhamdhere et al. propose an interpretability method to break down a DNN prediction score as sum of its hidden unit contributions, in the form of a metric called conductance. Analyzing conductances of DNN hidden units, we find out there is a difference in how wrong and correct predictions are inferred. We identify distinguishable patterns of hidden unit activations for wrong and correct predictions. We then use an error detector in the form of a binary classifier on top of the DNN to automatically discriminate wrong and correct predictions of the DNN based on their hidden unit activations. Detected wrong predictions are discarded, increasing the model robustness. A different approach to distinguish wrong and correct predictions of DNNs is proposed by Wang et al. whose method is based on the premise that input samples leading a DNN into making wrong predictions are less stable to the DNN weight changes than correctly classified input samples. In our study, we compare both methods and find out by combining them that better detection of wrong predictions can be achieved.
Decision Explanation and Feature Importance for Invertible Networks
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier can be viewed as a two-stage model: (1) invertible transformation from input space to the feature space; (2) a linear classifier in the feature space. We can determine the decision boundary of a linear classifier in the feature space; since the transform is invertible, we can invert the decision boundary from the feature space to the input space. Furthermore, we propose to determine the projection of a data point onto the decision boundary, and define explanation as the difference between data and its projection. Finally, we propose to locally approximate a neural network with its first-order Taylor expansion, and define feature importance using a local linear model. We provide the implementation of our method: \url{https://github.com/juntang-zhuang/explain_invertible}.
AV Speech Enhancement Challenge using a Real Noisy Corpus
Venue: arXiv cs.SD 2019
Year: 2019

This paper presents, a first of its kind, audio-visual (AV) speech enhacement challenge in real-noisy settings. A detailed description of the AV challenge, a novel real noisy AV corpus (ASPIRE), benchmark speech enhancement task, and baseline performance results are outlined. The latter are based on training a deep neural architecture on a synthetic mixture of Grid corpus and ChiME3 noises (consisting of bus, pedestrian, cafe, and street noises) and testing on the ASPIRE corpus. Subjective evaluations of five different speech enhancement algorithms (including SEAGN, spectrum subtraction (SS) , log-minimum mean-square error (LMMSE), audio-only CochleaNet, and AV CochleaNet) are presented as baseline results. The aim of the multi-modal challenge is to provide a timely opportunity for comprehensive evaluation of novel AV speech enhancement algorithms, using our new benchmark, real-noisy AV corpus and specified performance metrics. This will promote AV speech processing research globally, stimulate new ground-breaking multi-modal approaches, and attract interest from companies, academics and researchers working in AV speech technologies and applications. We encourage participants (through a challenge website sign-up) from both the speech and hearing research communities, to benefit from their complementary approaches to AV speech in noise processing.
Conjure Documentation, Release 2.3.0
Venue: arXiv cs.OH 2019
Year: 2019

Conjure is an automated modelling tool for Constraint Programming. In this documentation, you will find the following: A brief introduction to Conjure, installation instructions, a description of how to use Conjure through its command line user interface, a list of Conjure's features, a description of Conjure's input language Essence, and a collection of simple demonstrations of Conjure's use.
Beacons in Dense Wi-Fi Networks: How to Befriend with Neighbors in the 5G World?
Venue: arXiv cs.NI 2019
Year: 2019

To address 5G challenges, IEEE 802.11 is currently developing new amendments to the Wi-Fi standard, the most promising of which is 802.11ax. A key scenario considered by the developers of this amendment is dense and overlapped networks typically present in residential buildings, offices, airports, stadiums, and other places of a modern city. Being crucial for Wi-Fi hotspots, the hidden station problem becomes even more challenging for dense and overlapped networks, where even access points (APs) can be hidden. In this case, user stations can experience continuous collisions of beacons sent by different APs, which can cause disassociation and break Internet access. In this paper, we show that beacon collisions are rather typical for residential networks and may lead to unexpected and irreproducible malfunction. We investigate how often beacon collisions occur, and describe a number of mechanisms which can be used to avoid beacon collisions in dense deployment. Specifically, we pay much attention to those mechanisms which are currently under consideration of the IEEE 802.11ax group.
Learning Multi-Stage Sparsification for Maximum Clique Enumeration
Venue: arXiv cs.LG 2019
Year: 2019

We propose a multi-stage learning approach for pruning the search space of maximum clique enumeration, a fundamental computationally difficult problem arising in various network analysis tasks. In each stage, our approach learns the characteristics of vertices in terms of various neighborhood features and leverage them to prune the set of vertices that are likely not contained in any maximum clique. Furthermore, we demonstrate that our approach is domain independent -- the same small set of features works well on graph instances from different domain. Compared to the state-of-the-art heuristics and preprocessing strategies, the advantages of our approach are that (i) it does not require any estimate on the maximum clique size at runtime and (ii) we demonstrate it to be effective also for dense graphs. In particular, for dense graphs, we typically prune around 30 \% of the vertices resulting in speedups of up to 53 times for state-of-the-art solvers while generally preserving the size of the maximum clique (though some maximum cliques may be lost). For large real-world sparse graphs, we routinely prune over 99 \% of the vertices resulting in several tenfold speedups at best, typically with no impact on solution quality.
Augmenting learning using symmetry in a biologically-inspired domain
Venue: arXiv cs.LG 2019
Year: 2019

Invariances to translation, rotation and other spatial transformations are a hallmark of the laws of motion, and have widespread use in the natural sciences to reduce the dimensionality of systems of equations. In supervised learning, such as in image classification tasks, rotation, translation and scale invariances are used to augment training datasets. In this work, we use data augmentation in a similar way, exploiting symmetry in the quadruped domain of the DeepMind control suite (Tassa et al. 2018) to add to the trajectories experienced by the actor in the actor-critic algorithm of Abdolmaleki et al. (2018). In a data-limited regime, the agent using a set of experiences augmented through symmetry is able to learn faster. Our approach can be used to inject knowledge of invariances in the domain and task to augment learning in robots, and more generally, to speed up learning in realistic robotics applications.
Scalable String Reconciliation by Recursive Content-Dependent Shingling
Venue: arXiv cs.IT 2019
Year: 2019

We consider the problem of reconciling similar, but remote, strings with minimum communication complexity. This "string reconciliation" problem is a fundamental building block for a variety of networking applications, including those that maintain large-scale distributed networks and perform remote file synchronization. We present the novel Recursive Content-Dependent Shingling (RCDS) protocol that is computationally practical for large strings and scales linearly with the edit distance between the remote strings. We provide comparisons to the performance of Rsync, one of the most popular file synchronization tools in active use. Our experiments show that, with minimal engineering, RCDS outperforms the heavily optimized Rsync in reconciling release revisions for about 51% of the 5000 top starred git repositories on GitHub. The improvement is particularly evident for repositories that see frequent, but small, updates.
On the Parameterized Complexity of Reconfiguration of Connected Dominating Sets
Venue: arXiv cs.DS 2019
Year: 2019

In a reconfiguration version of an optimization problem $\mathcal{Q}$ the input is an instance of $\mathcal{Q}$ and two feasible solutions $S$ and $T$. The objective is to determine whether there exists a step-by-step transformation between $S$ and $T$ such that all intermediate steps also constitute feasible solutions. In this work, we study the parameterized complexity of the \textsc{Connected Dominating Set Reconfiguration} problem (\textsc{CDS-R)}. It was shown in previous work that the \textsc{Dominating Set Reconfiguration} problem (\textsc{DS-R}) parameterized by $k$, the maximum allowed size of a dominating set in a reconfiguration sequence, is fixed-parameter tractable on all graphs that exclude a biclique $K_{d,d}$ as a subgraph, for some constant $d \geq 1$. We show that the additional connectivity constraint makes the problem much harder, namely, that \textsc{CDS-R} is \textsf{W}$[1]$-hard parameterized by $k+\ell$, the maximum allowed size of a dominating set plus the length of the reconfiguration sequence, already on $5$-degenerate graphs. On the positive side, we show that \textsc{CDS-R} parameterized by $k$ is fixed-parameter tractable, and in fact admits a polynomial kernel on planar graphs.
Essentia: Mining Domain-Specific Paraphrases with Word-Alignment Graphs
Venue: arXiv cs.CL 2019
Year: 2019

Paraphrases are important linguistic resources for a wide variety of NLP applications. Many techniques for automatic paraphrase mining from general corpora have been proposed. While these techniques are successful at discovering generic paraphrases, they often fail to identify domain-specific paraphrases (e.g., {staff, concierge} in the hospitality domain). This is because current techniques are often based on statistical methods, while domain-specific corpora are too small to fit statistical methods. In this paper, we present an unsupervised graph-based technique to mine paraphrases from a small set of sentences that roughly share the same topic or intent. Our system, Essentia, relies on word-alignment techniques to create a word-alignment graph that merges and organizes tokens from input sentences. The resulting graph is then used to generate candidate paraphrases. We demonstrate that our system obtains high-quality paraphrases, as evaluated by crowd workers. We further show that the majority of the identified paraphrases are domain-specific and thus complement existing paraphrase databases.
Automated Weed Detection in Aerial Imagery with Context
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we demonstrate the ability to discriminate between cultivated maize plant and grass or grass-like weed image segments using the context surrounding the image segments. While convolutional neural networks have brought state of the art accuracies within object detection, errors arise when objects in different classes share similar features. This scenario often occurs when objects in images are viewed at too small of a scale to discern distinct differences in features, causing images to be incorrectly classified or localized. To solve this problem, we will explore using context when classifying image segments. This technique involves feeding a convolutional neural network a central square image along with a border of its direct surroundings at train and test times. This means that although images are labelled at a smaller scale to preserve accurate localization, the network classifies the images and learns features that include the wider context. We demonstrate the benefits of this context technique in the object detection task through a case study of grass (foxtail) and grass-like (yellow nutsedge) weed detection in maize fields. In this standard situation, adding context alone nearly halved the error of the neural network from 7.1% to 4.3%. After only one epoch with context, the network also achieved a higher accuracy than the network without context did after 50 epochs. The benefits of using the context technique are likely to particularly evident in agricultural contexts in which parts (such as leaves) of several plants may appear similar when not taking into account the context in which those parts appear.
NB-IoT Small Cell: A 3GPP Perspective
Venue: arXiv cs.NI 2019
Year: 2019

Narrowband Internet of Things (NB-IoT) technology was introduced in 3GPP Release 13 to accommodate device-generated traffic over cellular networks. There have been efforts in Release 14 to deliver further improvements to facilitate the deployment of Internet of Things (IoT) use cases, such as coverage enhancement, and support for low cost, low complexity and low power consumption device. Small cell NB-IoT is added as an attractive feature in 3GPP Release 15. In this article, we provide an overview of this feature and try to shed light on major aspects of small cell deployment in NB-IoT systems.
Side-Channel Hardware Trojan for Provably-Secure SCA-Protected Implementations
Venue: arXiv cs.CR 2019
Year: 2019

Hardware Trojans have drawn the attention of academia, industry and government agencies. Effective detection mechanisms and countermeasures against such malicious designs can only be developed when there is a deep understanding of how hardware Trojans can be built in practice, in particular Trojans specifically designed to avoid detection. In this work, we present a mechanism to introduce an extremely stealthy hardware Trojan into cryptographic primitives equipped with provably-secure first-order side-channel countermeasures. Once the Trojan is triggered, the malicious design exhibits exploitable side-channel leakage, leading to successful key recovery attacks. Generally, such a Trojan requires neither addition nor removal of any logic which makes it extremely hard to detect. On ASICs, it can be inserted by subtle manipulations at the sub-transistor level and on FPGAs by changing the routing of particular signals, leading to \textbf{zero} logic overhead. The underlying concept is based on modifying a securely-masked hardware implementation in such a way that running the device at a particular clock frequency violates one of its essential properties, leading to exploitable leakage. We apply our technique to a Threshold Implementation of the PRESENT block cipher realized in two different CMOS technologies, and show that triggering the Trojan makes the ASIC prototypes vulnerable.
Quantifying Voter Biases in Online Platforms: An Instrumental Variable Approach
Venue: arXiv cs.SI 2019
Year: 2019

In content-based online platforms, use of aggregate user feedback (say, the sum of votes) is commonplace as the "gold standard" for measuring content quality. Use of vote aggregates, however, is at odds with the existing empirical literature, which suggests that voters are susceptible to different biases -- reputation (e.g., of the poster), social influence (e.g., votes thus far), and position (e.g., answer position). Our goal is to quantify, in an observational setting, the degree of these biases in online platforms. Specifically, what are the causal effects of different impression signals -- such as the reputation of the contributing user, aggregate vote thus far, and position of content -- on a participant's vote on content? We adopt an instrumental variable (IV) framework to answer this question. We identify a set of candidate instruments, carefully analyze their validity, and then use the valid instruments to reveal the effects of the impression signals on votes. Our empirical study using log data from Stack Exchange websites shows that the bias estimates from our IV approach differ from the bias estimates from the ordinary least squares (OLS) method. In particular, OLS underestimates reputation bias (1.6--2.2x for gold badges) and position bias (up to 1.9x for the initial position) and overestimates social influence bias (1.8--2.3x for initial votes). The implications of our work include: redesigning user interface to avoid voter biases; making changes to platforms' policy to mitigate voter biases; detecting other forms of biases in online platforms.
Deploying the NASA Valkyrie Humanoid for IED Response: An Initial Approach and Evaluation Summary
Venue: arXiv cs.RO 2019
Year: 2019

As part of a feasibility study, this paper shows the NASA Valkyrie humanoid robot performing an end-to-end improvised explosive device (IED) response task. To demonstrate and evaluate robot capabilities, sub-tasks highlight different locomotion, manipulation, and perception requirements: traversing uneven terrain, passing through a narrow passageway, opening a car door, retrieving a suspected IED, and securing the IED in a total containment vessel (TCV). For each sub-task, a description of the technical approach and the hidden challenges that were overcome during development are presented. The discussion of results, which explicitly includes existing limitations, is aimed at motivating continued research and development to enable practical deployment of humanoid robots for IED response. For instance, the data shows that operator pauses contribute to 50\% of the total completion time, which implies that further work is needed on user interfaces for increasing task completion efficiency.
Accelerating Deep Learning by Focusing on the Biggest Losers
Venue: arXiv cs.LG 2019
Year: 2019

This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Selective-Backprop uses the output of a training example's forward pass to decide whether to use that example to compute gradients and update parameters, or to skip immediately to the next example. By reducing the number of computationally-expensive backpropagation steps performed, Selective-Backprop accelerates training. Evaluation on CIFAR10, CIFAR100, and SVHN, across a variety of modern image models, shows that Selective-Backprop converges to target error rates up to 3.5x faster than with standard SGD and between 1.02--1.8x faster than a state-of-the-art importance sampling approach. Further acceleration of 26% can be achieved by using stale forward pass results for selection, thus also skipping forward passes of low priority examples.
Variational Temporal Abstraction
Venue: arXiv cs.LG 2019
Year: 2019

We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data. We propose the Variational Temporal Abstraction (VTA), a hierarchical recurrent state space model that can infer the latent temporal structure and thus perform the stochastic state transition hierarchically. We also propose to apply this model to implement the jumpy-imagination ability in imagination-augmented agent-learning in order to improve the efficiency of the imagination. In experiments, we demonstrate that our proposed method can model 2D and 3D visual sequence datasets with interpretable temporal structure discovery and that its application to jumpy imagination enables more efficient agent-learning in a 3D navigation task.