OpenRaiser/Intern-Atlas huggingface dataset:
CONAN: Complementary Pattern Augmentation for Rare Disease Detection
Venue: arXiv cs.LG 2019
Year: 2019

Rare diseases affect hundreds of millions of people worldwide but are hard to detect since they have extremely low prevalence rates (varying from 1/1,000 to 1/200,000 patients) and are massively underdiagnosed. How do we reliably detect rare diseases with such low prevalence rates? How to further leverage patients with possibly uncertain diagnosis to improve detection? In this paper, we propose a Complementary pattern Augmentation (CONAN) framework for rare disease detection. CONAN combines ideas from both adversarial training and max-margin classification. It first learns self-attentive and hierarchical embedding for patient pattern characterization. Then, we develop a complementary generative adversarial networks (GAN) model to generate candidate positive and negative samples from the uncertain patients by encouraging a max-margin between classes. In addition, CONAN has a disease detector that serves as the discriminator during the adversarial training for identifying rare diseases. We evaluated CONAN on two disease detection tasks. For low prevalence inflammatory bowel disease (IBD) detection, CONAN achieved .96 precision recall area under the curve (PR-AUC) and 50.1% relative improvement over best baseline. For rare disease idiopathic pulmonary fibrosis (IPF) detection, CONAN achieves .22 PR-AUC with 41.3% relative improvement over the best baseline.
Domain-Aware Dynamic Networks
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks with more parameters and FLOPs have higher capacity and generalize better to diverse domains. But to be deployed on edge devices, the model's complexity has to be constrained due to limited compute resource. In this work, we propose a method to improve the model capacity without increasing inference-time complexity. Our method is based on an assumption of data locality: for an edge device, within a short period of time, the input data to the device are sampled from a single domain with relatively low diversity. Therefore, it is possible to utilize a specialized, low-complexity model to achieve good performance in that input domain. To leverage this, we propose Domain-aware Dynamic Network (DDN), which is a high-capacity dynamic network in which each layer contains multiple weights. During inference, based on the input domain, DDN dynamically combines those weights into one single weight that specializes in the given domain. This way, DDN can keep the inference-time complexity low but still maintain a high capacity. Experiments show that without increasing the parameters, FLOPs, and actual latency, DDN achieves up to 2.6\% higher AP50 than a static network on the BDD100K object-detection benchmark.
Heuristic Strategies in Uncertain Approval Voting Environments
Venue: arXiv cs.GT 2019
Year: 2019

In many collective decision making situations, agents vote to choose an alternative that best represents the preferences of the group. Agents may manipulate the vote to achieve a better outcome by voting in a way that does not reflect their true preferences. In real world voting scenarios, people often do not have complete information about other voter preferences and it can be computationally complex to identify a strategy that will maximize their expected utility. In such situations, it is often assumed that voters will vote truthfully rather than expending the effort to strategize. However, being truthful is just one possible heuristic that may be used. In this paper, we examine the effectiveness of heuristics in single winner and multi-winner approval voting scenarios with missing votes. In particular, we look at heuristics where a voter ignores information about other voting profiles and makes their decisions based solely on how much they like each candidate. In a behavioral experiment, we show that people vote truthfully in some situations and prioritize high utility candidates in others. We examine when these behaviors maximize expected utility and show how the structure of the voting environment affects both how well each heuristic performs and how humans employ these heuristics.
Negative Translations for Affine and Lukasiewicz Logic
Venue: arXiv cs.LO 2019
Year: 2019

We investigate four well-known negative translations of classical logic into intuitionistic logic within a substructural setting. We find that in affine logic the translation schemes due to Kolmogorov and Gödel both satisfy Troelstra's criteria for a negative translation. On the other hand, the schemes of Glivenko and Gentzen both fail for affine logic, but for different reasons: one can extend affine logic to make Glivenko work and Gentzen fail and vice versa. By contrast, in the setting of Lukasiewicz logic, we can prove a general result asserting that a wide class of formula translations including those of Kolmogorov, Gödel, Gentzen and Glivenko not only satisfy Troelstra's criteria with respect to a natural intuitionistic fragment of Lukasiewicz logic but are all equivalent.
Tag Recommendation by Word-Level Tag Sequence Modeling
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we transform tag recommendation into a word-based text generation problem and introduce a sequence-to-sequence model. The model inherits the advantages of LSTM-based encoder for sequential modeling and attention-based decoder with local positional encodings for learning relations globally. Experimental results on Zhihu datasets illustrate the proposed model outperforms other state-of-the-art text classification based methods.
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation
Venue: arXiv cs.LG 2019
Year: 2019

Recent advances in the sparse neural network literature have made it possible to prune many large feed forward and convolutional networks with only a small quantity of data. Yet, these same techniques often falter when applied to the problem of recovering sparse recurrent networks. These failures are quantitative: when pruned with recent techniques, RNNs typically obtain worse performance than they do under a simple random pruning scheme. The failures are also qualitative: the distribution of active weights in a pruned LSTM or GRU network tend to be concentrated in specific neurons and gates, and not well dispersed across the entire architecture. We seek to rectify both the quantitative and qualitative issues with recurrent network pruning by introducing a new recurrent pruning objective derived from the spectrum of the recurrent Jacobian. Our objective is data efficient (requiring only 64 data points to prune the network), easy to implement, and produces 95% sparse GRUs that significantly improve on existing baselines. We evaluate on sequential MNIST, Billion Words, and Wikitext.
Federated Learning with Autotuned Communication-Efficient Secure Aggregation
Venue: arXiv cs.DC 2019
Year: 2019

Federated Learning enables mobile devices to collaboratively learn a shared inference model while keeping all the training data on a user's device, decoupling the ability to do machine learning from the need to store the data in the cloud. Existing work on federated learning with limited communication demonstrates how random rotation can enable users' model updates to be quantized much more efficiently, reducing the communication cost between users and the server. Meanwhile, secure aggregation enables the server to learn an aggregate of at least a threshold number of device's model contributions without observing any individual device's contribution in unaggregated form. In this paper, we highlight some of the challenges of setting the parameters for secure aggregation to achieve communication efficiency, especially in the context of the aggressively quantized inputs enabled by random rotation. We then develop a recipe for auto-tuning communication-efficient secure aggregation, based on specific properties of random rotation and secure aggregation -- namely, the predictable distribution of vector entries post-rotation and the modular wrapping inherent in secure aggregation. We present both theoretical results and initial experiments.
Sub-pixel matching method for low-resolution thermal stereo images
Venue: arXiv cs.CV 2019
Year: 2019

In the context of a localization and tracking application, we developed a stereo vision system based on cheap low-resolution 80x60 pixels thermal cameras. We proposed a threefold sub-pixel stereo matching framework (called ST for Subpixel Thermal): 1) robust features extraction method based on phase congruency, 2) rough matching of these features in pixel precision, and 3) refined matching in sub-pixel accuracy based on local phase coherence. We performed experiments on our very low-resolution thermal images (acquired using a stereo system we manufactured) as for high-resolution images from a benchmark dataset. Even if phase congruency computation time is high, it was able to extract two times more features than state-of-the-art methods such as ORB or SURF. We proposed a modified version of the phase correlation applied in the phase congruency feature space for sub-pixel matching. Using simulated stereo, we investigated how the phase congruency threshold and the sub-image size of sub-pixel matching can influence the accuracy. We then proved that given our stereo setup and the resolution of our images, being wrong of 1 pixel leads to a 500 mm error in the Z position of the point. Finally, we showed that our method could extract four times more matches than a baseline method ORB + OpenCV KNN matching on low-resolution images. Moreover, our matches were more robust. More precisely, when projecting points of a standing person, ST got a standard deviation of 300 mm when ORB + OpenCV KNN gave more than 1000 mm.
Learning Rate Dropout
Venue: arXiv cs.CV 2019
Year: 2019

The performance of a deep neural network is highly dependent on its training, and finding better local optimal solutions is the goal of many optimization algorithms. However, existing optimization algorithms show a preference for descent paths that converge slowly and do not seek to avoid bad local optima. In this work, we propose Learning Rate Dropout (LRD), a simple gradient descent technique for training related to coordinate descent. LRD empirically aids the optimizer to actively explore in the parameter space by randomly setting some learning rates to zero; at each iteration, only parameters whose learning rate is not 0 are updated. As the learning rate of different parameters is dropped, the optimizer will sample a new loss descent path for the current update. The uncertainty of the descent path helps the model avoid saddle points and bad local minima. Experiments show that LRD is surprisingly effective in accelerating training while preventing overfitting.
Pebble-Intervals Automata and FO2 with Two Orders (Extended Version)
Venue: arXiv cs.LO 2019
Year: 2019

We introduce a novel automata model, called pebble-intervals automata (PIA), and study its power and closure properties. PIAs are tailored for a decidable fragment of FO that is important for reasoning about structures that use data values from infinite domains: the two-variable fragment with one total preorder and its induced successor relation, one linear order, and an arbitrary number of unary relations. We prove that the string projection of every language of data words definable in the logic is accepted by a pebble-intervals automaton A, and obtain as a corollary an automata-theoretic proof of the EXPSPACE upper bound for finite satisfiability due to Schwentick and Zeume.
Scalable Graph Algorithms
Venue: arXiv cs.DS 2019
Year: 2019

Processing large complex networks recently attracted considerable interest. Complex graphs are useful in a wide range of applications from technological networks to biological systems like the human brain. Sometimes these networks are composed of billions of entities that give rise to emerging properties and structures. Analyzing these structures aids us in gaining new insights about our surroundings. As huge networks become abundant, there is a need for scalable algorithms to perform analysis. A prominent example is the PageRank algorithm, which is one of the measures used by web search engines such as Google to rank web pages displayed to the user. In order to find these patterns, massive amounts of data have to be acquired and processed. Designing and evaluating scalable graph algorithms to handle these data sets is a crucial task on the road to understanding the underlying systems. This habilitation thesis is a summary a broad spectrum of scalable graph algorithms that I developed over the last six years with many coauthors. In general, this research is based on four pillars: multilevel algorithms, practical kernelization, parallelization and memetic algorithms that are highly interconnected. Experiments conducted indicate that our algorithms find better solutions and/or are much more scalable than the previous state-of-the-art.
Idle Time Optimization for Target Assignment and Path Finding in Sortation Centers
Venue: arXiv cs.AI 2019
Year: 2019

In this paper, we study the one-shot and lifelong versions of the Target Assignment and Path Finding problem in automated sortation centers, where each agent needs to constantly assign itself a sorting station, move to its assigned station without colliding with obstacles or other agents, wait in the queue of that station to obtain a parcel for delivery, and then deliver the parcel to a sorting bin. The throughput of such centers is largely determined by the total idle time of all stations since their queues can frequently become empty. To address this problem, we first formalize and study the one-shot version that assigns stations to a set of agents and finds collision-free paths for the agents to their assigned stations. We present efficient algorithms for this task based on a novel min-cost max-flow formulation that minimizes the total idle time of all stations in a fixed time window. We then demonstrate how our algorithms for solving the one-shot problem can be applied to solving the lifelong problem as well. Experimentally, we believe to be the first researchers to consider real-world automated sortation centers using an industrial simulator with realistic data and a kinodynamic model of real robots. On this simulator, we showcase the benefits of our algorithms by demonstrating their efficiency and effectiveness for up to 350 agents.
Towards Efficient Integration of Blockchain for IoT Security: The Case Study of IoT Remote Access
Venue: arXiv cs.CR 2019
Year: 2019

The booming Internet of Things (IoT) market has drawn tremendous interest from cyber attackers. The centralized cloud-based IoT service architecture has serious limitations in terms of security, availability, and scalability, and is subject to single points of failure (SPOF). Recently, accommodating IoT services on blockchains has become a trend for better security, privacy, and reliability. However, blockchain's shortcomings of high cost, low throughput, and long latency make it unsuitable for IoT applications. In this paper, we take a retrospection of existing blockchain-based IoT solutions and propose a framework for efficient blockchain and IoT integration. Following the framework, we design a novel blockchain-assisted decentralized IoT remote accessing system, RS-IoT, which has the advantage of defending IoT devices against zero-day attacks without relying on any trusted third-party. By introducing incentives and penalties enforced by smart contracts, our work enables "an economic approach" to thwarting the majority of attackers who aim to achieve monetary gains. Our work presents an example of how blockchain can be used to ensure the fairness of service trading in a decentralized environment and punish misbehaviors objectively. We show the security of RS-IoT via detailed security analyses. Finally, we demonstrate its scalability, efficiency, and usability through a proof-of-concept implementation on the Ethereum testnet blockchain.
Who Asked Us? How the Theory of Computing Answers Questions about Analysis
Venue: arXiv cs.CC 2019
Year: 2019

Algorithmic fractal dimensions -- constructs of computability theory -- have recently been used to answer open questions in classical geometric measure theory, questions of mathematical analysis whose statements do not involve computability theory or logic. We survey these developments and the prospects for future such results.
Training Distributed Deep Recurrent Neural Networks with Mixed Precision on GPU Clusters
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we evaluate training of deep recurrent neural networks with half-precision floats. We implement a distributed, data-parallel, synchronous training algorithm by integrating TensorFlow and CUDA-aware MPI to enable execution across multiple GPU nodes and making use of high-speed interconnects. We introduce a learning rate schedule facilitating neural network convergence at up to $O(100)$ workers. Strong scaling tests performed on clusters of NVIDIA Pascal P100 GPUs show linear runtime and logarithmic communication time scaling for both single and mixed precision training modes. Performance is evaluated on a scientific dataset taken from the Joint European Torus (JET) tokamak, containing multi-modal time series of sensory measurements leading up to deleterious events called plasma disruptions, and the benchmark Large Movie Review Dataset~\cite{imdb}. Half-precision significantly reduces memory and network bandwidth, allowing training of state-of-the-art models with over 70 million trainable parameters while achieving a comparable test set performance as single precision.
Not All Attention Is Needed: Gated Attention Network for Sequence Data
Venue: arXiv cs.LG 2019
Year: 2019

Although deep neural networks generally have fixed network structures, the concept of dynamic mechanism has drawn more and more attention in recent years. Attention mechanisms compute input-dependent dynamic attention weights for aggregating a sequence of hidden states. Dynamic network configuration in convolutional neural networks (CNNs) selectively activates only part of the network at a time for different inputs. In this paper, we combine the two dynamic mechanisms for text classification tasks. Traditional attention mechanisms attend to the whole sequence of hidden states for an input sentence, while in most cases not all attention is needed especially for long sequences. We propose a novel method called Gated Attention Network (GA-Net) to dynamically select a subset of elements to attend to using an auxiliary network, and compute attention weights to aggregate the selected elements. It avoids a significant amount of unnecessary computation on unattended elements, and allows the model to pay attention to important parts of the sequence. Experiments in various datasets show that the proposed method achieves better performance compared with all baseline models with global or local attention while requiring less computation and achieving better interpretability. It is also promising to extend the idea to more complex attention-based models, such as transformers and seq-to-seq models.
Online Knowledge Distillation with Diverse Peers
Venue: arXiv cs.LG 2019
Year: 2019

Distillation is an effective knowledge-transfer technique that uses predicted distributions of a powerful teacher model as soft targets to train a less-parameterized student model. A pre-trained high capacity teacher, however, is not always available. Recently proposed online variants use the aggregated intermediate predictions of multiple student models as targets to train each student model. Although group-derived targets give a good recipe for teacher-free distillation, group members are homogenized quickly with simple aggregation functions, leading to early saturated solutions. In this work, we propose Online Knowledge Distillation with Diverse peers (OKDDip), which performs two-level distillation during training with multiple auxiliary peers and one group leader. In the first-level distillation, each auxiliary peer holds an individual set of aggregation weights generated with an attention-based mechanism to derive its own targets from predictions of other auxiliary peers. Learning from distinct target distributions helps to boost peer diversity for effectiveness of group-based distillation. The second-level distillation is performed to transfer the knowledge in the ensemble of auxiliary peers further to the group leader, i.e., the model used for inference. Experimental results show that the proposed framework consistently gives better performance than state-of-the-art approaches without sacrificing training or inference complexity, demonstrating the effectiveness of the proposed two-level distillation framework.
End to End Trainable Active Contours via Differentiable Rendering
Venue: arXiv cs.CV 2019
Year: 2019

We present an image segmentation method that iteratively evolves a polygon. At each iteration, the vertices of the polygon are displaced based on the local value of a 2D shift map that is inferred from the input image via an encoder-decoder architecture. The main training loss that is used is the difference between the polygon shape and the ground truth segmentation mask. The network employs a neural renderer to create the polygon from its vertices, making the process fully differentiable. We demonstrate that our method outperforms the state of the art segmentation networks and deep active contour solutions in a variety of benchmarks, including medical imaging and aerial images. Our code is available at https://github.com/shirgur/ACDRNet.
Using Massive MIMO Arrays for Joint Communication and Sensing
Venue: arXiv cs.IT 2019
Year: 2019

One of the trends that is gaining more and more importance in the field of beyond-5G and 6G wireless communication systems is the investigation on systems that jointly perform communication and sensing of the environment. This paper proposes to use a base station (BS), that we call \textit{radar-BS}, equipped with a large-scale antenna array to execute, using the same frequency range, communication with mobile users and sensing/surveillance of the surrounding environment through radar scanning. The massive antenna array can indeed both operate as a MIMO radar with co-located antennas -- transmitting radar signals pointing at positive elevation angles -- and perform signal-space beamforming to communicate with users mainly based on the ground. Our results show that using a massive MIMO radar-BS the communication and the radar system can coexist with little mutual interference.
AD-EYE: A Co-simulation Platform for Early Verification of Functional Safety Concepts
Venue: arXiv cs.RO 2019
Year: 2019

Automated Driving is revolutionizing many of the traditional ways of operation in the automotive industry. The impact on safety engineering of automotive functions is arguably one of the most important changes. There has been a need to re-think the impact of the partial or complete absence of the human driver (in terms of a supervisory entity) in not only newly developed functions but also in the qualification of the use of legacy functions in new contexts. The scope of the variety of scenarios that a vehicle may encounter even within a constrained Operational Design Domain, and the highly dynamic nature of Automated Driving, mean that new methods such as simulation can greatly aid the process of safety engineering. This paper discusses the need for early verification of the Functional Safety Concepts (FSCs), details the information typically available at this stage in the product lifecycle, and proposes a co-simulation platform named AD-EYE designed for exploiting the possibilities in an industrial context by evaluating design decisions and refining Functional Safety Requirements based on a reusable scenario database. Leveraging our prior experiences in developing FSCs for Automated Driving functions, and the preliminary implementation of co-simulation platform, we demonstrate the advantages and identify the limitations of using simulations for refinement and early FSC verification using examples of types of requirements that could benefit from our methodology.