OpenRaiser/Intern-Atlas huggingface dataset:
Understanding the Interactions of Workloads and DRAM Types: A Comprehensive Experimental Study
Venue: arXiv cs.AR 2019
Year: 2019

It has become increasingly difficult to understand the complex interaction between modern applications and main memory, composed of DRAM chips. Manufacturers are now selling and proposing many different types of DRAM, with each DRAM type catering to different needs (e.g., high throughput, low power, high memory density). At the same time, the memory access patterns of prevalent and emerging workloads are rapidly diverging, as these applications manipulate larger data sets in very different ways. As a result, the combined DRAM-workload behavior is often difficult to intuitively determine today, which can hinder memory optimizations in both hardware and software. In this work, we identify important families of workloads, as well as prevalent types of DRAM chips, and rigorously analyze the combined DRAM--workload behavior. To this end, we perform a comprehensive experimental study of the interaction between nine different DRAM types and 115 modern applications and multiprogrammed workloads. We draw 12 key observations from our characterization, enabled in part by our development of new metrics that take into account contention between memory requests due to hardware design. Notably, we find that (1) newer DRAM types such as DDR4 and HMC often do not outperform older types such as DDR3, due to higher access latencies and, in the case of HMC, poor exploitation of locality; (2) there is no single DRAM type that can cater to all components of a heterogeneous system (e.g., GDDR5 significantly outperforms other memories for multimedia acceleration, while HMC significantly outperforms other memories for network acceleration); and (3) there is still a strong need to lower DRAM latency, but unfortunately the current design trend of commodity DRAM is toward higher latencies to obtain other benefits. We hope that the trends we identify can drive optimizations in both hardware and software design.
An Adequate While-Language for Hybrid Computation
Venue: arXiv cs.LO 2019
Year: 2019

Hybrid computation combines discrete and continuous dynamics in the form of an entangled mixture inherently present both in various natural phenomena, and in applications ranging from control theory to microbiology. The emergent behaviours bear signs of both computational and physical processes, and thus present difficulties not only for analysis, but also for describing them adequately in a structural, well-founded way. Here, we introduce a language for hybrid computation, inspired by the fine-grain call-by-value paradigm, and equip it with a denotational and computationally adequate denotational semantics. Our denotational semantics crucially relies on a hybrid monad supporting an (Elgot) iteration operator, we developed elsewhere. As an intermediate step we introduce a more lightweight duration semantics furnished with analogous results and drawing on a new duration monad that we introduce as a lightweight counterpart to the hybrid monad.
Knowledge-based Analysis for Mortality Prediction from CT Images
Venue: arXiv cs.CV 2019
Year: 2019

Recent studies have highlighted the high correlation between cardiovascular diseases (CVD) and lung cancer, and both are associated with significant morbidity and mortality. Low-Dose CT (LCDT) scans have led to significant improvements in the accuracy of lung cancer diagnosis and thus the reduction of cancer deaths. However, the high correlation between lung cancer and CVD has not been well explored for mortality prediction. This paper introduces a knowledge-based analytical method using deep convolutional neural network (CNN) for all-cause mortality prediction. The underlying approach combines structural image features extracted from CNNs, based on LDCT volume in different scale, and clinical knowledge obtained from quantitative measurements, to comprehensively predict the mortality risk of lung cancer screening subjects. The introduced method is referred to here as the Knowledge-based Analysis of Mortality Prediction Network, or KAMP-Net. It constitutes a collaborative framework that utilizes both imaging features and anatomical information, instead of completely relying on automatic feature extraction. Our work demonstrates the feasibility of incorporating quantitative clinical measurements to assist CNNs in all-cause mortality prediction from chest LDCT images. The results of this study confirm that radiologist defined features are an important complement to CNNs to achieve a more comprehensive feature extraction. Thus, the proposed KAMP-Net has shown to achieve a superior performance when compared to other methods. Our code is available at https://github.com/DIAL-RPI/KAMP-Net.
Adversarial Augmentation for Enhancing Classification of Mammography Images
Venue: arXiv cs.CV 2019
Year: 2019

Supervised deep learning relies on the assumption that enough training data is available, which presents a problem for its application to several fields, like medical imaging. On the example of a binary image classification task (breast cancer recognition), we show that pretraining a generative model for meaningful image augmentation helps enhance the performance of the resulting classifier. By augmenting the data, performance on downstream classification tasks could be improved even with a relatively small training set. We show that this "adversarial augmentation" yields promising results compared to classical image augmentation on the example of breast cancer classification.
To Retransmit or Not: Real-Time Remote Estimation in Wireless Networked Control
Venue: arXiv cs.IT 2019
Year: 2019

Real-time remote estimation is critical for mission-critical applications including industrial automation, smart grid, and the tactile Internet. In this paper, we propose a hybrid automatic repeat request (HARQ)-based real-time remote estimation framework for linear time-invariant (LTI) dynamic systems. Considering the estimation quality of such a system, there is a fundamental tradeoff between the reliability and freshness of the sensor's measurement transmission. When a failed transmission occurs, the sensor can either retransmit the previous old measurement such that the receiver can obtain a more reliable old measurement, or transmit a new but less reliable measurement. To design the optimal decision, we formulate a new problem to optimize the sensor's online decision policy, i.e., to retransmit or not, depending on both the current estimation quality of the remote estimator and the current number of retransmissions of the sensor, so as to minimize the long-term remote estimation mean-squared error (MSE). This problem is non-trivial. In particular, it is not clear what the condition is in terms of the communication channel quality and the LTI system parameters, to ensure that the long-term estimation MSE can be bounded. We give a sufficient condition of the existence of a stationary and deterministic optimal policy that stabilizes the remote estimation system and minimizes the MSE. Also, we prove that the optimal policy has a switching structure, and derive a low-complexity suboptimal policy. Our numerical results show that the proposed optimal policy notably improves the performance of the remote estimation system compared to the conventional non-HARQ policy.
Atrial Scar Quantification via Multi-scale CNN in the Graph-cuts Framework
Venue: arXiv cs.CV 2019
Year: 2019

Late gadolinium enhancement magnetic resonance imaging (LGE MRI) appears to be a promising alternative for scar assessment in patients with atrial fibrillation (AF). Automating the quantification and analysis of atrial scars can be challenging due to the low image quality. In this work, we propose a fully automated method based on the graph-cuts framework, where the potentials of the graph are learned on a surface mesh of the left atrium (LA) using a multi-scale convolutional neural network (MS-CNN). For validation, we have employed fifty-eight images with manual delineations. MS-CNN, which can efficiently incorporate both the local and global texture information of the images, has been shown to evidently improve the segmentation accuracy of the proposed graph-cuts based method. The segmentation could be further improved when the contribution between the t-link and n-link weights of the graph is balanced. The proposed method achieves a mean accuracy of 0.856 +- 0.033 and mean Dice score of 0.702 +- 0.071 for LA scar quantification. Compared with the conventional methods, which are based on the manual delineation of LA for initialization, our method is fully automatic and has demonstrated significantly better Dice score and accuracy (p < 0.01). The method is promising and can be useful in diagnosis and prognosis of AF.
Bayesian optimisation under uncertain inputs
Venue: arXiv cs.LG 2019
Year: 2019

Bayesian optimisation (BO) has been a successful approach to optimise functions which are expensive to evaluate and whose observations are noisy. Classical BO algorithms, however, do not account for errors about the location where observations are taken, which is a common issue in problems with physical components. In these cases, the estimation of the actual query location is also subject to uncertainty. In this context, we propose an upper confidence bound (UCB) algorithm for BO problems where both the outcome of a query and the true query location are uncertain. The algorithm employs a Gaussian process model that takes probability distributions as inputs. Theoretical results are provided for both the proposed algorithm and a conventional UCB approach within the uncertain-inputs setting. Finally, we evaluate each method's performance experimentally, comparing them to other input noise aware BO approaches on simulated scenarios involving synthetic and real data.
Public Sphere 2.0: Targeted Commenting in Online News Media
Venue: arXiv cs.IR 2019
Year: 2019

With the increase in online news consumption, to maximize advertisement revenue, news media websites try to attract and retain their readers on their sites. One of the most effective tools for reader engagement is commenting, where news readers post their views as comments against the news articles. Traditionally, it has been assumed that the comments are mostly made against the full article. In this work, we show that present commenting landscape is far from this assumption. Because the readers lack the time to go over an entire article, most of the comments are relevant to only particular sections of an article. In this paper, we build a system which can automatically classify comments against relevant sections of an article. To implement that, we develop a deep neural network based mechanism to find comments relevant to any section and a paragraph wise commenting interface to showcase them. We believe that such a data driven commenting system can help news websites to further increase reader engagement.
Low Subpacketization Coded Caching via Projective Geometry for Broadcast and D2D networks
Venue: arXiv cs.IT 2019
Year: 2019

Coded caching was introduced as a technique of systematically exploiting locally available storage at the clients to increase the channel throughput via coded transmissions. Most known coded caching schemes in literature enable large gains in terms of the rate, however at the cost of subpacketization that is exponential in $K^{\frac{1}{r}}$ ($K$ being the number of clients, $r$ some positive integer). Building upon recent prior work for coded caching design via line graphs and finite-field projective geometries, we present a new scheme in this work which achieves a subexponential (in $K$) subpacketization of $q^{O((log_qK)^2)}$ and rate $Θ\left(\frac{K}{(log_qK)^2}\right)$, for large $K$, and the cached fraction $\frac{M}{N}$ being upper bounded by a constant $\frac{2}{q^{α-1}}$ (for some prime power $q$ and constant $α\geq 2$) . Apart from this asymptotic improvement, we show that through some numerical comparisons that our present scheme has much lower subpacketization than previous comparable schemes, with some increase in the rate of the delivery scheme, for the same memory requirements. For instance, we obtain practically relevant subpacketization levels such as $10^2 - 10^7$ for $10^2 - 10^4$ number of clients. Leveraging prior results on adapting coded caching schemes for the error-free broadcast channel to device to device networks, we obtain a low-subpacketization scheme for D2D networks also, and give numerical comparison for the same with prior work.
With Great Speed Come Small Buffers: Space-Bandwidth Tradeoffs for Routing
Venue: arXiv cs.DS 2019
Year: 2019

We consider the Adversarial Queuing Theory (AQT) model, where packet arrivals are subject to a maximum average rate $0\leρ\le1$ and burstiness $σ\ge0$. In this model, we analyze the size of buffers required to avoid overflows in the basic case of a path. Our main results characterize the space required by the average rate and the number of distinct destinations: we show that $O(k d^{1/k})$ space suffice, where $d$ is the number of distinct destinations and $k=\lfloor 1/ρ\rfloor$; and we show that $Ω(\frac 1 k d^{1/k})$ space is necessary. For directed trees, we describe an algorithm whose buffer space requirement is at most $1 + d' + σ$ where $d'$ is the maximum number of destinations on any root-leaf path.
Time and Energy-Optimal Lane Change Maneuvers for Cooperating Connected Automated Vehicles
Venue: arXiv cs.SY 2019
Year: 2019

We derive optimal control policies for a Connected and Automated Vehicle (CAV) cooperating with neighboring CAVs to implement a highway lane change maneuver. We optimize the maneuver time and subsequently minimize the associated energy consumption of all cooperating vehicles in this maneuver. We prove structural properties of the optimal policies which simplify the solution derivations and lead to analytical optimal control expressions. The solutions, when they exist, are guaranteed to satisfy safety constraints for all vehicles involved in the maneuver. Simulation results show the effectiveness of the proposed solution and significant performance improvements compared to maneuvers performed by human-driven vehicles.
Evaluating the Search Phase of Neural Architecture Search
Venue: arXiv cs.LG 2019
Year: 2019

Neural Architecture Search (NAS) aims to facilitate the design of deep networks for new tasks. Existing techniques rely on two stages: searching over the architecture space and validating the best architecture. NAS algorithms are currently compared solely based on their results on the downstream task. While intuitive, this fails to explicitly evaluate the effectiveness of their search strategies. In this paper, we propose to evaluate the NAS search phase. To this end, we compare the quality of the solutions obtained by NAS search policies with that of random architecture selection. We find that: (i) On average, the state-of-the-art NAS algorithms perform similarly to the random policy; (ii) the widely-used weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We believe that our evaluation framework will be key to designing NAS strategies that consistently discover architectures superior to random ones.
Policies for allocation of information in task-oriented groups: elitism and egalitarianism outperform welfarism
Venue: arXiv cs.MA 2019
Year: 2019

Communication or influence networks are probably the most controllable of all factors that are known to impact on the problem-solving capability of task-forces. In the case connections are costly, it is necessary to implement a policy to allocate them to the individuals. Here we use an agent-based model to study how distinct allocation policies affect the performance of a group of agents whose task is to find the global maxima of NK fitness landscapes. Agents cooperate by broadcasting messages informing on their fitness and use this information to imitate the fittest agent in their influence neighborhoods. The larger the influence neighborhood of an agent, the more links, and hence information, the agent receives. We find that the elitist policy in which agents with above-average fitness have their influence neighborhoods amplified, whereas agents with below-average fitness have theirs deflated, is optimal for smooth landscapes, provided the group size is not too small. For rugged landscapes, however, the elitist policy can perform very poorly for certain group sizes. In addition, we find that the egalitarian policy, in which the size of the influence neighborhood is the same for all agents, is optimal for both smooth and rugged landscapes in the case of small groups. The welfarist policy, in which the actions of the elitist policy are reversed, is always suboptimal, i.e., depending on the group size it is outperformed by either the elitist or the egalitarian policies.
Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure
Venue: arXiv cs.LG 2019
Year: 2019

Recent efforts show that neural networks are vulnerable to small but intentional perturbations on input features in visual classification tasks. Due to the additional consideration of connections between examples (\eg articles with citation link tend to be in the same class), graph neural networks could be more sensitive to the perturbations, since the perturbations from connected examples exacerbate the impact on a target example. Adversarial Training (AT), a dynamic regularization technique, can resist the worst-case perturbations on input features and is a promising choice to improve model robustness and generalization. However, existing AT methods focus on standard classification, being less effective when training models on graph since it does not model the impact from connected examples. In this work, we explore adversarial training on graph, aiming to improve the robustness and generalization of models learned on graph. We propose Graph Adversarial Training (GraphAT), which takes the impact from connected examples into account when learning to construct and resist perturbations. We give a general formulation of GraphAT, which can be seen as a dynamic regularization scheme based on the graph structure. To demonstrate the utility of GraphAT, we employ it on a state-of-the-art graph neural network model --- Graph Convolutional Network (GCN). We conduct experiments on two citation graphs (Citeseer and Cora) and a knowledge graph (NELL), verifying the effectiveness of GraphAT which outperforms normal training on GCN by 4.51% in node classification accuracy. Codes are available via: https://github.com/fulifeng/GraphAT.
Lung Cancer Detection using Co-learning from Chest CT Images and Clinical Demographics
Venue: arXiv cs.CV 2019
Year: 2019

Early detection of lung cancer is essential in reducing mortality. Recent studies have demonstrated the clinical utility of low-dose computed tomography (CT) to detect lung cancer among individuals selected based on very limited clinical information. However, this strategy yields high false positive rates, which can lead to unnecessary and potentially harmful procedures. To address such challenges, we established a pipeline that co-learns from detailed clinical demographics and 3D CT images. Toward this end, we leveraged data from the Consortium for Molecular and Cellular Characterization of Screen-Detected Lesions (MCL), which focuses on early detection of lung cancer. A 3D attention-based deep convolutional neural net (DCNN) is proposed to identify lung cancer from the chest CT scan without prior anatomical location of the suspicious nodule. To improve upon the non-invasive discrimination between benign and malignant, we applied a random forest classifier to a dataset integrating clinical information to imaging data. The results show that the AUC obtained from clinical demographics alone was 0.635 while the attention network alone reached an accuracy of 0.687. In contrast when applying our proposed pipeline integrating clinical and imaging variables, we reached an AUC of 0.787 on the testing dataset. The proposed network both efficiently captures anatomical information for classification and also generates attention maps that explain the features that drive performance.
Accelerated Steady-State Torque Computation for Induction Machines using Parallel-In-Time Algorithms
Venue: arXiv cs.CE 2019
Year: 2019

This paper focuses on efficient steady-state computations of induction machines. In particular, the periodic Parareal algorithm with initial-value coarse problem (PP-IC) is considered for acceleration of classical time-stepping simulations via non-intrusive parallelization in time domain, i.e., existing implementations can be reused. Superiority of this parallel-in-time method is in its direct applicability to time-periodic problems, compared to, e.g, the standard Parareal method, which only solves an initial-value problem, starting from a prescribed initial value. PP-IC is exploited here to obtain the steady state of several operating points of an induction motor, developed by Robert Bosch GmbH. Numerical experiments show that acceleration up to several dozens of times can be obtained, depending on availability of parallel processing units. Comparison of PP-IC with existing time-periodic explicit error correction method highlights better robustness and efficiency of the considered time-parallel approach.
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Venue: arXiv cs.LG 2019
Year: 2019

Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models are composed of modular building blocks that are flexible and easily extensible, and experiment configurations are centralized and highly customizable. Distributed training and quantized inference are supported directly within the framework, and it contains existing implementations of a large number of utilities, helper functions, and the newest research ideas. Lingvo has been used in collaboration by dozens of researchers in more than 20 papers over the last two years. This document outlines the underlying design of Lingvo and serves as an introduction to the various pieces of the framework, while also offering examples of advanced features that showcase the capabilities of the framework.
LSwarm: Efficient Collision Avoidance for Large Swarms with Coverage Constraints in Complex Urban Scenes
Venue: arXiv cs.RO 2019
Year: 2019

In this paper, we address the problem of collision avoidance for a swarm of UAVs used for continuous surveillance of an urban environment. Our method, LSwarm, efficiently avoids collisions with static obstacles, dynamic obstacles and other agents in 3-D urban environments while considering coverage constraints. LSwarm computes collision avoiding velocities that (i) maximize the conformity of an agent to an optimal path given by a global coverage strategy and (ii) ensure sufficient resolution of the coverage data collected by each agent. Our algorithm is formulated based on ORCA (Optimal Reciprocal Collision Avoidance) and is scalable with respect to the size of the swarm. We evaluate the coverage performance of LSwarm in realistic simulations of a swarm of quadrotors in complex urban models. In practice, our approach can compute collision avoiding velocities for a swarm composed of tens to hundreds of agents in a few milliseconds on dense urban scenes consisting of tens of buildings.