OpenRaiser/Intern-Atlas huggingface dataset:
Latent Dirichlet Allocation in Generative Adversarial Networks
Venue: arXiv cs.LG 2019
Year: 2019

We study the problem of multimodal generative modelling of images based on generative adversarial networks (GANs). Despite the success of existing methods, they often ignore the underlying structure of vision data or its multimodal generation characteristics. To address this problem, we introduce the Dirichlet prior for multimodal image generation, which leads to a new Latent Dirichlet Allocation based GAN (LDAGAN). In detail, for the generative process modelling, LDAGAN defines a generative mode for each sample, determining which generative sub-process it belongs to. For the adversarial training, LDAGAN derives a variational expectation-maximization (VEM) algorithm to estimate model parameters. Experimental results on real-world datasets have demonstrated the outstanding performance of LDAGAN over other existing GANs.
Optimizing Throughput Performance in Distributed MIMO Wi-Fi Networks using Deep Reinforcement Learning
Venue: arXiv cs.IT 2019
Year: 2019

This paper explores the feasibility of leveraging concepts from deep reinforcement learning (DRL) to enable dynamic resource management in Wi-Fi networks implementing distributed multi-user MIMO (D-MIMO). D-MIMO is a technique by which a set of wireless access points are synchronized and grouped together to jointly serve multiple users simultaneously. This paper addresses two dynamic resource management problems pertaining to D-MIMO Wi-Fi networks: (i) channel assignment of D-MIMO groups, and (ii) deciding how to cluster access points to form D-MIMO groups, in order to maximize user throughput performance. These problems are known to be NP-Hard and only heuristic solutions exist in literature. We construct a DRL framework through which a learning agent interacts with a D-MIMO Wi-Fi network, learns about the network environment, and is successful in converging to policies which address the aforementioned problems. Through extensive simulations and on-line training based on D-MIMO Wi-Fi networks, this paper demonstrates the efficacy of DRL in achieving an improvement of 20% in user throughput performance compared to heuristic solutions, particularly when network conditions are dynamic. This work also showcases the effectiveness of DRL in meeting multiple network objectives simultaneously, for instance, maximizing throughput of users as well as fairness of throughput among them.
Deep learning incorporating biologically-inspired neural dynamics
Venue: arXiv cs.NE 2019
Year: 2019

Neural networks have become the key technology of artificial intelligence and have contributed to breakthroughs in several machine learning tasks, primarily owing to advances in deep learning applied to Artificial Neural Networks (ANNs). Simultaneously, Spiking Neural Networks (SNNs) incorporating biologically-feasible spiking neurons have held great promise because of their rich temporal dynamics and high-power efficiency. However, the developments in SNNs were proceeding separately from those in ANNs, effectively limiting the adoption of deep learning research insights. Here we show an alternative perspective on the spiking neuron that casts it as a particular ANN construct called Spiking Neural Unit (SNU), and a soft SNU (sSNU) variant that generalizes its dynamics to a novel recurrent ANN unit. SNUs bridge the biologically-inspired SNNs with ANNs and provide a methodology for seamless inclusion of spiking neurons in deep learning architectures. Furthermore, SNU enables highly-efficient in-memory acceleration of SNNs trained with backpropagation through time, implemented with the hardware in-the-loop. We apply SNUs to tasks ranging from hand-written digit recognition, language modelling, to music prediction. We obtain accuracy comparable to, or better than, that of state-of-the-art ANNs, and we experimentally verify the efficacy of the in-memory-based SNN realization for the music-prediction task using 52,800 phase-change memory devices. The new generation of neural units introduced in this paper incorporate biologically-inspired neural dynamics in deep learning. In addition, they provide a systematic methodology for training neuromorphic computing hardware. Thus, they open a new avenue for a widespread adoption of SNNs in practical applications.
Learning Constraints from Demonstrations
Venue: arXiv cs.RO 2019
Year: 2019

We extend the learning from demonstration paradigm by providing a method for learning unknown constraints shared across tasks, using demonstrations of the tasks, their cost functions, and knowledge of the system dynamics and control constraints. Given safe demonstrations, our method uses hit-and-run sampling to obtain lower cost, and thus unsafe, trajectories. Both safe and unsafe trajectories are used to obtain a consistent representation of the unsafe set via solving an integer program. Our method generalizes across system dynamics and learns a guaranteed subset of the constraint. We also provide theoretical analysis on what subset of the constraint can be learnable from safe demonstrations. We demonstrate our method on linear and nonlinear system dynamics, show that it can be modified to work with suboptimal demonstrations, and that it can also be used to learn constraints in a feature space.
Group-Attention Single-Shot Detector (GA-SSD): Finding Pulmonary Nodules in Large-Scale CT Images
Venue: arXiv cs.CV 2019
Year: 2019

Early diagnosis of pulmonary nodules (PNs) can improve the survival rate of patients and yet is a challenging task for radiologists due to the image noise and artifacts in computed tomography (CT) images. In this paper, we propose a novel and effective abnormality detector implementing the attention mechanism and group convolution on 3D single-shot detector (SSD) called group-attention SSD (GA-SSD). We find that group convolution is effective in extracting rich context information between continuous slices, and attention network can learn the target features automatically. We collected a large-scale dataset that contained 4146 CT scans with annotations of varying types and sizes of PNs (even PNs smaller than 3mm were annotated). To the best of our knowledge, this dataset is the largest cohort with relatively complete annotations for PNs detection. Our experimental results show that the proposed group-attention SSD outperforms the classic SSD framework as well as the state-of-the-art 3DCNN, especially on some challenging lesion types.
You Cannot Fix What You Cannot Find! An Investigation of Fault Localization Bias in Benchmarking Automated Program Repair Systems
Venue: arXiv cs.SE 2019
Year: 2019

Properly benchmarking Automated Program Repair (APR) systems should contribute to the development and adoption of the research outputs by practitioners. To that end, the research community must ensure that it reaches significant milestones by reliably comparing state-of-the-art tools for a better understanding of their strengths and weaknesses. In this work, we identify and investigate a practical bias caused by the fault localization (FL) step in a repair pipeline. We propose to highlight the different fault localization configurations used in the literature, and their impact on APR systems when applied to the Defects4J benchmark. Then, we explore the performance variations that can be achieved by `tweaking' the FL step. Eventually, we expect to create a new momentum for (1) full disclosure of APR experimental procedures with respect to FL, (2) realistic expectations of repairing bugs in Defects4J, as well as (3) reliable performance comparison among the state-of-the-art APR systems, and against the baseline performance results of our thoroughly assessed kPAR repair tool. Our main findings include: (a) only a subset of Defects4J bugs can be currently localized by commonly-used FL techniques; (b) current practice of comparing state-of-the-art APR systems (i.e., counting the number of fixed bugs) is potentially misleading due to the bias of FL configurations; and (c) APR authors do not properly qualify their performance achievement with respect to the different tuning parameters implemented in APR systems.
Momen(e)t: Flavor the Moments in Learning to Classify Shapes
Venue: arXiv cs.CG 2019
Year: 2019

A fundamental question in learning to classify 3D shapes is how to treat the data in a way that would allow us to construct efficient and accurate geometric processing and analysis procedures. Here, we restrict ourselves to networks that operate on point clouds. There were several attempts to treat point clouds as non-structured data sets by which a neural network is trained to extract discriminative properties. The idea of using 3D coordinates as class identifiers motivated us to extend this line of thought to that of shape classification by comparing attributes that could easily account for the shape moments. Here, we propose to add polynomial functions of the coordinates allowing the network to account for higher order moments of a given shape. Experiments on two benchmarks show that the suggested network is able to provide state of the art results and at the same token learn more efficiently in terms of memory and computational complexity.
Learning a Probabilistic Model for Diffeomorphic Registration
Venue: arXiv cs.CV 2019
Year: 2019

We propose to learn a low-dimensional probabilistic deformation model from data which can be used for registration and the analysis of deformations. The latent variable model maps similar deformations close to each other in an encoding space. It enables to compare deformations, generate normal or pathological deformations for any new image or to transport deformations from one image pair to any other image. Our unsupervised method is based on variational inference. In particular, we use a conditional variational autoencoder (CVAE) network and constrain transformations to be symmetric and diffeomorphic by applying a differentiable exponentiation layer with a symmetric loss function. We also present a formulation that includes spatial regularization such as diffusion-based filters. Additionally, our framework provides multi-scale velocity field estimations. We evaluated our method on 3-D intra-subject registration using 334 cardiac cine-MRIs. On this dataset, our method showed state-of-the-art performance with a mean DICE score of 81.2% and a mean Hausdorff distance of 7.3mm using 32 latent dimensions compared to three state-of-the-art methods while also demonstrating more regular deformation fields. The average time per registration was 0.32s. Besides, we visualized the learned latent space and show that the encoded deformations can be used to transport deformations and to cluster diseases with a classification accuracy of 83% after applying a linear projection.
Differential Evolution with Better and Nearest Option for Function Optimization
Venue: arXiv cs.NE 2019
Year: 2019

Differential evolution(DE) is a conventional algorithm with fast convergence speed. However, DE may be trapped in local optimal solution easily. Many researchers devote themselves to improving DE. In our previously work, whale swarm algorithm have shown its strong searching performance due to its niching based mutation strategy. Based on this fact, we propose a new DE algorithm called DE with Better and Nearest option (NbDE). In order to evaluate the performance of NbDE, NbDE is compared with several meta-heuristic algorithms on nine classical benchmark test functions with different dimensions. The results show that NbDE outperforms other algorithms in convergence speed and accuracy.
Large-Scale Optical Neural Networks based on Photoelectric Multiplication
Venue: arXiv cs.ET 2019
Year: 2019

Recent success in deep neural networks has generated strong interest in hardware accelerators to improve speed and energy consumption. This paper presents a new type of photonic accelerator based on coherent detection that is scalable to large ($N \gtrsim 10^6$) networks and can be operated at high (GHz) speeds and very low (sub-aJ) energies per multiply-and-accumulate (MAC), using the massive spatial multiplexing enabled by standard free-space optical components. In contrast to previous approaches, both weights and inputs are optically encoded so that the network can be reprogrammed and trained on the fly. Simulations of the network using models for digit- and image-classification reveal a "standard quantum limit" for optical neural networks, set by photodetector shot noise. This bound, which can be as low as 50 zJ/MAC, suggests performance below the thermodynamic (Landauer) limit for digital irreversible computation is theoretically possible in this device. The proposed accelerator can implement both fully-connected and convolutional networks. We also present a scheme for back-propagation and training that can be performed in the same hardware. This architecture will enable a new class of ultra-low-energy processors for deep learning.
Discriminative Supervised Hashing for Cross-Modal similarity Search
Venue: arXiv cs.LG 2019
Year: 2019

With the advantage of low storage cost and high retrieval efficiency, hashing techniques have recently been an emerging topic in cross-modal similarity search. As multiple modal data reflect similar semantic content, many researches aim at learning unified binary codes. However, discriminative hashing features learned by these methods are not adequate. This results in lower accuracy and robustness. We propose a novel hashing learning framework which jointly performs classifier learning, subspace learning and matrix factorization to preserve class-specific semantic content, termed Discriminative Supervised Hashing (DSH), to learn the discrimative unified binary codes for multi-modal data. Besides, reducing the loss of information and preserving the non-linear structure of data, DSH non-linearly projects different modalities into the common space in which the similarity among heterogeneous data points can be measured. Extensive experiments conducted on the three publicly available datasets demonstrate that the framework proposed in this paper outperforms several state-of -the-art methods.
Active Learning and CSI Acquisition for mmWave Initial Alignment
Venue: arXiv cs.IT 2019
Year: 2019

Millimeter wave (mmWave) communication with large antenna arrays is a promising technique to enable extremely high data rates due to the large available bandwidth in mmWave frequency bands. In addition, given the knowledge of an optimal directional beamforming vector, large antenna arrays have been shown to overcome both the severe signal attenuation in mmWave as well as the interference problem. However, fundamental limits on achievable learning rate of an optimal beamforming vector remain. This paper considers the problem of adaptive and sequential optimization of the beamforming vectors during the initial access phase of communication. With a single-path channel model, the problem is reduced to actively learning the Angle-of-Arrival (AoA) of the signal sent from the user to the Base Station (BS). Drawing on the recent results in the design of a hierarchical beamforming codebook [1], sequential measurement dependent noisy search strategies [2], and active learning from an imperfect labeler [3], an adaptive and sequential alignment algorithm is proposed. An upper bound on the expected search time of the proposed algorithm is derived via Extrinsic Jensen-Shannon Divergence. which demonstrates that the search time of the proposed algorithm asymptotically matches the performance of the noiseless bisection search up to a constant factor. Furthermore, the upper bound shows that the acquired AoA error probability decays exponentially fast with the search time with an exponent that is a decreasing function of the acquisition rate. Numerically, the proposed algorithm is compared with prior work where a significant improvement of the system communication rate is observed. Most notably, in the relevant regime of low (-10dB to 5dB) raw SNR, this establishes the first practically viable solution for initial access and, hence, the first demonstration of stand-alone mmWave communication
Atomic-SDN: Is Synchronous Flooding the Solution to Software-Defined Networking in IoT?
Venue: arXiv cs.NI 2019
Year: 2019

The adoption of Software Defined Networking (SDN) within traditional networks has provided operators the ability to manage diverse resources and easily reconfigure networks as requirements change. Recent research has extended this concept to IEEE 802.15.4 low-power wireless networks, which form a key component of the Internet of Things (IoT). However, the multiple traffic patterns necessary for SDN control makes it difficult to apply this approach to these highly challenging environments. This paper presents Atomic-SDN, a highly reliable and low-latency solution for SDN in low-power wireless. Atomic-SDN introduces a novel Synchronous Flooding (SF) architecture capable of dynamically configuring SF protocols to satisfy complex SDN control requirements, and draws from the authors' previous experiences in the IEEE EWSN Dependability Competition: where SF solutions have consistently outperformed other entries. Using this approach, Atomic-SDN presents considerable performance gains over other SDN implementations for low-power IoT networks. We evaluate Atomic-SDN through simulation and experimentation, and show how utilizing SF techniques provides latency and reliability guarantees to SDN control operations as the local mesh scales. We compare Atomic-SDN against other SDN implementations based on the IEEE 802.15.4 network stack, and establish that Atomic-SDN improves SDN control by orders-of-magnitude across latency, reliability, and energy-efficiency metrics.
Feedback, Dynamics, and Optimal Control in Climate Economics
Venue: arXiv cs.SY 2019
Year: 2019

For his work in the economics of climate change, Professor William Nordhaus was a co-recipient of the 2018 Nobel Memorial Prize for Economic Sciences. A core component of the work undertaken by Nordhaus is the Dynamic Integrated model of Climate and Economy, known as the DICE model. The DICE model is a discrete-time model with two control inputs and is primarily used in conjunction with a particular optimal control problem in order to estimate optimal pathways for reducing greenhouse gas emissions. In this paper, we provide a tutorial introduction to the DICE model and we indicate challenges and open problems of potential interest for the systems and control community.
On the Role of Age of Information in the Internet of Things
Venue: arXiv cs.IT 2019
Year: 2019

In this article, we provide an accessible introduction to the emerging idea of Age of Information (AoI) that quantifies freshness of information and explore its possible role in the efficient design of freshness-aware Internet of Things (IoT). We start by summarizing the concept of AoI and its variants with emphasis on the differences between AoI and other well-known performance metrics in the literature, such as throughput and delay. Building on this, we explore freshness-aware IoT design for a network in which IoT devices sense potentially different physical processes and are supposed to frequently update the status of these processes at a destination node (such as a cellular base station). Inspired by the recent interest, we also assume that these IoT devices are powered by wireless energy transfer by the destination node. For this setting, we investigate the optimal sampling policy that jointly optimizes wireless energy transfer and scheduling of update packet transmissions from IoT devices with the goal of minimizing long-term weighted sum-AoI. Using this, we characterize the achievable AoI region. We also compare this AoI-optimal policy with the one that maximizes average throughput (throughput-optimal policy), and demonstrate the impact of system state on their structures. Several promising directions for future research are also presented.
Towards Value-Sensitive Learning Analytics Design
Venue: arXiv cs.HC 2019
Year: 2019

To support ethical considerations and system integrity in learning analytics, this paper introduces two cases of applying the Value Sensitive Design methodology to learning analytics design. The first study applied two methods of Value Sensitive Design, namely stakeholder analysis and value analysis, to a conceptual investigation of an existing learning analytics tool. This investigation uncovered a number of values and value tensions, leading to design trade-offs to be considered in future tool refinements. The second study holistically applied Value Sensitive Design to the design of a recommendation system for the Wikipedia WikiProjects. To proactively consider values among stakeholders, we derived a multi-stage design process that included literature analysis, empirical investigations, prototype development, community engagement, iterative testing and refinement, and continuous evaluation. By reporting on these two cases, this paper responds to a need of practical means to support ethical considerations and human values in learning analytics systems. These two cases demonstrate that Value Sensitive Design could be a viable approach for balancing a wide range of human values, which tend to encompass and surpass ethical issues, in learning analytics design.
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Venue: arXiv cs.SE 2019
Year: 2019

Millions of open-source projects with numerous bug fixes are available in code repositories. This proliferation of software development histories can be leveraged to learn how to fix common programming bugs. To explore such a potential, we perform an empirical study to assess the feasibility of using Neural Machine Translation techniques for learning bug-fixing patches for real defects. First, we mine millions of bug-fixes from the change histories of projects hosted on GitHub, in order to extract meaningful examples of such bug-fixes. Next, we abstract the buggy and corresponding fixed code, and use them to train an Encoder-Decoder model able to translate buggy code into its fixed version. In our empirical investigation we found that such a model is able to fix thousands of unique buggy methods in the wild. Overall, this model is capable of predicting fixed patches generated by developers in 9-50% of the cases, depending on the number of candidate patches we allow it to generate. Also, the model is able to emulate a variety of different Abstract Syntax Tree operations and generate candidate patches in a split second.
Formal Specification and Verification of Smart Contracts for Azure Blockchain
Venue: arXiv cs.PL 2019
Year: 2019

Ensuring correctness of smart contracts is paramount to ensuring trust in blockchain-based systems. This paper studies the safety and security of smart contracts in the \emph{Azure Blockchain Workbench}, an enterprise Blockchain-as-a-Service offering from Microsoft. As part of this study, we formalize \emph{semantic conformance} of smart contracts against a state machine model with access-control policy and develop a highly-automated formal verifier for Solidity that can produce proofs as well as counterexamples. We have applied our verifier {\sc VeriSol} to analyze {\it all} contracts shipped with the Azure Blockchain Workbench, which includes application samples as well as a governance contract for Proof of Authority (PoA). We have found previously unknown bugs in these published smart contracts. After fixing these bugs, {\sc VeriSol} was able to successfully perform full verification for all of these contracts.