OpenRaiser/Intern-Atlas huggingface dataset:
Discrimination-aware Channel Pruning for Deep Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Channel pruning is one of the predominant approaches for deep model compression. Existing pruning methods either train from scratch with sparsity constraints on channels, or minimize the reconstruction error between the pre-trained feature maps and the compressed ones. Both strategies suffer from some limitations: the former kind is computationally expensive and difficult to converge, whilst the latter kind optimizes the reconstruction error but ignores the discriminative power of channels. To overcome these drawbacks, we investigate a simple-yet-effective method, called discrimination-aware channel pruning, to choose those channels that really contribute to discriminative power. To this end, we introduce additional losses into the network to increase the discriminative power of intermediate layers and then select the most discriminative channels for each layer by considering the additional loss and the reconstruction error. Last, we propose a greedy algorithm to conduct channel selection and parameter optimization in an iterative way. Extensive experiments demonstrate the effectiveness of our method. For example, on ILSVRC-12, our pruned ResNet-50 with 30% reduction of channels even outperforms the original model by 0.39% in top-1 accuracy.
On preserving non-discrimination when combining expert advice
Venue: arXiv cs.LG 2019
Year: 2019

We study the interplay between sequential decision making and avoiding discrimination against protected groups, when examples arrive online and do not follow distributional assumptions. We consider the most basic extension of classical online learning: "Given a class of predictors that are individually non-discriminatory with respect to a particular metric, how can we combine them to perform as well as the best predictor, while preserving non-discrimination?" Surprisingly we show that this task is unachievable for the prevalent notion of "equalized odds" that requires equal false negative rates and equal false positive rates across groups. On the positive side, for another notion of non-discrimination, "equalized error rates", we show that running separate instances of the classical multiplicative weights algorithm for each group achieves this guarantee. Interestingly, even for this notion, we show that algorithms with stronger performance guarantees than multiplicative weights cannot preserve non-discrimination.
Energy-Reliability Aware Link Optimization for Battery-Powered IoT Devices with Non-Ideal Power Amplifiers
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we study cross-layer optimization of low-power wireless links for reliability-aware applications while considering both the constraints and the non-ideal characteristics of the hardware in Internet-of-things (IoT) devices. Specifically, we define an energy consumption (EC) model that captures the energy cost---of transceiver circuitry, power amplifier, packet error statistics, packet overhead, etc.---in delivering a useful data bit. We derive the EC models for an ideal and two realistic non-linear power amplifier models. To incorporate packet error statistics, we develop a simple, in the form of elementary functions, and accurate closed-form packet error rate (PER) approximation in Rayleigh block-fading. Using the EC models, we derive energy-optimal yet reliability and hardware compliant conditions for limiting unconstrained optimal signal-to-noise ratio (SNR), and payload size. Together with these conditions, we develop a semi-analytic algorithm for resource-constrained IoT devices to jointly optimize parameters on physical (modulation size, SNR) and MAC (payload size and the number of retransmissions) layers in relation to link distance. Our results show that despite reliability constraints, the common notion---higher $M$-ary QAM modulations are energy optimal for short-range communication---prevails, and can provide up to 180% lifetime extension as compared to often employed OQPSK modulation in IoT devices. However, the reliability constraints reduce both their range and the energy efficiency, while non-ideal traditional PA reduces the range further by 50%, and diminishes the energy gains unless a better PA is employed.
Patch-Based Sparse Representation For Bacterial Detection
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose an unsupervised approach for bacterial detection in optical endomicroscopy images. This approach splits each image into a set of overlapping patches and assumes that observed intensities are linear combinations of the actual intensity values associated with background image structures, corrupted by additive Gaussian noise and potentially by a sparse outlier term modelling anomalies (which are considered to be candidate bacteria). The actual intensity term representing background structures is modelled as a linear combination of a few atoms drawn from a dictionary which is learned from bacteria-free data and then fixed while analyzing new images. The bacteria detection task is formulated as a minimization problem and an alternating direction method of multipliers (ADMM) is then used to estimate the unknown parameters. Simulations conducted using two ex vivo lung datasets show good detection and correlation performance between bacteria counts identified by a trained clinician and those of the proposed method.
Marginal Replay vs Conditional Replay for Continual Learning
Venue: arXiv cs.LG 2019
Year: 2019

We present a new replay-based method of continual classification learning that we term "conditional replay" which generates samples and labels together by sampling from a distribution conditioned on the class. We compare conditional replay to another replay-based continual learning paradigm (which we term "marginal replay") that generates samples independently of their class and assigns labels in a separate step. The main improvement in conditional replay is that labels for generated samples need not be inferred, which reduces the margin for error in complex continual classification learning tasks. We demonstrate the effectiveness of this approach using novel and standard benchmarks constructed from MNIST and FashionMNIST data, and compare to the regularization-based \textit{elastic weight consolidation} (EWC) method.
Efficient convex optimization for optimal PMU placement in large distribution grids
Venue: arXiv cs.SY 2019
Year: 2019

The small amount of measurements in distribution grids makes their monitoring more difficult. Topological observability may not be possible, and thus, pseudo-measurements are needed to perform state estimation, which is required to control elements such as distributed generation or transformers at distribution grids. Therefore, we consider the problem of optimal sensor placement to improve the state estimation accuracy in large-scale, 3-phase coupled, unbalanced distribution grids. This is an NP-hard optimization problem whose optimal solution is unpractical to obtain for large networks. Therefore, we develop a computationally efficient convex optimization algorithm to compute a lower bound on the possible value of the optimal solution, and thus check the gap between the bound and heuristic solutions. We test the method on a large test feeder, the standard IEEE 8500-node, to show the effectiveness of the approach.
A Comparative Measurement Study of Deep Learning as a Service Framework
Venue: arXiv cs.PF 2019
Year: 2019

Big data powered Deep Learning (DL) and its applications have blossomed in recent years, fueled by three technological trends: a large amount of digitized data openly accessible, a growing number of DL software frameworks in open source and commercial markets, and a selection of affordable parallel computing hardware devices. However, no single DL framework, to date, dominates in terms of performance and accuracy even for baseline classification tasks on standard datasets, making the selection of a DL framework an overwhelming task. This paper takes a holistic approach to conduct empirical comparison and analysis of four representative DL frameworks with three unique contributions. First, given a selection of CPU-GPU configurations, we show that for a specific DL framework, different configurations of its hyper-parameters may have a significant impact on both performance and accuracy of DL applications. Second, to the best of our knowledge, this study is the first to identify the opportunities for improving the training time performance and the accuracy of DL frameworks by configuring parallel computing libraries and tuning individual and multiple hyper-parameters. Third, we also conduct a comparative measurement study on the resource consumption patterns of four DL frameworks and their performance and accuracy implications, including CPU and memory usage, and their correlations to varying settings of hyper-parameters under different configuration combinations of hardware, parallel computing libraries. We argue that this measurement study provides in-depth empirical comparison and analysis of four representative DL frameworks, and offers practical guidance for service providers to deploying and delivering DL as a Service (DLaaS) and for application developers and DLaaS consumers to select the right DL frameworks for the right DL workloads.
On the End-Vertex Problem of Graph Searches
Venue: arXiv cs.DM 2019
Year: 2019

End vertices of graph searches can exhibit strong structural properties and are crucial for many graph algorithms. The problem of deciding whether a given vertex of a graph is an end-vertex of a particular search was first introduced by Corneil, Köhler and Lanlignel in 2010. There they showed that this problem is in fact NP-complete for LBFS on weakly chordal graphs. A similar result for BFS was obtained by Charbit, Habib and Mamcarz in 2014. Here, we prove that the end-vertex problem is NP-complete for MNS on weakly chordal graphs and for MCS on general graphs. Moreover, building on previous results, we show that this problem is linear for various searches on split and unit interval graphs.
Multi-Location Program Repair Strategies Learned from Past Successful Experience
Venue: arXiv cs.SE 2019
Year: 2019

Automated program repair (APR) has great potential to reduce the effort and time-consumption in software maintenance and becomes a hot topic in software engineering recently with many approaches being proposed. Multi-location program repair has always been a challenge in this field since its complexity in logic and structure. While some approaches do not claim to have the features for solving multi-location bugs, they generate correct patches for these defects in practice. In this paper, we first make an observation on multi-location bugs in Defects4J and divide them into two categories (i.e., similar and relevant multi-location bugs) based on the repair actions in their patches. We then summarize the situation of multi-location bugs in Defects4J fixed by current tools. We analyze the twenty-two patches generated by current tools and propose two feasible strategies for fixing multi-location bugs, illustrating them through two detailed case studies. At last, the experimental results prove the feasibility of our methods with the repair of two bugs that have never been fixed before. By learning from successful experience in the past, this paper points out possible ways ahead for multi-location program repair.
Randomized Work Stealing versus Sharing in Large-scale Systems with Non-exponential Job Sizes
Venue: arXiv cs.DC 2019
Year: 2019

Work sharing and work stealing are two scheduling paradigms to redistribute work when performing distributed computations. In work sharing, processors attempt to migrate pending jobs to other processors in the hope of reducing response times. In work stealing, on the other hand, underutilized processors attempt to steal jobs from other processors. Both paradigms generate a certain communication overhead and the question addressed in this paper is which of the two reduces the response time the most given that they use the same amount of communication overhead. Prior work presented explicit bounds, for large scale systems, on when randomized work sharing outperforms randomized work stealing in case of Poisson arrivals and exponential job durations and indicated that work sharing is best when the load is below $ϕ-1 \approx 0.6180$, with $ϕ$ being the golden ratio. In this paper we revisit this problem and study the impact of the job size distribution using a mean field model. We present an efficient method to determine the boundary between the regions where sharing or stealing is best for a given job size distribution, as well as bounds that apply to any (phase-type) job size distribution. The main insight is that work stealing benefits significantly from having more variable job sizes and work sharing may become inferior to work stealing for loads as small as $1/2 + ε$ for any $ε> 0$.
Exact Channel Synthesis
Venue: arXiv cs.IT 2019
Year: 2019

We consider the exact channel synthesis problem. This problem concerns the determination of the minimum amount of information required to create exact correlation remotely when there is a certain rate of randomness shared by two terminals. This problem generalizes an existing approximate version, in which the generated joint distribution is required to be close to a target distribution under the total variation (TV) distance measure (instead being exactly equal to the target distribution). We provide single-letter inner and outer bounds on the admissible region of the shared randomness rate and the communication rate for the exact channel synthesis problem. These two bounds coincide for doubly symmetric binary sources. We observe that for such sources, the admissible rate region for exact channel synthesis is strictly included in that for the TV-approximate version. We also extend the exact and TV-approximate channel synthesis problems to sources with countably infinite alphabets and continuous sources; the latter includes Gaussian sources. As by-products, lemmas concerning soft-covering under Rényi divergence measures are derived.
Acute and sub-acute stroke lesion segmentation from multimodal MRI
Venue: arXiv cs.CV 2019
Year: 2019

Acute stroke lesion segmentation tasks are of great clinical interest as they can help doctors make better informed treatment decisions. Magnetic resonance imaging (MRI) is time demanding but can provide images that are considered gold standard for diagnosis. Automated stroke lesion segmentation can provide with an estimate of the location and volume of the lesioned tissue, which can help in the clinical practice to better assess and evaluate the risks of each treatment. We propose a deep learning methodology for acute and sub-acute stroke lesion segmentation using multimodal MR imaging. The proposed method is evaluated using two public datasets from the 2015 Ischemic Stroke Lesion Segmentation challenge (ISLES 2015). These involve the tasks of sub-acute stroke lesion segmentation (SISS) and acute stroke penumbra estimation (SPES) from diffusion, perfusion and anatomical MRI modalities. The performance is compared against state-of-the-art methods with a blind online testing set evaluation on each of the challenges. At the time of submitting this manuscript, our approach is the first method in the online rankings for the SISS (DSC=0.59$\pm$0.31) and SPES sub-tasks (DSC=0.84$\pm$0.10). When compared with the rest of submitted strategies, we achieve top rank performance with a lower Hausdorff distance. Better segmentation results are obtained by leveraging the anatomy and pathophysiology of acute stroke lesions and using a combined approach to minimize the effects of class imbalance. The same training procedure is used for both tasks, showing the proposed methodology can generalize well enough to deal with different unrelated tasks and imaging modalities without training hyper-parameter tuning. A public version of the proposed method has been released to the scientific community at https://github.com/NIC-VICOROB/stroke-mri-segmentation.
Improving ICS Cyber Resilience through Optimal Diversification of Network Resources
Venue: arXiv cs.CR 2019
Year: 2019

Network diversity has been widely recognized as an effective defense strategy to mitigate the spread of malware. Optimally diversifying network resources can improve the resilience of a network against malware propagation. This work proposes an efficient method to compute such an optimal deployment, in the context of upgrading a legacy Industrial Control System with modern IT infrastructure. Our approach can tolerate various constraints when searching for an optimal diversification, such as outdated products and strict configuration policies. We explicitly measure the vulnerability similarity of products based on the CVE/NVD, to estimate the infection rate of malware between products. A Stuxnet-inspired case demonstrates our optimal diversification in practice, particularly when constrained by various requirements. We then measure the improved resilience of the diversified network in terms of a well-defined diversity metric and Mean-time-to-compromise (MTTC), to verify the effectiveness of our approach. We further evaluate three factors affecting the performance of the optimization, such as the network structure, the variety of products and constraints. Finally, we show the competitive scalability of our approach in finding optimal solutions within a couple of seconds to minutes for networks of large scales (up to 10,000 hosts) and high densities (up to 240,000 edges).
Online Planner Selection with Graph Neural Networks and Adaptive Scheduling
Venue: arXiv cs.AI 2019
Year: 2019

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference. Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based. The code is available at \url{https://github.com/matenure/GNN_planner}.
Consistent estimation of the max-flow problem: Towards unsupervised image segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Advances in the image-based diagnostics of complex biological and manufacturing processes have brought unsupervised image segmentation to the forefront of enabling automated, on the fly decision making. However, most existing unsupervised segmentation approaches are either computationally complex or require manual parameter selection (e.g., flow capacities in max-flow/min-cut segmentation). In this work, we present a fully unsupervised segmentation approach using a continuous max-flow formulation over the image domain while optimally estimating the flow parameters from the image characteristics. More specifically, we show that the maximum a posteriori estimate of the image labels can be formulated as a continuous max-flow problem given the flow capacities are known. The flow capacities are then iteratively obtained by employing a novel Markov random field prior over the image domain. We present theoretical results to establish the posterior consistency of the flow capacities. We compare the performance of our approach on two real-world case studies including brain tumor image segmentation and defect identification in additively manufactured components using electron microscopic images. Comparative results with several state-of-the-art supervised as well as unsupervised methods suggest that the present method performs statistically similar to the supervised methods, but results in more than 90% improvement in the Dice score when compared to the state-of-the-art unsupervised methods.
Temporal Regularization in Markov Decision Process
Venue: arXiv cs.LG 2019
Year: 2019

Several applications of Reinforcement Learning suffer from instability due to high variance. This is especially prevalent in high dimensional domains. Regularization is a commonly used technique in machine learning to reduce variance, at the cost of introducing some bias. Most existing regularization techniques focus on spatial (perceptual) regularization. Yet in reinforcement learning, due to the nature of the Bellman equation, there is an opportunity to also exploit temporal regularization based on smoothness in value estimates over trajectories. This paper explores a class of methods for temporal regularization. We formally characterize the bias induced by this technique using Markov chain concepts. We illustrate the various characteristics of temporal regularization via a sequence of simple discrete and continuous MDPs, and show that the technique provides improvement even in high-dimensional Atari games.
TRUFL: Distributed Trust Management framework in SDN
Venue: arXiv cs.CR 2019
Year: 2019

Software Defined Networking (SDN) has emerged as a revolutionary paradigm to manage cloud infrastructure. SDN lacks scalable trust setup and verification mechanism between Data Plane-Control Plane elements, Control Plane elements, and Control Plane-Application Plane. Trust management schemes like Public Key Infrastructure (PKI) used currently in SDN are slow for trust establishment in a larger cloud environment. We propose a distributed trust mechanism - TRUFL to establish and verify trust in SDN. The distributed framework utilizes parallelism in trust management, in effect faster transfer rates and reduced latency compared to centralized trust management. The TRUFL framework scales well with the number of OpenFlow rules when compared to existing research works.
Analyzing and learning the language for different types of harassment
Venue: arXiv cs.CL 2019
Year: 2019

Disclaimer: This paper is concerned with violent online harassment. To describe the subject at an adequate level of realism, examples of our collected tweets involve violent, threatening, vulgar and hateful speech language in the context of racial, sexual, political, appearance and intellectual harassment. The presence of a significant amount of harassment in user-generated content and its negative impact calls for robust automatic detection approaches. This requires that we can identify different forms or types of harassment. Earlier work has classified harassing language in terms of hurtfulness, abusiveness, sentiment, and profanity. However, to identify and understand harassment more accurately, it is essential to determine the context that represents the interrelated conditions in which they occur. In this paper, we introduce the notion of contextual type to harassment involving five categories: (i) sexual, (ii) racial, (iii) appearance-related, (iv) intellectual and (v) political. We utilize an annotated corpus from Twitter distinguishing these types of harassment. To study the context for each type that sheds light on the linguistic meaning, interpretation, and distribution, we conduct two lines of investigation: an extensive linguistic analysis, and a statistical distribution of unigrams. We then build type-ware classifiers to automate the identification of type-specific harassment. Our experiments demonstrate that these classifiers provide competitive accuracy for identifying and analyzing harassment on social media. We present extensive discussion and major observations about the effectiveness of type-aware classifiers using a detailed comparison setup providing insight into the role of type-dependent features.