OpenRaiser/Intern-Atlas huggingface dataset:
CNNPruner: Pruning Convolutional Neural Networks with Visual Analytics
Venue: arXiv cs.CV 2020
Year: 2020

Convolutional neural networks (CNNs) have demonstrated extraordinarily good performance in many computer vision tasks. The increasing size of CNN models, however, prevents them from being widely deployed to devices with limited computational resources, e.g., mobile/embedded devices. The emerging topic of model pruning strives to address this problem by removing less important neurons and fine-tuning the pruned networks to minimize the accuracy loss. Nevertheless, existing automated pruning solutions often rely on a numerical threshold of the pruning criteria, lacking the flexibility to optimally balance the trade-off between model size and accuracy. Moreover, the complicated interplay between the stages of neuron pruning and model fine-tuning makes this process opaque, and therefore becomes difficult to optimize. In this paper, we address these challenges through a visual analytics approach, named CNNPruner. It considers the importance of convolutional filters through both instability and sensitivity, and allows users to interactively create pruning plans according to a desired goal on model size or accuracy. Also, CNNPruner integrates state-of-the-art filter visualization techniques to help users understand the roles that different filters played and refine their pruning plans. Through comprehensive case studies on CNNs with real-world sizes, we validate the effectiveness of CNNPruner.
NeuroDiff: Scalable Differential Verification of Neural Networks using Fine-Grained Approximation
Venue: arXiv cs.LG 2020
Year: 2020

As neural networks make their way into safety-critical systems, where misbehavior can lead to catastrophes, there is a growing interest in certifying the equivalence of two structurally similar neural networks. For example, compression techniques are often used in practice for deploying trained neural networks on computationally- and energy-constrained devices, which raises the question of how faithfully the compressed network mimics the original network. Unfortunately, existing methods either focus on verifying a single network or rely on loose approximations to prove the equivalence of two networks. Due to overly conservative approximation, differential verification lacks scalability in terms of both accuracy and computational cost. To overcome these problems, we propose NeuroDiff, a symbolic and fine-grained approximation technique that drastically increases the accuracy of differential verification while achieving many orders-of-magnitude speedup. NeuroDiff has two key contributions. The first one is new convex approximations that more accurately bound the difference neurons of two networks under all possible inputs. The second one is judicious use of symbolic variables to represent neurons whose difference bounds have accumulated significant error. We also find that these two techniques are complementary, i.e., when combined, the benefit is greater than the sum of their individual benefits. We have evaluated NeuroDiff on a variety of differential verification tasks. Our results show that NeuroDiff is up to 1000X faster and 5X more accurate than the state-of-the-art tool.
Optimal Targeting in Super-Modular Games
Venue: arXiv cs.GT 2020
Year: 2020

We study an optimal targeting problem for super-modular games with binary actions and finitely many players. The considered problem consists in the selection of a subset of players of minimum size such that, when the actions of these players are forced to a controlled value while the others are left to repeatedly play a best response action, the system will converge to the greatest Nash equilibrium of the game. Our main contributions consist in showing that the problem is NP-complete and in proposing an efficient iterative algorithm with provable convergence properties for its solution. We discuss in detail the special case of network coordination games and its relation with the notion of cohesiveness. Finally, we show with simulations the strength of our approach with respect to naive heuristics based on classical network centrality measures.
A Predictive Interference Management Algorithm for URLLC in Beyond 5G Networks
Venue: arXiv cs.IT 2020
Year: 2020

Interference mitigation is a major design challenge in wireless systems,especially in the context of ultra-reliable low-latency communication (URLLC) services. Conventional average-based interference management schemes are not suitable for URLLC as they do not accurately capture the tail information of the interference distribution. This letter proposes a novel interference prediction algorithm that considers the entire interference distribution instead of only the mean. The key idea is to model the interference variation as a discrete state space discrete-time Markov chain. The state transition probability matrix is then used to estimate the state evolution in time, and allocate radio resources accordingly. The proposed scheme is found to meet the target reliability requirements in a low-latency single-shot transmission system considering realistic system assumptions, while requiring only ~25% more resources than the optimum case with perfect interference knowledge.
DR2S : Deep Regression with Region Selection for Camera Quality Evaluation
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we tackle the problem of estimating a camera capability to preserve fine texture details at a given lighting condition. Importantly, our texture preservation measurement should coincide with human perception. Consequently, we formulate our problem as a regression one and we introduce a deep convolutional network to estimate texture quality score. At training time, we use ground-truth quality scores provided by expert human annotators in order to obtain a subjective quality measure. In addition, we propose a region selection method to identify the image regions that are better suited at measuring perceptual quality. Finally, our experimental evaluation shows that our learning-based approach outperforms existing methods and that our region selection algorithm consistently improves the quality estimation.
DGTN: Dual-channel Graph Transition Network for Session-based Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

The task of session-based recommendation is to predict user actions based on anonymous sessions. Recent research mainly models the target session as a sequence or a graph to capture item transitions within it, ignoring complex transitions between items in different sessions that have been generated by other users. These item transitions include potential collaborative information and reflect similar behavior patterns, which we assume may help with the recommendation for the target session. In this paper, we propose a novel method, namely Dual-channel Graph Transition Network (DGTN), to model item transitions within not only the target session but also the neighbor sessions. Specifically, we integrate the target session and its neighbor (similar) sessions into a single graph. Then the transition signals are explicitly injected into the embedding by channel-aware propagation. Experiments on real-world datasets demonstrate that DGTN outperforms other state-of-the-art methods. Further analysis verifies the rationality of dual-channel item transition modeling, suggesting a potential future direction for session-based recommendation.
Bandits Under The Influence (Extended Version)
Venue: arXiv cs.LG 2020
Year: 2020

Recommender systems should adapt to user interests as the latter evolve. A prevalent cause for the evolution of user interests is the influence of their social circle. In general, when the interests are not known, online algorithms that explore the recommendation space while also exploiting observed preferences are preferable. We present online recommendation algorithms rooted in the linear multi-armed bandit literature. Our bandit algorithms are tailored precisely to recommendation scenarios where user interests evolve under social influence. In particular, we show that our adaptations of the classic LinREL and Thompson Sampling algorithms maintain the same asymptotic regret bounds as in the non-social case. We validate our approach experimentally using both synthetic and real datasets.
Crafting Adversarial Examples for Deep Learning Based Prognostics (Extended Version)
Venue: arXiv cs.LG 2020
Year: 2020

In manufacturing, unexpected failures are considered a primary operational risk, as they can hinder productivity and can incur huge losses. State-of-the-art Prognostics and Health Management (PHM) systems incorporate Deep Learning (DL) algorithms and Internet of Things (IoT) devices to ascertain the health status of equipment, and thus reduce the downtime, maintenance cost and increase the productivity. Unfortunately, IoT sensors and DL algorithms, both are vulnerable to cyber attacks, and hence pose a significant threat to PHM systems. In this paper, we adopt the adversarial example crafting techniques from the computer vision domain and apply them to the PHM domain. Specifically, we craft adversarial examples using the Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM) and apply them on the Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Convolutional Neural Network (CNN) based PHM models. We evaluate the impact of adversarial attacks using NASA's turbofan engine dataset. The obtained results show that all the evaluated PHM models are vulnerable to adversarial attacks and can cause a serious defect in the remaining useful life estimation. The obtained results also show that the crafted adversarial examples are highly transferable and may cause significant damages to PHM systems.
A Technical Review of Wireless security for the Internet of things: Software Defined Radio perspective
Venue: arXiv cs.CR 2020
Year: 2020

The increase of cyberattacks using IoT devices has exposed the vulnerabilities in the infrastructures that make up the IoT and have shown how small devices can affect networks and services functioning. This paper presents a review of the vulnerabilities of the wireless technologies that bear the IoT and assessing the experiences in implementing wireless attacks targeting the Internet of Things using Software-Defined Radio (SDR) technologies. A systematic literature review was conducted. The types of vulnerabilities and attacks that can affect the wireless technologies that stand the IoT ecosystem and SDR radio platforms were compared. On the IoT system model layer, perception layer was identified as the most vulnerable. Most attacks at this level occur due to limitations in hardware, physical exposure of devices, and heterogeneity of technologies. Future cybersecurity systems based on SDR radios have notable advantages due to their flexibility to adapt to new communication technologies and their potential for the development of advanced tools. However, cybersecurity challenges for the Internet of Things are so complex that it is needed to merge SDR hardware with cognitive techniques and intelligent techniques such as deep learning to adapt to rapid technological changes.
Connecting Distributed Pockets of EnergyFlexibility through Federated Computations:Limitations and Possibilities
Venue: arXiv cs.DC 2020
Year: 2020

Electric grids are traditionally operated as multi-entity systems with each entity managing a geographical region. Interest and demand for decarbonization and energy democratization is resulting in growing penetration of controllable energy resources. In turn, this process is increasing the number of grid entities. The paradigm shift is also fueled by increased adoption of intelligent sensors and actuators equipped with advanced processing and computing capabilities. While collaboration among power grid entities (agents) reduces energy cost and increases overall reliability, achieving effective collaboration is challenging. The main challenges stem from the heterogeneity of system agents and their collected information. Furthermore, the scale of data collection is constantly increasing and many grid entities have strict privacy requirements. Another challenge is the energy industry's common practice of keeping data in silos. Federated computation is an approach well suited to addressing these issues that are increasingly important for multi-agent energy systems. Through federated computation, agents collaboratively solve learning and optimization problems while respecting each agent's privacy and overcoming barriers of cross-device and cross-organization data isolation. In this paper, we first establish the need for federated computations to achieve energy optimization goals of the future power grid. We discuss practical challenges of performing multi-agent data processing in general. Then we address challenges that arise specifically for orchestrating operation of connected distributed energy resources in the Internet of Things. We conclude this paper by presenting a novel federated computation framework that addresses some of these issues, and we share examples of two initial field test setups in research demonstrations and commercial building applications with Grid Fruit LLC.
The Persian Dependency Treebank Made Universal
Venue: arXiv cs.CL 2020
Year: 2020

We describe an automatic method for converting the Persian Dependency Treebank (Rasooli et al, 2013) to Universal Dependencies. This treebank contains 29107 sentences. Our experiments along with manual linguistic analysis show that our data is more compatible with Universal Dependencies than the Uppsala Persian Universal Dependency Treebank (Seraji et al., 2016), and is larger in size and more diverse in vocabulary. Our data brings in a labeled attachment F-score of 85.2 in supervised parsing. Our delexicalized Persian-to-English parser transfer experiments show that a parsing model trained on our data is ~2% absolutely more accurate than that of Seraji et al. (2016) in terms of labeled attachment score.
DISPATCH: Design Space Exploration of Cyber-Physical Systems
Venue: arXiv cs.LG 2020
Year: 2020

Design of cyber-physical systems (CPSs) is a challenging task that involves searching over a large search space of various CPS configurations and possible values of components composing the system. Hence, there is a need for sample-efficient CPS design space exploration to select the system architecture and component values that meet the target system requirements. We address this challenge by formulating CPS design as a multi-objective optimization problem and propose DISPATCH, a two-step methodology for sample-efficient search over the design space. First, we use a genetic algorithm to search over discrete choices of system component values for architecture search and component selection or only component selection and terminate the algorithm even before meeting the system requirements, thus yielding a coarse design. In the second step, we use an inverse design to search over a continuous space to fine-tune the component values and meet the diverse set of system requirements. We use a neural network as a surrogate function for the inverse design of the system. The neural network, converted into a mixed-integer linear program, is used for active learning to sample component values efficiently in a continuous search space. We illustrate the efficacy of DISPATCH on electrical circuit benchmarks: two-stage and three-stage transimpedence amplifiers. Simulation results show that the proposed methodology improves sample efficiency by 5-14x compared to a prior synthesis method that relies on reinforcement learning. It also synthesizes circuits with the best performance (highest bandwidth/lowest area) compared to designs synthesized using reinforcement learning, Bayesian optimization, or humans.
Entropy, Computing and Rationality
Venue: arXiv cs.AI 2020
Year: 2020

Making decisions freely presupposes that there is some indeterminacy in the environment and in the decision making engine. The former is reflected on the behavioral changes due to communicating: few changes indicate rigid environments; productive changes manifest a moderate indeterminacy, but a large communicating effort with few productive changes characterize a chaotic environment. Hence, communicating, effective decision making and productive behavioral changes are related. The entropy measures the indeterminacy of the environment, and there is an entropy range in which communicating supports effective decision making. This conjecture is referred to here as the The Potential Productivity of Decisions. The computing engine that is causal to decision making should also have some indeterminacy. However, computations performed by standard Turing Machines are predetermined. To overcome this limitation an entropic mode of computing that is called here Relational-Indeterminate is presented. Its implementation in a table format has been used to model an associative memory. The present theory and experiment suggest the Entropy Trade-off: There is an entropy range in which computing is effective but if the entropy is too low computations are too rigid and if it is too high computations are unfeasible. The entropy trade-off of computing engines corresponds to the potential productivity of decisions of the environment. The theory is referred to an Interaction-Oriented Cognitive Architecture. Memory, perception, action and thought involve a level of indeterminacy and decision making may be free in such degree. The overall theory supports an ecological view of rationality. The entropy of the brain has been measured in neuroscience studies and the present theory supports that the brain is an entropic machine. The paper is concluded with a number of predictions that may be tested empirically.
Solving Gossip Problems using Answer Set Programming: An Epistemic Planning Approach
Venue: arXiv cs.AI 2020
Year: 2020

We investigate the use of Answer Set Programming to solve variations of gossip problems, by modeling them as epistemic planning problems.
Dynamic Multi-Agent Path Finding based on Conflict Resolution using Answer Set Programming
Venue: arXiv cs.AI 2020
Year: 2020

We study a dynamic version of multi-agent path finding problem (called D-MAPF) where existing agents may leave and new agents may join the team at different times. We introduce a new method to solve D-MAPF based on conflict-resolution. The idea is, when a set of new agents joins the team and there are conflicts, instead of replanning for the whole team, to replan only for a minimal subset of agents whose plans conflict with each other. We utilize answer set programming as part of our method for planning, replanning and identifying minimal set of conflicts.
A Machine Learning guided Rewriting Approach for ASP Logic Programs
Venue: arXiv cs.AI 2020
Year: 2020

Answer Set Programming (ASP) is a declarative logic formalism that allows to encode computational problems via logic programs. Despite the declarative nature of the formalism, some advanced expertise is required, in general, for designing an ASP encoding that can be efficiently evaluated by an actual ASP system. A common way for trying to reduce the burden of manually tweaking an ASP program consists in automatically rewriting the input encoding according to suitable techniques, for producing alternative, yet semantically equivalent, ASP programs. However, rewriting does not always grant benefits in terms of performance; hence, proper means are needed for predicting their effects with this respect. In this paper we describe an approach based on Machine Learning (ML) to automatically decide whether to rewrite. In particular, given an ASP program and a set of input facts, our approach chooses whether and how to rewrite input rules based on a set of features measuring their structural properties and domain information. To this end, a Multilayer Perceptrons model has then been trained to guide the ASP grounder I-DLV on rewriting input rules. We report and discuss the results of an experimental evaluation over a prototypical implementation.
Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning
Venue: arXiv cs.LG 2020
Year: 2020

Graph representation learning has attracted lots of attention recently. Existing graph neural networks fed with the complete graph data are not scalable due to limited computation and memory costs. Thus, it remains a great challenge to capture rich information in large-scale graph data. Besides, these methods mainly focus on supervised learning and highly depend on node label information, which is expensive to obtain in the real world. As to unsupervised network embedding approaches, they overemphasize node proximity instead, whose learned representations can hardly be used in downstream application tasks directly. In recent years, emerging self-supervised learning provides a potential solution to address the aforementioned problems. However, existing self-supervised works also operate on the complete graph data and are biased to fit either global or very local (1-hop neighborhood) graph structures in defining the mutual information based loss terms. In this paper, a novel self-supervised representation learning method via Subgraph Contrast, namely \textsc{Subg-Con}, is proposed by utilizing the strong correlation between central nodes and their sampled subgraphs to capture regional structure information. Instead of learning on the complete input graph data, with a novel data augmentation strategy, \textsc{Subg-Con} learns node representations through a contrastive loss defined based on subgraphs sampled from the original graph instead. Compared with existing graph representation learning approaches, \textsc{Subg-Con} has prominent performance advantages in weaker supervision requirements, model learning scalability, and parallelization. Extensive experiments verify both the effectiveness and the efficiency of our work compared with both classic and state-of-the-art graph representation learning approaches on multiple real-world large-scale benchmark datasets from different domains.
Learning Image Labels On-the-fly for Training Robust Classification Models
Venue: arXiv cs.CV 2020
Year: 2020

Current deep learning paradigms largely benefit from the tremendous amount of annotated data. However, the quality of the annotations often varies among labelers. Multi-observer studies have been conducted to study these annotation variances (by labeling the same data for multiple times) and its effects on critical applications like medical image analysis. This process indeed adds an extra burden to the already tedious annotation work that usually requires professional training and expertise in the specific domains. On the other hand, automated annotation methods based on NLP algorithms have recently shown promise as a reasonable alternative, relying on the existing diagnostic reports of those images that are widely available in the clinical system. Compared to human labelers, different algorithms provide labels with varying qualities that are even noisier. In this paper, we show how noisy annotations (e.g., from different algorithm-based labelers) can be utilized together and mutually benefit the learning of classification tasks. Specifically, the concept of attention-on-label is introduced to sample better label sets on-the-fly as the training data. A meta-training based label-sampling module is designed to attend the labels that benefit the model learning the most through additional back-propagation processes. We apply the attention-on-label scheme on the classification task of a synthetic noisy CIFAR-10 dataset to prove the concept, and then demonstrate superior results (3-5% increase on average in multiple disease classification AUCs) on the chest x-ray images from a hospital-scale dataset (MIMIC-CXR) and hand-labeled dataset (OpenI) in comparison to regular training paradigms.