OpenRaiser/Intern-Atlas huggingface dataset:
A Review of Confidentiality Threats Against Embedded Neural Network Models
Venue: arXiv cs.CR 2021
Year: 2021

Utilization of Machine Learning (ML) algorithms, especially Deep Neural Network (DNN) models, becomes a widely accepted standard in many domains more particularly IoT-based systems. DNN models reach impressive performances in several sensitive fields such as medical diagnosis, smart transport or security threat detection, and represent a valuable piece of Intellectual Property. Over the last few years, a major trend is the large-scale deployment of models in a wide variety of devices. However, this migration to embedded systems is slowed down because of the broad spectrum of attacks threatening the integrity, confidentiality and availability of embedded models. In this review, we cover the landscape of attacks targeting the confidentiality of embedded DNN models that may have a major impact on critical IoT systems, with a particular focus on model extraction and data leakage. We highlight the fact that Side-Channel Analysis (SCA) is a relatively unexplored bias by which model's confidentiality can be compromised. Input data, architecture or parameters of a model can be extracted from power or electromagnetic observations, testifying a real need from a security point of view.
Automatic Learning to Detect Concept Drift
Venue: arXiv cs.AI 2021
Year: 2021

Many methods have been proposed to detect concept drift, i.e., the change in the distribution of streaming data, due to concept drift causes a decrease in the prediction accuracy of algorithms. However, the most of current detection methods are based on the assessment of the degree of change in the data distribution, cannot identify the type of concept drift. In this paper, we propose Active Drift Detection with Meta learning (Meta-ADD), a novel framework that learns to classify concept drift by tracking the changed pattern of error rates. Specifically, in the training phase, we extract meta-features based on the error rates of various concept drift, after which a meta-detector is developed via a prototypical neural network by representing various concept drift classes as corresponding prototypes. In the detection phase, the learned meta-detector is fine-tuned to adapt to the corresponding data stream via stream-based active learning. Hence, Meta-ADD uses machine learning to learn to detect concept drifts and identify their types automatically, which can directly support drift understand. The experiment results verify the effectiveness of Meta-ADD.
Novel Algorithm for Computing All-Pairs Homogeneity-Arc Binary-State Undirected Network Reliability
Venue: arXiv cs.DM 2021
Year: 2021

Among various real-life emerging applications, wireless sensor networks, Internet of Things, smart grids, social networks, communication networks, transportation networks, and computer grid systems, etc., the binary-state network is the fundamental network structure and model with either working or failed binary components. The network reliability is an effective index for assessing the network function and performance. Hence, the network reliability between two specific nodes has been widely adopted and more efficient network reliability algorithm is always needed. To have complete information for a better decision, all-pairs network reliability thus arises correspondingly. In this study, a new algorithm called the all-pairs BAT is proposed by revising the binary-addition-tree algorithm (BAT) and the layered-search algorithm (LSA). From both the theoretical analysis and the practical experiments conducted on 20 benchmark problems, the proposed all-pairs BAT is more efficient than these algorithms by trying all combinations of any pairs of nodes.
Multiparty Interactive Coding over Networks of Intersecting Broadcast Links
Venue: arXiv cs.IT 2021
Year: 2021

We consider computations over networks with multiple broadcast channels that intersect at a single party. Each broadcast link suffers from random bit-flip noise that affects the receivers independently. We design interactive coding schemes that successfully perform any computation over these noisy networks and strive to reduce their communication overhead with respect to the original (noiseless) computation. A simple variant of a coding scheme by Rajagopalan and Schulman (STOC 1994) shows that any (noiseless) protocol of $R$ rounds can be reliably simulated in $O(R\log n)$ rounds over a network with $n=n_1n_2+1$ parties in which a single party is connected to $n_2$ noisy broadcast channels, each of which connects $n_1$ distinct parties. We design new coding schemes with improved overheads. Our approach divides the network into four regimes according to the relationship between $n_1$ and $n_2$. We employ a two-layer coding where the inner code protects each broadcast channel and is tailored to the specific conditions of the regime in consideration. The outer layer protects the computation in the network and is generally based on the scheme of Rajagopalan and Schulman, adapted to the case of broadcast channels. The overhead we obtain ranges from $O(\log\log n_2)$ to $O(\log n_2 \frac{\log\log n_1}{\log n_1})$ and beats the trivial $O(\log n)$ overhead in all four regimes.
Where and When: Space-Time Attention for Audio-Visual Explanations
Venue: arXiv cs.CV 2021
Year: 2021

Explaining the decision of a multi-modal decision-maker requires to determine the evidence from both modalities. Recent advances in XAI provide explanations for models trained on still images. However, when it comes to modeling multiple sensory modalities in a dynamic world, it remains underexplored how to demystify the mysterious dynamics of a complex multi-modal model. In this work, we take a crucial step forward and explore learnable explanations for audio-visual recognition. Specifically, we propose a novel space-time attention network that uncovers the synergistic dynamics of audio and visual data over both space and time. Our model is capable of predicting the audio-visual video events, while justifying its decision by localizing where the relevant visual cues appear, and when the predicted sounds occur in videos. We benchmark our model on three audio-visual video event datasets, comparing extensively to multiple recent multi-modal representation learners and intrinsic explanation models. Experimental results demonstrate the clear superior performance of our model over the existing methods on audio-visual video event recognition. Moreover, we conduct an in-depth study to analyze the explainability of our model based on robustness analysis via perturbation tests and pointing games using human annotations.
Robustness Enhancement of Object Detection in Advanced Driver Assistance Systems (ADAS)
Venue: arXiv cs.CV 2021
Year: 2021

A unified system integrating a compact object detector and a surrounding environmental condition classifier for enhancing the robustness of object detection scheme in advanced driver assistance systems (ADAS) is proposed in this paper. ADAS are invented to improve traffic safety and effectiveness in autonomous driving systems where object detection plays an extremely important role. However, modern object detectors integrated in ADAS are still unstable due to high latency and the variation of the environmental contexts in the deployment phase. Our system is proposed to address the aforementioned problems. The proposed system includes two main components: (1) a compact one-stage object detector which is expected to be able to perform at a comparable accuracy compared to state-of-the-art object detectors, and (2) an environmental condition detector that helps to send a warning signal to the cloud in case the self-driving car needs human actions due to the significance of the situation. The empirical results prove the reliability and the scalability of the proposed system to realistic scenarios.
FPT algorithms for packing $k$-safe spanning rooted sub(di)graphs
Venue: arXiv cs.DS 2021
Year: 2021

We study three problems introduced by Bang-Jensen and Yeo [Theor. Comput. Sci. 2015] and by Bang-Jensen, Havet, and Yeo [Discret. Appl. Math. 2016] about finding disjoint "balanced" spanning rooted substructures in graphs and digraphs, which generalize classic packing problems. Namely, given a positive integer $k$, a digraph $D=(V,A)$, and a root $r \in V$, we consider the problem of finding two arc-disjoint $k$-safe spanning $r$-arborescences and the problem of finding two arc-disjoint $(r,k)$-flow branchings. We show that both these problems are FPT with parameter $k$, improving on existing XP algorithms. The latter of these results answers a question of Bang-Jensen, Havet, and Yeo [Discret. Appl. Math. 2016]. Further, given an integer $k$, a graph $G=(V,E)$, and $r \in V$, we consider the problem of finding two arc-disjoint $(r,k)$-safe spanning trees. We show that this problem is also FPT with parameter $k$, again improving on a previous XP algorithm. Our main technical contribution is to prove that the existence of such spanning substructures is equivalent to the existence of substructures with size and maximum (out-)degree both bounded by a (linear or quadratic) function of $k$, which may be of independent interest.
Regret Bounds for Stochastic Shortest Path Problems with Linear Function Approximation
Venue: arXiv cs.LG 2021
Year: 2021

We propose an algorithm that uses linear function approximation (LFA) for stochastic shortest path (SSP). Under minimal assumptions, it obtains sublinear regret, is computationally efficient, and uses stationary policies. To our knowledge, this is the first such algorithm in the LFA literature (for SSP or other formulations). Our algorithm is a special case of a more general one, which achieves regret square root in the number of episodes given access to a certain computation oracle.
Orienting Point Clouds with Dipole Propagation
Venue: arXiv cs.GR 2021
Year: 2021

Establishing a consistent normal orientation for point clouds is a notoriously difficult problem in geometry processing, requiring attention to both local and global shape characteristics. The normal direction of a point is a function of the local surface neighborhood; yet, point clouds do not disclose the full underlying surface structure. Even assuming known geodesic proximity, calculating a consistent normal orientation requires the global context. In this work, we introduce a novel approach for establishing a globally consistent normal orientation for point clouds. Our solution separates the local and global components into two different sub-problems. In the local phase, we train a neural network to learn a coherent normal direction per patch (i.e., consistently oriented normals within a single patch). In the global phase, we propagate the orientation across all coherent patches using a dipole propagation. Our dipole propagation decides to orient each patch using the electric field defined by all previously orientated patches. This gives rise to a global propagation that is stable, as well as being robust to nearby surfaces, holes, sharp features and noise.
Deep Reinforcement Learning for Adaptive Exploration of Unknown Environments
Venue: arXiv cs.LG 2021
Year: 2021

Performing autonomous exploration is essential for unmanned aerial vehicles (UAVs) operating in unknown environments. Often, these missions start with building a map for the environment via pure exploration and subsequently using (i.e. exploiting) the generated map for downstream navigation tasks. Accomplishing these navigation tasks in two separate steps is not always possible or even disadvantageous for UAVs deployed in outdoor and dynamically changing environments. Current exploration approaches either use a priori human-generated maps or use heuristics such as frontier-based exploration. Other approaches use learning but focus only on learning policies for specific tasks by either using sample inefficient random exploration or by making impractical assumptions about full map availability. In this paper, we develop an adaptive exploration approach to trade off between exploration and exploitation in one single step for UAVs searching for areas of interest (AoIs) in unknown environments using Deep Reinforcement Learning (DRL). The proposed approach uses a map segmentation technique to decompose the environment map into smaller, tractable maps. Then, a simple information gain function is repeatedly computed to determine the best target region to search during each iteration of the process. DDQN and A2C algorithms are extended with a stack of LSTM layers and trained to generate optimal policies for the exploration and exploitation, respectively. We tested our approach in 3 different tasks against 4 baselines. The results demonstrate that our proposed approach is capable of navigating through randomly generated environments and covering more AoI in less time steps compared to the baselines.
Learning 3D Granular Flow Simulations
Venue: arXiv cs.LG 2021
Year: 2021

Recently, the application of machine learning models has gained momentum in natural sciences and engineering, which is a natural fit due to the abundance of data in these fields. However, the modeling of physical processes from simulation data without first principle solutions remains difficult. Here, we present a Graph Neural Networks approach towards accurate modeling of complex 3D granular flow simulation processes created by the discrete element method LIGGGHTS and concentrate on simulations of physical systems found in real world applications like rotating drums and hoppers. We discuss how to implement Graph Neural Networks that deal with 3D objects, boundary conditions, particle - particle, and particle - boundary interactions such that an accurate modeling of relevant physical quantities is made possible. Finally, we compare the machine learning based trajectories to LIGGGHTS trajectories in terms of particle flows and mixing entropies.
Viewport-Aware Dynamic 360° Video Segment Categorization
Venue: arXiv cs.MM 2021
Year: 2021

Unlike conventional videos, 360° videos give freedom to users to turn their heads, watch and interact with the content owing to its immersive spherical environment. Although these movements are arbitrary, similarities can be observed between viewport patterns of different users and different videos. Identifying such patterns can assist both content and network providers to enhance the 360° video streaming process, eventually increasing the end-user Quality of Experience (QoE). But a study on how viewport patterns display similarities across different video content, and their potential applications has not yet been done. In this paper, we present a comprehensive analysis of a dataset of 88 360° videos and propose a novel video categorization algorithm that is based on similarities of viewports. First, we propose a novel viewport clustering algorithm that outperforms the existing algorithms in terms of clustering viewports with similar positioning and speed. Next, we develop a novel and unique dynamic video segment categorization algorithm that shows notable improvement in similarity for viewport distributions within the clusters when compared to that of existing static video categorizations.
Sampling From the Wasserstein Barycenter
Venue: arXiv cs.LG 2021
Year: 2021

This work presents an algorithm to sample from the Wasserstein barycenter of absolutely continuous measures. Our method is based on the gradient flow of the multimarginal formulation of the Wasserstein barycenter, with an additive penalization to account for the marginal constraints. We prove that the minimum of this penalized multimarginal formulation is achieved for a coupling that is close to the Wasserstein barycenter. The performances of the algorithm are showcased in several settings.
ABET Accreditation: A Way Forward for PDC Education
Venue: arXiv cs.DC 2021
Year: 2021

With parallel and distributed computing (PDC) now wide-spread, modern computing programs must incorporate PDC within the curriculum. ACM and IEEE Computer Society's Computer Science curricular guidelines have recommended exposure to PDC concepts since 2013. More recently, a variety of initiatives have made PDC curricular content, lectures, and labs freely available for undergraduate computer science programs. Despite these efforts, progress in ensuring computer science students graduate with sufficient PDC exposure has been uneven. This paper discusses the impact of ABET's revised criteria that have required exposure to PDC to achieve accreditation for computer science programs since 2018. The authors reviewed 20 top ABET-accredited computer science programs and analyzed how they covered the required PDC components in their curricula. Using their own institutions as case studies, the authors examine in detail how three different ABET-accredited computer science programs covered PDC using different approaches, yet meeting the PDC requirements of these ABET criteria. The paper also shows how ACM/IEEE Computer Society curricular guidelines for computer engineering and software engineering programs, along with ABET accreditation criteria, can cover PDC.
Information Complexity and Generalization Bounds
Venue: arXiv cs.LG 2021
Year: 2021

We present a unifying picture of PAC-Bayesian and mutual information-based upper bounds on the generalization error of randomized learning algorithms. As we show, Tong Zhang's information exponential inequality (IEI) gives a general recipe for constructing bounds of both flavors. We show that several important results in the literature can be obtained as simple corollaries of the IEI under different assumptions on the loss function. Moreover, we obtain new bounds for data-dependent priors and unbounded loss functions. Optimizing the bounds gives rise to variants of the Gibbs algorithm, for which we discuss two practical examples for learning with neural networks, namely, Entropy- and PAC-Bayes- SGD. Further, we use an Occam's factor argument to show a PAC-Bayesian bound that incorporates second-order curvature information of the training loss.
Reinforcement Learning for Scalable Logic Optimization with Graph Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Logic optimization is an NP-hard problem commonly approached through hand-engineered heuristics. We propose to combine graph convolutional networks with reinforcement learning and a novel, scalable node embedding method to learn which local transforms should be applied to the logic graph. We show that this method achieves a similar size reduction as ABC on smaller circuits and outperforms it by 1.5-1.75x on larger random graphs.
A Survey on End-User Robot Programming
Venue: arXiv cs.RO 2021
Year: 2021

As robots interact with a broader range of end-users, end-user robot programming has helped democratize robot programming by empowering end-users who may not have experience in robot programming to customize robots to meet their individual contextual needs. This article surveys work on end-user robot programming, with a focus on end-user program specification. It describes the primary domains, programming phases, and design choices represented by the end-user robot programming literature. The survey concludes by highlighting open directions for further investigation to enhance and widen the reach of end-user robot programming systems.
Surveilling Surveillance: Estimating the Prevalence of Surveillance Cameras with Street View Data
Venue: arXiv cs.CY 2021
Year: 2021

The use of video surveillance in public spaces -- both by government agencies and by private citizens -- has attracted considerable attention in recent years, particularly in light of rapid advances in face-recognition technology. But it has been difficult to systematically measure the prevalence and placement of cameras, hampering efforts to assess the implications of surveillance on privacy and public safety. Here, we combine computer vision, human verification, and statistical analysis to estimate the spatial distribution of surveillance cameras. Specifically, we build a camera detection model and apply it to 1.6 million street view images sampled from 10 large U.S. cities and 6 other major cities around the world, with positive model detections verified by human experts. After adjusting for the estimated recall of our model, and accounting for the spatial coverage of our sampled images, we are able to estimate the density of surveillance cameras visible from the road. Across the 16 cities we consider, the estimated number of surveillance cameras per linear kilometer ranges from 0.2 (in Los Angeles) to 0.9 (in Seoul). In a detailed analysis of the 10 U.S. cities, we find that cameras are concentrated in commercial, industrial, and mixed zones, and in neighborhoods with higher shares of non-white residents -- a pattern that persists even after adjusting for land use. These results help inform ongoing discussions on the use of surveillance technology, including its potential disparate impacts on communities of color.
An Optimal Algorithm for Triangle Counting in the Stream
Venue: arXiv cs.DS 2021
Year: 2021

We present a new algorithm for approximating the number of triangles in a graph $G$ whose edges arrive as an arbitrary order stream. If $m$ is the number of edges in $G$, $T$ the number of triangles, $Δ_E$ the maximum number of triangles which share a single edge, and $Δ_V$ the maximum number of triangles which share a single vertex, then our algorithm requires space: \[ \widetilde{O}\left(\frac{m}{T}\cdot \left(Δ_E + \sqrt{Δ_V}\right)\right) \] Taken with the $Ω\left(\frac{m Δ_E}{T}\right)$ lower bound of Braverman, Ostrovsky, and Vilenchik (ICALP 2013), and the $Ω\left( \frac{m \sqrt{Δ_V}}{T}\right)$ lower bound of Kallaugher and Price (SODA 2017), our algorithm is optimal up to log factors, resolving the complexity of a classic problem in graph streaming.
Intensity Harmonization for Airborne LiDAR
Venue: arXiv cs.CV 2021
Year: 2021

Constructing a point cloud for a large geographic region, such as a state or country, can require multiple years of effort. Often several vendors will be used to acquire LiDAR data, and a single region may be captured by multiple LiDAR scans. A key challenge is maintaining consistency between these scans, which includes point density, number of returns, and intensity. Intensity in particular can be very different between scans, even in areas that are overlapping. Harmonizing the intensity between scans to remove these discrepancies is expensive and time consuming. In this paper, we propose a novel method for point cloud harmonization based on deep neural networks. We evaluate our method quantitatively and qualitatively using a high quality real world LiDAR dataset. We compare our method to several baselines, including standard interpolation methods as well as histogram matching. We show that our method performs as well as the best baseline in areas with similar intensity distributions, and outperforms all baselines in areas with different intensity distributions. Source code is available at https://github.com/mvrl/lidar-harmonization .