OpenRaiser/Intern-Atlas huggingface dataset:
Ethical issues with using Internet of Things devices in citizen science research: A scoping review
Venue: arXiv cs.CY 2021
Year: 2021

Our chapter presents a scoping review of published scientific studies or case studies of scientific studies that utilise both citizen scientists and Internet of Things devices. Specifically, we selected studies where the authors had included at least a short discussion of the ethical issues encountered during the research process. Having conducted a search of five databases (IEEE Xplore, Scopus, Web of Science, ProQuest, and PubMed), we identified 631 potential results. Following abstract and title screening, and then full text eligibility assessment, we identified 34 published articles that matched our criteria. We then analysed the full text for these articles inductively and deductively, coding ethical issues into three main categories. These categories were autonomy and data privacy, data quality, and intellectual property. We also analysed the full text of these articles to see what strategies researchers took to resolve these ethical issues, as well as any legal implications raised. Following this analysis, our discussion provides recommendations for researchers who wish to integrate citizen scientists and Internet of Things devices into their research. First, all citizen science projects should integrate a data privacy protocol to protect the confidentiality of participants. Secondly, scientific researchers should consider any potential issues of data quality, including whether compromises might be required, before establishing a project. Finally, all intellectual property issues should be clarified both at the start of the project and during its lifecycle. Researchers should also consider any ethical issues that might flow from the use of commercially available Internet of Things devices for research.
Adversarial Immunization for Certifiable Robustness on Graphs
Venue: arXiv cs.LG 2021
Year: 2021

Despite achieving strong performance in semi-supervised node classification task, graph neural networks (GNNs) are vulnerable to adversarial attacks, similar to other deep learning models. Existing researches focus on developing either robust GNN models or attack detection methods against adversarial attacks on graphs. However, little research attention is paid to the potential and practice of immunization to adversarial attacks on graphs. In this paper, we propose and formulate the graph adversarial immunization problem, i.e., vaccinating an affordable fraction of node pairs, connected or unconnected, to improve the certifiable robustness of graph against any admissible adversarial attack. We further propose an effective algorithm, called AdvImmune, which optimizes with meta-gradient in a discrete way to circumvent the computationally expensive combinatorial optimization when solving the adversarial immunization problem. Experiments are conducted on two citation networks and one social network. Experimental results demonstrate that the proposed AdvImmune method remarkably improves the ratio of robust nodes by 12%, 42%, 65%, with an affordable immune budget of only 5% edges.
Towards Joint Learning of Optimal MAC Signaling and Wireless Channel Access
Venue: arXiv cs.IT 2021
Year: 2021

Communication protocols are the languages used by network nodes. Before a user equipment (UE) can exchange data with a base station (BS), it must first negotiate the conditions and parameters for that transmission. This negotiation is supported by signaling messages at all layers of the protocol stack. Each year, the mobile communications industry defines and standardizes these messages, which are designed by humans during lengthy technical (and often political) debates. Following this standardization effort, the development phase begins, wherein the industry interprets and implements the resulting standards. But is this massive development undertaking the only way to implement a given protocol? We address the question of whether radios can learn a pre-given target protocol as an intermediate step towards evolving their own. Furthermore, we train cellular radios to emerge a channel access policy that performs optimally under the constraints of the target protocol. We show that multi-agent reinforcement learning (MARL) and learning-to-communicate (L2C) techniques achieve this goal with gains over expert systems. Finally, we provide insight into the transferability of these results to scenarios never seen during training.
Jupyter Notebooks on GitHub: Characteristics and Code Clones
Venue: arXiv cs.SE 2021
Year: 2021

Jupyter notebooks has emerged as a standard tool for data science programming. Programs in Jupyter notebooks are different from typical programs as they are constructed by a collection of code snippets interleaved with text and visualisation. This allows interactive exploration and snippets may be executed in different order which may give rise to different results due to side-effects between snippets. Previous studies have shown the presence of considerable code duplication -- code clones -- in sources of traditional programs, in both so-called systems programming languages and so-called scripting languages. In this paper we present the first large-scale study of code cloning in Jupyter notebooks. We analyse a corpus of 2.7 million Jupyter notebooks hosted on GitHJub, representing 37 million individual snippets and 227 million lines of code. We study clones at the level of individual snippets, and study the extent to which snippets are recurring across multiple notebooks. We study both identical clones and approximate clones and conduct a small-scale ocular inspection of the most common clones. We find that code cloning is common in Jupyter notebooks -- more than 70% of all code snippets are exact copies of other snippets (with possible differences in white spaces), and around 50% of all notebooks do not have any unique snippet, but consists solely of snippets that are also found elsewhere. In notebooks written in Python, at least 80% of all snippets are approximate clones and the prevalence of code cloning is higher in Python than in other languages. We further find that clones between different repositories are far more common than clones within the same repository. However, the most common individual repository from which a Jupyter notebook contains clones is the repository in which itself resides.
Differentiable Joint Pruning and Quantization for Hardware Efficiency
Venue: arXiv cs.LG 2021
Year: 2021

We present a differentiable joint pruning and quantization (DJPQ) scheme. We frame neural network compression as a joint gradient-based optimization problem, trading off between model pruning and quantization automatically for hardware efficiency. DJPQ incorporates variational information bottleneck based structured pruning and mixed-bit precision quantization into a single differentiable loss function. In contrast to previous works which consider pruning and quantization separately, our method enables users to find the optimal trade-off between both in a single training procedure. To utilize the method for more efficient hardware inference, we extend DJPQ to integrate structured pruning with power-of-two bit-restricted quantization. We show that DJPQ significantly reduces the number of Bit-Operations (BOPs) for several networks while maintaining the top-1 accuracy of original floating-point models (e.g., 53x BOPs reduction in ResNet18 on ImageNet, 43x in MobileNetV2). Compared to the conventional two-stage approach, which optimizes pruning and quantization independently, our scheme outperforms in terms of both accuracy and BOPs. Even when considering bit-restricted quantization, DJPQ achieves larger compression ratios and better accuracy than the two-stage approach.
A Greedy Algorithm for the Social Golfer and the Oberwolfach Problem
Venue: arXiv cs.DM 2021
Year: 2021

Inspired by the increasing popularity of Swiss-system tournaments in sports, we study the problem of predetermining the number of rounds that can be guaranteed in a Swiss-system tournament. Matches of these tournaments are usually determined in a myopic round-based way dependent on the results of previous rounds. Together with the hard constraint that no two players meet more than once during the tournament, at some point it might become infeasible to schedule a next round. For tournaments with $n$ players and match sizes of $k\geq2$ players, we prove that we can always guarantee $\lfloor \frac{n}{k(k-1)} \rfloor$ rounds. We show that this bound is tight. This provides a simple polynomial time constant factor approximation algorithm for the social golfer problem. We extend the results to the Oberwolfach problem. We show that a simple greedy approach guarantees at least $\lfloor \frac{n+4}{6} \rfloor$ rounds for the Oberwolfach problem. This yields a polynomial time $\frac{1}{3+ε}$-approximation algorithm for any fixed $ε>0$ for the Oberwolfach problem. Assuming that El-Zahar's conjecture is true, we improve the bound on the number of rounds to be essentially tight.
A strain-gradient formulation for fiber reinforced polymers: Hybrid phase-field model for porous-ductile fracture
Venue: arXiv cs.CE 2021
Year: 2021

A novel numerical approach to analyze the mechanical behavior within composite materials including the inelastic regime up to final failure is presented. Therefore, a second-gradient theory is combined with phase-field methods to fracture. In particular, we assume that the polymeric matrix material undergoes ductile fracture, whereas continuously embedded fibers undergo brittle fracture as it is typical e.g. for roving glass reinforced thermoplastics. A hybrid phase-field approach is developed and applied along with a modified Gurson-Tvergaard-Needelman GTN-type plasticity model accounting for a temperature-dependent growth of voids on microscale. The mechanical response of the arising microstructure of the woven fabric gives rise to additional higher-order terms, representing homogenized bending contributions of the fibers. Eventually, a series of tests is conducted for this physically comprehensive multifield formulation to investigate different kinds and sequences of failure within long fiber reinforced polymers.
Book Success Prediction with Pretrained Sentence Embeddings and Readability Scores
Venue: arXiv cs.CL 2021
Year: 2021

Predicting the potential success of a book in advance is vital in many applications. This could help both publishers and readers in their decision-making process whether or not a book is worth publishing and reading, respectively. In this paper, we propose a model that leverages pretrained sentence embeddings along with various readability scores for book success prediction. Unlike previous methods, the proposed method requires no count-based, lexical, or syntactic features. Instead, we use a convolutional neural network over pretrained sentence embeddings and leverage different readability scores through a simple concatenation operation. Our proposed model outperforms strong baselines for this task by as large as 6.4\% F1-score points. Moreover, our experiments show that according to our model, only the first 1K sentences are good enough to predict the potential success of books.
Interpretable Anomaly Detection with DIFFI: Depth-based Isolation Forest Feature Importance
Venue: arXiv cs.LG 2021
Year: 2021

Anomaly Detection is an unsupervised learning task aimed at detecting anomalous behaviours with respect to historical data. In particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called `Anomaly Score') and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection, due to its proven effectiveness and low computational complexity. A major problem affecting Isolation Forest is represented by the lack of interpretability, an effect of the inherent randomness governing the splits performed by the Isolation Trees, the building blocks of the Isolation Forest. In this paper we propose effective, yet computationally inexpensive, methods to define feature importance scores at both global and local level for the Isolation Forest. Moreover, we define a procedure to perform unsupervised feature selection for Anomaly Detection problems based on our interpretability method; such procedure also serves the purpose of tackling the challenging task of feature importance evaluation in unsupervised anomaly detection. We assess the performance on several synthetic and real-world datasets, including comparisons against state-of-the-art interpretability techniques, and make the code publicly available to enhance reproducibility and foster research in the field.
On Linear Convergence of Policy Gradient Methods for Finite MDPs
Venue: arXiv cs.LG 2021
Year: 2021

We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been some recent work viewing this setting as an instance of smooth non-linear optimization problems and showing sub-linear convergence rates with small step-sizes. Here, we take a different perspective based on connections with policy iteration and show that many variants of policy gradient methods succeed with large step-sizes and attain a linear rate of convergence.
MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning
Venue: arXiv cs.LG 2021
Year: 2021

Graph Neural Networks (GNNs) have recently caught great attention and achieved significant progress in graph-level applications. In this paper, we propose a framework for graph neural networks with multiresolution Haar-like wavelets, or MathNet, with interrelated convolution and pooling strategies. The underlying method takes graphs in different structures as input and assembles consistent graph representations for readout layers, which then accomplishes label prediction. To achieve this, the multiresolution graph representations are first constructed and fed into graph convolutional layers for processing. The hierarchical graph pooling layers are then involved to downsample graph resolution while simultaneously remove redundancy within graph signals. The whole workflow could be formed with a multi-level graph analysis, which not only helps embed the intrinsic topological information of each graph into the GNN, but also supports fast computation of forward and adjoint graph transforms. We show by extensive experiments that the proposed framework obtains notable accuracy gains on graph classification and regression tasks with performance stability. The proposed MathNet outperforms various existing GNN models, especially on big data sets.
Simplifying Dependent Reductions in the Polyhedral Model
Venue: arXiv cs.PL 2021
Year: 2021

A Reduction -- an accumulation over a set of values, using an associative and commutative operator -- is a common computation in many numerical computations, including scientific computations, machine learning, computer vision, and financial analytics. Contemporary polyhedral-based compilation techniques make it possible to optimize reductions, such as prefix sums, in which each component of the reduction's output potentially shares computation with another component in the reduction. Therefore an optimizing compiler can identify the computation shared between multiple components and generate code that computes the shared computation only once. These techniques, however, do not support reductions that -- when phrased in the language of the polyhedral model -- span multiple dependent statements. In such cases, existing approaches can generate incorrect code that violates the data dependences of the original, unoptimized program. In this work, we identify and formalize the optimization of dependent reductions as an integer bilinear program. We present a heuristic optimization algorithm that uses an affine sequential schedule of the program to determine how to simplfy reductions yet still preserve the program's dependences. We demonstrate that the algorithm provides optimal complexity for a set of benchmark programs from the literature on probabilistic inference algorithms, whose performance critically relies on simplifying these reductions. The complexities for 10 of the 11 programs improve siginifcantly by factors at least of the sizes of the input data, which are in the range of $10^4$ to $10^6$ for typical real application inputs. We also confirm the significance of the improvement by showing speedups in wall-clock time that range from $1.1\text{x}$ to over $10^6\text{x}$.
SOCRATES: Towards a Unified Platform for Neural Network Analysis
Venue: arXiv cs.LG 2021
Year: 2021

Studies show that neural networks, not unlike traditional programs, are subject to bugs, e.g., adversarial samples that cause classification errors and discriminatory instances that demonstrate the lack of fairness. Given that neural networks are increasingly applied in critical applications (e.g., self-driving cars, face recognition systems and personal credit rating systems), it is desirable that systematic methods are developed to analyze (e.g., test or verify) neural networks against desirable properties. Recently, a number of approaches have been developed for analyzing neural networks. These efforts are however scattered (i.e., each approach tackles some restricted classes of neural networks against certain particular properties), incomparable (i.e., each approach has its own assumptions and input format) and thus hard to apply, reuse or extend. In this project, we aim to build a unified framework for developing techniques to analyze neural networks. Towards this goal, we develop a platform called SOCRATES which supports a standardized format for a variety of neural network models, an assertion language for property specification as well as multiple neural network analysis algorithms including two novel ones for falsifying and probabilistic verification of neural network models. SOCRATES is extensible and thus existing approaches can be easily integrated. Experiment results show that our platform can handle a wide range of networks models and properties. More importantly, it provides a platform for synergistic research on neural network analysis.
Breaking the Communication-Privacy-Accuracy Trilemma
Venue: arXiv cs.LG 2021
Year: 2021

Two major challenges in distributed learning and estimation are 1) preserving the privacy of the local samples; and 2) communicating them efficiently to a central server, while achieving high accuracy for the end-to-end task. While there has been significant interest in addressing each of these challenges separately in the recent literature, treatments that simultaneously address both challenges are still largely missing. In this paper, we develop novel encoding and decoding mechanisms that simultaneously achieve optimal privacy and communication efficiency in various canonical settings. In particular, we consider the problems of mean estimation and frequency estimation under $\varepsilon$-local differential privacy and $b$-bit communication constraints. For mean estimation, we propose a scheme based on Kashin's representation and random sampling, with order-optimal estimation error under both constraints. For frequency estimation, we present a mechanism that leverages the recursive structure of Walsh-Hadamard matrices and achieves order-optimal estimation error for all privacy levels and communication budgets. As a by-product, we also construct a distribution estimation mechanism that is rate-optimal for all privacy regimes and communication constraints, extending recent work that is limited to $b=1$ and $\varepsilon=O(1)$. Our results demonstrate that intelligent encoding under joint privacy and communication constraints can yield a performance that matches the optimal accuracy achievable under either constraint alone.
Autonomous Removal of Perspective Distortion of Elevator Button Images based on Corner Detection
Venue: arXiv cs.CV 2021
Year: 2021

Elevator button recognition is a critical function to realize the autonomous operation of elevators. However, challenging image conditions and various image distortions make it difficult to recognize buttons accurately. To fill this gap, we propose a novel deep learning-based approach, which aims to autonomously correct perspective distortions of elevator button images based on button corner detection results. First, we leverage a novel image segmentation model and the Hough Transform method to obtain button segmentation and button corner detection results. Then, pixel coordinates of standard button corners are used as reference features to estimate camera motions for correcting perspective distortions. Fifteen elevator button images are captured from different angles of view as the dataset. The experimental results demonstrate that our proposed approach is capable of estimating camera motions and removing perspective distortions of elevator button images with high accuracy.
Speculative Interference Attacks: Breaking Invisible Speculation Schemes
Venue: arXiv cs.AR 2021
Year: 2021

Recent security vulnerabilities that target speculative execution (e.g., Spectre) present a significant challenge for processor design. The highly publicized vulnerability uses speculative execution to learn victim secrets by changing cache state. As a result, recent computer architecture research has focused on invisible speculation mechanisms that attempt to block changes in cache state due to speculative execution. Prior work has shown significant success in preventing Spectre and other vulnerabilities at modest performance costs. In this paper, we introduce speculative interference attacks, which show that prior invisible speculation mechanisms do not fully block these speculation-based attacks. We make two key observations. First, misspeculated younger instructions can change the timing of older, bound-to-retire instructions, including memory operations. Second, changing the timing of a memory operation can change the order of that memory operation relative to other memory operations, resulting in persistent changes to the cache state. Using these observations, we demonstrate (among other attack variants) that secret information accessed by mis-speculated instructions can change the order of bound-to-retire loads. Load timing changes can therefore leave secret-dependent changes in the cache, even in the presence of invisible speculation mechanisms. We show that this problem is not easy to fix: Speculative interference converts timing changes to persistent cache-state changes, and timing is typically ignored by many cache-based defenses. We develop a framework to understand the attack and demonstrate concrete proof-of-concept attacks against invisible speculation mechanisms. We provide security definitions sufficient to block speculative interference attacks; describe a simple defense mechanism with a high performance cost; and discuss how future research can improve its performance.