OpenRaiser/Intern-Atlas huggingface dataset:
Touchtone leakage attacks via smartphone sensors: mitigation without hardware modification
Venue: arXiv cs.CR 2021
Year: 2021

Smartphone motion sensors provide a concealed mechanism for eavesdropping on acoustic information, like touchtones, emitted by a device. Eavesdropping on touchtones exposes credit card information, banking pins, and social security card numbers to malicious 3rd party apps requiring only motion sensor data. This paper's primary contribution is an analysis rooted in physics and signal processing theory of several eavesdropping mitigations, which could be implemented in a smartphone update. We verify our analysis imperially to show how previously suggested mitigations, i.e. a low-pass filter, can undesirably reduce the motion sensor data to all applications by 83% but only reduce an advanced adversary's accuracy by less than one percent. Other designs, i.e. anti-aliasing filters, can fully preserve the motion sensor data to support benign application functionality while reducing attack accuracy by 50.1%. We intend for this analysis to motivate the need for deployable mitigations against acoustic leakage on smartphone motion sensors, including but not limited to touchtones, while also providing a basis for future mitigations to improve upon.
Temporal Information and Event Markup Language: TIE-ML Markup Process and Schema Version 1.0
Venue: arXiv cs.CL 2021
Year: 2021

Temporal Information and Event Markup Language (TIE-ML) is a markup strategy and annotation schema to improve the productivity and accuracy of temporal and event related annotation of corpora to facilitate machine learning based model training. For the annotation of events, temporal sequencing, and durations, it is significantly simpler by providing an extremely reduced tag set for just temporal relations and event enumeration. In comparison to other standards, as for example the Time Markup Language (TimeML), it is much easier to use by dropping sophisticated formalisms, theoretical concepts, and annotation approaches. Annotations of corpora using TimeML can be mapped to TIE-ML with a loss, and TIE-ML annotations can be fully mapped to TimeML with certain under-specification.
Smart at what cost? Characterising Mobile Deep Neural Networks in the wild
Venue: arXiv cs.LG 2021
Year: 2021

With smartphones' omnipresence in people's pockets, Machine Learning (ML) on mobile is gaining traction as devices become more powerful. With applications ranging from visual filters to voice assistants, intelligence on mobile comes in many forms and facets. However, Deep Neural Network (DNN) inference remains a compute intensive workload, with devices struggling to support intelligence at the cost of responsiveness.On the one hand, there is significant research on reducing model runtime requirements and supporting deployment on embedded devices. On the other hand, the strive to maximise the accuracy of a task is supported by deeper and wider neural networks, making mobile deployment of state-of-the-art DNNs a moving target. In this paper, we perform the first holistic study of DNN usage in the wild in an attempt to track deployed models and match how these run on widely deployed devices. To this end, we analyse over 16k of the most popular apps in the Google Play Store to characterise their DNN usage and performance across devices of different capabilities, both across tiers and generations. Simultaneously, we measure the models' energy footprint, as a core cost dimension of any mobile deployment. To streamline the process, we have developed gaugeNN, a tool that automates the deployment, measurement and analysis of DNNs on devices, with support for different frameworks and platforms. Results from our experience study paint the landscape of deep learning deployments on smartphones and indicate their popularity across app developers. Furthermore, our study shows the gap between bespoke techniques and real-world deployments and the need for optimised deployment of deep learning models in a highly dynamic and heterogeneous ecosystem.
Identifying Reasoning Flaws in Planning-Based RL Using Tree Explanations
Venue: arXiv cs.AI 2021
Year: 2021

Enabling humans to identify potential flaws in an agent's decision making is an important Explainable AI application. We consider identifying such flaws in a planning-based deep reinforcement learning (RL) agent for a complex real-time strategy game. In particular, the agent makes decisions via tree search using a learned model and evaluation function over interpretable states and actions. This gives the potential for humans to identify flaws at the level of reasoning steps in the tree, even if the entire reasoning process is too complex to understand. However, it is unclear whether humans will be able to identify such flaws due to the size and complexity of trees. We describe a user interface and case study, where a small group of AI experts and developers attempt to identify reasoning flaws due to inaccurate agent learning. Overall, the interface allowed the group to identify a number of significant flaws of varying types, demonstrating the promise of this approach.
Guidelines for the Computational Testing of Machine Learning approaches to Vehicle Routing Problems
Venue: arXiv cs.LG 2021
Year: 2021

Despite the extensive research efforts and the remarkable results obtained on Vehicle Routing Problems (VRP) by using algorithms proposed by the Machine Learning community that are partially or entirely based on data-driven analysis, most of these approaches are still seldom employed by the Operations Research (OR) community. Among the possible causes, we believe, the different approach to the computational evaluation of the proposed methods may play a major role. With the current work, we want to highlight a number of challenges (and possible ways to handle them) arising during the computational studies of heuristic approaches to VRPs that, if appropriately addressed, may produce a computational study having the characteristics of those presented in OR papers, thus hopefully promoting the collaboration between the two communities.
Stochastic geometric modelling and simulation of cellular systems for coverage probability characterization
Venue: arXiv cs.NI 2021
Year: 2021

Stochastic geometry (SG) has been successfully used as a modelling tool for cellular networks to characterize the coverage probability in both the downlink (DL) and uplink (UL) systems, under the assumption that the base stations (BS) are deployed as a Poisson point process. In the present article, we extend this use and provide further results for interference limited and Rayleigh fading networks, culminating in a multifaceted contribution. First, we compactly model the two systems at once, allowing parallels to be drawn and contrast to be created. Also, for DL we manage to obtain two closed form expressions for two special cases. Moreover, for UL, notorious for being difficult, we develop a clever approximation that overcomes the difficulty, yielding excellent results. Additionally, we present two efficient Monte Carlo simulation algorithms, designed primarily to validate the models, but can be of great use for SG modelling of communications systems in general. Finally, we prove two theorems at odds with popular belief in cellular communications research. Specifically, we prove that under the SG model, the coverage probability in both DL and UL is independent of BS density. Based on this revelation, a plethora of results in the literature have to be re-examined to rid them of a parameter that has been proven superfluous.
Visually Grounded Concept Composition
Venue: arXiv cs.CV 2021
Year: 2021

We investigate ways to compose complex concepts in texts from primitive ones while grounding them in images. We propose Concept and Relation Graph (CRG), which builds on top of constituency analysis and consists of recursively combined concepts with predicate functions. Meanwhile, we propose a concept composition neural network called Composer to leverage the CRG for visually grounded concept learning. Specifically, we learn the grounding of both primitive and all composed concepts by aligning them to images and show that learning to compose leads to more robust grounding results, measured in text-to-image matching accuracy. Notably, our model can model grounded concepts forming at both the finer-grained sentence level and the coarser-grained intermediate level (or word-level). Composer leads to pronounced improvement in matching accuracy when the evaluation data has significant compound divergence from the training data.
Adaptive Multi-layer Contrastive Graph Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

We present Adaptive Multi-layer Contrastive Graph Neural Networks (AMC-GNN), a self-supervised learning framework for Graph Neural Network, which learns feature representations of sample data without data labels. AMC-GNN generates two graph views by data augmentation and compares different layers' output embeddings of Graph Neural Network encoders to obtain feature representations, which could be used for downstream tasks. AMC-GNN could learn the importance weights of embeddings in different layers adaptively through the attention mechanism, and an auxiliary encoder is introduced to train graph contrastive encoders better. The accuracy is improved by maximizing the representation's consistency of positive pairs in the early layers and the final embedding space. Our experiments show that the results can be consistently improved by using the AMC-GNN framework, across four established graph benchmarks: Cora, Citeseer, Pubmed, DBLP citation network datasets, as well as four newly proposed datasets: Co-author-CS, Co-author-Physics, Amazon-Computers, Amazon-Photo.
Efficient Reinforced Feature Selection via Early Stopping Traverse Strategy
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we propose a single-agent Monte Carlo based reinforced feature selection (MCRFS) method, as well as two efficiency improvement strategies, i.e., early stopping (ES) strategy and reward-level interactive (RI) strategy. Feature selection is one of the most important technologies in data prepossessing, aiming to find the optimal feature subset for a given downstream machine learning task. Enormous research has been done to improve its effectiveness and efficiency. Recently, the multi-agent reinforced feature selection (MARFS) has achieved great success in improving the performance of feature selection. However, MARFS suffers from the heavy burden of computational cost, which greatly limits its application in real-world scenarios. In this paper, we propose an efficient reinforcement feature selection method, which uses one agent to traverse the whole feature set, and decides to select or not select each feature one by one. Specifically, we first develop one behavior policy and use it to traverse the feature set and generate training data. And then, we evaluate the target policy based on the training data and improve the target policy by Bellman equation. Besides, we conduct the importance sampling in an incremental way, and propose an early stopping strategy to improve the training efficiency by the removal of skew data. In the early stopping strategy, the behavior policy stops traversing with a probability inversely proportional to the importance sampling weight. In addition, we propose a reward-level interactive strategy to improve the training efficiency via reward-level external advice. Finally, we design extensive experiments on real-world data to demonstrate the superiority of the proposed method.
A Communication Security Game on Switched Systems for Autonomous Vehicle Platoons
Venue: arXiv cs.GT 2021
Year: 2021

Vehicle-to-vehicle communication enables autonomous platoons to boost traffic efficiency and safety, while ensuring string stability with a constant spacing policy. However, communication-based controllers are susceptible to a range of cyber-attacks. In this paper, we propose a distributed attack mitigation defense framework with a dual-mode control system reconfiguration scheme to prevent a compromised platoon member from causing collisions via message falsification attacks. In particular, we model it as a switched system consisting of a communication-based cooperative controller and a sensor-based local controller and derive conditions to achieve global uniform exponential stability (GUES) as well as string stability in the sense of platoon operation. The switching decision comes from game-theoretic analysis of the attacker and the defender's interactions. In this framework, the attacker acts as a leader that chooses whether to engage in malicious activities and the defender decides which control system to deploy with the help of an anomaly detector. Imperfect detection reports associate the game with imperfect information. A dedicated state constraint further enhances safety against bounded but aggressive message modifications in which a bounded solution may still violate practical constraint e.g. vehicles nearly crashing. Our formulation uniquely combines switched systems with security games to strategically improve the safety of such autonomous vehicle systems.
Unsupervised Domain Adaptation in Semantic Segmentation Based on Pixel Alignment and Self-Training
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes an unsupervised cross-modality domain adaptation approach based on pixel alignment and self-training. Pixel alignment transfers ceT1 scans to hrT2 modality, helping to reduce domain shift in the training segmentation model. Self-training adapts the decision boundary of the segmentation network to fit the distribution of hrT2 scans. Experiment results show that PAST has outperformed the non-UDA baseline significantly, and it received rank-2 on CrossMoDA validation phase Leaderboard with a mean Dice score of 0.8395.
Hierarchical Character Tagger for Short Text Spelling Error Correction
Venue: arXiv cs.CL 2021
Year: 2021

State-of-the-art approaches to spelling error correction problem include Transformer-based Seq2Seq models, which require large training sets and suffer from slow inference time; and sequence labeling models based on Transformer encoders like BERT, which involve token-level label space and therefore a large pre-defined vocabulary dictionary. In this paper we present a Hierarchical Character Tagger model, or HCTagger, for short text spelling error correction. We use a pre-trained language model at the character level as a text encoder, and then predict character-level edits to transform the original text into its error-free form with a much smaller label space. For decoding, we propose a hierarchical multi-task approach to alleviate the issue of long-tail label distribution without introducing extra model parameters. Experiments on two public misspelling correction datasets demonstrate that HCTagger is an accurate and much faster approach than many existing models.
Conflicting Privacy Preference Signals in the Wild
Venue: arXiv cs.HC 2021
Year: 2021

Privacy preference signals allow users to express preferences over how their personal data is processed. These signals become important in determining privacy outcomes when they reference an enforceable legal basis, as is the case with recent signals such as the Global Privacy Control and the Transparency & Consent Framework. However, the coexistence of multiple privacy preference signals creates ambiguity as users may transmit more than one signal. This paper collects evidence about ambiguity flowing from the aforementioned two signals and the historic Do Not Track signal. We provide the first empirical evidence that ambiguous signals are sent by web users in the wild. We also show that preferences stored in the browser are reliable predictors of privacy preferences expressed in web dialogs. Finally, we provide the first evidence that popular cookie dialogs are blocked by the majority of users who adopted the Do Not Track and Global Privacy Control standards. These empirical results inform forthcoming legal debates about how to interpret privacy preference signals.
Persistent homology and the shape of evolutionary games
Venue: arXiv cs.GT 2021
Year: 2021

For nearly three decades, spatial games have produced a wealth of insights to the study of behavior and its relation to population structure. However, as different rules and factors are added or altered, the dynamics of spatial models often become increasingly complicated to interpret. To tackle this problem, we introduce persistent homology as a rigorous framework that can be used to both define and compute higher-order features of data in a manner which is invariant to parameter choices, robust to noise, and independent of human observation. Our work demonstrates its relevance for spatial games by showing how topological features of simulation data that persist over different spatial scales reflect the stability of strategies in 2D lattice games. To do so, we analyze the persistent homology of scenarios from two games: a Prisoner's Dilemma and a SIRS epidemic model. The experimental results show how the method accurately detects features that correspond to real aspects of the game dynamics. Unlike other tools that study dynamics of spatial systems, persistent homology can tell us something meaningful about population structure while remaining neutral about the underlying structure itself. Regardless of game complexity, since strategies either succeed or fail to conform to shapes of a certain topology there is much potential for the method to provide novel insights for a wide variety of spatially extended systems in biology, social science, and physics.
Towards event aggregation for reducing the volume of logged events during IKC stages of APT attacks
Venue: arXiv cs.CR 2021
Year: 2021

Nowadays, targeted attacks like Advanced Persistent Threats (APTs) has become one of the major concern of many enterprise networks. As a common approach to counter these attacks, security staff deploy a variety of security and non-security sensors at different lines of defense (Network, Host, and Application) to track the attacker's behaviors during their kill chain. However, one of the drawbacks of this approach is the huge amount of events raised by heterogeneous security and non-security sensors which makes it difficult to analyze logged events for later processing i.e. event correlation for timely detection of APT attacks. Till now, some research papers have been published on event aggregation for reducing the volume of logged low-level events. However, most research works have been provided a method to aggregate the events of a single-type and homogeneous event source i.e. NIDS. In addition, their main focus is only on the degree to which the event volume is reduced, while the amount of security information lost during the event aggregation process is also very important. In this paper, we propose a three-phase event aggregation method to reduce the volume of logged heterogeneous events during APT attacks considering the lowest rate of loss of security information. To this aim, at first, low-level events of the sensors are clustered into some similar event groups and then, after filtering noisy event clusters, the remained clusters are summarized based on an Attribute-Oriented Induction (AOI) method in a controllable manner to reduce the unimportant or duplicated events. The method has been evaluated on the three publicly available datasets: SotM34, Bryant, and LANL. The experimental results show that the method is efficient enough in event aggregation and can reduce events volume up to 99.7\% with an acceptable level of information loss ratio (ILR).
Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks
Venue: arXiv cs.AR 2021
Year: 2021

Emerging edge computing platforms often contain machine learning (ML) accelerators that can accelerate inference for a wide range of neural network (NN) models. These models are designed to fit within the limited area and energy constraints of the edge computing platforms, each targeting various applications (e.g., face detection, speech recognition, translation, image captioning, video analytics). To understand how edge ML accelerators perform, we characterize the performance of a commercial Google Edge TPU, using 24 Google edge NN models (which span a wide range of NN model types) and analyzing each NN layer within each model. We find that the Edge TPU suffers from three major shortcomings: (1) it operates significantly below peak computational throughput, (2) it operates significantly below its theoretical energy efficiency, and (3) its memory system is a large energy and performance bottleneck. Our characterization reveals that the one-size-fits-all, monolithic design of the Edge TPU ignores the high degree of heterogeneity both across different NN models and across different NN layers within the same NN model, leading to the shortcomings we observe. We propose a new acceleration framework called Mensa. Mensa incorporates multiple heterogeneous edge ML accelerators (including both on-chip and near-data accelerators), each of which caters to the characteristics of a particular subset of NN models and layers. During NN inference, for each NN layer, Mensa decides which accelerator to schedule the layer on, taking into account both the optimality of each accelerator for the layer and layer-to-layer communication costs. Averaged across all 24 Google edge NN models, Mensa improves energy efficiency and throughput by 3.0x and 3.1x over the Edge TPU, and by 2.4x and 4.3x over Eyeriss~v2, a state-of-the-art accelerator.
Worrisome Patterns in Developers: A Survey in Cryptography
Venue: arXiv cs.CR 2021
Year: 2021

We surveyed 97 developers who had used cryptography in open-source projects, in the hope of identifying developer security and cryptography practices. We asked them about individual and company-level practices, and divided respondents into three groups (i.e., high, medium, and low) based on their level of knowledge. We found differences between the high-profile developers and the other two groups. For instance, high-profile developers have more years of experience in programming, have attended more security and cryptography courses, have more background in security, are highly concerned about security, and tend to use security tools more than the other two groups. Nevertheless, we observed worrisome patterns among all participants such as the high usage of unreliable sources like Stack Overflow, and the low rate of security tool usage.
UFO-ViT: High Performance Linear Vision Transformer without Softmax
Venue: arXiv cs.CV 2021
Year: 2021

Vision transformers have become one of the most important models for computer vision tasks. Although they outperform prior works, they require heavy computational resources on a scale that is quadratic to $N$. This is a major drawback of the traditional self-attention (SA) algorithm. Here, we propose the Unit Force Operated Vision Transformer (UFO-ViT), a novel SA mechanism that has linear complexity. The main approach of this work is to eliminate nonlinearity from the original SA. We factorize the matrix multiplication of the SA mechanism without complicated linear approximation. By modifying only a few lines of code from the original SA, the proposed models outperform most transformer-based models on image classification and dense prediction tasks on most capacity regimes.
StoryDB: Broad Multi-language Narrative Dataset
Venue: arXiv cs.CL 2021
Year: 2021

This paper presents StoryDB - a broad multi-language dataset of narratives. StoryDB is a corpus of texts that includes stories in 42 different languages. Every language includes 500+ stories. Some of the languages include more than 20 000 stories. Every story is indexed across languages and labeled with tags such as a genre or a topic. The corpus shows rich topical and language variation and can serve as a resource for the study of the role of narrative in natural language processing across various languages including low resource ones. We also demonstrate how the dataset could be used to benchmark three modern multilanguage models, namely, mDistillBERT, mBERT, and XLM-RoBERTa.