OpenRaiser/Intern-Atlas huggingface dataset:
Novel Human-Object Interaction Detection via Adversarial Domain Generalization
Venue: arXiv cs.CV 2020
Year: 2020

We study in this paper the problem of novel human-object interaction (HOI) detection, aiming at improving the generalization ability of the model to unseen scenarios. The challenge mainly stems from the large compositional space of objects and predicates, which leads to the lack of sufficient training data for all the object-predicate combinations. As a result, most existing HOI methods heavily rely on object priors and can hardly generalize to unseen combinations. To tackle this problem, we propose a unified framework of adversarial domain generalization to learn object-invariant features for predicate prediction. To measure the performance improvement, we create a new split of the HICO-DET dataset, where the HOIs in the test set are all unseen triplet categories in the training set. Our experiments show that the proposed framework significantly increases the performance by up to 50% on the new split of HICO-DET dataset and up to 125% on the UnRel dataset for auxiliary evaluation in detecting novel HOIs.
COVID-19 Contact Tracing: Eight Privacy Questions Explored
Venue: arXiv cs.CY 2020
Year: 2020

We respond to a recent short paper by de Motjoye et el. on privacy issues with Covid-19 tracking. Their paper, which we discuss here, is structured around three "toy protocols" for the design of an app which can maximise the utility of contact tracing information while minimising the more general risk to privacy. On this basis, the paper proceeds to introduce eight questions against which they should be assessed. The questions raised and the protocols proposed effectively amount to the creation of a game with different categories of players able to make different moves. It is therefore possible to analyse the model in terms of optimal game design.
Transformer-based Context-aware Sarcasm Detection in Conversation Threads from Social Media
Venue: arXiv cs.CL 2020
Year: 2020

We present a transformer-based sarcasm detection model that accounts for the context from the entire conversation thread for more robust predictions. Our model uses deep transformer layers to perform multi-head attentions among the target utterance and the relevant context in the thread. The context-aware models are evaluated on two datasets from social media, Twitter and Reddit, and show 3.1% and 7.0% improvements over their baselines. Our best models give the F1-scores of 79.0% and 75.0% for the Twitter and Reddit datasets respectively, becoming one of the highest performing systems among 36 participants in this shared task.
Mechanisms for Outsourcing Computation via a Decentralized Market
Venue: arXiv cs.DC 2020
Year: 2020

As the number of personal computing and IoT devices grows rapidly, so does the amount of computational power that is available at the edge. Since many of these devices are often idle, there is a vast amount of computational power that is currently untapped, and which could be used for outsourcing computation. Existing solutions for harnessing this power, such as volunteer computing (e.g., BOINC), are centralized platforms in which a single organization or company can control participation and pricing. By contrast, an open market of computational resources, where resource owners and resource users trade directly with each other, could lead to greater participation and more competitive pricing. To provide an open market, we introduce MODiCuM, a decentralized system for outsourcing computation. MODiCuM deters participants from misbehaving-which is a key problem in decentralized systems-by resolving disputes via dedicated mediators and by imposing enforceable fines. However, unlike other decentralized outsourcing solutions, MODiCuM minimizes computational overhead since it does not require global trust in mediation results. We provide analytical results proving that MODiCuM can deter misbehavior, and we evaluate the overhead of MODiCuM using experimental results based on an implementation of our platform.
LQR-Assisted Whole-Body Control of a Wheeled Bipedal Robot with Kinematic Loops
Venue: arXiv cs.RO 2020
Year: 2020

We present a hierarchical whole-body controller leveraging the full rigid body dynamics of the wheeled bipedal robot Ascento. We derive closed-form expressions for the dynamics of its kinematic loops in a way that readily generalizes to more complex systems. The rolling constraint is incorporated using a compact analytic solution based on rotation matrices. The non-minimum phase balancing dynamics are accounted for by including a linear-quadratic regulator as a motion task. Robustness when driving curves is increased by regulating the lean angle as a function of the zero-moment point. The proposed controller is computationally lightweight and significantly extends the rough-terrain capabilities and robustness of the system, as we demonstrate in several experiments.
Fine-Grain Few-Shot Vision via Domain Knowledge as Hyperspherical Priors
Venue: arXiv cs.CV 2020
Year: 2020

Prototypical networks have been shown to perform well at few-shot learning tasks in computer vision. Yet these networks struggle when classes are very similar to each other (fine-grain classification) and currently have no way of taking into account prior knowledge (through the use of tabular data). Using a spherical latent space to encode prototypes, we can achieve few-shot fine-grain classification by maximally separating the classes while incorporating domain knowledge as informative priors. We describe how to construct a hypersphere of prototypes that embed a-priori domain information, and demonstrate the effectiveness of the approach on challenging benchmark datasets for fine-grain classification, with top results for one-shot classification and 5x speedups in training time.
Attention-guided Context Feature Pyramid Network for Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

For object detection, how to address the contradictory requirement between feature map resolution and receptive field on high-resolution inputs still remains an open question. In this paper, to tackle this issue, we build a novel architecture, called Attention-guided Context Feature Pyramid Network (AC-FPN), that exploits discriminative information from various large receptive fields via integrating attention-guided multi-path features. The model contains two modules. The first one is Context Extraction Module (CEM) that explores large contextual information from multiple receptive fields. As redundant contextual relations may mislead localization and recognition, we also design the second module named Attention-guided Module (AM), which can adaptively capture the salient dependencies over objects by using the attention mechanism. AM consists of two sub-modules, i.e., Context Attention Module (CxAM) and Content Attention Module (CnAM), which focus on capturing discriminative semantics and locating precise positions, respectively. Most importantly, our AC-FPN can be readily plugged into existing FPN-based models. Extensive experiments on object detection and instance segmentation show that existing models with our proposed CEM and AM significantly surpass their counterparts without them, and our model successfully obtains state-of-the-art results. We have released the source code at https://github.com/Caojunxu/AC-FPN.
From Witch's Shot to Music Making Bones -- Resources for Medical Laymen to Technical Language and Vice Versa
Venue: arXiv cs.CL 2020
Year: 2020

Many people share information in social media or forums, like food they eat, sports activities they do or events which have been visited. This also applies to information about a person's health status. Information we share online unveils directly or indirectly information about our lifestyle and health situation and thus provides a valuable data resource. If we can make advantage of that data, applications can be created that enable e.g. the detection of possible risk factors of diseases or adverse drug reactions of medications. However, as most people are not medical experts, language used might be more descriptive rather than the precise medical expression as medics do. To detect and use those relevant information, laymen language has to be translated and/or linked to the corresponding medical concept. This work presents baseline data sources in order to address this challenge for German. We introduce a new data set which annotates medical laymen and technical expressions in a patient forum, along with a set of medical synonyms and definitions, and present first baseline results on the data.
When Program Analysis Meets Bytecode Search: Targeted and Efficient Inter-procedural Analysis of Modern Android Apps in BackDroid
Venue: arXiv cs.CR 2020
Year: 2020

Widely-used Android static program analysis tools, e.g., Amandroid and FlowDroid, perform the whole-app inter-procedural analysis that is comprehensive but fundamentally difficult to handle modern (large) apps. The average app size has increased three to four times over five years. In this paper, we explore a new paradigm of targeted inter-procedural analysis that can skip irrelevant code and focus only on the flows of security-sensitive sink APIs. To this end, we propose a technique called on-the-fly bytecode search, which searches the disassembled app bytecode text just in time when a caller needs to be located. In this way, it guides targeted (and backward) inter-procedural analysis step by step until reaching entry points, without relying on a whole-app graph. Such search-based inter-procedural analysis, however, is challenging due to Java polymorphism, callbacks, asynchronous flows, static initializers, and inter-component communication in Android apps. We overcome these unique obstacles in our context by proposing a set of bytecode search mechanisms that utilize flexible searches and forward object taint analysis. Atop of this new inter-procedural analysis, we further adjust the traditional backward slicing and forward constant propagation to provide the complete dataflow tracking of sink API calls. We have implemented a prototype called BackDroid and compared it with Amandroid in analyzing 3,178 modern popular apps for crypto and SSL misconfigurations. The evaluation shows that for such sink-based problems, BackDroid is 37 times faster (2.13 v.s. 78.15 minutes) and has no timed-out failure (v.s. 35% in Amandroid), while maintaining close or even better detection effectiveness.
Routing Oblivious Measurement Analytics
Venue: arXiv cs.NI 2020
Year: 2020

Network-wide traffic analytics are often needed for various network monitoring tasks. These measurements are often performed by collecting samples at network switches, which are then sent to the controller for aggregation. However, performing such analytics without ``overcounting'' flows or packets that traverse multiple measurement switches is challenging. Therefore, existing solutions often simplify the problem by making assumptions on the routing or measurement switch placement. We introduce AROMA, a measurement infrastructure that generates a uniform sample of packets and flows regardless of the topology, workload and routing. Therefore, AROMA can be deployed in many settings, and can also work in the data plane using programmable PISA switches. The AROMA infrastructure includes controller algorithms that approximate a variety of essential measurement tasks while providing formal accuracy guarantees. Using extensive simulations on real-world network traces, we show that our algorithms are competitively accurate compared to the best existing solutions despite the fact that they make no assumptions on the underlying network or the placement of measurement switches.
Underwater object detection using Invert Multi-Class Adaboost with deep learning
Venue: arXiv cs.CV 2020
Year: 2020

In recent years, deep learning based methods have achieved promising performance in standard object detection. However, these methods lack sufficient capabilities to handle underwater object detection due to these challenges: (1) Objects in real applications are usually small and their images are blurry, and (2) images in the underwater datasets and real applications accompany heterogeneous noise. To address these two problems, we first propose a novel neural network architecture, namely Sample-WeIghted hyPEr Network (SWIPENet), for small object detection. SWIPENet consists of high resolution and semantic rich Hyper Feature Maps which can significantly improve small object detection accuracy. In addition, we propose a novel sample-weighted loss function which can model sample weights for SWIPENet, which uses a novel sample re-weighting algorithm, namely Invert Multi-Class Adaboost (IMA), to reduce the influence of noise on the proposed SWIPENet. Experiments on two underwater robot picking contest datasets URPC2017 and URPC2018 show that the proposed SWIPENet+IMA framework achieves better performance in detection accuracy against several state-of-the-art object detection approaches.
PhyAAt: Physiology of Auditory Attention to Speech Dataset
Venue: arXiv cs.HC 2020
Year: 2020

Auditory attention to natural speech is a complex brain process. Its quantification from physiological signals can be valuable to improving and widening the range of applications of current brain-computer-interface systems, however it remains a challenging task. In this article, we present a dataset of physiological signals collected from an experiment on auditory attention to natural speech. In this experiment, auditory stimuli consisting of reproductions of English sentences in different auditory conditions were presented to 25 non-native participants, who were asked to transcribe the sentences. During the experiment, 14 channel electroencephalogram, galvanic skin response, and photoplethysmogram signals were collected from each participant. Based on the number of correctly transcribed words, an attention score was obtained for each auditory stimulus presented to subjects. A strong correlation ($p<<0.0001$) between the attention score and the auditory conditions was found. We also formulate four different predictive tasks involving the collected dataset and develop a feature extraction framework. The results for each predictive task are obtained using a Support Vector Machine with spectral features, and are better than chance level. The dataset has been made publicly available for further research, along with a python library - phyaat to facilitate the preprocessing, modeling, and reproduction of the results presented in this paper. The dataset and other resources are shared on webpage - https://phyaat.github.io.
ManifoldPlus: A Robust and Scalable Watertight Manifold Surface Generation Method for Triangle Soups
Venue: arXiv cs.GR 2020
Year: 2020

We present ManifoldPlus, a method for robust and scalable conversion of triangle soups to watertight manifolds. While many algorithms in computer graphics require the input mesh to be a watertight manifold, in practice many meshes designed by artists are often for visualization purposes, and thus have non-manifold structures such as incorrect connectivity, ambiguous face orientation, double surfaces, open boundaries, self-intersections, etc. Existing methods suffer from problems in the inputs with face orientation and zero-volume structures. Additionally most methods do not scale to meshes of high complexity. In this paper, we propose a method that extracts exterior faces between occupied voxels and empty voxels, and uses a projection-based optimization method to accurately recover a watertight manifold that resembles the reference mesh. Compared to previous methods, our methodology is simpler. It does not rely on face normals of the input triangle soups and can accurately recover zero-volume structures. Our algorithm is scalable, because it employs an adaptive Gauss-Seidel method for shape optimization, in which each step is an easy-to-solve convex problem. We test ManifoldPlus on ModelNet10 and AccuCity datasets to verify that our methods can generate watertight meshes ranging from object-level shapes to city-level models. Furthermore, through our experimental evaluations, we show that our method is more robust, efficient and accurate than the state-of-the-art. Our implementation is publicly available.
ShapeAdv: Generating Shape-Aware Adversarial 3D Point Clouds
Venue: arXiv cs.CV 2020
Year: 2020

We introduce ShapeAdv, a novel framework to study shape-aware adversarial perturbations that reflect the underlying shape variations (e.g., geometric deformations and structural differences) in the 3D point cloud space. We develop shape-aware adversarial 3D point cloud attacks by leveraging the learned latent space of a point cloud auto-encoder where the adversarial noise is applied in the latent space. Specifically, we propose three different variants including an exemplar-based one by guiding the shape deformation with auxiliary data, such that the generated point cloud resembles the shape morphing between objects in the same category. Different from prior works, the resulting adversarial 3D point clouds reflect the shape variations in the 3D point cloud space while still being close to the original one. In addition, experimental evaluations on the ModelNet40 benchmark demonstrate that our adversaries are more difficult to defend with existing point cloud defense methods and exhibit a higher attack transferability across classifiers. Our shape-aware adversarial attacks are orthogonal to existing point cloud based attacks and shed light on the vulnerability of 3D deep neural networks.
miniKanren as a Tool for Symbolic Computation in Python
Venue: arXiv cs.PL 2020
Year: 2020

In this article, we give a brief overview of the current state and future potential of symbolic computation within the Python statistical modeling and machine learning community. We detail the use of miniKanren as an underlying framework for term rewriting and symbolic mathematics, as well as its ability to orchestrate the use of existing Python libraries. We also discuss the relevance and potential of relational programming for implementing more robust, portable, domain-specific "math-level" optimizations--with a slight focus on Bayesian modeling. Finally, we describe the work going forward and raise some questions regarding potential cross-overs between statistical modeling and programming language theory.
Projection to Fairness in Statistical Learning
Venue: arXiv cs.LG 2020
Year: 2020

In the context of regression, we consider the fundamental question of making an estimator fair while preserving its prediction accuracy as much as possible. To that end, we define its projection to fairness as its closest fair estimator in a sense that reflects prediction accuracy. Our methodology leverages tools from optimal transport to construct efficiently the projection to fairness of any given estimator as a simple post-processing step. Moreover, our approach precisely quantifies the cost of fairness, measured in terms of prediction accuracy.
Efficient Offline Monitoring of Linear Temporal Logic with Bit Vectors
Venue: arXiv cs.LO 2020
Year: 2020

A bitmap is a data structure designed to compactly represent sets of integers; it provides very fast operations for querying and manipulating such sets, exploiting bit-level parallelism. In this paper, we describe a technique for the offline verification of arbitrary expressions of Linear Temporal Logic using bitmap manipulation. An event trace is first preprocessed and transformed into a set of bitmaps. The LTL expression is then evaluated through a recursive procedure manipulating these bitmaps. Experimental results show that, for complex LTL formulas containing almost 20 operators, event traces can be evaluated at a throughput of millions of events per second.
Reliability and Performance Assessment of Federated Learning on Clinical Benchmark Data
Venue: arXiv cs.LG 2020
Year: 2020

As deep learning have been applied in a clinical context, privacy concerns have increased because of the collection and processing of a large amount of personal data. Recently, federated learning (FL) has been suggested to protect personal privacy because it does not centralize data during the training phase. In this study, we assessed the reliability and performance of FL on benchmark datasets including MNIST and MIMIC-III. In addition, we attempted to verify FL on datasets that simulated a realistic clinical data distribution. We implemented FL that uses a client and server architecture and tested client and server FL on modified MNIST and MIMIC-III datasets. FL delivered reliable performance on both imbalanced and extremely skewed distributions (i.e., the difference of the number of patients and the characteristics of patients in each hospital). Therefore, FL can be suitable to protect privacy when applied to medical data.
Rethinking Blockchains in the Internet of Things Era from a Wireless Communication Perspective
Venue: arXiv cs.CR 2020
Year: 2020

Due to the rapid development of Internet of Things (IoT), a massive number of devices are connected to the Internet. For these distributed devices in IoT networks, how to ensure their security and privacy becomes a significant challenge. The blockchain technology provides a promising solution to protect the data integrity, provenance, privacy, and consistency for IoT networks. In blockchains, communication is a prerequisite for participants, which are distributed in the system, to reach consensus. However, in IoT networks, most of the devices communicate through wireless links, which are not always reliable. Hence, the communication reliability of IoT devices influences the system security. In this article, we rethink the roles of communication and computing in blockchains by accounting for communication reliability. We analyze the tradeoff between communication reliability and computing power in blockchain security, and present a lower bound to the computing power that is needed to conduct an attack with a given communication reliability. Simulation results show that adversarial nodes can succeed in tampering a block with less computing power by hindering the propagation of blocks from other nodes.
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers
Venue: arXiv cs.CL 2020
Year: 2020

Following the major success of neural language models (LMs) such as BERT or GPT-2 on a variety of language understanding tasks, recent work focused on injecting (structured) knowledge from external resources into these models. While on the one hand, joint pretraining (i.e., training from scratch, adding objectives based on external knowledge to the primary LM objective) may be prohibitively computationally expensive, post-hoc fine-tuning on external knowledge, on the other hand, may lead to the catastrophic forgetting of distributional knowledge. In this work, we investigate models for complementing the distributional knowledge of BERT with conceptual knowledge from ConceptNet and its corresponding Open Mind Common Sense (OMCS) corpus, respectively, using adapter training. While overall results on the GLUE benchmark paint an inconclusive picture, a deeper analysis reveals that our adapter-based models substantially outperform BERT (up to 15-20 performance points) on inference tasks that require the type of conceptual knowledge explicitly present in ConceptNet and OMCS. All code and experiments are open sourced under: https://github.com/wluper/retrograph .