OpenRaiser/Intern-Atlas huggingface dataset:
Querying Labelled Data with Scenario Programs for Sim-to-Real Validation
Venue: arXiv cs.CV 2021
Year: 2021

Simulation-based testing of autonomous vehicles (AVs) has become an essential complement to road testing to ensure safety. Consequently, substantial research has focused on searching for failure scenarios in simulation. However, a fundamental question remains: are AV failure scenarios identified in simulation meaningful in reality, i.e., are they reproducible on the real system? Due to the sim-to-real gap arising from discrepancies between simulated and real sensor data, a failure scenario identified in simulation can be either a spurious artifact of the synthetic sensor data or an actual failure that persists with real sensor data. An approach to validate simulated failure scenarios is to identify instances of the scenario in a corpus of real data, and check if the failure persists on the real data. To this end, we propose a formal definition of what it means for a labelled data item to match an abstract scenario, encoded as a scenario program using the SCENIC probabilistic programming language. Using this definition, we develop a querying algorithm which, given a scenario program and a labelled dataset, finds the subset of data matching the scenario. Experiments demonstrate that our algorithm is accurate and efficient on a variety of realistic traffic scenarios, and scales to a reasonable number of agents.
A Blockchain-Enabled Incentivised Framework for Cyber Threat Intelligence Sharing in ICS
Venue: arXiv cs.CR 2021
Year: 2021

In recent years Industrial Control Systems (ICS) have been targeted increasingly by sophisticated cyberattacks. Improving ICS security has drawn significant attention in the literature that emphasises the importance of Cyber Threat Intelligence (CTI) sharing in accelerating detection, mitigation, and prevention of cyberattacks. However, organisations are reluctant to exchange CTI due to fear of exposure, reputational damage, and lack of incentives. Furthermore, there has been limited discussion about the factors influencing participation in sharing CTI about ICS. The existing CTI-sharing platforms rely on centralised trusted architectures that suffer from a single point of failure and risk companies' privacy as the central node maintains CTI details. In this paper, we address the needs of organisations involved in the management and protection of ICS and present a novel framework that facilitates secure, private, and incentivised exchange of CTI related to ICS using blockchain. We propose a new blockchain-enabled framework that facilitates the secure dissemination of CTI data among multiple stakeholders in ICS. We provide the framework design, technical development and evaluate the framework's feasibility in a real-world application environment using practical use-case scenarios. Our proposed design shows a more practical and efficient framework for a CTI sharing network for ICS, including the bestowal and acknowledgment of data privacy, trust barriers, and security issues ingrained in this domain.
CAMA: Energy and Memory Efficient Automata Processing in Content-Addressable Memories
Venue: arXiv cs.AR 2021
Year: 2021

Accelerating finite automata processing is critical for advancing real-time analytic in pattern matching, data mining, bioinformatics, intrusion detection, and machine learning. Recent in-memory automata accelerators leveraging SRAMs and DRAMs have shown exciting improvements over conventional digital designs. However, the bit-vector representation of state transitions used by all SOTA designs is only optimal in processing worst-case completely random patterns, while a significant amount of memory and energy is wasted in running most real-world benchmarks. We present CAMA, a Content-Addressable Memory (CAM) enabled Automata accelerator for processing homogeneous non-deterministic finite automata (NFA). A radically different state representation scheme, along with co-designed novel circuits and data encoding schemes, greatly reduces energy, memory, and chip area for most realistic NFAs. CAMA is holistically optimized with the following major contributions: (1) a 16x256 8-transistor (8T) CAM array for state matching, replacing the 256x256 6T SRAM array or two 16x256 6T SRAM banks in SOTA designs; (2) a novel encoding scheme that enables content searching within 8T SRAMs and adapts to different applications; (3) a reconfigurable and scalable architecture that improves efficiency on all tested benchmarks, without losing support for any NFA that is compatible with SOTA designs; (4) an optimization framework that automates the choice of encoding schemes and maps a given NFA to the proposed hardware. Two versions of CAMA, one optimized for energy (CAMA-E) and the other for throughput (CAMA-T), are comprehensively evaluated in a 28nm CMOS process, and across 21 real-world and synthetic benchmarks. CAMA-E achieves 2.1x, 2.8x, and 2.04x lower energy than CA, 2-stride Impala, and eAP. CAMA-T shows 2.68x, 3.87x and 2.62x higher average compute density than 2-stride Impala, CA, and eAP.
Unequal Opportunities in Multi-hop Referral Programs
Venue: arXiv cs.SI 2021
Year: 2021

As modern social networks allow for faster and broader interactions with friends and acquaintances, online referral programs that promote sales through existing users are becoming increasingly popular. Because it is all too common that online networks reproduce historical structural bias, members of disadvantaged groups often benefit less from such referral opportunities. For instance, one-hop referral programs that distribute rewards only among pairs of friends or followers may offer less rewards and opportunities to minorities in networks where it was proved that their degrees is statistically smaller. Here, we examine the fairness of general referral programs, increasingly popular forms of marketing in which an existing referrer is encouraged to initiate the recruitment of new referred users over multiple hops. While this clearly expands opportunities for rewards, it remains unclear whether it helps addressing fairness concerns, or make them worse. We show, from studying 4 real-world networks and performing theoretical analysis on networks created with minority-majority affiliations and homophily, that the change of bias in multi-hop referral programs highly depends on the network structures and the referral strategies. Specifically, under three different constrained referral strategies which limit the number of referrals each person can share to a fixed number, we show that even with no explicit intention to discriminate and without access to sensitive attributes such as gender and race, certain referral strategies can still amplify the structural biases further when higher hops are allowed. Moreover, when there is no constraint on the number of referrals each person can distribute and when the effect of referral strategies is removed, we prove a precise condition under which the bias in 1-hop referral programs is amplified in higher-hop referral programs.
Unsupervised Statistical Learning for Die Analysis in Ancient Numismatics
Venue: arXiv cs.CV 2021
Year: 2021

Die analysis is an essential numismatic method, and an important tool of ancient economic history. Yet, manual die studies are too labor-intensive to comprehensively study large coinages such as those of the Roman Empire. We address this problem by proposing a model for unsupervised computational die analysis, which can reduce the time investment necessary for large-scale die studies by several orders of magnitude, in many cases from years to weeks. From a computer vision viewpoint, die studies present a challenging unsupervised clustering problem, because they involve an unknown and large number of highly similar semantic classes of imbalanced sizes. We address these issues through determining dissimilarities between coin faces derived from specifically devised Gaussian process-based keypoint features in a Bayesian distance clustering framework. The efficacy of our method is demonstrated through an analysis of 1135 Roman silver coins struck between 64-66 C.E..
Multistage Online Maxmin Allocation of Indivisible Entities
Venue: arXiv cs.DS 2021
Year: 2021

We consider an online allocation problem that involves a set $P$ of $n$ players and a set $E$ of $m$ indivisible entities over discrete time steps $1,2,\ldots,τ$. At each time step $t \in [1,τ]$, for every entity $e \in E$, there is a restriction list $L_t(e)$ that prescribes the subset of players to whom $e$ can be assigned and a non-negative value $v_t(e,p)$ of $e$ to every player $p \in P$. The sets $P$ and $E$ are fixed beforehand. The sets $L_t(\cdot)$ and values $v_t(\cdot,\cdot)$ are given in an online fashion. An allocation is a distribution of $E$ among $P$, and we are interested in the minimum total value of the entities received by a player according to the allocation. In the static case, it is NP-hard to find an optimal allocation the maximizes this minimum value. On the other hand, $ρ$-approximation algorithms have been developed for certain values of $ρ\in (0,1]$. We propose a $w$-lookahead algorithm for the multistage online maxmin allocation problem for any fixed $w \geq 1$ in which the restriction lists and values of entities may change between time steps, and there is a fixed stability reward for an entity to be assigned to the same player from one time step to the next. The objective is to maximize the sum of the minimum values and stability rewards over the time steps $1, 2, \ldots, τ$. Our algorithm achieves a competitive ratio of $(1-c)ρ$, where $c$ is the positive root of the equation $wc^2 = ρ(w+1)(1-c)$. When $w = 1$, it is greater than $\fracρ{4ρ+2} + \fracρ{10}$, which improves upon the previous ratio of $\fracρ{4ρ+2 - 2^{1-τ}(2ρ+1)}$ obtained for the case of 1-lookahead.
Tool as Embodiment for Recursive Manipulation
Venue: arXiv cs.RO 2021
Year: 2021

Humans and many animals exhibit a robust capability to manipulate diverse objects, often directly with their bodies and sometimes indirectly with tools. Such flexibility is likely enabled by the fundamental consistency in underlying physics of object manipulation such as contacts and force closures. Inspired by viewing tools as extensions of our bodies, we present Tool-As-Embodiment (TAE), a parameterization for tool-based manipulation policies that treat hand-object and tool-object interactions in the same representation space. The result is a single policy that can be applied recursively on robots to use end effectors to manipulate objects, and use objects as tools, i.e. new end-effectors, to manipulate other objects. By sharing experiences across different embodiments for grasping or pushing, our policy exhibits higher performance than if separate policies were trained. Our framework could utilize all experiences from different resolutions of tool-enabled embodiments to a single generic policy for each manipulation skill. Videos at https://sites.google.com/view/recursivemanipulation
Deep Measurement Updates for Bayes Filters
Venue: arXiv cs.CV 2021
Year: 2021

Measurement update rules for Bayes filters often contain hand-crafted heuristics to compute observation probabilities for high-dimensional sensor data, like images. In this work, we propose the novel approach Deep Measurement Update (DMU) as a general update rule for a wide range of systems. DMU has a conditional encoder-decoder neural network structure to process depth images as raw inputs. Even though the network is trained only on synthetic data, the model shows good performance at evaluation time on real-world data. With our proposed training scheme primed data training , we demonstrate how the DMU models can be trained efficiently to be sensitive to condition variables without having to rely on a stochastic information bottleneck. We validate the proposed methods in multiple scenarios of increasing complexity, beginning with the pose estimation of a single object to the joint estimation of the pose and the internal state of an articulated system. Moreover, we provide a benchmark against Articulated Signed Distance Functions(A-SDF) on the RBO dataset as a baseline comparison for articulation state estimation.
NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging
Venue: arXiv cs.CL 2021
Year: 2021

Named entity recognition (NER) models generally perform poorly when large training datasets are unavailable for low-resource domains. Recently, pre-training a large-scale language model has become a promising direction for coping with the data scarcity issue. However, the underlying discrepancies between the language modeling and NER task could limit the models' performance, and pre-training for the NER task has rarely been studied since the collected NER datasets are generally small or large but with low quality. In this paper, we construct a massive NER corpus with a relatively high quality, and we pre-train a NER-BERT model based on the created dataset. Experimental results show that our pre-trained model can significantly outperform BERT as well as other strong baselines in low-resource scenarios across nine diverse domains. Moreover, a visualization of entity representations further indicates the effectiveness of NER-BERT for categorizing a variety of entities.
On-Device Spatial Attention based Sequence Learning Approach for Scene Text Script Identification
Venue: arXiv cs.CV 2021
Year: 2021

Automatic identification of script is an essential component of a multilingual OCR engine. In this paper, we present an efficient, lightweight, real-time and on-device spatial attention based CNN-LSTM network for scene text script identification, feasible for deployment on resource constrained mobile devices. Our network consists of a CNN, equipped with a spatial attention module which helps reduce the spatial distortions present in natural images. This allows the feature extractor to generate rich image representations while ignoring the deformities and thereby, enhancing the performance of this fine grained classification task. The network also employs residue convolutional blocks to build a deep network to focus on the discriminative features of a script. The CNN learns the text feature representation by identifying each character as belonging to a particular script and the long term spatial dependencies within the text are captured using the sequence learning capabilities of the LSTM layers. Combining the spatial attention mechanism with the residue convolutional blocks, we are able to enhance the performance of the baseline CNN to build an end-to-end trainable network for script identification. The experimental results on several standard benchmarks demonstrate the effectiveness of our method. The network achieves competitive accuracy with state-of-the-art methods and is superior in terms of network size, with a total of just 1.1 million parameters and inference time of 2.7 milliseconds.
Weakly-Supervised Video Object Grounding via Causal Intervention
Venue: arXiv cs.CV 2021
Year: 2021

We target at the task of weakly-supervised video object grounding (WSVOG), where only video-sentence annotations are available during model learning. It aims to localize objects described in the sentence to visual regions in the video, which is a fundamental capability needed in pattern analysis and machine learning. Despite the recent progress, existing methods all suffer from the severe problem of spurious association, which will harm the grounding performance. In this paper, we start from the definition of WSVOG and pinpoint the spurious association from two aspects: (1) the association itself is not object-relevant but extremely ambiguous due to weak supervision, and (2) the association is unavoidably confounded by the observational bias when taking the statistics-based matching strategy in existing methods. With this in mind, we design a unified causal framework to learn the deconfounded object-relevant association for more accurate and robust video object grounding. Specifically, we learn the object-relevant association by causal intervention from the perspective of video data generation process. To overcome the problems of lacking fine-grained supervision in terms of intervention, we propose a novel spatial-temporal adversarial contrastive learning paradigm. To further remove the accompanying confounding effect within the object-relevant association, we pursue the true causality by conducting causal intervention via backdoor adjustment. Finally, the deconfounded object-relevant association is learned and optimized under a unified causal framework in an end-to-end manner. Extensive experiments on both IID and OOD testing sets of three benchmarks demonstrate its accurate and robust grounding performance against state-of-the-arts.
Learning Oriented Remote Sensing Object Detection via Naive Geometric Computing
Venue: arXiv cs.CV 2021
Year: 2021

Detecting oriented objects along with estimating their rotation information is one crucial step for analyzing remote sensing images. Despite that many methods proposed recently have achieved remarkable performance, most of them directly learn to predict object directions under the supervision of only one (e.g. the rotation angle) or a few (e.g. several coordinates) groundtruth values individually. Oriented object detection would be more accurate and robust if extra constraints, with respect to proposal and rotation information regression, are adopted for joint supervision during training. To this end, we innovatively propose a mechanism that simultaneously learns the regression of horizontal proposals, oriented proposals, and rotation angles of objects in a consistent manner, via naive geometric computing, as one additional steady constraint (see Figure 1). An oriented center prior guided label assignment strategy is proposed for further enhancing the quality of proposals, yielding better performance. Extensive experiments demonstrate the model equipped with our idea significantly outperforms the baseline by a large margin to achieve a new state-of-the-art result without any extra computational burden during inference. Our proposed idea is simple and intuitive that can be readily implemented. Source codes and trained models are involved in supplementary files.
Empirical evaluation of shallow and deep learning classifiers for Arabic sentiment analysis
Venue: arXiv cs.CL 2021
Year: 2021

This work presents a detailed comparison of the performance of deep learning models such as convolutional neural networks (CNN), long short-term memory (LSTM), gated recurrent units (GRU), their hybrids, and a selection of shallow learning classifiers for sentiment analysis of Arabic reviews. Additionally, the comparison includes state-of-the-art models such as the transformer architecture and the araBERT pre-trained model. The datasets used in this study are multi-dialect Arabic hotel and book review datasets, which are some of the largest publicly available datasets for Arabic reviews. Results showed deep learning outperforming shallow learning for binary and multi-label classification, in contrast with the results of similar work reported in the literature. This discrepancy in outcome was caused by dataset size as we found it to be proportional to the performance of deep learning models. The performance of deep and shallow learning techniques was analyzed in terms of accuracy and F1 score. The best performing shallow learning technique was Random Forest followed by Decision Tree, and AdaBoost. The deep learning models performed similarly using a default embedding layer, while the transformer model performed best when augmented with araBERT.
Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning
Venue: arXiv cs.CV 2021
Year: 2021

Face anti-spoofing (FAS) plays a vital role in preventing face recognition systems from presentation attacks. Existing face anti-spoofing datasets lack diversity due to the insufficient identity and insignificant variance, which limits the generalization ability of FAS model. In this paper, we propose Dual Spoof Disentanglement Generation (DSDG) framework to tackle this challenge by "anti-spoofing via generation". Depending on the interpretable factorized latent disentanglement in Variational Autoencoder (VAE), DSDG learns a joint distribution of the identity representation and the spoofing pattern representation in the latent space. Then, large-scale paired live and spoofing images can be generated from random noise to boost the diversity of the training set. However, some generated face images are partially distorted due to the inherent defect of VAE. Such noisy samples are hard to predict precise depth values, thus may obstruct the widely-used depth supervised optimization. To tackle this issue, we further introduce a lightweight Depth Uncertainty Module (DUM), which alleviates the adverse effects of noisy samples by depth uncertainty learning. DUM is developed without extra-dependency, thus can be flexibly integrated with any depth supervised network for face anti-spoofing. We evaluate the effectiveness of the proposed method on five popular benchmarks and achieve state-of-the-art results under both intra- and inter- test settings. The codes are available at https://github.com/JDAI-CV/FaceX-Zoo/tree/main/addition_module/DSDG.
Collective discrete optimisation as judgment aggregation
Venue: arXiv cs.AI 2021
Year: 2021

Many important collective decision-making problems can be seen as multi-agent versions of discrete optimisation problems. Participatory budgeting, for instance, is the collective version of the knapsack problem; other examples include collective scheduling, and collective spanning trees. Rather than developing a specific model, as well as specific algorithmic techniques, for each of these problems, we propose to represent and solve them in the unifying framework of judgment aggregation with weighted issues. We provide a modular definition of collective discrete optimisation (CDO) rules based on coupling a set scoring function with an operator, and we show how they generalise several existing procedures developed for specific CDO problems. We also give an implementation based on integer linear programming (ILP) and test it on the problem of collective spanning trees.
Meta Arcade: A Configurable Environment Suite for Meta-Learning
Venue: arXiv cs.LG 2021
Year: 2021

Most approaches to deep reinforcement learning (DRL) attempt to solve a single task at a time. As a result, most existing research benchmarks consist of individual games or suites of games that have common interfaces but little overlap in their perceptual features, objectives, or reward structures. To facilitate research into knowledge transfer among trained agents (e.g. via multi-task and meta-learning), more environment suites that provide configurable tasks with enough commonality to be studied collectively are needed. In this paper we present Meta Arcade, a tool to easily define and configure custom 2D arcade games that share common visuals, state spaces, action spaces, game components, and scoring mechanisms. Meta Arcade differs from prior environments in that both task commonality and configurability are prioritized: entire sets of games can be constructed from common elements, and these elements are adjustable through exposed parameters. We include a suite of 24 predefined games that collectively illustrate the possibilities of this framework and discuss how these games can be configured for research applications. We provide several experiments that illustrate how Meta Arcade could be used, including single-task benchmarks of predefined games, sample curriculum-based approaches that change game parameters over a set schedule, and an exploration of transfer learning between games.