OpenRaiser/Intern-Atlas huggingface dataset:
Adversarial Attacks in a Multi-view Setting: An Empirical Study of the Adversarial Patches Inter-view Transferability
Venue: arXiv cs.CR 2021
Year: 2021

While machine learning applications are getting mainstream owing to a demonstrated efficiency in solving complex problems, they suffer from inherent vulnerability to adversarial attacks. Adversarial attacks consist of additive noise to an input which can fool a detector. Recently, successful real-world printable adversarial patches were proven efficient against state-of-the-art neural networks. In the transition from digital noise based attacks to real-world physical attacks, the myriad of factors affecting object detection will also affect adversarial patches. Among these factors, view angle is one of the most influential, yet under-explored. In this paper, we study the effect of view angle on the effectiveness of an adversarial patch. To this aim, we propose the first approach that considers a multi-view context by combining existing adversarial patches with a perspective geometric transformation in order to simulate the effect of view angle changes. Our approach has been evaluated on two datasets: the first dataset which contains most real world constraints of a multi-view context, and the second dataset which empirically isolates the effect of view angle. The experiments show that view angle significantly affects the performance of adversarial patches, where in some cases the patch loses most of its effectiveness. We believe that these results motivate taking into account the effect of view angles in future adversarial attacks, and open up new opportunities for adversarial defenses.
Language Models As or For Knowledge Bases
Venue: arXiv cs.CL 2021
Year: 2021

Pre-trained language models (LMs) have recently gained attention for their potential as an alternative to (or proxy for) explicit knowledge bases (KBs). In this position paper, we examine this hypothesis, identify strengths and limitations of both LMs and KBs, and discuss the complementary nature of the two paradigms. In particular, we offer qualitative arguments that latent LMs are not suitable as a substitute for explicit KBs, but could play a major role for augmenting and curating KBs.
Perceptions and attitudes of Children and Young People to Artificial Intelligence in Medicine
Venue: arXiv cs.HC 2021
Year: 2021

There is increasing interest in Artificial Intelligence and its application to medicine. Perceptions are less well-known, notably amongst children and young people. 21 members of a Young Persons Advisory Group for research, recommend creating an enabling environment with children and young people, through educational workshops with practical examples that use Artificial Intelligence to help, but not replace humans, address issues, build trust, and effectively communicate about potential opportunities.
An Analysis of COVID-19 Knowledge Graph Construction and Applications
Venue: arXiv cs.SI 2021
Year: 2021

The construction and application of knowledge graphs have seen a rapid increase across many disciplines in recent years. Additionally, the problem of uncovering relationships between developments in the COVID-19 pandemic and social media behavior is of great interest to researchers hoping to curb the spread of the disease. In this paper we present a knowledge graph constructed from COVID-19 related tweets in the Los Angeles area, supplemented with federal and state policy announcements and disease spread statistics. By incorporating dates, topics, and events as entities, we construct a knowledge graph that describes the connections between these useful information. We use natural language processing and change point analysis to extract tweet-topic, tweet-date, and event-date relations. Further analysis on the constructed knowledge graph provides insight into how tweets reflect public sentiments towards COVID-19 related topics and how changes in these sentiments correlate with real-world events.
NFT-K: Non-Fungible Tangent Kernels
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks have become essential for numerous applications due to their strong empirical performance such as vision, RL, and classification. Unfortunately, these networks are quite difficult to interpret, and this limits their applicability in settings where interpretability is important for safety, such as medical imaging. One type of deep neural network is neural tangent kernel that is similar to a kernel machine that provides some aspect of interpretability. To further contribute interpretability with respect to classification and the layers, we develop a new network as a combination of multiple neural tangent kernels, one to model each layer of the deep neural network individually as opposed to past work which attempts to represent the entire network via a single neural tangent kernel. We demonstrate the interpretability of this model on two datasets, showing that the multiple kernels model elucidates the interplay between the layers and predictions.
MELONS: generating melody with long-term structure using transformers and structure graph
Venue: arXiv cs.SD 2021
Year: 2021

The creation of long melody sequences requires effective expression of coherent musical structure. However, there is no clear representation of musical structure. Recent works on music generation have suggested various approaches to deal with the structural information of music, but generating a full-song melody with clear long-term structure remains a challenge. In this paper, we propose MELONS, a melody generation framework based on a graph representation of music structure which consists of eight types of bar-level relations. MELONS adopts a multi-step generation method with transformer-based networks by factoring melody generation into two sub-problems: structure generation and structure conditional melody generation. Experimental results show that MELONS can produce structured melodies with high quality and rich contents.
Cross Domain Emotion Recognition using Few Shot Knowledge Transfer
Venue: arXiv cs.CL 2021
Year: 2021

Emotion recognition from text is a challenging task due to diverse emotion taxonomies, lack of reliable labeled data in different domains, and highly subjective annotation standards. Few-shot and zero-shot techniques can generalize across unseen emotions by projecting the documents and emotion labels onto a shared embedding space. In this work, we explore the task of few-shot emotion recognition by transferring the knowledge gained from supervision on the GoEmotions Reddit dataset to the SemEval tweets corpus, using different emotion representation methods. The results show that knowledge transfer using external knowledge bases and fine-tuned encoders perform comparably as supervised baselines, requiring minimal supervision from the task dataset.
Achieving safe minimum circle circumnavigation around multiple targets: a dynamic compensation approach
Venue: arXiv cs.MA 2021
Year: 2021

Minimum circle circumnavigation is proposed in this paper, which is of special value in target monitoring, capturing and/or attacking. In this paper, a safe minimum circle circumnavigation of multiple targets based on bearing measurements is studied. In contrast with the traditional circumnavigation problem, with the new pattern, one agent is able to enclose multiple targets along a minimum circle with the desired enclosing distance and tangential speed. To achieve the minimum circle circumnavigation, an algorithm including dynamic compensators and a control protocol is proposed, by which collision is avoided between the agent and the multiple targets during the whole moving process. Moreover, the control protocol developed for a single agent is further extended to the scenarios of multiple agents by adding a coordination mechanism into the tangential velocity term, which drives the agents to distribute evenly on each expected circular orbit with the same radius or different radius. Simulations results illustrate the effectiveness of the proposed methods.
Confidential Token-Based License Management
Venue: arXiv cs.CR 2021
Year: 2021

In a global economy with many competitive participants, licensing and tracking of 3D printed parts is desirable if not mandatory for many use-cases. We investigate a blockchain-based approach, as blockchains provide many attractive features, like decentralized architecture and high security assurances. An often neglected aspect of the product life-cycle management is the confidentiality of transactions to hide valuable business information from competitors. To solve the combined problem of trust and confidentiality, we present a confidential licensing and tracking system which works on any publicly verifiable, token-based blockchain that supports tokens of different types representing licenses or attributes of parts. Together with the secure integration of a unique eID in each part, our system provides an efficient, immutable and authenticated transaction log scalable to thousands of transactions per second. With our confidential Token-Based License Management system (cTLM), large industries such as automotive or aviation can license and trace all parts confidentially.
Adaptive F-FFT Demodulation for ICI Mitigation in Differential Underwater Acoustic OFDM Systems
Venue: arXiv cs.IT 2021
Year: 2021

This paper addresses the problem of frequency-domain inter-carrier interference (ICI) mitigation for differential orthogonal frequency-division multiplexing (OFDM) systems. The classical fractional fast Fourier transform (F-FFT), adopting the fixed sampling interval, would suffer from the limited accuracy of ICI mitigation and low adaptability in dynamic Doppler spread. To target the above challenges, we propose an adaptive fractional Fourier transform (A-FFT) demodulation method, in which an estimation algorithm based on the coordinate descent approach is designed to compute the fiducial frequency offset without increasing pilots. By means of compensating ICI at fractions of the fiducial frequency offset adapted to the time-varying Doppler shift, the A-FFT has the capability of tracking Doppler fluctuations over the underwater acoustic channels, thus extending the application range of frequency-domain ICI mitigation. Simulation results show that the A-FFT is significantly superior to the existing classical methods, the partial fast Fourier transform (P-FFT) and the F-FFT, for both medium and high Doppler factors and large carrier numbers in terms of the mean squared error (MSE). Numerically, the MSE of the A-FFT is reduced by $\bf{39.88\%-72.14\%}$ compared to that of the F-FFT with the input signal-to-noise ratio ranging from 10 dB to 30 dB at a Doppler factor of $\bf{2.5\times 10^{-4}}$ and a carrier number of 1024, while the P-FFT even cannot work well.
ViSeRet: A simple yet effective approach to moment retrieval via fine-grained video segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Video-text retrieval has many real-world applications such as media analytics, surveillance, and robotics. This paper presents the 1st place solution to the video retrieval track of the ICCV VALUE Challenge 2021. We present a simple yet effective approach to jointly tackle two video-text retrieval tasks (video retrieval and video corpus moment retrieval) by leveraging the model trained only on the video retrieval task. In addition, we create an ensemble model that achieves the new state-of-the-art performance on all four datasets (TVr, How2r, YouCook2r, and VATEXr) presented in the VALUE Challenge.
Decentralized sliding-mode control laws for the bearing-based formation tracking problem
Venue: arXiv cs.MA 2021
Year: 2021

This paper studies the time-varying bearing-based tracking of leader-follower formations. The desired constraints between agents are specified by bearing vectors, and several leaders are moving with a bounded reference velocity. Each followers can measure the relative positions of its neighbors, its own velocities, and receive information from their neighbors. Under the assumptions that the desired formation is infinitesimally bearing rigid and the local reference frames of followers are aligned with each other, two control laws are presented in this paper based on sliding mode control approach. Stability analyses are given based on Lyapunov stability theory and supported by numerical simulations.
Privacy preserving local analysis of digital trace data: A proof-of-concept
Venue: arXiv cs.CR 2021
Year: 2021

We present PORT, a software platform for local data extraction and analysis of digital trace data. While digital trace data collected by private and public parties hold a huge potential for social-scientific discovery, their most useful parts have been unattainable for academic researchers due to privacy concerns and prohibitive API access. However, the EU General Data Protection Regulation (GDPR) grants all citizens the right to an electronic copy of their personal data. All major data controllers, such as social media platforms, banks, online shops, loyalty card systems and public transportation cards comply with this right by providing their clients with a `Data Download Package' (DDP). Previously, a conceptual workflow was introduced allowing citizens to donate their data to scientific- researchers. In this workflow, citizens' DDPs are processed locally on their machines before they are asked to provide informed consent to share a subset of the processed data with the researchers. In this paper, we present the newly developed software PORT that implements the local processing part of this workflow, protecting privacy by shielding sensitive data from any contact with outside observers -- including the researchers themselves. Thus, PORT enables a host of potential applications of social data science to hitherto unobtainable data.
Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking
Venue: arXiv cs.CV 2021
Year: 2021

On the way towards general Visual Question Answering (VQA) systems that are able to answer arbitrary questions, the need arises for evaluation beyond single-metric leaderboards for specific datasets. To this end, we propose a browser-based benchmarking tool for researchers and challenge organizers, with an API for easy integration of new models and datasets to keep up with the fast-changing landscape of VQA. Our tool helps test generalization capabilities of models across multiple datasets, evaluating not just accuracy, but also performance in more realistic real-world scenarios such as robustness to input noise. Additionally, we include metrics that measure biases and uncertainty, to further explain model behavior. Interactive filtering facilitates discovery of problematic behavior, down to the data sample level. As proof of concept, we perform a case study on four models. We find that state-of-the-art VQA models are optimized for specific tasks or datasets, but fail to generalize even to other in-domain test sets, for example they cannot recognize text in images. Our metrics allow us to quantify which image and question embeddings provide most robustness to a model. All code is publicly available.
Deploying Containerized QuantEx Quantum Simulation Software on HPC Systems
Venue: arXiv cs.DC 2021
Year: 2021

The simulation of quantum circuits using the tensor network method is very computationally demanding and requires significant High Performance Computing (HPC) resources to find an efficient contraction order and to perform the contraction of the large tensor networks. In addition, the researchers want a workflow that is easy to customize, reproduce and migrate to different HPC systems. In this paper, we discuss the issues associated with the deployment of the QuantEX quantum computing simulation software within containers on different HPC systems. Also, we compare the performance of the containerized software with the software running on bare metal.
CLIP4Caption ++: Multi-CLIP for Video Caption
Venue: arXiv cs.CV 2021
Year: 2021

This report describes our solution to the VALUE Challenge 2021 in the captioning task. Our solution, named CLIP4Caption++, is built on X-Linear/X-Transformer, which is an advanced model with encoder-decoder architecture. We make the following improvements on the proposed CLIP4Caption++: We employ an advanced encoder-decoder model architecture X-Transformer as our main framework and make the following improvements: 1) we utilize three strong pre-trained CLIP models to extract the text-related appearance visual features. 2) we adopt the TSN sampling strategy for data enhancement. 3) we involve the video subtitle information to provide richer semantic information. 3) we introduce the subtitle information, which fuses with the visual features as guidance. 4) we design word-level and sentence-level ensemble strategies. Our proposed method achieves 86.5, 148.4, 64.5 CIDEr scores on VATEX, YC2C, and TVC datasets, respectively, which shows the superior performance of our proposed CLIP4Caption++ on all three datasets.
Continual Learning with Differential Privacy
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we focus on preserving differential privacy (DP) in continual learning (CL), in which we train ML models to learn a sequence of new tasks while memorizing previous tasks. We first introduce a notion of continual adjacent databases to bound the sensitivity of any data record participating in the training process of CL. Based upon that, we develop a new DP-preserving algorithm for CL with a data sampling strategy to quantify the privacy risk of training data in the well-known Averaged Gradient Episodic Memory (A-GEM) approach by applying a moments accountant. Our algorithm provides formal guarantees of privacy for data records across tasks in CL. Preliminary theoretical analysis and evaluations show that our mechanism tightens the privacy loss while maintaining a promising model utility.
Homogeneous Learning: Self-Attention Decentralized Deep Learning
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning (FL) has been facilitating privacy-preserving deep learning in many walks of life such as medical image classification, network intrusion detection, and so forth. Whereas it necessitates a central parameter server for model aggregation, which brings about delayed model communication and vulnerability to adversarial attacks. A fully decentralized architecture like Swarm Learning allows peer-to-peer communication among distributed nodes, without the central server. One of the most challenging issues in decentralized deep learning is that data owned by each node are usually non-independent and identically distributed (non-IID), causing time-consuming convergence of model training. To this end, we propose a decentralized learning model called Homogeneous Learning (HL) for tackling non-IID data with a self-attention mechanism. In HL, training performs on each round's selected node, and the trained model of a node is sent to the next selected node at the end of each round. Notably, for the selection, the self-attention mechanism leverages reinforcement learning to observe a node's inner state and its surrounding environment's state, and find out which node should be selected to optimize the training. We evaluate our method with various scenarios for an image classification task. The result suggests that HL can produce a better performance compared with standalone learning and greatly reduce both the total training rounds by 50.8% and the communication cost by 74.6% compared with random policy-based decentralized learning for training on non-IID data.
Using Document Similarity Methods to create Parallel Datasets for Code Translation
Venue: arXiv cs.CL 2021
Year: 2021

Translating source code from one programming language to another is a critical, time-consuming task in modernizing legacy applications and codebases. Recent work in this space has drawn inspiration from the software naturalness hypothesis by applying natural language processing techniques towards automating the code translation task. However, due to the paucity of parallel data in this domain, supervised techniques have only been applied to a limited set of popular programming languages. To bypass this limitation, unsupervised neural machine translation techniques have been proposed to learn code translation using only monolingual corpora. In this work, we propose to use document similarity methods to create noisy parallel datasets of code, thus enabling supervised techniques to be applied for automated code translation without having to rely on the availability or expensive curation of parallel code datasets. We explore the noise tolerance of models trained on such automatically-created datasets and show that these models perform comparably to models trained on ground truth for reasonable levels of noise. Finally, we exhibit the practical utility of the proposed method by creating parallel datasets for languages beyond the ones explored in prior work, thus expanding the set of programming languages for automated code translation.
Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World
Venue: arXiv cs.RO 2021
Year: 2021

Legged robots are physically capable of traversing a wide range of challenging environments, but designing controllers that are sufficiently robust to handle this diversity has been a long-standing challenge in robotics. Reinforcement learning presents an appealing approach for automating the controller design process and has been able to produce remarkably robust controllers when trained in a suitable range of environments. However, it is difficult to predict all likely conditions the robot will encounter during deployment and enumerate them at training-time. What if instead of training controllers that are robust enough to handle any eventuality, we enable the robot to continually learn in any setting it finds itself in? This kind of real-world reinforcement learning poses a number of challenges, including efficiency, safety, and autonomy. To address these challenges, we propose a practical robot reinforcement learning system for fine-tuning locomotion policies in the real world. We demonstrate that a modest amount of real-world training can substantially improve performance during deployment, and this enables a real A1 quadrupedal robot to autonomously fine-tune multiple locomotion skills in a range of environments, including an outdoor lawn and a variety of indoor terrains.
UrbanNet: Leveraging Urban Maps for Long Range 3D Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Relying on monocular image data for precise 3D object detection remains an open problem, whose solution has broad implications for cost-sensitive applications such as traffic monitoring. We present UrbanNet, a modular architecture for long range monocular 3D object detection with static cameras. Our proposed system combines commonly available urban maps along with a mature 2D object detector and an efficient 3D object descriptor to accomplish accurate detection at long range even when objects are rotated along any of their three axes. We evaluate UrbanNet on a novel challenging synthetic dataset and highlight the advantages of its design for traffic detection in roads with changing slope, where the flat ground approximation does not hold. Data and code are available at https://github.com/TRAILab/UrbanNet
Evaluation of Latent Space Disentanglement in the Presence of Interdependent Attributes
Venue: arXiv cs.SD 2021
Year: 2021

Controllable music generation with deep generative models has become increasingly reliant on disentanglement learning techniques. However, current disentanglement metrics, such as mutual information gap (MIG), are often inadequate and misleading when used for evaluating latent representations in the presence of interdependent semantic attributes often encountered in real-world music datasets. In this work, we propose a dependency-aware information metric as a drop-in replacement for MIG that accounts for the inherent relationship between semantic attributes.