OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Grasping with a "Soft" Drone: From Theory to Practice
Venue: arXiv cs.RO 2021
Year: 2021

Rigid grippers used in existing aerial manipulators require precise positioning to achieve successful grasps and transmit large contact forces that may destabilize the drone. This limits the speed during grasping and prevents "dynamic grasping", where the drone attempts to grasp an object while moving. On the other hand, biological systems (e.g., birds) rely on compliant and soft parts to dampen contact forces and compensate for grasping inaccuracy, enabling impressive feats. This paper presents the first prototype of a soft drone -- a quadrotor where traditional (i.e., rigid) landing gears are replaced with a soft tendon-actuated gripper to enable aggressive grasping. We provide three key contributions. First, we describe our soft drone prototype, including electro-mechanical design, software infrastructure, and fabrication. Second, we review the set of algorithms we use for trajectory optimization and control of the drone and the soft gripper; the algorithms combine state-of-the-art techniques for quadrotor control (i.e., an adaptive geometric controller) with advanced soft robotics models (i.e., a quasi-static finite element model). Finally, we evaluate our soft drone in physics simulations (using SOFA and Unity) and in real tests in a motion-capture room. Our drone is able to dynamically grasp objects of unknown shape where baseline approaches fail. Our physical prototype ensures consistent performance, achieving 91.7% successful grasps across 23 trials. We showcase dynamic grasping results in the video attachment. Video Attachment: https://youtu.be/mqbj8mEyCdk
Multi-Format Contrastive Learning of Audio Representations
Venue: arXiv cs.SD 2021
Year: 2021

Recent advances suggest the advantage of multi-modal training in comparison with single-modal methods. In contrast to this view, in our work we find that similar gain can be obtained from training with different formats of a single modality. In particular, we investigate the use of the contrastive learning framework to learn audio representations by maximizing the agreement between the raw audio and its spectral representation. We find a significant gain using this multi-format strategy against the single-format counterparts. Moreover, on the downstream AudioSet and ESC-50 classification task, our audio-only approach achieves new state-of-the-art results with a mean average precision of 0.376 and an accuracy of 90.5%, respectively.
Physical Activity Analysis of College Students During the COVID-19 Pandemic Using Smartphones
Venue: arXiv cs.CY 2021
Year: 2021

Owing to the pandemic caused by the coronavirus disease of 2019 (COVID-19), several universities have closed their campuses for preventing the spread of infection. Consequently, the university classes are being held over the Internet, and students attend these classes from their homes. While the COVID-19 pandemic is expected to be prolonged, the online-centric lifestyle has raised concerns about secondary health issues caused by reduced physical activity (PA). However, the actual status of PA among university students has not yet been examined in Japan. Hence, in this study, we collected daily PA data (including the data corresponding to the number of steps taken and the data associated with six types of activities) by employing smartphones and thereby analyzed the changes in the PA of university students. The PA data were collected over a period of ten weeks from 305 first-year university students who were attending a mandatory class of physical education at the university. The obtained results indicate that compared to the average number of steps taken before the COVID-19 pandemic (6474.87 steps), the average number of steps taken after the COVID-19 pandemic (3522.5 steps) has decreased by 45.6%. Furthermore, the decrease in commuting time (7 AM to 10 AM), classroom time, and extracurricular activity time (11 AM to 12 AM) has led to a decrease in PA on weekdays owing to reduced unplanned exercise opportunities and has caused an increase in the duration of being in the stationary state in the course of daily life.
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
Venue: arXiv cs.CV 2021
Year: 2021

Multi-modal pre-training models have been intensively explored to bridge vision and language in recent years. However, most of them explicitly model the cross-modal interaction between image-text pairs, by assuming that there exists strong semantic correlation between the text and image modalities. Since this strong assumption is often invalid in real-world scenarios, we choose to implicitly model the cross-modal correlation for large-scale multi-modal pre-training, which is the focus of the Chinese project `WenLan' led by our team. Specifically, with the weak correlation assumption over image-text pairs, we propose a two-tower pre-training model called BriVL within the cross-modal contrastive learning framework. Unlike OpenAI CLIP that adopts a simple contrastive learning method, we devise a more advanced algorithm by adapting the latest method MoCo into the cross-modal scenario. By building a large queue-based dictionary, our BriVL can incorporate more negative samples in limited GPU resources. We further construct a large Chinese multi-source image-text dataset called RUC-CAS-WenLan for pre-training our BriVL model. Extensive experiments demonstrate that the pre-trained BriVL model outperforms both UNITER and OpenAI CLIP on various downstream tasks.
Utility of Traffic Information in Dynamic Routing: Is Sharing Information Always Useful?
Venue: arXiv cs.MA 2021
Year: 2021

Real-time traffic information can be utilized to enhance the efficiency of transportation networks by dynamically updating routing plans to mitigate traffic jams. However, an interesting question is whether the network coordinator should broadcast real-time traffic information to all network users or communicate it selectively to some of them. Which option enhances the network efficiency more? In this work, we demonstrate using simulation experiments that sharing real-time traffic information with all network users is sub-optimal, and it is actually better to share the information with a majority subset of the total population in order to improve the overall network performance. This result is valid under the assumption that each network user decides its route to destination locally.
Privacy-preserving Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Privacy considerations and bias in datasets are quickly becoming high-priority issues that the computer vision community needs to face. So far, little attention has been given to practical solutions that do not involve collection of new datasets. In this work, we show that for object detection on COCO, both anonymizing the dataset by blurring faces, as well as swapping faces in a balanced manner along the gender and skin tone dimension, can retain object detection performances while preserving privacy and partially balancing bias.
DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spaces
Venue: arXiv cs.CL 2021
Year: 2021

Recent research efforts in NLP have demonstrated that distributional word vector spaces often encode stereotypical human biases, such as racism and sexism. With word representations ubiquitously used in NLP models and pipelines, this raises ethical issues and jeopardizes the fairness of language technologies. While there exists a large body of work on bias measures and debiasing methods, to date, there is no platform that would unify these research efforts and make bias measuring and debiasing of representation spaces widely accessible. In this work, we present DebIE, the first integrated platform for (1) measuring and (2) mitigating bias in word embeddings. Given an (i) embedding space (users can choose between the predefined spaces or upload their own) and (ii) a bias specification (users can choose between existing bias specifications or create their own), DebIE can (1) compute several measures of implicit and explicit bias and modify the embedding space by executing two (mutually composable) debiasing models. DebIE's functionality can be accessed through four different interfaces: (a) a web application, (b) a desktop application, (c) a REST-ful API, and (d) as a command-line application. DebIE is available at: debie.informatik.uni-mannheim.de.
A Quadratic Actor Network for Model-Free Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

In this work we discuss the incorporation of quadratic neurons into policy networks in the context of model-free actor-critic reinforcement learning. Quadratic neurons admit an explicit quadratic function approximation in contrast to conventional approaches where the the non-linearity is induced by the activation functions. We perform empiric experiments on several MuJoCo continuous control tasks and find that when quadratic neurons are added to MLP policy networks those outperform the baseline MLP whilst admitting a smaller number of parameters. The top returned reward is in average increased by $5.8\%$ while being about $21\%$ more sample efficient. Moreover, it can maintain its advantage against added action and observation noise.
Topological Data Analysis of Korean Music in Jeongganbo: A Cycle Structure
Venue: arXiv cs.SD 2021
Year: 2021

Jeongganbo is a unique music representation invented by Sejong the Great. Contrary to the western music notation, the pitch of each note is encrypted and the length is visualized directly in a matrix form in Jeongganbo. We use topological data analysis (TDA) to analyze the Korean music written in Jeongganbo for Suyeonjang, Songuyeo, and Taryong, those well-known pieces played at the palace and among noble community. We are particularly interested in the cycle structure. We first define and determine the node elements of each music, characterized uniquely with its pitch and length. Then we transform the music into a graph and define the distance between the nodes as their adjacent occurrence rate. The graph is used as a point cloud whose homological structure is investigated by measuring the hole structure in each dimension. We identify cycles of each music, match those in Jeongganbo, and show how those cycles are interconnected. The main discovery of this work is that the cycles of Suyeonjang and Songuyeo, categorized as a special type of cyclic music known as Dodeuri, frequently overlap each other when appearing in the music while the cycles found in Taryong, which does not belong to Dodeuri class, appear individually.
Developing and evaluating an human-automation shared control takeover strategy based on Human-in-the-loop driving simulation
Venue: arXiv cs.HC 2021
Year: 2021

The purpose of this paper is to develop a shared control takeover strategy for smooth and safety control transition from an automation driving system to the human driver and to approve its positive impacts on drivers' behavior and attitudes. A "human-in-the-loop" driving simulator experiment was conducted to evaluate the impact of the proposed shared control takeover strategy under different disengagement conditions. Results of thirty-two drivers showed shared control takeover strategy could improve safety performance at the aggregated level, especially at non-driving related disengagements. For more urgent disengagements caused by another vehicle's sudden brake, a shared control strategy enlarges individual differences. The primary reason is that some drivers had higher self-reported mental workloads in response to the shared control takeover strategy. Therefore, shared control between driver and automation can involve driver's training to avoid mental overload when developing takeover strategies.
Diagnosing Vulnerability of Variational Auto-Encoders to Adversarial Attacks
Venue: arXiv cs.CR 2021
Year: 2021

In this work, we explore adversarial attacks on the Variational Autoencoders (VAE). We show how to modify data point to obtain a prescribed latent code (supervised attack) or just get a drastically different code (unsupervised attack). We examine the influence of model modifications ($β$-VAE, NVAE) on the robustness of VAEs and suggest metrics to quantify it.
Intelligent behavior depends on the ecological niche: Scaling up AI to human-like intelligence in socio-cultural environments
Venue: arXiv cs.AI 2021
Year: 2021

This paper outlines a perspective on the future of AI, discussing directions for machines models of human-like intelligence. We explain how developmental and evolutionary theories of human cognition should further inform artificial intelligence. We emphasize the role of ecological niches in sculpting intelligent behavior, and in particular that human intelligence was fundamentally shaped to adapt to a constantly changing socio-cultural environment. We argue that a major limit of current work in AI is that it is missing this perspective, both theoretically and experimentally. Finally, we discuss the promising approach of developmental artificial intelligence, modeling infant development through multi-scale interaction between intrinsically motivated learning, embodiment and a fastly changing socio-cultural environment. This paper takes the form of an interview of Pierre-Yves Oudeyer by Mandred Eppe, organized within the context of a KI - K{ü}nstliche Intelligenz special issue in developmental robotics.
HumanGAN: A Generative Model of Humans Images
Venue: arXiv cs.CV 2021
Year: 2021

Generative adversarial networks achieve great performance in photorealistic image synthesis in various domains, including human images. However, they usually employ latent vectors that encode the sampled outputs globally. This does not allow convenient control of semantically-relevant individual parts of the image, and is not able to draw samples that only differ in partial aspects, such as clothing style. We address these limitations and present a generative model for images of dressed humans offering control over pose, local body part appearance and garment style. This is the first method to solve various aspects of human image generation such as global appearance sampling, pose transfer, parts and garment transfer, and parts sampling jointly in a unified framework. As our model encodes part-based latent appearance vectors in a normalized pose-independent space and warps them to different poses, it preserves body and clothing appearance under varying posture. Experiments show that our flexible and general generative method outperforms task-specific baselines for pose-conditioned image generation, pose transfer and part sampling in terms of realism and output resolution.
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
Venue: arXiv cs.CL 2021
Year: 2021

Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. As a result, these models fail to generalize to real-world out-of-distribution data. In this work, we show that the words in the NLU training set can be modeled as a long-tailed distribution. There are two findings: 1) NLU models have strong preference for features located at the head of the long-tailed distribution, and 2) Shortcut features are picked up during very early few iterations of the model training. These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample. Based on this shortcut measurement, we propose a shortcut mitigation framework LTGR, to suppress the model from making overconfident predictions for samples with large shortcut degree. Experimental results on three NLU benchmarks demonstrate that our long-tailed distribution explanation accurately reflects the shortcut learning behavior of NLU models. Experimental analysis further indicates that LTGR can improve the generalization accuracy on OOD data, while preserving the accuracy on in-distribution data.
Stochastic-HMDs: Adversarial Resilient Hardware Malware Detectors through Voltage Over-scaling
Venue: arXiv cs.CR 2021
Year: 2021

Machine learning-based hardware malware detectors (HMDs) offer a potential game changing advantage in defending systems against malware. However, HMDs suffer from adversarial attacks, can be effectively reverse-engineered and subsequently be evaded, allowing malware to hide from detection. We address this issue by proposing a novel HMDs (Stochastic-HMDs) through approximate computing, which makes HMDs' inference computation-stochastic, thereby making HMDs resilient against adversarial evasion attacks. Specifically, we propose to leverage voltage overscaling to induce stochastic computation in the HMDs model. We show that such a technique makes HMDs more resilient to both black-box adversarial attack scenarios, i.e., reverse-engineering and transferability. Our experimental results demonstrate that Stochastic-HMDs offer effective defense against adversarial attacks along with by-product power savings, without requiring any changes to the hardware/software nor to the HMDs' model, i.e., no retraining or fine tuning is needed. Moreover, based on recent results in probably approximately correct (PAC) learnability theory, we show that Stochastic-HMDs are provably more difficult to reverse engineer.
The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshop
Venue: arXiv cs.CV 2021
Year: 2021

This document describes the details and the motivation behind a new dataset we collected for the semi-supervised recognition challenge~\cite{semi-aves} at the FGVC7 workshop at CVPR 2020. The dataset contains 1000 species of birds sampled from the iNat-2018 dataset for a total of nearly 150k images. From this collection, we sample a subset of classes and their labels, while adding the images from the remaining classes to the unlabeled set of images. The presence of out-of-domain data (novel classes), high class-imbalance, and fine-grained similarity between classes poses significant challenges for existing semi-supervised recognition techniques in the literature. The dataset is available here: \url{https://github.com/cvl-umass/semi-inat-2020}
Preregistering NLP Research
Venue: arXiv cs.CL 2021
Year: 2021

Preregistration refers to the practice of specifying what you are going to do, and what you expect to find in your study, before carrying out the study. This practice is increasingly common in medicine and psychology, but is rarely discussed in NLP. This paper discusses preregistration in more detail, explores how NLP researchers could preregister their work, and presents several preregistration questions for different kinds of studies. Finally, we argue in favour of registered reports, which could provide firmer grounds for slow science in NLP research. The goal of this paper is to elicit a discussion in the NLP community, which we hope to synthesise into a general NLP preregistration form in future research.
Smart and Reconfigurable Wireless Communications: From IRS Modeling to Algorithm Design
Venue: arXiv cs.IT 2021
Year: 2021

Intelligent reflecting surfaces (IRSs) have been introduced into wireless communications systems due to their great potential to smartly customize and reconfigure radio propagation environments in a cost-effective manner. Despite the promising advantages of IRSs, academic research on IRSs is still in its infancy. In particular, the design and analysis of IRS-assisted wireless communication systems critically depend on an accurate and tractable modeling of the IRS. In this article, we first present and compare three IRS models, namely the conventional independent diffusive scatterer-based model, physics-based model, and impedance network-based model, in terms of their accuracy, tractability, and hardware complexity. Besides, a new framework based on partitioning the IRS into tiles and employing codebooks of transmission modes is introduced to enable scalable IRS optimization. Then, we investigate the impact of the three considered IRS models on system design, where several crucial technical challenges for the efficient design of IRS-assisted wireless systems are identified and the corresponding solutions are unraveled. Furthermore, to illustrate the properties of the considered models and the efficiency of the proposed solution concepts, IRS-assisted secure wireless systems and simultaneous wireless information and power transfer (SWIPT) systems are studied in more detail. Finally, several promising future research directions for IRS-assisted wireless systems are highlighted.
IRS-Assisted Ambient Backscatter Communications Utilizing Deep Reinforcement Learning
Venue: arXiv cs.IT 2021
Year: 2021

We consider an ambient backscatter communication (AmBC) system aided by an intelligent reflecting surface (IRS). The optimization of the IRS to assist AmBC is extremely difficult when there is no prior channel knowledge, for which no design solutions are currently available. We utilize a deep reinforcement learning-based framework to jointly optimize the IRS and reader beamforming, with no knowledge of the channels or ambient signal. We show that the proposed framework can facilitate effective AmBC communication with a detection performance comparable to several benchmarks under full channel knowledge.
PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching
Venue: arXiv cs.CV 2021
Year: 2021

Supervised learning with deep convolutional neural networks (DCNNs) has seen huge adoption in stereo matching. However, the acquisition of large-scale datasets with well-labeled ground truth is cumbersome and labor-intensive, making supervised learning-based approaches often hard to implement in practice. To overcome this drawback, we propose a robust and effective self-supervised stereo matching approach, consisting of a pyramid voting module (PVM) and a novel DCNN architecture, referred to as OptStereo. Specifically, our OptStereo first builds multi-scale cost volumes, and then adopts a recurrent unit to iteratively update disparity estimations at high resolution; while our PVM can generate reliable semi-dense disparity images, which can be employed to supervise OptStereo training. Furthermore, we publish the HKUST-Drive dataset, a large-scale synthetic stereo dataset, collected under different illumination and weather conditions for research purposes. Extensive experimental results demonstrate the effectiveness and efficiency of our self-supervised stereo matching approach on the KITTI Stereo benchmarks and our HKUST-Drive dataset. PVStereo, our best-performing implementation, greatly outperforms all other state-of-the-art self-supervised stereo matching approaches. Our project page is available at sites.google.com/view/pvstereo.
Modeling of Resource Allocation Mechanisms in Distributed Computing Systems using Petri Nets and Stochastic Activity Networks (SAN): a Review and Reo-based Suggestion
Venue: arXiv cs.DC 2021
Year: 2021

Resource allocation is crucial in the distributed systems. It is a key step in designing the mechanisms of systems for determining the resource allocation mechanism, it is important for obtaining the desired efficiency in the system, plus it is vital for predicting and preventing Deadlocks. Various models of Petri Net (Stochastic PN, Colored PN, Generalized PN, etc.) are used for modeling, simulation, execution, and solving the problems of resource allocation. SAN models are used for modeling the problems pertinent to resource allocation. First, we shall address the basic concepts pertinent to these models and the resource allocation problem (introduction chapter), then, some applications of the Petri Net and SAN models in the distributed computational systems or systems based on them shall be studied. Finally, the issues and findings will be concluded.