OpenRaiser/Intern-Atlas huggingface dataset:
Why Robust Natural Language Understanding is a Challenge
Venue: arXiv cs.CL 2022
Year: 2022

With the proliferation of Deep Machine Learning into real-life applications, a particular property of this technology has been brought to attention: robustness Neural Networks notoriously present low robustness and can be highly sensitive to small input perturbations. Recently, many methods for verifying networks' general properties of robustness have been proposed, but they are mostly applied in Computer Vision. In this paper we propose a Verification specification for Natural Language Understanding classification based on larger regions of interest, and we discuss the challenges of such task. We observe that, although the data is almost linearly separable, the verifier struggles to output positive results and we explain the problems and implications.
On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling
Venue: arXiv cs.ET 2022
Year: 2022

Synaptic memory consolidation has been heralded as one of the key mechanisms for supporting continual learning in neuromorphic Artificial Intelligence (AI) systems. Here we report that a Fowler-Nordheim (FN) quantum-tunneling device can implement synaptic memory consolidation similar to what can be achieved by algorithmic consolidation models like the cascade and the elastic weight consolidation (EWC) models. The proposed FN-synapse not only stores the synaptic weight but also stores the synapse's historical usage statistic on the device itself. We also show that the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and we demonstrate that a network comprising FN-synapses outperforms a comparable EWC network for a small benchmark continual learning task. With an energy footprint of femtojoules per synaptic update, we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and persistent learning.
BoT-SORT: Robust Associations Multi-Pedestrian Tracking
Venue: arXiv cs.CV 2022
Year: 2022

The goal of multi-object tracking (MOT) is detecting and tracking all the objects in a scene, while keeping a unique identifier for each object. In this paper, we present a new robust state-of-the-art tracker, which can combine the advantages of motion and appearance information, along with camera-motion compensation, and a more accurate Kalman filter state vector. Our new trackers BoT-SORT, and BoT-SORT-ReID rank first in the datasets of MOTChallenge [29, 11] on both MOT17 and MOT20 test sets, in terms of all the main MOT metrics: MOTA, IDF1, and HOTA. For MOT17: 80.5 MOTA, 80.2 IDF1, and 65.0 HOTA are achieved. The source code and the pre-trained models are available at https://github.com/NirAharon/BOT-SORT
Online vs. Offline Adaptive Domain Randomization Benchmark
Venue: arXiv cs.RO 2022
Year: 2022

Physics simulators have shown great promise for conveniently learning reinforcement learning policies in safe, unconstrained environments. However, transferring the acquired knowledge to the real world can be challenging due to the reality gap. To this end, several methods have been recently proposed to automatically tune simulator parameters with posterior distributions given real data, for use with domain randomization at training time. These approaches have been shown to work for various robotic tasks under different settings and assumptions. Nevertheless, existing literature lacks a thorough comparison of existing adaptive domain randomization methods with respect to transfer performance and real-data efficiency. In this work, we present an open benchmark for both offline and online methods (SimOpt, BayRn, DROID, DROPO), to shed light on which are most suitable for each setting and task at hand. We found that online methods are limited by the quality of the currently learned policy for the next iteration, while offline methods may sometimes fail when replaying trajectories in simulation with open-loop commands. The code used will be released at https://github.com/gabrieletiboni/adr-benchmark.
There and Back Again: On Applying Data Reduction Rules by Undoing Others
Venue: arXiv cs.DS 2022
Year: 2022

Data reduction rules are an established method in the algorithmic toolbox for tackling computationally challenging problems. A data reduction rule is a polynomial-time algorithm that, given a problem instance as input, outputs an equivalent, typically smaller instance of the same problem. The application of data reduction rules during the preprocessing of problem instances allows in many cases to considerably shrink their size, or even solve them directly. Commonly, these data reduction rules are applied exhaustively and in some fixed order to obtain irreducible instances. It was often observed that by changing the order of the rules, different irreducible instances can be obtained. We propose to "undo" data reduction rules on irreducible instances, by which they become larger, and then subsequently apply data reduction rules again to shrink them. We show that this somewhat counter-intuitive approach can lead to significantly smaller irreducible instances. The process of undoing data reduction rules is not limited to "rolling back" data reduction rules applied to the instance during preprocessing. Instead, we formulate so-called backward rules, which essentially undo a data reduction rule, but without using any information about which data reduction rules were applied to it previously. In particular, based on the example of Vertex Cover we propose two methods applying backward rules to shrink the instances further. In our experiments we show that this way smaller irreducible instances consisting of real-world graphs from the SNAP and DIMACS datasets can be computed.
longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attacks
Venue: arXiv cs.CL 2022
Year: 2022

Developing methods to adversarially challenge NLP systems is a promising avenue for improving both model performance and interpretability. Here, we describe the approach of the team "longhorns" on Task 1 of the The First Workshop on Dynamic Adversarial Data Collection (DADC), which asked teams to manually fool a model on an Extractive Question Answering task. Our team finished first, with a model error rate of 62%. We advocate for a systematic, linguistically informed approach to formulating adversarial questions, and we describe the results of our pilot experiments, as well as our official submission.
On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method
Venue: arXiv cs.CL 2022
Year: 2022

Most works on modeling the conversation history in Conversational Question Answering (CQA) report a single main result on a common CQA benchmark. While existing models show impressive results on CQA leaderboards, it remains unclear whether they are robust to shifts in setting (sometimes to more realistic ones), training data size (e.g. from large to small sets) and domain. In this work, we design and conduct the first large-scale robustness study of history modeling approaches for CQA. We find that high benchmark scores do not necessarily translate to strong robustness, and that various methods can perform extremely differently under different settings. Equipped with the insights from our study, we design a novel prompt-based history modeling approach, and demonstrate its strong robustness across various settings. Our approach is inspired by existing methods that highlight historic answers in the passage. However, instead of highlighting by modifying the passage token embeddings, we add textual prompts directly in the passage text. Our approach is simple, easy-to-plug into practically any model, and highly effective, thus we recommend it as a starting point for future model developers. We also hope that our study and insights will raise awareness to the importance of robustness-focused evaluation, in addition to obtaining high leaderboard scores, leading to better CQA systems.
Mapping the Design Space of Human-AI Interaction in Text Summarization
Venue: arXiv cs.HC 2022
Year: 2022

Automatic text summarization systems commonly involve humans for preparing data or evaluating model performance, yet, there lacks a systematic understanding of humans' roles, experience, and needs when interacting with or being assisted by AI. From a human-centered perspective, we map the design opportunities and considerations for human-AI interaction in text summarization and broader text generation tasks. We first conducted a systematic literature review of 70 papers, developing a taxonomy of five interactions in AI-assisted text generation and relevant design dimensions. We designed text summarization prototypes for each interaction. We then interviewed 16 users, aided by the prototypes, to understand their expectations, experience, and needs regarding efficiency, control, and trust with AI in text summarization and propose design considerations accordingly.
On Non-Random Missing Labels in Semi-Supervised Learning
Venue: arXiv cs.CV 2022
Year: 2022

Semi-Supervised Learning (SSL) is fundamentally a missing label problem, in which the label Missing Not At Random (MNAR) problem is more realistic and challenging, compared to the widely-adopted yet naive Missing Completely At Random assumption where both labeled and unlabeled data share the same class distribution. Different from existing SSL solutions that overlook the role of "class" in causing the non-randomness, e.g., users are more likely to label popular classes, we explicitly incorporate "class" into SSL. Our method is three-fold: 1) We propose Class-Aware Propensity (CAP) that exploits the unlabeled data to train an improved classifier using the biased labeled data. 2) To encourage rare class training, whose model is low-recall but high-precision that discards too many pseudo-labeled data, we propose Class-Aware Imputation (CAI) that dynamically decreases (or increases) the pseudo-label assignment threshold for rare (or frequent) classes. 3) Overall, we integrate CAP and CAI into a Class-Aware Doubly Robust (CADR) estimator for training an unbiased SSL model. Under various MNAR settings and ablations, our method not only significantly outperforms existing baselines but also surpasses other label bias removal SSL methods. Please check our code at: https://github.com/JoyHuYY1412/CADR-FixMatch.
ComDensE : Combined Dense Embedding of Relation-aware and Common Features for Knowledge Graph Completion
Venue: arXiv cs.AI 2022
Year: 2022

Real-world knowledge graphs (KG) are mostly incomplete. The problem of recovering missing relations, called KG completion, has recently become an active research area. Knowledge graph (KG) embedding, a low-dimensional representation of entities and relations, is the crucial technique for KG completion. Convolutional neural networks in models such as ConvE, SACN, InteractE, and RGCN achieve recent successes. This paper takes a different architectural view and proposes ComDensE which combines relation-aware and common features using dense neural networks. In the relation-aware feature extraction, we attempt to create relational inductive bias by applying an encoding function specific to each relation. In the common feature extraction, we apply the common encoding function to all input embeddings. These encoding functions are implemented using dense layers in ComDensE. ComDensE achieves the state-of-the-art performance in the link prediction in terms of MRR, HIT@1 on FB15k-237 and HIT@1 on WN18RR compared to the previous baseline approaches. We conduct an extensive ablation study to examine the effects of the relation-aware layer and the common layer of the ComDensE. Experimental results illustrate that the combined dense architecture as implemented in ComDensE achieves the best performance.
A Data Science Pipeline for Algorithmic Trading: A Comparative Study of Applications for Finance and Cryptoeconomics
Venue: arXiv cs.HC 2022
Year: 2022

Recent advances in Artificial Intelligence (AI) have made algorithmic trading play a central role in finance. However, current research and applications are disconnected information islands. We propose a generally applicable pipeline for designing, programming, and evaluating the algorithmic trading of stock and crypto assets. Moreover, we demonstrate how our data science pipeline works with respect to four conventional algorithms: the moving average crossover, volume-weighted average price, sentiment analysis, and statistical arbitrage algorithms. Our study offers a systematic way to program, evaluate, and compare different trading strategies. Furthermore, we implement our algorithms through object-oriented programming in Python3, which serves as open-source software for future academic research and applications.
GSCLIP : A Framework for Explaining Distribution Shifts in Natural Language
Venue: arXiv cs.CL 2022
Year: 2022

Helping end users comprehend the abstract distribution shifts can greatly facilitate AI deployment. Motivated by this, we propose a novel task, dataset explanation. Given two image data sets, dataset explanation aims to automatically point out their dataset-level distribution shifts with natural language. Current techniques for monitoring distribution shifts provide inadequate information to understand datasets with the goal of improving data quality. Therefore, we introduce GSCLIP, a training-free framework to solve the dataset explanation task. In GSCLIP, we propose the selector as the first quantitative evaluation method to identify explanations that are proper to summarize dataset shifts. Furthermore, we leverage this selector to demonstrate the superiority of a generator based on language model generation. Systematic evaluation on natural data shift verifies that GSCLIP, a combined system of a hybrid generator group and an efficient selector is not only easy-to-use but also powerful for dataset explanation at scale.
Interpretable Melody Generation from Lyrics with Discrete-Valued Adversarial Training
Venue: arXiv cs.SD 2022
Year: 2022

Generating melody from lyrics is an interesting yet challenging task in the area of artificial intelligence and music. However, the difficulty of keeping the consistency between input lyrics and generated melody limits the generation quality of previous works. In our proposal, we demonstrate our proposed interpretable lyrics-to-melody generation system which can interact with users to understand the generation process and recreate the desired songs. To improve the reliability of melody generation that matches lyrics, mutual information is exploited to strengthen the consistency between lyrics and generated melodies. Gumbel-Softmax is exploited to solve the non-differentiability problem of generating discrete music attributes by Generative Adversarial Networks (GANs). Moreover, the predicted probabilities output by the generator is utilized to recommend music attributes. Interacting with our lyrics-to-melody generation system, users can listen to the generated AI song as well as recreate a new song by selecting from recommended music attributes.
Learning-Based Near-Orthogonal Superposition Code for MIMO Short Message Transmission
Venue: arXiv cs.IT 2022
Year: 2022

Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning-based near-orthogonal superposition (NOS) coding scheme is proposed to transmit short messages in multiple-input multiple-output (MIMO) channels for mMTC applications. In the proposed MIMO-NOS scheme, a neural network-based encoder is optimized via end-to-end learning with a corresponding neural network-based detector/decoder in a superposition-based auto-encoder framework including a MIMO channel. The proposed MIMO-NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector and reshaped for the space-time transmission. For the receiver, we propose a novel looped K-best tree-search algorithm with cyclic redundancy check (CRC) assistance to enhance the error correcting ability in the block-fading MIMO channel. Simulation results show the proposed MIMO-NOS scheme outperforms maximum likelihood (ML) MIMO detection combined with a polar code with CRC-assisted list decoding by 1-2 dB in various MIMO systems for short (32-64 bit) message transmission.
Learning-Aided Beam Prediction in mmWave MU-MIMO Systems for High-Speed Railway
Venue: arXiv cs.IT 2022
Year: 2022

The problem of beam alignment and tracking in high mobility scenarios such as high-speed railway (HSR) becomes extremely challenging, since large overhead cost and significant time delay are introduced for fast time-varying channel estimation. To tackle this challenge, we propose a learning-aided beam prediction scheme for HSR networks, which predicts the beam directions and the channel amplitudes within a period of future time with fine time granularity, using a group of observations. Concretely, we transform the problem of high-dimensional beam prediction into a two-stage task, i.e., a low-dimensional parameter estimation and a cascaded hybrid beamforming operation. In the first stage, the location and speed of a certain terminal are estimated by maximum likelihood criterion, and a data-driven data fusion module is designed to improve the final estimation accuracy and robustness. Then, the probable future beam directions and channel amplitudes are predicted, based on the HSR scenario priors including deterministic trajectory, motion model, and channel model. Furthermore, we incorporate a learnable non-linear mapping module into the overall beam prediction to allow non-linear tracks. Both of the proposed learnable modules are model-based and have a good interpretability. Compared to the existing beam management scheme, the proposed beam prediction has (near) zero overhead cost and time delay. Simulation results verify the effectiveness of the proposed scheme.
Transmissive RIS for 6G Communications: Design, Prototyping, and Experimental Demonstrations
Venue: arXiv cs.IT 2022
Year: 2022

Reconfigurable intelligent surface (RIS) has been widely considered as a key technique to improve spectral efficiency for 6G communications. Compared with most existing research that only focuses on the reflective RIS, the design and prototyping of a novel transmissive RIS are presented in this paper, and its enhancement to the RIS-aided communication system is experimentally demonstrated. The 2-bit transmissive RIS element utilizes the penetration structure, which combines a 1-bit current reversible dipole and a 90° digital phase shifter based on a quadrature hybrid coupler. A transmissive RIS prototype with 16$\times$16 elements is designed, fabricated, and measured to verify the proposed design. The measured phase shift and insertion loss of the RIS element validate the 2-bit phase modulation capability. Being illuminated by a horn feed, the prototype achieves a maximum broadside gain of 22.0 dBi at 27 GHz, and the two-dimensional beamforming capability with scan angles up to $\pm$60° is validated. The experimental results of the RIS-aided communication system verify that by introducing the extra gain and beam steering capability, the transmissive RIS is able to achieve a higher data rate, reduce the transmit power, improve the transmission capability through obstacles, and dynamically adapt to the signal propagation direction.
The Body Scaling Effect and Its Impact on Physics Plausibility
Venue: arXiv cs.MM 2022
Year: 2022

In this study we investigated the effect of body ownership illusion-based body scaling on physics plausibility in Virtual Reality (VR). Our interest was in examining whether body ownership illusion-based body scaling could affect the plausibility of rigid body dynamics similarly to altering VR users' scale by manipulating their virtual interpupillary distance and viewpoint height. The procedure involved the conceptual replication of two previous studies. We investigated physics plausibility with 40 participants under two conditions. In our synchronous condition, we used visuo-tactile stimuli to elicit a body ownership illusion of inhabiting an invisible doll-sized body on participants reclining on an exam table. Our asynchronous condition was otherwise similar, but the visuo-tactile stimuli were provided asynchronously to prevent the onset of the body ownership illusion. We were interested in whether the correct approximation of physics (true physics) or physics that are incorrect and appearing as if the environment is five times larger instead (movie physics) appear more realistic to participants as a function of body scale. We found that movie physics did appear more realistic to participants under the body ownership illusion condition. However, our hypothesis that true physics would appear more realistic in the asynchronous condition was unsupported. Our exploratory analyses revealed that movie physics were perceived as plausible under both conditions. Moreover, we were not able to replicate previous findings from literature concerning object size estimations while inhabiting a small invisible body. However, we found a significant opposite effect regarding size estimations; the object sizes were on average underestimated during the synchronous visuo-tactile condition when compared to the asynchronous condition.
Domain Adaptive Pretraining for Multilingual Acronym Extraction
Venue: arXiv cs.CL 2022
Year: 2022

This paper presents our findings from participating in the multilingual acronym extraction shared task SDU@AAAI-22. The task consists of acronym extraction from documents in 6 languages within scientific and legal domains. To address multilingual acronym extraction we employed BiLSTM-CRF with multilingual XLM-RoBERTa embeddings. We pretrained the XLM-RoBERTa model on the shared task corpus to further adapt XLM-RoBERTa embeddings to the shared task domain(s). Our system (team: SMR-NLP) achieved competitive performance for acronym extraction across all the languages.
Signed ego network model and its application to Twitter
Venue: arXiv cs.SI 2022
Year: 2022

The Ego Network Model (ENM) describes how individuals organise their social relations in concentric circles (typically five) of decreasing intimacy, and it has been found almost ubiquitously in social networks, both offline and online. The ENM gauges the tie strength between peers in terms of interaction frequency, which is easy to measure and provides a good proxy for the time spent nurturing the relationship. However, advances in signed network analysis have shown that positive and negative relations play very different roles in network dynamics. For this reason, this work sets out to investigate the ENM when including signed relations. The main contributions of this paper are twofold: firstly, a novel method of signing relationships between individuals using sentiment analysis and, secondly, an investigation of the properties of Signed Ego Networks (Ego Networks with signed connections). Signed Ego Networks are then extracted for the users of eight different Twitter datasets composed of both specialised users (e.g. journalists) and generic users. We find that negative links are over-represented in the active part of the Ego Networks of all types of users, suggesting that Twitter users tend to engage regularly with negative connections. Further, we observe that negative relationships are overwhelmingly predominant in the Ego Network circles of specialised users, hinting at very polarised online interactions for this category of users. In addition, negative relationships are found disproportionately more at the more intimate levels of the ENM for journalists, while their percentages are stable across the circles of the other Twitter users