Reviewerly/exHarmony huggingface dataset:
Multi-branch Attentive Transformer

While the multi-branch architecture is one of the key ingredients to the success of computer vision tasks, it has not been well investigated in natural language processing, especially sequence learning tasks. In this work, we propose a simple yet effective variant of Transformer called multi-branch attentive Transformer (briefly, MAT), where the attention layer is the average of multiple branches and each branch is an independent multi-head attention layer. We leverage two training techniques to regularize the training: drop-branch, which randomly drops individual branches during training, and proximal initialization, which uses a pre-trained Transformer model to initialize multiple branches. Experiments on machine translation, code generation and natural language understanding demonstrate that such a simple variant of Transformer brings significant improvements. Our code is available at \url{https://github.com/HA-Transformer}.
Beyond Triplet Loss: Meta Prototypical N-tuple Loss for Person  Re-identification

Person Re-identification (ReID) aims at matching a person of interest across images. In convolutional neural network (CNN) based approaches, loss design plays a vital role in pulling closer features of the same identity and pushing far apart features of different identities. In recent years, triplet loss achieves superior performance and is predominant in ReID. However, triplet loss considers only three instances of two classes in per-query optimization (with an anchor sample as query) and it is actually equivalent to a two-class classification. There is a lack of loss design which enables the joint optimization of multiple instances (of multiple classes) within per-query optimization for person ReID. In this paper, we introduce a multi-class classification loss, i.e., N-tuple loss, to jointly consider multiple (N) instances for per-query optimization. This in fact aligns better with the ReID test/inference process, which conducts the ranking/comparisons among multiple instances. Furthermore, for more efficient multi-class classification, we propose a new meta prototypical N-tuple loss. With the multi-class classification incorporated, our model achieves the state-of-the-art performance on the benchmark person ReID datasets.
Evaluating Transport Protocols on 5G for Mobile Augmented Reality

Mobile Augmented Reality (MAR) mixes physical environments with user-interactive virtual annotations. Immersive MAR experiences are supported by computation-intensive tasks which rely on offloading mechanisms to ease device workloads. However, this introduces additional network traffic which in turn influences the motion-to-photon latency (a determinant of user-perceived quality of experience). Therefore, a proper transport protocol is crucial to minimise transmission latency and ensure sufficient throughput to support MAR performance. Relatedly, 5G, a potential MAR supporting technology, is widely believed to be smarter, faster, and more efficient than its predecessors. However, the suitability and performance of existing transport protocols in MAR in the 5G context has not been explored. Therefore, we present an evaluation of popular transport protocols, including UDP, TCP, MPEG-TS, RTP, and QUIC, with a MAR system on a real-world 5G testbed. We also compare with their 5G performance with LTE and WiFi. Our evaluation results indicate that TCP has the lowest round-trip-time on 5G, with a median of $15.09\pm0.26$ ms, while QUIC appears to perform better on LTE. Through an additional test with varying signal quality (specifically, degrading secondary synchronisation signal reference signal received quality), we discover that protocol performance appears to be significantly impacted by signal quality.
Automatic Text Summarization of COVID-19 Medical Research Articles using  BERT and GPT-2

With the COVID-19 pandemic, there is a growing urgency for medical community to keep up with the accelerating growth in the new coronavirus-related literature. As a result, the COVID-19 Open Research Dataset Challenge has released a corpus of scholarly articles and is calling for machine learning approaches to help bridging the gap between the researchers and the rapidly growing publications. Here, we take advantage of the recent advances in pre-trained NLP models, BERT and OpenAI GPT-2, to solve this challenge by performing text summarization on this dataset. We evaluate the results using ROUGE scores and visual inspection. Our model provides abstractive and comprehensive information based on keywords extracted from the original articles. Our work can help the the medical community, by providing succinct summaries of articles for which the abstract are not already available.
Multi-Center Federated Learning

Federated learning has received great attention for its capability to train a large-scale model in a decentralized manner without needing to access user data directly. It helps protect the users' private data from centralized collecting. Unlike distributed machine learning, federated learning aims to tackle non-IID data from heterogeneous sources in various real-world applications, such as those on smartphones. Existing federated learning approaches usually adopt a single global model to capture the shared knowledge of all users by aggregating their gradients, regardless of the discrepancy between their data distributions. However, due to the diverse nature of user behaviors, assigning users' gradients to different global models (i.e., centers) can better capture the heterogeneity of data distributions across users. Our paper proposes a novel multi-center aggregation mechanism for federated learning, which learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. We formulate the problem as a joint optimization that can be efficiently solved by a stochastic expectation maximization (EM) algorithm. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.
Multi-modal Datasets for Super-resolution

Nowdays, most datasets used to train and evaluate super-resolution models are single-modal simulation datasets. However, due to the variety of image degradation types in the real world, models trained on single-modal simulation datasets do not always have good robustness and generalization ability in different degradation scenarios. Previous work tended to focus only on true-color images. In contrast, we first proposed real-world black-and-white old photo datasets for super-resolution (OID-RW), which is constructed using two methods of manually filling pixels and shooting with different cameras. The dataset contains 82 groups of images, including 22 groups of character type and 60 groups of landscape and architecture. At the same time, we also propose a multi-modal degradation dataset (MDD400) to solve the super-resolution reconstruction in real-life image degradation scenarios. We managed to simulate the process of generating degraded images by the following four methods: interpolation algorithm, CNN network, GAN network and capturing videos with different bit rates. Our experiments demonstrate that not only the models trained on our dataset have better generalization capability and robustness, but also the trained images can maintain better edge contours and texture features.
On the Effect of Dropping Layers of Pre-trained Transformer Models

Transformer-based NLP models are trained using hundreds of millions or even billions of parameters, limiting their applicability in computationally constrained environments. While the number of parameters generally correlates with performance, it is not clear whether the entire network is required for a downstream task. Motivated by the recent work on pruning and distilling pre-trained models, we explore strategies to drop layers in pre-trained models, and observe the effect of pruning on downstream GLUE tasks. We were able to prune BERT, RoBERTa and XLNet models up to 40%, while maintaining up to 98% of their original performance. Additionally we show that our pruned models are on par with those built using knowledge distillation, both in terms of size and performance. Our experiments yield interesting observations such as, (i) the lower layers are most critical to maintain downstream task performance, (ii) some tasks such as paraphrase detection and sentence similarity are more robust to the dropping of layers, and (iii) models trained using a different objective function exhibit different learning patterns and w.r.t the layer dropping.
TextGAIL: Generative Adversarial Imitation Learning for Text Generation

Generative Adversarial Networks (GANs) for text generation have recently received many criticisms, as they perform worse than their MLE counterparts. We suspect previous text GANs' inferior performance is due to the lack of a reliable guiding signal in their discriminators. To address this problem, we propose a generative adversarial imitation learning framework for text generation that uses large pre-trained language models to provide more reliable reward guidance. Our approach uses contrastive discriminator, and proximal policy optimization (PPO) to stabilize and improve text generation performance. For evaluation, we conduct experiments on a diverse set of unconditional and conditional text generation tasks. Experimental results show that TextGAIL achieves better performance in terms of both quality and diversity than the MLE baseline. We also validate our intuition that TextGAIL's discriminator demonstrates the capability of providing reasonable rewards with an additional task.
A Novel Framework with Information Fusion and Neighborhood Enhancement  for User Identity Linkage

User identity linkage across social networks is an essential problem for cross-network data mining. Since network structure, profile and content information describe different aspects of users, it is critical to learn effective user representations that integrate heterogeneous information. This paper proposes a novel framework with INformation FUsion and Neighborhood Enhancement (INFUNE) for user identity linkage. The information fusion component adopts a group of encoders and decoders to fuse heterogeneous information and generate discriminative node embeddings for preliminary matching. Then, these embeddings are fed to the neighborhood enhancement component, a novel graph neural network, to produce adaptive neighborhood embeddings that reflect the overlapping degree of neighborhoods of varying candidate user pairs. The importance of node embeddings and neighborhood embeddings are weighted for final prediction. The proposed method is evaluated on real-world social network data. The experimental results show that INFUNE significantly outperforms existing state-of-the-art methods.
Self-driving simulation scene reconstruction using self-supervised depth completion

The rapid development and testing of autonomous driving technology heavily rely on several complex and diverse simulation scenarios. Nowadays, the most common way to generate simulation scenes is using computer graphic techniques or game engines. However, this method requires manual creation of various on-scene elements, such as vehicles, pedestrians, traffic signs. Moreover, material properties, have limited scalability, and require a lot of time and effort. In this paper, we propose a simple and effective method to reconstruct realistic self-driving scenes, more realistic and accurate than other methods that generate scenes using real data. Our method uses self-supervised depth completion to fuse multimodal information from image and LiDAR data, preserving both the rich textures and colors in the scene while retaining accurate distances and shapes. Furthermore, we provide valid values for places in LiDAR with no data by enriching and enhancing the detail and realism of the reconstruction results. Additionally, our depth completion neural network is also self-supervised, which avoids the difficulty of labeling large data. Extensive experiments on the KITTI urban and Cross-country datasets show that our method outperforms other state-of-the-art methods in both depth completion accuracy and scene reconstruction effect.
Sliced at SemEval-2022 Task 11: Bigger, Better? Massively Multilingual LMs for Multilingual Complex NER on an Academic GPU Budget

Massively multilingual language models (MMLMs) have become a widely-used language representation method, and multiple large MMLMs were proposed in recent years.A trend is to train MMLMs on larger text corpora or with more layers.In this paper we set out to evaluate recent popular MMLMs on detecting semantically ambiguous and complex named entities with an academic GPU budget.Our submission of a single model for 11 languages on the SemEval Task 11 MultiCoNER shows that a fine-tuned XLM-R large outperforms the more recent RemBERT, ranking 9th from 26 submissions in the multilingual track.Compared to Rem-BERT, the XLM-R model has the additional advantage to fit on a slice of a multi-instance GPU.As contrary to expectations and recent findings, we found RemBERT to not be the best MMLM, we further set out to investigate this discrepancy with additional experiments on multilingual Wikipedia NER data.While we expected RemBERT to have an edge on that dataset as it is closer to its pre-training data, surprisingly, our results show that this is not the case.
Exploiting Social Media Content for Self-Supervised Style Transfer

Recent research on style transfer takes inspiration from unsupervised neural machine translation (UNMT), learning from large amounts of non-parallel data by exploiting cycle consistency loss, back-translation, and denoising autoencoders.By contrast, the use of selfsupervised NMT (SSNMT), which leverages (near) parallel instances hidden in non-parallel data more efficiently than UNMT, has not yet been explored for style transfer.In this paper we present a novel Self-Supervised Style Transfer (3ST) model, which augments SS-NMT with UNMT methods in order to identify and efficiently exploit supervisory signals in non-parallel social media posts.We compare 3ST with state-of-the-art (SOTA) style transfer models across civil rephrasing, formality and polarity tasks.We show that 3ST is able to balance the three major objectives (fluency, content preservation, attribute transfer accuracy) the best, outperforming SOTA models on averaged performance across their tested tasks in automatic and human evaluation.
Towards Debiasing Translation Artifacts

Cross-lingual natural language processing relies on translation, either by humans or machines, at different levels, from translating training data to translating test sets.However, compared to original texts in the same language, translations possess distinct qualities referred to as translationese.Previous research has shown that these translation artifacts influence the performance of a variety of crosslingual tasks.In this work, we propose a novel approach to reducing translationese by extending an established bias-removal technique.We use the Iterative Null-space Projection (INLP) algorithm, and show by measuring classification accuracy before and after debiasing, that translationese is reduced at both sentence and word level.We evaluate the utility of debiasing translationese on a natural language inference (NLI) task, and show that by reducing this bias, NLI accuracy improves.To the best of our knowledge, this is the first study to debias translationese as represented in latent embedding space.
"Wait, I'm Still Talking!" Predicting the Dialogue Interaction Behavior  Using Imagine-Then-Arbitrate Model

Producing natural and accurate responses like human beings is the ultimate goal of intelligent dialogue agents. So far, most of the past works concentrate on selecting or generating one pertinent and fluent response according to current query and its context. These models work on a one-to-one environment, making one response to one utterance each round. However, in real human-human conversations, human often sequentially sends several short messages for readability instead of a long message in one turn. Thus messages will not end with an explicit ending signal, which is crucial for agents to decide when to reply. So the first step for an intelligent dialogue agent is not replying but deciding if it should reply at the moment. To address this issue, in this paper, we propose a novel Imagine-then-Arbitrate (ITA) neural dialogue model to help the agent decide whether to wait or to make a response directly. Our method has two imaginator modules and an arbitrator module. The two imaginators will learn the agent's and user's speaking style respectively, generate possible utterances as the input of the arbitrator, combining with dialogue history. And the arbitrator decides whether to wait or to make a response to the user directly. To verify the performance and effectiveness of our method, we prepared two dialogue datasets and compared our approach with several popular models. Experimental results show that our model performs well on addressing ending prediction issue and outperforms baseline models.
Interpreting video features: a comparison of 3D convolutional networks  and convolutional LSTM networks

A number of techniques for interpretability have been presented for deep learning in computer vision, typically with the goal of understanding what the networks have based their classification on. However, interpretability for deep video architectures is still in its infancy and we do not yet have a clear concept of how to decode spatiotemporal features. In this paper, we present a study comparing how 3D convolutional networks and convolutional LSTM networks learn features across temporally dependent frames. This is the first comparison of two video models that both convolve to learn spatial features but have principally different methods of modeling time. Additionally, we extend the concept of meaningful perturbation introduced by \cite{MeaningFulPert} to the temporal dimension, to identify the temporal part of a sequence most meaningful to the network for a classification decision. Our findings indicate that the 3D convolutional model concentrates on shorter events in the input sequence, and places its spatial focus on fewer, contiguous areas.
First Steps Towards Automatic Question Generation and Assessment of LL(1) Grammars

Automatic question generation and the assessment of procedural knowledge is still a challenging research topic. This article focuses on case of it, the LL(1) grammar design. This is a well known technique for construct a top-down parser. There are many tools that given a context-free grammar can construct the LL(1) tables, but they are not designed for assessment. This article describes an application that covers all the tasks needed to automatize the assessment process.
HITSZ-HLT at SemEval-2022 Task 10: A Span-Relation Extraction Framework for Structured Sentiment Analysis

This paper describes our system that participated in the SemEval-2022 Task 10: Structured Sentiment Analysis, which aims to extract opinion tuples from texts.A full opinion tuple generally contains an opinion holder, an opinion target, the sentiment expression, and the corresponding polarity.The complex structure of the opinion tuple makes the task challenging.To address this task, we formalize it as a span-relation extraction problem and propose a two-stage extraction framework accordingly.In the first stage, we employ the span module to enumerate spans and then recognize the type of every span.In the second stage, we employ the relation module to determine the relation between spans.Our system achieves competitive results and ranks among the top-10 systems in almost subtasks.
On the Evaluation of BDD Requirements with Text-based Metrics: The ETCS-L3 Case Study

A proper requirement definition phase is of a paramount importance in software engineering. It is the first and prime mean to realize efficient and reliable systems. System requirements are usually formulated and expressed in natural language, given its universality and ease of communication and writing. Unfortunately, natural language can be a source of ambiguity, complexity and omissions, which may cause system failures. Among the different approaches proposed by the software engineering community, Behavioural-Driven Development (BDD) is affirming as a valid, practical method to structure effective and non-ambiguous requirement specifications. The paper tackles with the problem of measuring requirements in BDD by assessing some traditional Natural Language Processing-related metrics with respect to a sample excerpt of requirement specification rewritten according to the BDD criteria. This preliminary assessment is made on the ERTMS-ETCS Level 3 case study whose specification, up to this date, is not managed by a standardisation body. The paper demonstrates the necessity of novel metrics able to cope with the BDD specification paradigms.
Improving Arabic Hate Speech Identification Using Online Machine Learning and Deep Learning Models

Due to the rising use of social media platforms on a global scale to interact and express thoughts freely, the spread of hate speech has become very noticeable on these platforms. Governments, organizations, and academic institutions have all spent substantially on discovering effective solutions to handle this issue. Numerous researches have been performed in several languages to find automated methods for identifying hate speech, but there has been minimal work done in Arabic. The findings of a performance evaluation of two machine learning models, namely the passive-aggressive classifier (PAC) and the Bidirectional Gated Recurrent Unit (Bi-GRU) augmented with an attention layer, are investigated in this work. Proposed models are developed and evaluated using a multi-platform Arabic hate speech dataset. We employ term frequency-inverse document frequency (TF-IDF) and Arabic word embeddings for feature extraction techniques after running a variety of pre-processing steps. The experimental results reveal that the two proposed models (PAC, Bi-GRU with attention layer) provide an accuracy of 98.4% and 99.1%, respectively, outperforming existing methods reported in the literature.
Adaptive Multi-attention for Image Sentence Generator Using C-LSTM

Capturing image feature and multi-object region of an image and transferring it into a Natural Language Sentence is a research issue needs to be addressed with natural language processing. Technically, the attention mechanism will force every word representation to an corresponding image region, however at times it do neglect certain words like ‘the’ in the description text, as it misleads the text interpretation. The captioning of an image involves not only detecting the features from various images, but also decoding the collaborations between the items into significant image text. The focus of the suggested work, predicts the image sentence in a more detailed way for every region/frame of an image. To overcome, an image feature extraction is carried out using CNN and LSTM for the image text generation with the help of adaptive attention mechanism, which will be add in the layer of LSTM to predict better image sentence is constructed. The above mentioned deep network methods have been analyzed using two output combination. Experiments have been implemented using Flickr8k dataset. The implementation analysis illustrates that adaptive attention performs significantly better than without adaptive attention of image sentence model and generates more meaningful captions compared to any of the individual models used. From the results on test images, the suggested network gives the accuracy, bleu score with and without using adaptive attention in the LSTM of 81.53, 61.94 and 73.53, 57.94%.