Reviewerly/exHarmony huggingface dataset:
A Baseline Readability Model for Cebuano

In this study, we developed the first baseline readability model for the Cebuano language.Cebuano is the second most-used native language in the Philippines with about 27.5 million speakers.As the baseline, we extracted traditional or surface-based features, syllable patterns based from Cebuano's documented orthography, and neural embeddings from the multilingual BERT model.Results show that the use of the first two handcrafted linguistic features obtained the best performance trained on an optimized Random Forest model with approximately 87% across all metrics.The feature sets and algorithm used also is similar to previous results in readability assessment for the Filipino language-showing potential of crosslingual application.To encourage more work for readability assessment in Philippine languages such as Cebuano, we open-sourced both code and data 1 .
Improving Constituent Representation with Hypertree Neural Networks

Many natural language processing tasks involve text spans and thus high-quality span representations are needed to enhance neural approaches to these tasks.Most existing methods of span representation are based on simple derivations (such as max-pooling) from word representations and do not utilize compositional structures of natural language.In this paper, we aim to improve representations of constituent spans using a novel hypertree neural networks (HTNN) that is structured with constituency parse trees.Each node in the HTNN represents a constituent of the input sentence and each hyperedge represents a composition of smaller child constituents into a larger parent constituent.In each update iteration of the HTNN, the representation of each constituent is computed based on all the hyperedges connected to it, thus incorporating both bottom-up and top-down compositional information.We conduct comprehensive experiments to evaluate HTNNs against other span representation models and the results show the effectiveness of HTNN.
DD-TIG at SemEval-2022 Task 5: Investigating the Relationships Between Multimodal and Unimodal Information in Misogynous Memes Detection and Classification

This paper describes our submission for task 5 Multimedia Automatic Misogyny Identification (MAMI) at SemEval-2022.The task is designed to detect and classify misogynous memes.To utilize both textual and visual information presented in a meme, we investigate several of the most recent visuallanguage transformer-based multimodal models and choose ERNIE-ViL-Large as our base model.For subtask A, with observations of models' overfitting on unimodal patterns, strategies are proposed to mitigate problems of biased words and template memes.For subtask B, we transform this multi-label problem into a multi-class one and experiment with oversampling and complementary techniques.Our approach places 2nd for subtask A and 5th for subtask B in this competition.
Improving Neural Models for Radiology Report Retrieval with Lexicon-based Automated Annotation

Many clinical informatics tasks that are based on electronic health records (EHR) need relevant patient cohorts to be selected based on findings, symptoms and diseases.Frequently, these conditions are described in radiology reports which can be retrieved using information retrieval (IR) methods.The latest of these techniques utilize neural IR models such as BERT trained on clinical text.However, these methods still lack semantic understanding of the underlying clinical conditions as well as ruled out findings, resulting in poor precision during retrieval.In this paper we combine clinical finding detection with supervised query match learning.Specifically, we use lexicon-driven concept detection to detect relevant findings in sentences.These findings are used as queries to train a Sentence-BERT (SBERT) model using triplet loss on matched and unmatched query-sentence pairs.We show that the proposed supervised training task remarkably improves the retrieval performance of SBERT.The trained model generalizes well to unseen queries and reports from different collections.
Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative

In this paper, we investigate ensemble methods for fine-tuning transformer-based pretrained models for clinical natural language processing tasks, specifically temporal relation extraction from the clinical narrative.Our experimental results on the THYME data show that ensembling as a fine-tuning strategy can further boost model performance over single learners optimized for hyperparameters.Dynamic snapshot ensembling is particularly beneficial as it finetunes a wide array of parameters and results in a 2.8% absolute improvement in F1 over the base single learner.
On Assessing and Developing Spoken ’Grammatical Error Correction’ Systems

Spoken 'grammatical error correction' (SGEC) is an important process to provide feedback for second language learning.Due to a lack of end-to-end training data, SGEC is often implemented as a cascaded, modular system, consisting of speech recognition, disfluency removal, and grammatical error correction (GEC).This cascaded structure enables efficient use of training data for each module.It is, however, difficult to compare and evaluate the performance of individual modules as preceeding modules may introduce errors.For example the GEC module input depends on the output of nonnative speech recognition and disfluency detection, both challenging tasks for learner data.This paper focuses on the assessment and development of SGEC systems.We first discuss metrics for evaluating SGEC, both individual modules and the overall system.The systemlevel metrics enable tuning for optimal system performance.A known issue in cascaded systems is error propagation between modules.To mitigate this problem semi-supervised approaches and self-distillation are investigated.Lastly, when SGEC system gets deployed it is important to give accurate feedback to users.Thus, we apply filtering to remove edits with low-confidence, aiming to improve overall feedback precision.The performance metrics are examined on a Linguaskill multi-level data set, which includes the original non-native speech, manual transcriptions and reference grammatical error corrections, to enable system analysis and development.
AlexU-AL at SemEval-2022 Task 6: Detecting Sarcasm in Arabic Text Using Deep Learning Techniques

Sarcasm detection is an important task in Natural Language Understanding.Sarcasm is a form of verbal irony that occurs when there is a discrepancy between the literal and intended meanings of an expression.In this paper, we use the tweets of the Arabic dataset provided by SemEval-2022 task 6 to train deep learning classifiers to solve the sub-tasks A and C associated with the dataset.Sub-task A is to determine if the tweet is sarcastic or not.For sub-task C, given a sarcastic text and its non-sarcastic rephrase, i.e. two texts that convey the same meaning, determine which is the sarcastic one.In our solution, we utilize finetuned MARBERT (Abdul-Mageed et al., 2021) model with an added single linear layer on top for classification.The proposed solution achieved 0.5076 F1-sarcastic in Arabic sub-task A, accuracy of 0.7450 and F-score of 0.7442 in Arabic sub-task C. We achieved the 2 nd and the 9 th places for Arabic sub-tasks A and C respectively.
Interpreting Emoji with Emoji

We study the extent to which emoji can be used to add interpretability to embeddings of text and emoji.To do so, we extend the POLAR-framework that transforms word embeddings to interpretable counterparts and apply it to word-emoji embeddings trained on four years of messaging data from the Jodel social network.We devise a crowdsourced human judgement experiment to study six usecases, evaluating against words only, what role emoji can play in adding interpretability to word embeddings.That is, we use a revised POLAR approach interpreting words and emoji with words, emoji or both according to human judgement.We find statistically significant trends demonstrating that emoji can be used to interpret other emoji very well.
Mitigating Learnerese Effects for CEFR Classification

The role of an author's L1 in SLA can be challenging for automated CEFR classification, in that texts from different L1 groups may be too heterogeneous to combine them as training data.We experiment with recent debiasing approaches by attempting to devoid textual representations of L1 features.This results in a more homogeneous group when aggregating CEFRannotated texts from different L1 groups, leading to better classification performance.Using iterative null-space projection, we marginally improve classification performance for a linear classifier by 1 point.An MLP (e.g.non-linear) classifier remains unaffected by this procedure.We discuss possible directions of future work to attempt to increase this performance gain.
Zuo Zhuan Ancient Chinese Dataset for Word Sense Disambiguation

Word Sense Disambiguation (WSD) is a core task in Natural Language Processing (NLP).Ancient Chinese has rarely been used in WSD tasks, however, as no public dataset for ancient Chinese WSD tasks exists.Creation of an ancient Chinese dataset is considered a significant challenge because determining the most appropriate sense in a context is difficult and time-consuming owing to the different usages in ancient and modern Chinese.Actually, no public dataset for ancient Chinese WSD tasks exists.To solve the problem of ancient Chinese WSD, we annotate part of Pre-Qin (221 BC) text Zuo Zhuan using a copyright-free dictionary to create a public sense-tagged dataset.Then, we apply a simple Nearest Neighbors (k-NN) method using a pre-trained language model to the dataset.Our code and dataset will be available on GitHub 1 .
ANALISIS USABILITY MENGGUNAKAN METODE HEURISTIC EVALUATION PADA APLIKASI TOKO ONLINE

This study tries to analyze the Usability of the Shopee applications based on the heuristic evaluation method in order to determine the shortcomings of the Shopee applications based on ten heuristic principles. The purpose of this study is to determine the level of user satisfaction in the Shopee applications, as well as to provide advice to those concerned in this case the Shopee applications by recommending the results of the usability evaluation test on the Shopee applications. In this study, the method used to perform Usability analysis on the Shopee applications is by using a questionnaire via Google Form, from the questionnaire a calculation basis is to determine Usability or the user interface in using the Shopee applications. Based on the results of the average percentage of respondents using the heuristic evaluation method on the Shopee applications, the Shopee application has a percentage value of 97.35%. From these results it is known that the level of Usability on the Shopee applications is very good.
Nighttime Traffic Light Detection with Arrow Signal Recognition Using a Vision and Learning Based Approach

With the rapid development of autonomous driving technologies, traffic light detection and recognition is one of the indispensable techniques for intelligent vehicles. This paper presents vision and learning based methods for nighttime traffic light detection with arrow signal recognition. In our two-stage framework, the traffic lights are first detected from the fusion of high illumination encoding and tone-mapped images. An SVM is then used to classify the arrow signals based on the log-polar transform of green light images. Since there is no public traffic light dataset exclusively for the night scenes, we collect our data for network training and testing. The proposed techniques are carried out on the real scene images for performance evaluation. In the experiments, the high accuracy recognition results have demonstrated the feasibility of our algorithms for the nighttime traffic light detection.
UAV Inspection Monitoring and Management System based on Wireless Sensing Technology

With the continuous progress of society, the economic level has also been greatly improved, and people are paying more and more attention to intelligent and humanized management in their lives. At the same time, environmental problems and hidden safety hazards are increasing, posing a serious threat to the safety of human life and property. People also have a higher pursuit for the quality of life. In order to ensure that personal, property and other related information are not leaked, and to ensure that the national security construction is not affected, security is one of the important issues. However, traditional monitoring and management work still has various problems. For example, due to the limitations of human eyes and monitoring equipment, the perception of changes in the surrounding environment is not timely and accurate, which leads to false alarms or accidents, inaccurate or incomplete data collection, and so on. The drone inspection technology can break through these limitations and reduce labor costs. This paper adopts experimental investigation method and data analysis method, and aims to further discover obstacles in the inspection work of UA V and optimize the UA V inspection monitoring and management system through the research of wireless sensing technology. According to the research results, the test system has a high packet loss rate when there is no routing node, and the probability of sending data packets again is high, which greatly increases the energy load of the measurement node. In the case of better signal quality, the retransmission rate of the system is also lower, and the energy loss is relatively low.
Emoji semantics/pragmatics: investigating commitment and lying

This paper presents the results of two experiments investigating the directness of emoji in constituting speaker meaning. This relationship is examined in two ways, with Experiment 1 testing whether speakers are committed to meanings they communicate via a single emoji and Experiment 2 testing whether that speaker is taken to have lied if that meaning is false and intended to deceive. Results indicate that emoji with high meaning agreement in general (i.e., pictorial representations of concrete objects or foods) reliably commit the speaker to that meaning and can constitute lying. Expressive emoji representing facial expressions and emotional states demonstrate a range of commitment and lie ratings: those with high meaning agreement constitute more commitment and more of a lie than those with less meaning agreement in the first place. Emoji can constitute speaker commitment and they can be lies, but this result does not apply uniformly to all emoji and is instead tied to agreement, conventionality, and lexicalization.
SentSpace: Large-Scale Benchmarking and Evaluation of Text using Cognitively Motivated Lexical, Syntactic, and Semantic Features

SentSpace is a modular framework for streamlined evaluation of text.SentSpace characterizes textual input using diverse lexical, syntactic, and semantic features derived from corpora and psycholinguistic experiments.Core sentence features fall into three primary feature spaces: 1) Lexical, 2) Contextual, and 3) Embeddings.To aid in the analysis of computed features, SentSpace provides a web interface for interactive visualization and comparison with text from large corpora.The modular design of SentSpace allows researchers to easily integrate their own feature computation into the pipeline while benefiting from a common framework for evaluation and visualization.In this manuscript we will describe the design of SentSpace, its core feature spaces, and demonstrate an example use case by comparing human-written and machine-generated (GPT2-XL) sentences to each other.We find that while GPT2-XL-generated text appears fluent at the surface level, psycholinguistic norms and measures of syntactic processing reveal key differences between text produced by humans and machines.Thus, SentSpace provides a broad set of cognitively motivated linguistic features for evaluation of text within natural language processing, cognitive science, as well as the social sciences.
Knowledge extraction from aeronautical messages (NOTAMs) with self-supervised language models for aircraft pilots

During their pre-flight briefings, aircraft pilots must analyze a long list of NOTAMs (NOtice To AirMen) indicating potential hazards along the flight route, sometimes up to 100 pages for long-haul flights.NOTAM free-text fields typically have a very special phrasing, with lots of acronyms and domain-specific vocabulary, which makes it differ significantly from standard English.In this paper, we pretrain language models derived from BERT on circa 1 million unlabeled NOTAMs and reuse the learnt representations on three downstream tasks valuable for pilots: criticality prediction, named entity recognition and translation into a structured language called Airlang.This selfsupervised approach, where smaller amounts of labeled data are enough for task-specific finetuning, is well suited in the aeronautical context since expert annotations are expensive and time-consuming.We present evaluation scores across the tasks showing a high potential for an operational usability of such models (by pilots, airlines or service providers), which is a first to the best of our knowledge.
YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models

This paper presents a deep learning system that contends at SemEval-2022 Task 5.The goal is to detect the existence of misogynous memes in sub-task A. At the same time, the advanced multi-label sub-task B categorizes the misogyny of misogynous memes into one of four types: stereotype, shaming, objectification, and violence.The Ensemble technique has been used for three multi-modal deep learning models: two MMBT models and VisualBERT.Our proposed system ranked 17 th place out of 83 participant teams with an F1-score of 0.722 in sub-task A, which shows a significant performance improvement over the baseline model's F1-score of 0.65.
Robust Conversational Agents against Imperceptible Toxicity Triggers

Warning: this paper contains content that may be offensive or upsetting.Recent research in Natural Language Processing (NLP) has advanced the development of various toxicity detection models with the intention of identifying and mitigating toxic language from existing systems.Despite the abundance of research in this area, less attention has been given to adversarial attacks that force the system to generate toxic language and the defense against them.Existing work to generate such attacks is either based on humangenerated attacks which is costly and not scalable or, in case of automatic attacks, the attack vector does not conform to human-like language, which can be detected using a language model loss.In this work, we propose attacks against conversational agents that are imperceptible, i.e., they fit the conversation in terms of coherency, relevancy, and fluency, while they are effective and scalable, i.e., they can automatically trigger the system into generating toxic language.We then propose a defense mechanism against such attacks which not only mitigates the attack but also attempts to maintain the conversational flow.Through automatic and human evaluations, we show that our defense is effective at avoiding toxic language generation even against imperceptible toxicity triggers while the generated language fits the conversation in terms of coherency and relevancy.Lastly, we establish the generalizability of such a defense mechanism on language generation models beyond conversational agents.
Multilingualism Encourages Recursion: a Transfer Study with mBERT

The present work constitutes an attempt to investigate the relational structures learnt by mBERT, a multilingual transformer-based network, with respect to different cross-linguistic regularities proposed in the fields of theoretical and quantitative linguistics.We pursued this objective by relying on a zero-shot transfer experiment, evaluating the model's ability to generalize its native task to artificial languages that could either respect or violate some proposed language universal, and comparing its performance to the output of BERT, a monolingual model with an identical configuration.We created four artificial corpora through a Probabilistic Context-Free Grammar by manipulating the distribution of tokens and the structure of their dependency relations.We showed that while both models were favoured by a Zipfian distribution of the tokens and by the presence of head-dependency type structures, the multilingual transformer network exhibited a stronger reliance on hierarchical cues compared to its monolingual counterpart.
Tesla at SemEval-2022 Task 4: Patronizing and Condescending Language Detection using Transformer-based Models with Data Augmentation

This paper describes our system for Task 4 of SemEval 2022: Patronizing and Condescending Language (PCL) Detection.For sub-task 1, where the objective is to classify a text as PCL or non-PCL, we use a T5 Model finetuned on the dataset.For sub-task 2, which is a multi-label classification problem, we use a RoBERTa model fine-tuned on the dataset.Given that the key challenge in this task is classification on an imbalanced dataset, our models rely on an augmented dataset that we generate using paraphrasing.We found that these two models yield the best results out of all the other approaches we tried.
Distantly Supervised Aspect Clustering And Naming For E-Commerce Reviews

Product aspect extraction from reviews is a critical task for e-commerce services to understand customer preferences and pain points.While aspect phrases extraction and sentiment analysis have received a lot of attention, clustering of aspect phrases and assigning human readable names to clusters in e-commerce reviews is an extremely important and challenging problem due to the scale of the reviews that makes human review infeasible.In this paper, we propose fully automated methods for clustering aspect words and generating human readable names for the clusters without any manually labeled data.We train transformer based sentence embeddings that are aware of unique e-commerce language characteristics (eg.incomplete sentences, spelling and grammar errors, vernacular etc.).We also train transformer based sequence to sequence models to generate human readable aspect names from clusters.Both the models are trained using heuristic based distant supervision.Additionally, the models are used to improve each other.Extensive empirical testing showed that the clustering model improves the Silhouette Score by 64% when compared to the state-ofthe-art baseline and the aspect naming model achieves a high ROUGE-L score of 0.79.
AISFG: Abundant Information Slot Filling Generator

As an essential component of task-oriented dialogue systems, slot filling requires enormous labeled training data in a certain domain.However, in most cases, there is little or no target domain training data is available in the training stage.Thus, cross-domain slot filling has to cope with the data scarcity problem by zero/few-shot learning.Previous researches on zero/few-shot cross-domain slot filling focus on slot descriptions and examples while ignoring the slot type ambiguity and example ambiguity issues.To address these problems, we propose Abundant Information Slot Filling Generator (AISFG), a generative model with a novel query template that incorporates domain descriptions, slot descriptions, and examples with context.Experimental results show that our model outperforms state-of-the-art approaches in zero/few-shot slot filling task.