Miracle-dz/UniFAR-FTUs huggingface dataset:
Unsupervised Deep Language and Dialect Identification for Short Texts

Automatic Language Identification (LI) or Dialect Identification (DI) of short texts of closely related languages or dialects, is one of the primary steps in many natural language processing pipelines. Language identification is considered a solved task in many cases; however, in the case of very closely related languages, or in an unsupervised scenario (where the languages are not known in advance), performance is still poor. In this paper, we propose the Unsupervised Deep Language and Dialect Identification (UDLDI) method, which can simultaneously learn sentence embeddings and cluster assignments from short texts. The UDLDI model understands the sentence constructions of languages by applying attention to character relations which helps to optimize the clustering of languages. We have performed our experiments on three short-text datasets for different language families, each consisting of closely related languages or dialects, with very minimal training sets. Our experimental evaluations on these datasets have shown significant improvement over state-of-the-art unsupervised methods and our model has outperformed state-of-the-art LI and DI systems in supervised settings.
PRHLT-UPV at SemEval-2020 Task 12: BERT for Multilingual Offensive Language Detection

The present paper describes the system submitted by the PRHLT-UPV team for the task 12 of SemEval-2020: OffensEval 2020. The official title of the task is Multilingual Offensive Language Identification in Social Media, and aims to identify offensive language in texts. The languages included in the task are English, Arabic, Danish, Greek and Turkish. We propose a model based on the BERT architecture for the analysis of texts in English. The approach leverages knowledge within a pre-trained model and performs fine-tuning for the particular task. In the analysis of the other languages the Multilingual BERT is used, which has been pre-trained for a large number of languages. In the experiments, the proposed method for English texts is compared with other approaches to analyze the relevance of the architecture used. Furthermore, simple models for the other languages are evaluated to compare them with the proposed one. The experimental results show that the model based on BERT outperforms other approaches. The main contribution of this work lies in this study, despite not obtaining the first positions in most cases of the competition ranking.
Fake news detection for the Russian language

In this paper, we trained and compared different models for fake news detection in Russian. For this task, we used such language features as bag-of-n-grams and bag of Rhetorical Structure Theory features, and BERT embeddings. We also compared the score of our models with the human score on this task and showed that our models deal with fake news detection better. We investigated the nature of fake news by dividing it into two non-overlapping classes: satire and fake news. As a result, we obtained the set of models for fake news detection; the best of these models achieved 0.889 F1-score on the test set for 2 classes and 0.9076 F1-score on 3 classes task.
DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection

This paper describes our team work and submission for the SemEval 2020 (Sub-Task A) “Offensive Eval: Identifying and Categorizing Offensive Arabic Language in Arabic Social Media”. Our two baseline models were based on different levels of representation: character vs. word level. In word level based representation we implemented a convolutional neural network model and a bi-directional GRU model. In character level based representation we implemented a hyper CNN and LSTM model. All of these models have been further augmented with attention layers for a better performance on our task. We also experimented with three types of static word embeddings: word2vec, FastText, and Glove, in addition to emoji embeddings, and compared the performance of the different deep learning models on the dataset provided by this task. The bi-directional GRU model with attention has achieved the highest score (0.85% F1 score) among all other models.
Style versus Content: A distinction without a (learnable) difference?

Textual style transfer involves modifying the style of a text while preserving its content. This assumes that it is possible to separate style from content. This paper investigates whether this separation is possible. We use sentiment transfer as our case study for style transfer analysis. Our experimental methodology frames style transfer as a multi-objective problem, balancing style shift with content preservation and fluency. Due to the lack of parallel data for style transfer we employ a variety of adversarial encoder-decoder networks in our experiments. Also, we use of a probing methodology to analyse how these models encode style-related features in their latent spaces. The results of our experiments which are further confirmed by a human evaluation reveal the inherent trade-off between the multiple style transfer objectives which indicates that style cannot be usefully separated from content within these style-transfer systems.
A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

Various methods have already been proposed for learning entity embeddings from text descriptions. Such embeddings are commonly used for inferring properties of entities, for recommendation and entity-oriented search, and for injecting background knowledge into neural architectures, among others. Entity embeddings essentially serve as a compact encoding of a similarity relation, but similarity is an inherently multi-faceted notion. By representing entities as single vectors, existing methods leave it to downstream applications to identify these different facets, and to select the most relevant ones. In this paper, we propose a model that instead learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. We use a mixture-of-experts formulation to jointly learn these facet-specific embeddings. The individual entity embeddings are learned using a variant of the GloVe model, which has the advantage that we can easily identify which properties are modelled well in which of the learned embeddings. This is exploited by an associated gating network, which uses pre-trained word vectors to encourage the properties that are modelled by a given embedding to be semantically coherent, i.e. to encourage each of the individual embeddings to capture a meaningful facet.
UAIC1860 at SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles

The “Detection of Propaganda Techniques in News Articles” task at the SemEval 2020 competition focuses on detecting and classifying propaganda, pervasive in news article. In this paper, we present a system able to evaluate on sentence level, three traditional text representation techniques for these study goals, using: tf*idf, word and character n-grams. Firstly, we built a binary classifier able to provide corresponding propaganda labels, propaganda or non-propaganda. Secondly, we build a multilabel multiclass model to identify applied propaganda.
Annotating argumentation in Swedish social media

This paper presents a small study of annotating argumentation in Swedish social media. Annotators were asked to annotate spans of argumentation in 9 threads from two discussion forums. At the post level, Cohen’s k and Krippendorff’s alpha 0.48 was achieved. When manually inspecting the annotations the annotators seemed to agree when conditions in the guidelines were explicitly met, but implicit argumentation and opinions, resulting in annotators having to interpret what’s missing in the text, caused disagreements.
Bridging the Gap in Multilingual Semantic Role Labeling: a Language-Agnostic Approach

Recent research indicates that taking advantage of complex syntactic features leads to favorable results in Semantic Role Labeling. Nonetheless, an analysis of the latest state-of-the-art multilingual systems reveals the difficulty of bridging the wide gap in performance between high-resource (e.g., English) and low-resource (e.g., German) settings. To overcome this issue, we propose a fully language-agnostic model that does away with morphological and syntactic features to achieve robustness across languages. Our approach outperforms the state of the art in all the languages of the CoNLL-2009 benchmark dataset, especially whenever a scarce amount of training data is available. Our objective is not to reject approaches that rely on syntax, rather to set a strong and consistent language-independent baseline for future innovations in Semantic Role Labeling. We release our model code and checkpoints at https://github.com/SapienzaNLP/multi-srl.
LITL at SMM4H: An Old-school Feature-based Classifier for Identifying Adverse Effects in Tweets

This paper describes our participation to the SMM4H shared task 2. We designed a rule-based classifier that estimates whether a tweet mentions an adverse effect associated to a medication. Our system addresses English and French, and is based on a number of specific word lists and features. These cues were mostly obtained through an extensive corpus analysis of the provided training data. Different weighting schemes were tested (manually tuned or based on a logistic regression), the best one achieving a F1 score of 0.31 for English and 0.15 for French.
Processing the diffusion-weighted magnetic resonance imaging of the PING dataset

Diffusion-weighted magnetic resonance imaging (dMRI) allows for the in-vivo assessment of anatomical white matter in the brain, thus allowing the depiction of structural connectivity. Using structural processing techniques and related methods, a growing body of literature has illustrated that connectomics is a crucial aspect to assessing the brain in health and disease. The Pediatric Imaging Neurocognition and Genetics (PING) dataset was collected and released openly to contribute to the assessment of typical brain development in a pediatric sample. This current work details the processing of diffusion-weighted images from the PING dataset, including rigorous quality assessment and fine-tuning of parameters at every step, to increase the accessibility of these data for connectomic analysis. This processing provides state-of-the-art diffusion measures, both classical diffusion tensor imaging (DTI) and more advanced HARDI-based metrics, enabling the evaluation not only of structural white matter but also of integrated multimodal analyses, i.e. combining structural information from dMRI with functional or gray matter analyses.
Zyy1510 Team at SemEval-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text with Sub-word Level Representations

This paper reports the zyy1510 team’s work in the International Workshop on Semantic Evaluation (SemEval-2020) shared task on Sentiment analysis for Code-Mixed (Hindi-English, English-Spanish) Social Media Text. The purpose of this task is to determine the polarity of the text, dividing it into one of the three labels positive, negative and neutral. To achieve this goal, we propose an ensemble model of word n-grams-based Multinomial Naive Bayes (MNB) and sub-word level representations in LSTM (Sub-word LSTM) to identify the sentiments of code-mixed data of Hindi-English and English-Spanish. This ensemble model combines the advantage of rich sequential patterns and the intermediate features after convolution from the LSTM model, and the polarity of keywords from the MNB model to obtain the final sentiment score. We have tested our system on Hindi-English and English-Spanish code-mixed social media data sets released for the task. Our model achieves the F1 score of 0.647 in the Hindi-English task and 0.682 in the English-Spanish task, respectively.
Neural Approaches for Natural Language Interfaces to Databases: A Survey

A natural language interface to databases (NLIDB) enables users without technical expertise to easily access information from relational databases. Interest in NLIDBs has resurged in the past years due to the availability of large datasets and improvements to neural sequence-to-sequence models. In this survey we focus on the key design decisions behind current state of the art neural approaches, which we group into encoder and decoder improvements. We highlight the three most important directions, namely linking question tokens to database schema elements (schema linking), better architectures for encoding the textual query taking into account the schema (schema encoding), and improved generation of structured queries using autoregressive neural models (grammar-based decoders). To foster future research, we also present an overview of the most important NLIDB datasets, together with a comparison of the top performing neural models and a short insight into recent non deep learning solutions.
Harnessing Cross-lingual Features to Improve Cognate Detection for Low-resource Languages

Cognates are variants of the same lexical form across different languages; for example “fonema” in Spanish and “phoneme” in English are cognates, both of which mean “a unit of sound”. The task of automatic detection of cognates among any two languages can help downstream NLP tasks such as Cross-lingual Information Retrieval, Computational Phylogenetics, and Machine Translation. In this paper, we demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian Languages. Our approach introduces the use of context from a knowledge graph to generate improved feature representations for cognate detection. We, then, evaluate the impact of our cognate detection mechanism on neural machine translation (NMT), as a downstream task. We evaluate our methods to detect cognates on a challenging dataset of twelve Indian languages, namely, Sanskrit, Hindi, Assamese, Oriya, Kannada, Gujarati, Tamil, Telugu, Punjabi, Bengali, Marathi, and Malayalam. Additionally, we create evaluation datasets for two more Indian languages, Konkani and Nepali. We observe an improvement of up to 18% points, in terms of F-score, for cognate detection. Furthermore, we observe that cognates extracted using our method help improve NMT quality by up to 2.76 BLEU. We also release our code, newly constructed datasets and cross-lingual models publicly.
yama, a mutant allele of Mov10l1, disrupts retrotransposon silencing and piRNA biogenesis

Piwi-interacting RNAs (piRNAs) play critical roles in protecting germline genome integrity and promoting normal spermiogenic differentiation. In mammals, there are two populations of piRNAs: pre-pachytene and pachytene piRNAs. Transposon-rich pre-pachytene piRNAs are expressed in fetal and perinatal germ cells and are required for retrotransposon silencing, whereas transposon-poor pachytene piRNAs are expressed in spermatocytes and round spermatids and regulate mRNA transcript levels. MOV10L1, a germ cell-specific RNA helicase, is essential for the production of both populations of piRNAs. Although the requirement of the RNA helicase domain located in the MOV10L1 C-terminal region for piRNA biogenesis is well known, its large N-terminal region remains mysterious. Here we report a novel Mov10l1 mutation in the Mov10l1 N-terminal region named yama. The yama mutation results in a single amino acid substitution V229E. The yama mutation causes meiotic arrest, de-repression of transposable elements, and male sterility because of defects in pre-pachytene piRNA biogenesis. Moreover, restricting the Mov10l1 mutation effects to later stages in germ cell development by combining with a postnatal conditional deletion of a complementing wild-type allele causes absence of pachytene piRNAs, accumulation of piRNA precursors, polar conglomeration of piRNA pathway proteins in spermatocytes, and spermiogenic arrest. Mechanistically, the V229E substitution in MOV10L1 reduces its interaction with PLD6, an endonuclease that generates the 5′ ends of piRNA intermediates. Our results uncover an important role for the MOV10L1-PLD6 interaction in piRNA biogenesis throughout male germ cell development. Author Summary Small non-coding RNAs play critical roles in silencing of exogenous viruses, endogenous retroviruses, and transposable elements, and also play multifaceted roles in controlling gene expression. Piwi-interacting RNAs (piRNAs) are found in gonads in diverse species from flies to humans. An evolutionarily conserved function of piRNAs is to silence transposable elements through an adaptive mechanism and thus to protect the germline genome integrity. In mammals, piRNAs also provide a poorly understood function to regulate postmeiotic differentiation of spermatids. More than two dozen proteins are involved in the piRNA pathway. MOV10L1, a germ-cell-specific RNA helicase, binds to piRNA precursors to initiate piRNA biogenesis. Here we have identified a single amino acid substitution (V229E) in MOV10L1 in the yama mutant. When constitutively expressed as the only source of MOV10L1 throughout germ cell development, the yama mutation abolishes piRNA biogenesis, de-silences transposable elements, and causes meiotic arrest. When the mutant phenotype is instead revealed only later in germ cell development by conditionally inactivating a complementing wild-type copy of the gene, the point mutant abolishes formation of later classes of piRNAs and again disrupts germ cell development. Point mutations in MOV10L1 may thus contribute to male infertility in humans.
Is Killed More Significant than Fled? A Contextual Model for Salient Event Detection

Identifying the key events in a document is critical to holistically understanding its important information. Although measuring the salience of events is highly contextual, most previous work has used a limited representation of events that omits essential information. In this work, we propose a highly contextual model of event salience that uses a rich representation of events, incorporates document-level information and allows for interactions between latent event encodings. Our experimental results on an event salience dataset demonstrate that our model improves over previous work by an absolute 2-4% on standard metrics, establishing a new state-of-the-art performance for the task. We also propose a new evaluation metric that addresses flaws in previous evaluation methodologies. Finally, we discuss the importance of salient event detection for the downstream task of summarization.
ECNU at SemEval-2020 Task 7: Assessing Humor in Edited News Headlines Using BiLSTM with Attention

In this paper we describe our system submitted to SemEval 2020 Task 7: “Assessing Humor in Edited News Headlines”. We participated in all subtasks, in which the main goal is to predict the mean funniness of the edited headline given the original and the edited headline. Our system involves two similar sub-networks, which generate vector representations for the original and edited headlines respectively. And then we do a subtract operation of the outputs from two sub-networks to predict the funniness of the edited headline.
Hacking the diversity of SARS-CoV-2 and SARS-like coronaviruses in human, bat and pangolin populations

In 2019, a novel coronavirus, SARS-CoV-2/nCoV-19, emerged in Wuhan, China, and has been responsible for the current COVID-19 pandemic. The evolutionary origins of the virus remain elusive and understanding its complex mutational signatures could guide vaccine design and development. As part of the international “CoronaHack” in April 2020 (https://www.coronahack.co.uk/), we employed a collection of contemporary methodologies to compare the genomic sequences of coronaviruses isolated from human (SARS-CoV-2;n=163), bat (bat-CoV;n=215) and pangolin (pangolin-CoV;n=7) available in public repositories. Following de novo gene annotation prediction, analysis on gene-gene similarity network, codon usage bias and variant discovery were carried out. Strong host-associated divergences were noted in ORF3a, ORF6, ORF7a, ORF8 and S, and in codon usage bias profiles. Lastly, we have characterised several high impact variants (inframe insertion/deletion or stop gain) in bat-CoV and pangolin-CoV populations, some of which are found in the same amino acid position and may be highlighting loci of potential functional relevance.
Pharyngeal Pumping and Tissue-Specific Transgenic P-Glycoprotein Expression Influence Macrocyclic Lactone Susceptibility in Caenorhabditis elegans

Macrocyclic lactone (ML) resistance has emerged in many parasitic nematodes including the pathogenic horse roundworm Parascaris univalens. The underlying mechanism of ML resistance and the drug penetration routes into the nematodes remain to be elucidated. Drug efflux by P-glycoproteins is considered a potential resistance mechanism but conclusive functional evidence is lacking. To this end, we used a motility assay modified to stimulate pharyngeal pumping (PP) by bacteria or serotonin and tissue-specific expression of Pun- PGP-9 in the free-living model nematode Caenorhabditis elegans. Here, stimulation of PP was identified as an important factor for C. elegans ML susceptibility, increasing the EC50 values of ivermectin by up to 11.1-fold and of moxidectin by 1.2-fold. In this context, intestinal Pun-PGP-9 expression elicited a protective effect against ivermectin and moxidectin only in the presence of PP stimulation, increasing the EC50 values by approximately 3- to 4-fold (ivermectin) or by < 1.3-fold (moxidectin). Conversely, epidermal Pun-PGP-9 expression protected against moxidectin regardless of PP with EC50 fold changes below 1.5 but against ivermectin with a considerable 2.9-fold EC50 increase only when the drug is not actively ingested. Our results highlight the role of active drug ingestion by nematodes for susceptibility and provide conclusive functional evidence for a contribution of P-glycoproteins to ML resistance. Author Summary Parasitic nematode infections pose a serious threat to animal health, in particular in light of the widespread anthelmintic resistance in different nematode species. In equines, the roundworm Parascaris univalens is a major pathogen of foals, exhibiting widespread resistance against macrocyclic lactones (MLs). This represents a particular challenge to animal health, but the underlying mechanisms and drug penetration routes remain mostly unknown. P-glycoprotein ABC-transporters have been linked to ML resistance in several parasitic nematodes. Here we demonstrate by tissue-specific overexpression of Pun-PGP-9 in the free-living model nematode Caenorhabditis elegans their ability to reduce the susceptibility to two commonly used MLs, ivermectin and moxidectin. At the same time, active drug ingestion by pharyngeal pumping (PP) strongly enhanced ivermectin and moderately effects moxidectin susceptibility. In more detail, the effect of intestinal or epidermal Pun-PGP-9 was dependent on active drug ingestion. These observations indicate differences in the drug penetration routes between ML derivatives and allow novel insight into the functional role of P-glycoproteins.