Miracle-dz/UniFAR-FTUs huggingface dataset:
Segregated Co-activation Patterns in the Emergence of Decision Confidence During Visual Perception

Visual metacognition—the introspection and evaluation of one’s own visual perceptual processes—is measured through both decision confidence and “metacognitive efficiency. ” Metacognitive efficiency refers to an individual’s ability to accurately judge incorrect and correct decisions through confidence ratings given their task performance. Previous imaging studies in humans and nonhuman primates reported widely distributed brain regions being involved in decision confidence and metacognition. However, the neural correlates of metacognition are remarkably inconsistent across studies concerning spatial outline. Therefore, this study investigates the neural correlates of visual metacognition by examining co-activation across regions that scale with visual decision confidence. We hypothesized that interacting processes of perceptual and metacognitive performance contribute to the arising decision confidence in distributed, but segregable co-activating brain regions. To test this hypothesis, we performed task-fMRI in healthy humans during a visual backward masking task with four-scale, post-decision confidence ratings. We measured blood oxygenation covariation patterns, which served as a physiological proxy for co-activation across brain regions. Decision confidence ratings and an individual’s metacognitive efficiency served as behavioral measures for metacognition. We found three distinct co-activation clusters involved in decision confidence: the first included right-centered fronto-temporal-parietal regions, the second included left temporal and parietal regions, and the left basal forebrain (BF), and the third included cerebellar regions. The right fronto-temporal-parietal cluster including the supplementary eye field and the right basal forebrain showed stronger co-activation in subjects with higher metacognitive efficiency. Our results provide novel evidence for co-activation of widely distributed fronto-parieto-temporal regions involved in visual confidence. The supplementary eye field was the only region that activated for both decision confidence and metacognitive efficiency, suggesting the supplementary eye field plays a key role in visual metacognition. Our results link findings in electrophysiology studies and human fMRI studies and provide evidence that confidence estimates arise from the integration of multiple information processing pathways.
Multidrug Resistance and Virulence Factors of Escherichia coli Harboring Plasmid-Mediated Colistin Resistance: mcr-1 and mcr-3 Genes in Contracted Pig Farms in Thailand

The presence of the plasmid-mediated colistin resistance encoding mcr gene family in the Enterobacteriaceae is one of the crucial global concerns. The use of colistin in livestock rearing is believed to be the cause of mcr gene spreading and is of impact to public health. The objective of this research was to detect the frequency and virulent genes of mcr-positive Escherichia coli (MCRPE) in fecal samples from healthy pigs in a contract farming system across Thailand. A total of 696 pooled samples were derived from 80 farms, located in 49 provinces across six regions of Thailand. The colistin-resistant E. coli were identified by MALDI-TOF mass spectrometry and antimicrobial susceptibility testing by broth microdilution. The antibiogram was determined using an automated susceptibility machine, and the genetic characteristics were investigated for mcr-1–5 genes, phylogenetic group, replicon types, and virulent genes. In total, 31 of 696 samples were positive, with E. coli containing mcr-1 or combination of mcr-1 and mcr-3 with incidence of 4.45 and 0.43%. Phylogenetic groups A and B1 and the IncF and IncFIB replicon types were predominantly found in the MCRPE located in the central area, with multidrug-resistant traits against 3–14 types of antimicrobials. Additionally, 19 of 31 isolates identified as enterotoxigenic E. coli were with the stap and stb (enterotoxin-encoding genes). In conclusion, a low carriage rate of mcr-positive E. coli was detected in the large-scale farming of healthy pigs. The association between multidrug-resistant MCRPE and their pathogenic potential should be of concern.
Histogenetic Radial Models as Aids to Understanding Complex Brain Structures: The Amygdalar Radial Model as a Recent Example

The radial dimension expands during central nervous system development after the proliferative neuroepithelium is molecularly patterned. The process is associated with neurogenesis, radial glia scaffolding, and migration of immature neurons into the developing mantle stratum. Radial histogenetic units, defined as a delimited neural polyclone whose cells share the same molecular profile, are molded during these processes, and usually become roughly stratified into periventricular, intermediate, and superficial (subpial) strata wherein neuronal cell types may differ and be distributed in various patterns. Cell-cell adhesion or repulsion phenomena together with interaction with local intercellular matrix cues regulate the acquisition of nuclear, reticular, or layer histogenetic forms in such strata. Finally, the progressive addition of inputs and outputs soon follows the purely neurogenetic and radial migratory phase. Frequently there is heterochrony in the radial development of adjacent histogenetic units, apart of peculiarities in differentiation due to non-shared aspects of the respective molecular profiles. Tangential migrations may add complexity to radial unit cytoarchitecture and function. The study of the contributions of such genetically controlled radial histogenetic units to the emerging complex neural structure is a key instrument to understand central nervous system morphology and function. One recent example in this scenario is the recently proposed radial model of the mouse pallial amygdala. This is theoretically valid generally in mammals (Garcia-Calero et al., 2020), and subdivides the nuclear complex of the pallial amygdala into five main radial units. The approach applies a novel ad hoc amygdalar section plane, given the observed obliquity of the amygdalar radial glial framework. The general relevance of radial unit studies for clarifying structural analysis of all complex brain regions such as the pallial amygdala is discussed, with additional examples.
Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation

This paper is concerned with improving dialogue generation models through injection of knowledge, e.g., content relevant to the post that can increase the quality of responses. Past research extends the training of the generative models by incorporating statistical properties of posts, responses and related knowledge, without explicitly assessing the knowledge quality. In our work, we demonstrate the importance of knowledge relevance and adopt a two-phase approach. We first apply a novel method, Transformer & Post based Posterior Approximation (TPPA) to select knowledge, and then use the Transformer with Expanded Decoder (TED) model to generate responses from both the post and the knowledge. TPPA method processes posts, post related knowledge, and response related knowledge at both word and sentence level. Our experiments with the TED generative model demonstrate the effectiveness of TPPA as it outperforms a set of strong baseline models. Our TPPA method is extendable and supports further optimization of knowledge retrieval and injection.
Service registration chatbot: collecting and comparing dialogues from AMT workers and service’s users

Crowdsourcing is the go-to solution for data collection and annotation in the context of NLP tasks. Nevertheless, crowdsourced data is noisy by nature; the source is often unknown and additional validation work is performed to guarantee the dataset’s quality. In this article, we compare two crowdsourcing sources on a dialogue paraphrasing task revolving around a chatbot service. We observe that workers hired on crowdsourcing platforms produce lexically poorer and less diverse rewrites than service users engaged voluntarily. Notably enough, on dialogue clarity and optimality, the two paraphrase sources’ human-perceived quality does not differ significantly. Furthermore, for the chatbot service, the combined crowdsourced data is enough to train a transformer-based Natural Language Generation (NLG) system. To enable similar services, we also release tools for collecting data and training the dialogue-act-based transformer-based NLG module.
Identifying and Measuring Annotator Bias Based on Annotators’ Demographic Characteristics

Machine learning is recently used to detect hate speech and other forms of abusive language in online platforms. However, a notable weakness of machine learning models is their vulnerability to bias, which can impair their performance and fairness. One type is annotator bias caused by the subjective perception of the annotators. In this work, we investigate annotator bias using classification models trained on data from demographically distinct annotator groups. To do so, we sample balanced subsets of data that are labeled by demographically distinct annotators. We then train classifiers on these subsets, analyze their performances on similarly grouped test sets, and compare them statistically. Our findings show that the proposed approach successfully identifies bias and that demographic features, such as first language, age, and education, correlate with significant performance differences.
Fast End-to-end Coreference Resolution for Korean

Recently, end-to-end neural network-based approaches have shown significant improvements over traditional pipeline-based models in English coreference resolution. However, such advancements came at a cost of computational complexity and recent works have not focused on tackling this problem. Hence, in this paper, to cope with this issue, we propose BERT-SRU-based Pointer Networks that leverages the linguistic property of head-final languages. Applying this model to the Korean coreference resolution, we significantly reduce the coreference linking search space. Combining this with Ensemble Knowledge Distillation, we maintain state-of-the-art performance 66.9% of CoNLL F1 on ETRI test set while achieving 2x speedup (30 doc/sec) in document processing time.
Simple Hierarchical Multi-Task Neural End-To-End Entity Linking for Biomedical Text

Recognising and linking entities is a crucial first step to many tasks in biomedical text analysis, such as relation extraction and target identification. Traditionally, biomedical entity linking methods rely heavily on heuristic rules and predefined, often domain-specific features. The features try to capture the properties of entities and complex multi-step architectures to detect, and subsequently link entity mentions. We propose a significant simplification to the biomedical entity linking setup that does not rely on any heuristic methods. The system performs all the steps of the entity linking task jointly in either single or two stages. We explore the use of hierarchical multi-task learning, using mention recognition and entity typing tasks as auxiliary tasks. We show that hierarchical multi-task models consistently outperform single-task models when trained tasks are homogeneous. We evaluate the performance of our models on the biomedical entity linking benchmarks using MedMentions and BC5CDR datasets. We achieve state-of-theart results on the challenging MedMentions dataset, and comparable results on BC5CDR.
Learning Informative Representations of Biomedical Relations with Latent Variable Models

Extracting biomedical relations from large corpora of scientific documents is a challenging natural language processing task. Existing approaches usually focus on identifying a relation either in a single sentence (mention-level) or across an entire corpus (pair-level). In both cases, recent methods have achieved strong results by learning a point estimate to represent the relation; this is then used as the input to a relation classifier. However, the relation expressed in text between a pair of biomedical entities is often more complex than can be captured by a point estimate. To address this issue, we propose a latent variable model with an arbitrarily flexible distribution to represent the relation between an entity pair. Additionally, our model provides a unified architecture for both mention-level and pair-level relation extraction. We demonstrate that our model achieves results competitive with strong baselines for both tasks while having fewer parameters and being significantly faster to train. We make our code publicly available.
Event Extraction as Multi-turn Question Answering

Event extraction, which aims to identify event triggers of pre-defined event types and their arguments of specific roles, is a challenging task in NLP. Most traditional approaches formulate this task as classification problems, with event types or argument roles taken as golden labels. Such approaches fail to model rich interactions among event types and arguments of different roles, and cannot generalize to new types or roles. This work proposes a new paradigm that formulates event extraction as multi-turn question answering. Our approach, MQAEE, casts the extraction task into a series of reading comprehension problems, by which it extracts triggers and arguments successively from a given sentence. A history answer embedding strategy is further adopted to model question answering history in the multi-turn process. By this new formulation, MQAEE makes full use of dependency among arguments and event types, and generalizes well to new types with new argument roles. Empirical results on ACE 2005 shows that MQAEE outperforms current state-of-the-art, pushing the final F1 of argument extraction to 53.4% (+2.0%). And it also has a good generalization ability, achieving competitive performance on 13 new event types even if trained only with a few samples of them.
Advancing Seq2seq with Joint Paraphrase Learning

We address the problem of model generalization for sequence to sequence (seq2seq) architectures. We propose going beyond data augmentation via paraphrase-optimized multi-task learning and observe that it is useful in correctly handling unseen sentential paraphrases as inputs. Our models greatly outperform SOTA seq2seq models for semantic parsing on diverse domains (Overnight - up to 3.2% and emrQA - 7%) and Nematus, the winning solution for WMT 2017, for Czech to English translation (CzENG 1.6 - 1.5 BLEU).
Effective Crowd-Annotation of Participants, Interventions, and Outcomes in the Text of Clinical Trial Reports

The search for Participants, Interventions, and Outcomes (PIO) in clinical trial reports is a critical task in Evidence Based Medicine. For an automatic PIO extraction, high-quality corpora are needed. Obtaining such a corpus from crowdworkers, however, has been shown to be ineffective since (i) workers usually lack domain-specific expertise to conduct the task with sufficient quality, and (ii) the standard approach of annotating entire abstracts of trial reports as one task-instance (i.e. HIT) leads to an uneven distribution in task effort. In this paper, we switch from entire abstract to sentence annotation, referred to as the SenBase approach. We build upon SenBase in SenSupport, where we compensate the lack of domain-specific expertise of crowdworkers by showing for each task-instance similar sentences that are already annotated by experts. Such tailored task-instance examples are retrieved via unsupervised semantic short-text similarity (SSTS) method – and we evaluate nine methods to find an effective solution for SenSupport. We compute the Cohen’s Kappa agreement between crowd-annotations and gold standard annotations and show that (i) both sentence-based approaches outperform a Baseline approach where entire abstracts are annotated; (ii) supporting annotators with tailored task-instance examples is the best performing approach with Kappa agreements of 0.78/0.75/0.69 for P, I, and O respectively.
Truth or Error? Towards systematic analysis of factual errors in abstractive summaries

This paper presents a typology of errors produced by automatic summarization systems. The typology was created by manually analyzing the output of four recent neural summarization systems. Our work is motivated by the growing awareness of the need for better summary evaluation methods that go beyond conventional overlap-based metrics. Our typology is structured into two dimensions. First, the Mapping Dimension describes surface-level errors and provides insight into word-sequence transformation issues. Second, the Meaning Dimension describes issues related to interpretation and provides insight into breakdowns in truth, i.e., factual faithfulness to the original text. Comparative analysis revealed that two neural summarization systems leveraging pre-trained models have an advantage in decreasing grammaticality errors, but not necessarily factual errors. We also discuss the importance of ensuring that summary length and abstractiveness do not interfere with evaluating summary quality.
Structural and Functional Decomposition for Personality Image Captioning in a Communication Game

Personality image captioning (PIC) aims to describe an image with a natural language caption given a personality trait. In this work, we introduce a novel formulation for PIC based on a communication game between a speaker and a listener. The speaker attempts to generate natural language captions while the listener encourages the generated captions to contain discriminative information about the input images and personality traits. In this way, we expect that the generated captions can be improved to naturally represent the images and express the traits. In addition, we propose to adapt the language model GPT2 to perform caption generation for PIC. This enables the speaker and listener to benefit from the language encoding capacity of GPT2. Our experiments show that the proposed model achieves the state-of-the-art performance for PIC.
Cross-Lingual Dependency Parsing by POS-Guided Word Reordering

We propose a novel approach to cross-lingual dependency parsing based on word reordering. The words in each sentence of a source language corpus are rearranged to meet the word order in a target language under the guidance of a part-of-speech based language model (LM). To obtain the highest reordering score under the LM, a population-based optimization algorithm and its genetic operators are designed to deal with the combinatorial nature of such word reordering. A parser trained on the reordered corpus then can be used to parse sentences in the target language. We demonstrate through extensive experimentation that our approach achieves better or comparable results across 25 target languages (1.73% increase in average), and outperforms a baseline by a significant margin on the languages that are greatly different from the source one. For example, when transferring the English parser to Hindi and Latin, our approach outperforms the baseline by 15.3% and 6.7% respectively.
Generalization to Mitigate Synonym Substitution Attacks

Studies have shown that deep neural networks (DNNs) are vulnerable to adversarial examples – perturbed inputs that cause DNN-based models to produce incorrect results. One robust adversarial attack in the NLP domain is the synonym substitution. In attacks of this variety, the adversary substitutes words with synonyms. Since synonym substitution perturbations aim to satisfy all lexical, grammatical, and semantic constraints, they are difficult to detect with automatic syntax check as well as by humans. In this paper, we propose a structure-free defensive method that is capable of improving the performance of DNN-based models with both clean and adversarial data. Our findings show that replacing the embeddings of the important words in the input samples with the average of their synonyms’ embeddings can significantly improve the generalization of DNN-based classifiers. By doing so, we reduce model sensitivity to particular words in the input samples. Our results indicate that the proposed defense is not only capable of defending against adversarial attacks, but is also capable of improving the performance of DNN-based models when tested on benign data. On average, the proposed defense improved the classification accuracy of the CNN and Bi-LSTM models by 41.30% and 55.66%, respectively, when tested under adversarial attacks. Extended investigation shows that our defensive method can improve the robustness of nonneural models, achieving an average of 17.62% and 22.93% classification accuracy increase on the SVM and XGBoost models, respectively. The proposed defensive method has also shown an average of 26.60% classification accuracy improvement when tested with the infamous BERT model. Our algorithm is generic enough to be applied in any NLP domain and to any model trained on any natural language.
Finding The Right One and Resolving it

One-anaphora has figured prominently in theoretical linguistic literature, but computational linguistics research on the phenomenon is sparse. Not only that, the long standing linguistic controversy between the determinative and the nominal anaphoric element one has propagated in the limited body of computational work on one-anaphora resolution, making this task harder than it is. In the present paper, we resolve this by drawing from an adequate linguistic analysis of the word one in different syntactic environments - once again highlighting the significance of linguistic theory in Natural Language Processing (NLP) tasks. We prepare an annotated corpus marking actual instances of one-anaphora with their textual antecedents, and use the annotations to experiment with state-of-the art neural models for one-anaphora resolution. Apart from presenting a strong neural baseline for this task, we contribute a gold-standard corpus, which is, to the best of our knowledge, the biggest resource on one-anaphora till date.
Item Response Theory for Efficient Human Evaluation of Chatbots

Conversational agent quality is currently assessed using human evaluation, and often requires an exorbitant number of comparisons to achieve statistical significance. In this paper, we introduce Item Response Theory (IRT) for chatbot evaluation, using a paired comparison in which annotators judge which system responds better to the next turn of a conversation. IRT is widely used in educational testing for simultaneously assessing the ability of test takers and the quality of test questions. It is similarly well suited for chatbot evaluation since it allows the assessment of both models and the prompts used to evaluate them. We use IRT to efficiently assess chatbots, and show that different examples from the evaluation set are better suited for comparing high-quality (nearer to human performance) than low-quality systems. Finally, we use IRT to reduce the number of evaluation examples assessed by human annotators while retaining discriminative power.
Tensorized Embedding Layers

The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding weight matrices can be enormous, which precludes their deployment in a limited resource setting. We introduce a novel way of parameterizing embedding layers based on the Tensor Train decomposition, which allows compressing the model significantly at the cost of a negligible drop or even a slight gain in performance. We evaluate our method on a wide range of benchmarks in natural language processing and analyze the trade-off between performance and compression ratios for a wide range of architectures, from MLPs to LSTMs and Transformers.