Miracle-dz/UniFAR-FTUs huggingface dataset:
Improving Backchannel Prediction Leveraging Sequential and Attentive Context Awareness

Backchannels, which refer to short and often affirmative or empathetic responses from a listener during a conversation, play a crucial role in effective communication. In this paper, we introduce CABP(Context-Aware Backchannel Prediction), a sequential and attentive context approach aimed at enhancing backchannel prediction performance. Additionally, CABP leverages the pretrained wav2vec model for encoding audio signal. Experimental results show that CABP performs better than context-free models, with performance improvements of 1.3% and 1.8% in Korean and English datasets, respectively. Furthermore, when utilizing the pretrained wav2vec model, CABP consistently demonstrates the best performance, achieving performance improvements of 4.4% and 3.1% in Korean and English datasets.
Exploring Instructive Prompts for Large Language Models in the Extraction of Evidence for Supporting Assigned Suicidal Risk Levels

Monitoring and predicting the expression of suicidal risk in individuals’ social media posts is a central focus in clinical NLP. Yet, existing approaches frequently lack a crucial explainability component necessary for extracting evidence related to an individual’s mental health state. We describe the CSIRO Data61 team’s evidence extraction system submitted to the CLPsych 2024 shared task. The task aims to investigate the zero-shot capabilities of open-source LLM in extracting evidence regarding an individual’s assigned suicide risk level from social media discourse. The results are assessed against ground truth evidence annotated by psychological experts, with an achieved recall-oriented BERTScore of 0.919. Our findings suggest that LLMs showcase strong feasibility in the extraction of information supporting the evaluation of suicidal risk in social media discourse. Opportunities for refinement exist, notably in crafting concise and effective instructions to guide the extraction process.
Identifying Narrative Content in Podcast Transcripts

As one of the oldest forms of human communication, narratives appear across a variety of genres and media. Computational methods have been applied to study narrativity in novels, social media, and patient records, leading to new approaches and insights. However, other types of media are growing in popularity, like podcasts. Podcasts contain a multitude of spoken narratives that can provide a meaningful glimpse into how people share stories with one another.In this paper, we outline and apply methods to process English-language podcast transcripts and extract narrative content from conversations within each episode. We provide an initial analysis of the types of narrative content that exists within a wide range of podcasts, and compare our results to other established narrative analysis tools.Our annotations for narrativity and pretrained models can help to enable future research into narrativity within a large corpus of approximately 100,000 podcast episodes.
LLMs Simulate Big5 Personality Traits: Further Evidence

An empirical investigation into the simulation of the Big5 personality traits by large language models (LLMs), namely Llama-2, GPT-4, and Mixtral, is presented. We analyze the personality traits simulated by these models and their stability. This contributes to the broader understanding of the capabilities of LLMs to simulate personality traits and the respective implications for personalized human-computer interaction.
Prompting Fairness: Learning Prompts for Debiasing Large Language Models

Large language models are prone to internalize social biases due to the characteristics of the data used for their self-supervised training scheme. Considering their recent emergence and wide availability to the general public, it is mandatory to identify and alleviate these biases to avoid perpetuating stereotypes towards underrepresented groups. We present a novel prompt-tuning method for reducing biases in encoder models such as BERT or RoBERTa. Unlike other methods, we only train a small set of additional reusable token embeddings that can be concatenated to any input sequence to reduce bias in the outputs. We particularize this method to gender bias by providing a set of templates used for training the prompts. Evaluations on two benchmarks show that our method is on par with the state of the art while having a limited impact on language modeling ability.
ReBERT at HSD-2Lang 2024: Fine-Tuning BERT with AdamW for Hate Speech Detection in Arabic and Turkish

Identifying hate speech is a challenging specialization in the natural language processing field (NLP). Particularly in fields with differing linguistics, it becomes more demanding to construct a well-performing classifier for the betterment of the community. In this paper, we leveraged the performances of pre-trained models on the given hate speech detection dataset. By conducting a hyperparameter search, we computed the feasible setups for fine-tuning and trained effective classifiers that performed well in both subtasks in the HSD-2Lang 2024 contest.
How to use Language Models for Synthetic Text Generation in Cerebrovascular Disease-specific Medical Reports

The quantity and quality of data have a significant impact on the performance of artificial intelligence (AI). However, in the biomedical domain, data often contains sensitive information such as personal details, making it challenging to secure enough data for medical AI. Consequently, there is a growing interest in synthetic data generation for medical AI. However, research has primarily focused on medical images, with little given to text-based data such as medical records. Therefore, this study explores the application of language models (LMs) for synthetic text generation in low-resource domains like medical records. It compares the results of synthetic text generation based on different LMs. To achieve this, we focused on two criteria for LM-based synthetic text generation of medical records using two keywords entered by the user: 1) the impact of the LM’s knowledge, 2) the impact of the LM’s size. Additionally, we objectively evaluated the generated synthetic text, including representative metrics such as BLUE and ROUGE, along with clinician’s evaluations.
Assessing Authenticity and Anonymity of Synthetic User-generated Content in the Medical Domain

Since medical text cannot be shared easily due to privacy concerns, synthetic data bears much potential for natural language processing applications. In the context of social media and user-generated messages about drug intake and adverse drug effects, this work presents different methods to examine the authenticity of synthetic text. We conclude that the generated tweets are untraceable and show enough authenticity from the medical point of view to be used as a replacement for a real Twitter corpus. However, original data might still be the preferred choice as they contain much more diversity.
SentenceLDA: Discriminative and Robust Document Representation with Sentence Level Topic Model

A subtle difference in context results in totally different nuances even for lexically identical words. On the other hand, two words can convey similar meanings given a homogeneous context. As a result, considering only word spelling information is not sufficient to obtain quality text representation. We propose SentenceLDA, a sentence-level topic model. We combine modern SentenceBERT and classical LDA to extend the semantic unit from word to sentence. By extending the semantic unit, we verify that SentenceLDA returns more discriminative document representation than other topic models, while maintaining LDA’s elegant probabilistic interpretability. We also verify the robustness of SentenceLDA by comparing the inference results on original and paraphrased texts. Additionally, we implement one possible application of SentenceLDA on corpus-level key opinion mining by applying SentenceLDA on an argumentative corpus, DebateSum.
Beyond Tech@DravidianLangTech2024 : Fake News Detection in Dravidian Languages Using Machine Learning

In the digital age, identifying fake news is essential when fake information travels quickly via social media platforms. This project employs machine learning techniques, including Random Forest, Logistic Regression, and Decision Tree, to distinguish between real and fake news. With the rise of news consumption on social media, it becomes essential to authenticate information shared on platforms like YouTube comments. The research emphasizes the need to stop spreading harmful rumors and focuses on authenticating news articles. The proposed model utilizes machine learning and natural language processing, specifically Support Vector Machines, to aggregate and determine the authenticity of news. To address the challenges of detecting fake news in this paper, describe the Machine Learning (ML) models submitted to ‘Fake News Detection in Dravidian Languages” at DravidianLangTech@EACL 2024 shared task. Four different models, namely: Naive Bayes, Support Vector Machine (SVM), Random forest, and Decision tree.
Overview of the Hate Speech Detection in Turkish and Arabic Tweets (HSD-2Lang) Shared Task at CASE 2024

This paper offers an overview of Hate Speech Detection in Turkish and Arabic Tweets (HSD-2Lang) Shared Task at CASE workshop to be held jointly with EACL 2024. The task was divided into two subtasks: Subtask A, targeting hate speech detection in various Turkish contexts, and Subtask B, addressing hate speech detection in Arabic with limited data. The shared task attracted significant attention with 33 teams that registered and 10 teams that participated in at least one task. In this paper, we provide the details of the tasks and the approaches adopted by the participant along with an analysis of the results obtained from this shared task.
Forged-GAN-BERT: Authorship Attribution for LLM-Generated Forged Novels

The advancement of generative Large Language Models (LLMs), capable of producing human-like texts, introduces challenges related to the authenticity of the text documents. This requires exploring potential forgery scenarios within the context of authorship attribution, especially in the literary domain. Particularly,two aspects of doubted authorship may arise in novels, as a novel may be imposed by a renowned author or include a copied writing style of a well-known novel. To address these concerns, we introduce Forged-GAN-BERT, a modified GANBERT-based model to improve the classification of forged novels in two data-augmentation aspects: via the Forged Novels Generator (i.e., ChatGPT) and the generator in GAN. Compared to other transformer-based models, the proposed Forged-GAN-BERT model demonstrates an improved performance with F1 scores of 0.97 and 0.71 for identifying forged novels in single-author and multi-author classification settings. Additionally, we explore different prompt categories for generating the forged novels to analyse the quality of the generated texts using different similarity distance measures, including ROUGE-1, Jaccard Similarity, Overlap Confident, and Cosine Similarity.
Exploring the impact of noise in low-resource ASR for Tamil

The use of deep learning algorithms has resulted in significant progress in automatic speech recognition (ASR). Robust high-accuracy ASR models typically require thousands or tens of thousands of hours of speech data, but even the strongest models tend fail under noisy conditions. Unsurprisingly, the impact of noise on accuracy is more drastic in low-resource settings. In this paper, we investigate the impact of noise on ASR in a low-resource setting. We explore novel methods for developing noise-robust ASR models using a a small dataset for Tamil, a widely-spoken but under-resourced Dravidian languages. We add various noises to the audio data to determine the impact of different kinds of noise (e.g., punctuated vs. constant, man-made vs natural) We also explore the relationship between different data augmentation methods are better suited to handling different types of noise. Our results show that all noises, regardless of the type, had an impact on ASR performance, and that upgrading the architecture alone could not mitigate the impact of noise. SpecAugment, the most common data augmentation method, was not as helpful as raw data augmentation, in which noise is explicitly added to the training data. Raw data augmentation enhances ASR performance on both clean data and noise-mixed data.
Moderation in the Wild: Investigating User-Driven Moderation in Online Discussions

Effective content moderation is imperative for fostering healthy and productive discussions in online domains. Despite the substantial efforts of moderators, the overwhelming nature of discussion flow can limit their effectiveness. However, it is not only trained moderators who intervene in online discussions to improve their quality. “Ordinary” users also act as moderators, actively intervening to correct information of other users’ posts, enhance arguments, and steer discussions back on course.This paper introduces the phenomenon of user moderation, documenting and releasing UMOD, the first dataset of comments in whichusers act as moderators. UMOD contains 1000 comment-reply pairs from the subreddit r/changemyview with crowdsourced annotations from a large annotator pool and with a fine-grained annotation schema targeting the functions of moderation, stylistic properties(aggressiveness, subjectivity, sentiment), constructiveness, as well as the individual perspectives of the annotators on the task. The releaseof UMOD is complemented by two analyses which focus on the constitutive features of constructiveness in user moderation and on thesources of annotator disagreements, given the high subjectivity of the task.
HyperTraPS-CT: Inference and prediction for accumulation pathways with flexible data and model structures

Accumulation processes, where many potentially coupled features are acquired over time, occur throughout the sciences, from evolutionary biology to disease progression, and particularly in the study of cancer progression. Existing methods for learning the dynamics of such systems typically assume limited (often pairwise) relationships between feature subsets, cross-sectional or untimed observations, small feature sets, or discrete orderings of events. Here we introduce HyperTraPS-CT (Hypercubic Transition Path Sampling in Continuous Time) to compute posterior distributions on continuous-time dynamics of many, arbitrarily coupled, traits in unrestricted state spaces, accounting for uncertainty in observations and their timings. We demonstrate the capacity of HyperTraPS-CT to deal with cross-sectional, longitudinal, and phylogenetic data, which may have no, uncertain, or precisely specified sampling times. HyperTraPS-CT allows positive and negative interactions between arbitrary subsets of features (not limited to pairwise interactions), supporting Bayesian and maximum-likelihood inference approaches to identify these interactions, consequent pathways, and predictions of future and unobserved features. We also introduce a range of visualisations for the inferred outputs of these processes and demonstrate model selection and regularisation for feature interactions. We apply this approach to case studies on the accumulation of mutations in cancer progression and the acquisition of anti-microbial resistance genes in tuberculosis, demonstrating its flexibility and capacity to produce predictions aligned with applied priorities.
LINC complex alterations are a hallmark of sporadic and familial ALS/FTD

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disorder that primarily affects motor neurons, leading to progressive muscle weakness and loss of voluntary muscle control. While the exact cause of ALS is not fully understood, emerging research suggests that dysfunction of the nuclear envelope (NE) may contribute to disease pathogenesis and progression. The NE plays a role in ALS through several mechanisms, including nuclear pore defects, nucleocytoplasmic transport impairment, accumulation of mislocalized proteins, and nuclear morphology abnormalities. The LINC complex is the second biggest multi-protein complex in the NE and consists of the SUN1/2 proteins spanning the inner nuclear membrane and Nesprin proteins embedded in the outer membrane. The LINC complex, by interacting with both the nuclear lamina and the cytoskeleton, transmits mechanical forces to the nucleus regulating its morphology and functional homeostasis. In this study we show extensive alterations to the LINC complex in motor and cortical iPSC-derived neurons and spinal cord organoids carrying the ALS causative mutation in the C9ORF72 gene (C9). Importantly, we show that such alterations are present in vivo in a cohort of sporadic ALS and C9-ALS postmortem spinal cord and motor cortex biopsies. We also found that LINC complex disruption strongly correlated with nuclear morphological alterations occurring in ALS neurons, independently of TDP43 mislocalization. Altogether, our data establish morphological and functional alterations to the LINC complex as important events in ALS pathogenic cascade, making this pathway a possible target for both biomarker and therapy development.
Expanding and enriching the LncRNA gene landscape using the GeneCaRNA database

The GeneCaRNA human gene database is a member of the GeneCards Suite. It presents ∼280,000 human non-coding RNA genes, identified algorithmically from ∼690,000 RNAcentral’s transcripts. This expands by ∼tenfold the ncRNA gene count relative to other sources. GeneCaRNA thus contains ∼120,000 long non-coding RNAs (LncRNAs, >200 bases long), including ∼100,000 novel genes. The latter have sparse functional information, a vast terra incognita for future research. LncRNA genes are uniformly represented on all nuclear chromosomes, with 10 genes in mitochondrial DNA. Data obtained from MalaCards, another GeneCards Suite member, finds 1,547 genes associated with 1 to 50 diseases. ∼15% of the associations portray experimental evidence, with cancers tending to be multigenic. Preliminary text mining within GeneCaRNA discovers interactions of LncRNA transcripts with target gene products, with 25% being ncRNAs and 75% proteins. GeneCaRNA has a biological pathways section, which at present shows 131 pathways for 38 LncRNA genes, a basis for future expansion. Finally, our GeneHancer database provides regulatory elements for ∼110,000 LncRNA genes, offering pointers for co-regulated genes and genetic linkages from enhancers to diseases. We anticipate that the broad vista provided by GeneCaRNA will serve as an essential guide for further LncRNA research.
Putative Looping Factor ZNF143/ZFP143 is an Essential Transcriptional Regulator with No Looping Function

Interactions between distal loci, including those involving enhancers and promoters, are a central mechanism of gene regulation in mammals, yet the protein regulators of these interactions remain largely undetermined. The zinc finger transcription factor ZNF143/ZFP143 has been strongly implicated as a regulator of chromatin interactions, functioning either with or without CTCF. However, ZNF143/ZFP143’s role in this process and its function, either with or without CTCF, are not well understood. Here, we tagged both CTCF and ZNF143/ZFP143 with dual-purpose degron/imaging tags to combinatorially assess their looping function and effect on each other. We find that ZNF143/ZFP143 possesses no general looping function in mouse and human cells, and that it largely functions independently of CTCF. Instead, ZNF143/ZFP143 is an essential and highly conserved transcription factor possessing an extremely stable chromatin residence time (>20 min) that regulates an important subset of mitochondrial and ribosomal genes.
Explainable deep learning on 7500 whole genomes elucidates cancer-specific patterns of chromosomal instability

Chromosomal instability (CIN) refers to an increased rate of chromosomal changes within cells. It is highly prevalent in cancer cells and leads to abnormalities in chromosome number (aneuploidy) and structure. CIN contributes to genetic diversity within a tumour, which facilitates tumour progression, drug resistance, and metastasis. Here, we present a deep learning method and an exploration of the chromosome copy aberrations (CNAs) resultant from CIN, across 7,500 high-depth, whole genome sequences, representing 13 cancer types. We found that the types of CNAs can act as a highly specific classifier for primary site. Using an explainable AI approach, we revealed both established and novel loci that contributed to cancer type, and focusing on highly significant chromosome loci within cancer types, we demonstrated prognostic relevance. We outline how the developed methodology can provide several applications for researchers, including drug target and biomarker discovery, as well as the identification of cancers of unknown primary site.
Association between gene expression plasticity and regulatory network topology

What drives the evolution of gene network topology? During the last decades, many looked into it to find structures responsible for gene expression patterns. Gene expression plasticity for instance has been associated with many network structures, but both theoretical results and empirical observations were often equivocal. Our objective was to decipher the regulatory causes of gene expression plasticity, and more specifically, which structures in regulatory networks were relevant for the sensitivity to environmental factors. We sought the common regulatory structures associated with gene expression plasticity between predictions from an evolutionary simulation model and the global regulatory network from Escherichia coli. The combination of empirical and theoretical approaches and their strikingly similar results confirmed that selection promotes more regulation towards plastic genes and, as a consequence, plastic genes were more often regulated by feedforward loops than non-plastic genes. Selection tends to bias the distribution of regulatory loop motifs towards positive feedforward and diamond loops, but this enrichment in specific structures was the same in plastic and non-plastic genes. The impossibility to predict gene expression plasticity from the network regulatory structure opens interesting questions about the nature of the missing information in current systems biology databases. Significance statement Organisms can respond to their environment, and some of their traits are said to be plastic, i.e. dependent on environmental signals. At the individual level, understanding how gene expression varies in response to the environment is of critical importance. Some argued that this sensitivity could be deduced from the structure and motifs of gene regulatory networks, but theoretical and empirical studies could not reach a consensus. Here, we aim at disentangling the relationship between regulatory structure and gene expression plasticity, combining empirical data of Escherichia coli and population evolution modelling. Both approaches agree on a larger amount of regulators for plastic genes compared to non-plastic genes. There were clear signals of selection enriching networks in specific structures, such as feedforward loops, but this selection was independent from gene expression plasticity.