Miracle-dz/UniFAR-FTUs huggingface dataset:
Morphology Aware Source Term Masking for Terminology-Constrained NMT

Terminology-constrained NMT systems facilitate the forced translation of domain-specific vocabulary. A notable method in this context is the “copy-and-inflect” approach, which appends the target term lemmas of constraints to their corresponding source terms in the input sentence. In this work, we propose a novel adaptation of the “copy-and-inflect” method, referred to as “morph-masking”. Our method involves masking the source terms of the constraints from the input sentence while retaining essential grammatical information. Our approach is based on the hypothesis that “copy-and-inflect” systems have access to both source and target terms, allowing them to generate the correct surface form of the constraint by either translating the source term itself or properly inflecting the target term lemma. Through extensive validation of our method in two translation directions with different levels of source morphological complexity, Basque to Spanish and English to German, we have demonstrated that “morph-masking” is capable of providing a harder constraint signal, resulting in a notable improvement over the “copy-and-inflect” method (up to 38% in term accuracy), especially in challenging constraint scenarios.
byteLLM@LT-EDI-2024: Homophobia/Transphobia Detection in Social Media Comments - Custom Subword Tokenization with Subword2Vec and BiLSTM

This research focuses on Homophobia and Transphobia Detection in Dravidian languages, specifically Telugu, Kannada, Tamil, and Malayalam. Leveraging the Homophobia/ Transphobia Detection dataset, we propose an innovative approach employing a custom-designed tokenizer with a Bidirectional Long Short-Term Memory (BiLSTM) architecture. Our distinctive contribution lies in a tokenizer that reduces model sizes to below 7MB, improving efficiency and addressing real-time deployment challenges. The BiLSTM implementation demonstrates significant enhancements in hate speech detection accuracy, effectively capturing linguistic nuances. Low-size models efficiently alleviate inference challenges, ensuring swift real-time detection and practical deployment. This work pioneers a framework for hate speech detection, providing insights into model size, inference speed, and real-time deployment challenges in combatting online hate speech within Dravidian languages.
SarcEmp - Fine-tuning DialoGPT for Sarcasm and Empathy

Conversational models often face challenges such as a lack of emotional temperament and a limited sense of humor when interacting with users. To address these issues, we have selected relevant data and fine-tuned the model to (i) humanize the chatbot based on the user’s emotional response and the context of the conversation using a dataset based on empathy and (ii) enhanced conversations while incorporating humor/sarcasm for better user engagement. We aspire to achieve more personalized and enhanced user-computer interactions with the help of varied datasets involving sarcasm together with empathy on top of already available state-of-the-art conversational systems.
HAMiSoN-MTL at ClimateActivism 2024: Detection of Hate Speech, Targets, and Stance using Multi-task Learning

The automatic identification of hate speech constitutes an important task, playing a relevant role towards inclusivity. In these terms, the shared task on Climate Activism Stance and Hate Event Detection at CASE 2024 proposes the analysis of Twitter messages related to climate change activism for three subtasks. Subtasks A and C aim at detecting hate speech and establishing the stance of the tweet, respectively, while subtask B seeks to determine the target of the hate speech. In this paper, we describe our approach to the given subtasks. Our systems leverage transformer-based multi-task learning. Additionally, since the dataset contains a low number of tweets, we have studied the effect of adding external data to increase the learning of the model. With our approach we achieve the fourth position on subtask C on the final leaderboard, with minimal difference from the first position, showcasing the strength of multi-task learning.
JRC at ClimateActivism 2024: Lexicon-based Detection of Hate Speech

In this paper we describe the participation of the JRC team in the Sub-task A: “Hate Speech Detection” in the Shared task on Hate Speech and Stance Detection during Climate Activism at the CASE 2024 workshop. Our system is purely lexicon (keyword) based and does not use any statistical classifier. The system ranked 18 out of 22 participants with F1 of 0.83, only one point below a system, based on LLM. Our system also obtained one the highest achieved precision scores among all participating algo- rithms.
Syntactic dependency length shaped by strategic memory allocation

Human processing of nonlocal syntactic dependencies requires engagement of limited working memory for encoding, maintenance, and retrieval. This process creates an evolutionary pressure for language to be structured in a way that keeps the subparts of a dependency closer to each other, an efficiency principle termed dependency locality. The current study proposes that such a dependency locality pressure can be modulated by the surprisal of the antecedent, defined as the first part of a dependency, due to strategic allocation of working memory. In particular, antecedents with novel and unpredictable information are prioritized for memory encoding, receiving more robust representation against memory interference and decay, and thus are more capable of handling longer dependency length. We examine this claim by examining dependency corpora of six languages (Danish, English, Italian, Mandarin, Russian, and Spanish), with word surprisal generated from GPT-3 language model. In support of our hypothesis, we find evidence for a positive correlation between dependency length and the antecedent surprisal in most of the languages in our analyses.
Lidoma@DravidianLangTech 2024: Identifying Hate Speech in Telugu Code-Mixed: A BERT Multilingual

Over the past few years, research on hate speech and offensive content identification on social media has been ongoing. Since most people in the world are not native English speakers, unapproved messages are typically sent in code-mixed language. We accomplished collaborative work to identify the language of code-mixed text on social media in order to address the difficulties associated with it in the Telugu language scenario. Specifically, we participated in the shared task on the provided dataset by the Dravidian- LangTech Organizer for the purpose of identifying hate and non-hate content. The assignment is to classify each sentence in the provided text into two predetermined groups: hate or non-hate. We developed a model in Python and selected a BERT multilingual to do the given task. Using a train-development data set, we developed a model, which we then tested on test data sets. An average macro F1 score metric was used to measure the model’s performance. For the task, the model reported an average macro F1 of 0.6151.
Can Large Language Models Reason About Goal-Oriented Tasks?

Most adults can complete a sequence of steps to achieve a certain goal, such as making a sandwich or repairing a bicycle tire. In completing these goal-oriented tasks, or simply tasks in this paper, one must use sequential reasoning to understand the relationship between the sequence of steps and the goal. LLMs have shown impressive capabilities across various natural language understanding tasks. However, prior work has mainlyfocused on logical reasoning tasks (e.g. arithmetic, commonsense QA); how well LLMs can perform on more complex reasoning tasks like sequential reasoning is not clear. In this paper, we address this gap and conduct a comprehensive evaluation of how well LLMs are able to conduct this reasoning for tasks and how they scale w.r. t multiple dimensions(e.g. adaptive prompting strategies, number of in-context examples, varying complexity of the sequential task). Our findings reveal that while Chain of Thought (CoT) prompting can significantly enhance LLMs’ sequential reasoning in certain scenarios, it can also be detrimental in others, whereas Tree of Thoughts (ToT) reasoning is less effective for this type of task. Additionally, we discover that an increase in model size or in-context examples does not consistently lead to improved performance.
AttriSage: Product Attribute Value Extraction Using Graph Neural Networks

Extracting the attribute value of a product from the given product description is essential for ecommerce functions like product recommendations, search, and information retrieval. Therefore, understanding products in E-commerce. Greater accuracy certainly gives any retailer the edge. The burdensome aspect of this problem lies in the diversity of the products and their attributes and values. Existing solutions typically employ large language models or sequence-tagging approaches to capture the context of a given product description and extract attribute values. However, they do so with limited accuracy, which serves as the underlying motivation to explore a more comprehensive solution. Through this paper, we present a novel approach for attribute value extraction from product description leveraging graphs and graph neural networks. Our proposed method demonstrates improvements in attribute value extraction accuracy compared to the baseline sequence tagging approaches.
Computational modeling of semantic change

Languages change constantly over time, influenced by social, technological, cultural and political factors that affect how people express themselves. In particular, words can undergo the process of semantic change, which can be subtle and significantly impact the interpretation of texts. For example, the word terrific used to mean ‘causing terror’ and was as such synonymous to terrifying. Nowadays, speakers use the word in the sense of ‘excessive’ and even ‘amazing’. In Historical Linguistics, tools and methods have been developed to analyse this phenomenon, including systematic categorisations of the types of change, the causes and the mechanisms underlying the different types of change. However, traditional linguistic methods, while informative, are often based on small, carefully curated samples. Thanks to the availability of both large diachronic corpora, the computational means to model word meaning unsupervised, and evaluation benchmarks, we are seeing an increasing interest in the computational modelling of semantic change. This is evidenced by the increasing number of publications in this new domain as well as the organisation of initiatives and events related to this topic, such as four editions of the International Workshop on Computational Approaches to Historical Language Change LChange1, and several evaluation campaigns (Schlechtweg et al., 2020a; Basile et al., 2020b; Kutuzov et al.; Zamora-Reina et al., 2022).
Compounds in Universal Dependencies: A Survey in Five European Languages

In Universal Dependencies, compounds, which we understand as words containing two or more roots, are represented according to tokenization, which reflects the orthographic conventions of the language. A closed compound (e.g. \textit{waterfall}) corresponds to a single word in Universal Dependencies while a hyphenated compound (\textit{father-in-law}) and an open compound (\textit{apple pie}) to multiple words. The aim of this paper is to open a discussion on how to move towards a more consistent annotation of compounds.The solution we argue for is to represent the internal structure of all compound types analogously to syntactic phrases, which would not only increase the comparability of compounding within and across languages, but also allow comparisons of compounds and syntactic phrases.
Archetypes and Entropy: Theory-Driven Extraction of Evidence for Suicide Risk

Research on psychological risk factors for suicide has developed for decades. However, combining explainable theory with modern data-driven language model approaches is non-trivial. In this study, we propose and evaluate methods for identifying language patterns aligned with theories of suicide risk by combining theory-driven suicidal archetypes with language model-based and relative entropy-based approaches. Archetypes are based on prototypical statements that evince risk of suicidality while relative entropy considers the ratio of how unusual both a risk-familiar and unfamiliar model find the statements. While both approaches independently performed similarly, we find that combining the two significantly improved the performance in the shared task evaluations, yielding our combined system submission with a BERTScore Recall of 0.906. Consistent with the literature, we find that titles are highly informative as suicide risk evidence, despite the brevity. We conclude that a combination of theory- and data-driven methods are needed in the mental health space and can outperform more modern prompt-based methods.
Advancing Open-Domain Conversational Agents - Designing an Engaging System for Natural Multi-Turn Dialogue

This system paper describes our conversational AI agent developed for the SCI-CHAT competition. The goal is to build automated dialogue agents that can have natural, coherent conversations with humans over multiple turns. Our model is based on fine-tuning the Snorkel-Mistral-PairRM-DPO language model on podcast conversation transcripts. This allows the model to leverage Snorkel-Mistral-PairRMDPO’s linguistic knowledge while adapting it for multi-turn dialogue modeling using LoRA. During evaluation, human judges will converse with the agent on specified topics and provide ratings on response quality. Our system aims to demonstrate how large pretrained language models, when properly adapted and evaluated, can effectively converse on open-ended topics spanning multiple turns.
CEN_Amrita@LT-EDI 2024: A Transformer based Speech Recognition System for Vulnerable Individuals in Tamil

Speech recognition is known to be a specialized application of speech processing. Automatic speech recognition (ASR) systems are designed to perform the speech-to-text task. Although ASR systems have been the subject of extensive research, they still encounter certain challenges when speech variations arise. The speaker’s age, gender, vulnerability, and other factors are the main causes of the variations in speech. In this work, we propose a fine-tuned speech recognition model for recognising the spoken words of vulnerable individuals in Tamil. This research utilizes a dataset sourced from the LT-EDI@EACL2024 shared task. We trained and tested pre-trained ASR models, including XLS-R and Whisper. The findings highlight that the fine-tuned Whisper ASR model surpasses the XLSR, achieving a word error rate (WER) of 24.452, signifying its superior performance in recognizing speech from diverse individuals.
The gut contractile organoid: a novel model for studying the gut motility regulated by coordinating signals between interstitial cells of Cajal and smooth muscles

The gut undergoes peristaltic movements regulated by intricate cellular interactions. However, they have poorly been explored due to a lack of model system. We here developed a novel contractile organoid that is derived from the muscle layer of chicken embryonic hindgut. The organoid contained smooth muscle cells (SMCs) and interstitial cells of Cajal (ICCs; pacemaker) with few enteric neurons, and underwent periodic contractions. The organoid formed by self-organization with morphological arrangements of ICCs (internal) and SMCs (peripheral), allowing identification of these cells in live. GCaMP-Ca2+ imaging analyses revealed that Ca2+ transients between ICC-ICC, SMC-SMC or SMC-ICC were markedly coordinated. Pharmacological studies further showed that gap junctions play a role in ICC-to-SMC signaling, and also possible feedback from SMC’s contraction to ICC’s pace-making activities. In addition, two organoids with different rhythm became synchronized when mediated by SMCs, unveiling a novel contribution of SMCs to ICC’s pace-making. The gut contractile organoid developed in this study offers a useful model to understand the mechanisms underlying the rhythm coordination between/among ICCs and SMCs during gut peristaltic movements.
Environmental similarity between relatives reduces heritability of reproductive timing in wild great tits

Intraspecific variation is necessary for evolutionary change and population resilience, but the extent to which it contributes to either depends on the causes of this variation. Understanding the causes of individual variation in traits involved with reproductive timing is important in the face of environmental change, especially in systems where reproduction must coincide with seasonal resource availability. However, separating the genetic and environmental causes of variation is not straightforward, and there has been limited consideration of how small-scale environmental effects might lead to similarities between individuals that occupy similar environments, potentially biasing estimates of genetic heritability. In ecological systems, environments can often be complex in their spatial structure, and it may therefore be important to account for similarities in the environments experienced by individuals within a population beyond considering spatial distances alone. Here, we construct multi-matrix quantitative genetic animal models using over 11,000 breeding records (spanning 35 generations) of individually-marked great tits (Parus major) and information about breeding proximity and habitat characteristics to quantify the drivers of variability in two key seasonal reproductive timing traits. We show that the environment experienced by related individuals explains around a fifth of the variation seen in reproductive timing, and accounting for this leads to decreased estimates of heritability. Our results thus demonstrate that environmental sharing between relatives can strongly affect estimates of heritability and therefore alter our expectations of the evolutionary response to selection.
Neuroprotective role of Hippo signaling by microtubule stability control in C. elegans

The evolutionarily conserved Hippo (Hpo) pathway has been shown to impact early development and tumorigenesis by governing cell proliferation and apoptosis. However, its post-developmental roles are relatively unexplored. Here, we demonstrate its roles in post-mitotic cells by showing that defective Hpo signaling accelerates age-associated structural and functional decline of neurons in C. elegans. Loss of wts-1/LATS resulted in premature deformation of touch neurons and impaired touch responses in a yap-1/YAP-dependent manner. Decreased movement as well as microtubule destabilization by treatment with colchicine or disruption of microtubule stabilizing genes alleviated the neuronal deformation of wts-1 mutants. Colchicine exerted neuroprotective effects even during normal aging. In addition, the deficiency of a microtubule-severing enzyme spas-1 also led to precocious structural deformation. These results consistently suggest that hyper-stabilized microtubules in both wts-1-deficient neurons and normally aged neurons are detrimental to the maintenance of neuronal structural integrity. In summary, Hpo pathway governs the structural and functional maintenance of differentiated neurons by modulating microtubule stability, raising the possibility that the microtubule stability of fully developed neurons could be a promising target to delay neuronal aging. Our study provides potential therapeutic approaches to combat age- or disease-related neurodegeneration.
Preserved spontaneous mentalizing amid reduced intersubject variability in autism during a movie narrative

Individuals with autism may perform well in structured Theory of Mind (ToM) tasks that assess their ability to infer mental states, yet encounter challenges in everyday social interactions. Using fMRI and pupillometry, we investigated whether this discrepancy stems from reduced spontaneous mentalizing or broader difficulties in unstructured environments. Autistic adults and neurotypical controls viewed a nonverbal animated movie featuring events known to evoke mental state inferences. Both groups exhibited similar ToM network activations and pupil responses to these events, alongside comparable verbal accounts of the mental states portrayed. However, intersubject correlation analysis revealed a significant reduction in response variability among autistic participants throughout the movie’s complex visual narrative, affecting brain regions beyond the ToM network. We suggest that the primary challenge for individuals with autism may lie in the idiosyncratic exploration of narratives in unstructured settings, regardless of whether mental state inferences are involved.
Choroid plexus enlargement in acute neuroinflammation is tightly interrelated to the tyrosine receptor signalling

The choroid plexus (ChP) plays a crucial function in neuroinflammation of the central nervous system and in the immune response of the brain during neurodegeneration. Recent studies described a massive ChP enlargement in patients with multiple sclerosis (MS) and active disease courses, but also in several other neuroinflammatory and neurodegenerative conditions. Nevertheless, the exact basis and pathophysiology behind ChP hypertrophy remains unclear. This study was designed to evaluate the association of cerebrospinal fluid (CSF) proteomic spectra with brain MRI-derived volumetric measures of ChP in two independent cohorts of MS patients, and to translationally validate the related molecular mechanisms in the transcriptomic analysis of the ChP properties in a mouse model of experimental autoimmune encephalomyelitis (EAE). Our analysis revealed five enriched proteins (NTRK2, ADAM23, SCARB2, CPM, CNTN5) significantly associated with the ChP volumes in both of the MS cohorts. These proteins relate closely to mechanisms of cellular communication, function (e.g. transmembrane tyrosine receptor signalling (RTK) and vascular endothelial growth) and pathways involved in the regulation of cellular plasticity (e.g. neuron differentiation, axonal remodelling and myelin regulation) as depicted by molecular function analysis and validation of the results in the transcriptome from ChP tissue specific for EAE. This work provides conclusive new evidence for the role of ChP in the context of neuroinflammation and neurodegeneration, demonstrating the intriguing relationships between ChP enlargement, CSF dynamics, and the development of neuroinflammatory and neurodegenerative diseases. Our results are encouraging for the development of new therapeutic avenues (i.e. targeting RTK signalling). One sentence summary Tyrosine receptor signalling is tightly associated with choroid plexus enlargement and is key in CSF dynamics during a neuroinflammatory attack in MS