Reviewerly/exHarmony huggingface dataset:
Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue

The demand for multimodal dialogue systems has been rising in various domains, emphasizing the importance of interpreting multimodal inputs from conversational and situational contexts. One main challenge in multimodal dialogue understanding is multimodal object identification, which constitutes the ability to identify objects relevant to a multimodal user-system conversation. We explore three methods to tackle this problem and evaluate them on the largest situated dialogue dataset, SIMMC 2.1. Our best method, scene-dialogue alignment, improves the performance by ~20% F1-score compared to the SIMMC 2.1 baselines. We provide analysis and discussion regarding the limitation of our methods and the potential directions for future works.
Contextual Dynamic Prompting for Response Generation in Task-oriented Dialog Systems

Response generation is one of the critical components in task-oriented dialog systems. Existing studies have shown that large pre-trained language models can be adapted to this task. The typical paradigm of adapting such extremely large language models would be by fine-tuning on the downstream tasks which is not only time-consuming but also involves significant resources and access to fine-tuning data. Prompting (Schick and Schütze, 2020) has been an alternative to fine-tuning in many NLP tasks. In our work, we explore the idea of using prompting for response generation in task-oriented dialog systems. Specifically, we propose an approach that performs contextual dynamic prompting where the prompts are learnt from dialog contexts. We aim to distill useful prompting signals from the dialog context. On experiments with MultiWOZ 2.2 dataset (Zang et al., 2020), we show that contextual dynamic prompts improve response generation in terms of combined score (Mehri et al., 2019) by 3 absolute points, and an additional 17 points when dialog states are incorporated. Furthermore, we carried out human annotation on these conversations and found that agents which incorporate context are preferred over agents with vanilla prefix-tuning.
Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog

Traditional systems designed for task oriented dialog utilize knowledge present only in structured knowledge sources to generate responses. However, relevant information required to generate responses may also reside in unstructured sources, such as documents. Recent state of the art models such as HyKnow (Gao et al., 2021b) and SEKNOW (Gao et al., 2021a) aimed at overcoming these challenges make limiting assumptions about the knowledge sources. For instance, these systems assume that certain types of information, such as a phone number, is always present in a structured knowledge base (KB) while information about aspects such as entrance ticket prices, would always be available in documents.In this paper, we create a modified version of the MutliWOZ-based dataset prepared by (Gao et al., 2021a) to demonstrate how current methods have significant degradation in performance when strict assumptions about the source of information are removed. Then, in line with recent work exploiting pre-trained language models, we fine-tune a BART (Lewiset al., 2020) based model using prompts (Brown et al., 2020; Sun et al., 2021) for the tasks of querying knowledge sources, as well as, for response generation, without makingassumptions about the information present in each knowledge source. Through a series of experiments, we demonstrate that our model is robust to perturbations to knowledge modality (source of information), and that it can fuse information from structured as well as unstructured knowledge to generate responses.
Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers

Demographic factors (e.g., gender or age) shape our language. Previous work showed that incorporating demographic factors can consistently improve performance for various NLP tasks with traditional NLP models. In this work, we investigate whether these previous findings still hold with state-of-the-art pretrained Transformer-based language models (PLMs). We use three common specialization methods proven effective for incorporating external knowledge into pretrained Transformers (e.g., domain-specific or geographic knowledge). We adapt the language representations for the demographic dimensions of gender and age, using continuous language modeling and dynamic multi-task learning for adaptation, where we couple language modeling objectives with the prediction of demographic classes. Our results, when employing a multilingual PLM, show substantial gains in task performance across four languages (English, German, French, and Danish), which is consistent with the results of previous work. However, controlling for confounding factors – primarily domain and language proficiency of Transformer-based PLMs – shows that downstream performance gains from our demographic adaptation do not actually stem from demographic knowledge. Our results indicate that demographic specialization of PLMs, while holding promise for positive societal impact, still represents an unsolved problem for (modern) NLP.
Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge

Although named entity recognition (NER) helps us to extract domain-specific entities from text (e.g., artists in the music domain), it is costly to create a large amount of training data or a structured knowledge base to perform accurate NER in the target domain. Here, we propose self-adaptive NER, which retrieves external knowledge from unstructured text to learn the usages of entities that have not been learned well. To retrieve useful knowledge for NER, we design an effective two-stage model that retrieves unstructured knowledge using uncertain entities as queries. Our model predicts the entities in the input and then finds those of which the prediction is not confident. Then, it retrieves knowledge by using these uncertain entities as queries and concatenates the retrieved text to the original input to revise the prediction. Experiments on CrossNER datasets demonstrated that our model outperforms strong baselines by 2.35 points in F1 metric.
Development of pre-trained language models for clinical NLP in Spanish

Clinical natural language processing aims to tackle language and prediction tasks using text from medical practice, such as clinical notes, prescriptions, and discharge summaries. Several approaches have been tried to deal with these tasks. Since 2017, pre-trained language models (PLMs) have achieved state-of-the-art performance in many tasks. However, most works have been developed in English. This PhD research proposal addresses the development of PLMs for clinical NLP in Spanish. To carry out this study, we will build a clinical corpus big enough to implement a functional PLM. We will test several PLM architectures and evaluate them with language and prediction tasks. The novelty of this work lies in the use of only clinical text, while previous clinical PLMs have used a mix of general, biomedical, and clinical text.
Privacy-Preserving Natural Language Processing

Ivan Habernal, Fatemehsadat Mireshghallah, Patricia Thaine, Sepideh Ghanavati, Oluwaseyi Feyisetan. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts. 2023.
An Intra-Class Relation Guided Approach for Code Comment Generation

Code comments are critical for maintaining and comprehending software programs, but they are often missing, mismatched, or outdated in practice. Code comment generation task aims to automatically produce descriptive comments for code snippets. Recently, methods based on the neural encoder-decoder architecture have achieved impressive performance. These methods assume that all the information required to generate comments is encoded in the target function itself, yet in most realistic situations, it is hard to understand a function in isolation from the surrounding context. Furthermore, the global context may contain redundant information that should not be introduced. To address the above issues, we present a novel graph-based learning framework to capture various relations among functions in a class file. Our approach is based on a common real-world scenario in which only a few functions in the source file have human-written comments. Guided by intra-class function relations, our model incorporates contextual information extracted from both the source code and available comments to generate missing comments. We conduct experiments on a Java dataset collected from real-world projects. Experimental results show that the proposed method outperforms competitive baseline models on all automatic and human evaluation metrics.
Compositional Generalisation with Structured Reordering and Fertility Layers

Seq2seq models have been shown to struggle with compositional generalisation, i.e. generalising to new and potentially more complex structures than seen during training. Taking inspiration from grammar-based models that excel at compositional generalisation, we present a flexible end-to-end differentiable neural model that composes two structural operations: a fertility step, which we introduce in this work, and a reordering step based on previous work (Wang et al., 2021). To ensure differentiability, we use the expected value of each step, which we compute using dynamic programming. Our model outperforms seq2seq models by a wide margin on challenging compositional splits of realistic semantic parsing tasks that require generalisation to longer examples. It also compares favourably to other models targeting compositional generalisation.
AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference

Given a sentence “Abby told Brittney that she upset Courtney”, one would struggle to understand who “she” refers to, and ask for clarification. However, if the word “upset” were replaced with “hugged”, “she” unambiguously refers to Abby. We study if modern coreference resolution models are sensitive to such pronominal ambiguity. To this end, we construct AmbiCoref, a diagnostic corpus of minimal sentence pairs with ambiguous and unambiguous referents. Our examples generalize psycholinguistic studies of human perception of ambiguity around particular arrangements of verbs and their arguments. Analysis shows that (1) humans are less sure of referents in ambiguous AmbiCoref examples than unambiguous ones, and (2) most coreference models show little difference in output between ambiguous and unambiguous pairs. We release AmbiCoref as a diagnostic corpus for testing whether models treat ambiguity similarly to humans.
Unsupervised Cross-lingual Word Embedding Representation for English-isiZulu

In this study, we investigate the effectiveness of using cross-lingual word embeddings for zero-shot transfer learning between a language with an abundant resource, English, and a languagewith limited resource, isiZulu. IsiZulu is a part of the South African Nguni language family, which is characterised by complex agglutinating morphology. We use VecMap, an open source tool, to obtain cross-lingual word embeddings. To perform an extrinsic evaluation of the effectiveness of the embeddings, we train a news classifier on labelled English data in order to categorise unlabelled isiZulu data using zero-shot transfer learning. In our study, we found our model to have a weighted average F1-score of 0.34. Our findings demonstrate that VecMap generates modular word embeddings in the cross-lingual space that have an impact on the downstream classifier used for zero-shot transfer learning.
Diverse Content Selection for Educational Question Generation

Amir Hadifar, Semere Kiros Bitew, Johannes Deleu, Veronique Hoste, Chris Develder, Thomas Demeester. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop. 2023.
Spelling convention sensitivity in neural language models

We examine whether large neural language models, trained on very large collections of varied English text, learn the potentially long-distance dependency of British versus American spelling conventions, i.e., whether spelling is consistently one or the other within model-generated strings. In contrast to long-distance dependencies in non-surface underlying structure (e.g., syntax), spelling consistency is easier to measure both in LMs and the text corpora used to train them, which can provide additional insight into certain observed model behaviors. Using a set of probe words unique to either British or American English, we first establish that training corpora exhibit substantial (though not total) consistency. A large T5 language model does appear to internalize this consistency, though only with respect to observed lexical items (not nonce words with British/American spelling patterns). We further experiment with correcting for biases in the training data by fine-tuning T5 on synthetic data that has been debiased, and find that finetuned T5 remains only somewhat sensitive to spelling consistency. Further experiments show GPT2 to be similarly limited.
Our kind of people? Detecting populist references in political debates

This paper investigates the identification of populist rhetoric in text and presents a novel cross-lingual dataset for this task. Our work is based on the definition of populism as a “communication style of political actors that refers to the people” but also includes anti-elitism as another core feature of populism. Accordingly, we annotate references to The People and The Elite in German and English parliamentary debates with a hierarchical scheme. The paper describes our dataset and annotation procedure and reports inter-annotator agreement for this task. Next, we compare and evaluate different transformer-based model architectures on a German dataset and report results for zero-shot learning on a smaller English dataset. We then show that semi-supervised tri-training can improve results in the cross-lingual setting. Our dataset can be used to investigate how political actors talk about The Elite and The People and to study how populist rhetoric is used as a strategic device.
Pendampingan Pembelajaran Reading Di Pondok Pesantren Al Urwatul Wutsqo Jombang

Reading is very important to master because reading can open a window to the world, especially reading English texts where many important sources of information are written in English. This assistance aims to help students at the al Urwatul Wutsqo Bulurejo Jombang Islamic boarding school to improve their English reading skills. The results of the mentoring were very positive where there was an increase in reading skills with an average reading score of 47.81 before the implementation of QARs and 68.75 afterward. Likewise, from the paired sample test, the t count is greater than the t table (17.073 &gt; 2.1199).
Impact of Visual Image Quality on Lymphocyte Detection Using YOLOv5 and RetinaNet Algorithms

Lymphocytes, a type of leukocytes, play a vital role in the immune system. The precise quantification, spatial arrangement and phenotypic characterization of lymphocytes within haematological or histopathological images can serve as a diagnostic indicator of a particular lesion. Artificial neural networks, employed for the detection of lymphocytes, not only can provide support to the work of histopathologists but also enable better disease monitoring and faster analysis of the general immune system condition. In this study, the impact of visual quality on the performance of state-of-the-art algorithms for detecting lymphocytes in medical images was examined. Two datasets were used, and image modifications such as blur, sharpness, brightness, and contrast were applied to assess the performance of YOLOv5 and RetinaNet models. The study revealed that the visual quality of images exerts a substantial impact on the effectiveness of the deep learning methods in detecting lymphocytes accurately. These findings have significant implications for deep learning approaches used in digital pathology.
IMPROVING STUDENTS’ ABILITY IN WRITING ANNOUNCEMENT THROUGH GALLERY WALK TECHNIQUE AT THE TENTH GRADE OF SMA NEGERI 1 GUNUNGSITOLI IDANOI IN 2022/2023

This research was aimed to improve the students’ ability in writing announcement through Gallery Walk Technique at tenth grade of SMA Negeri 1 Gunungsitoli Idanoi in 2022/2023. The researcher employed a classroom action research design conducted over two cycles. The research focused on the students of X-MIPA. The instruments utilized for this study were a writing test, and an observation sheet. The results of each cycle revealed that in Cycle I, the technique was not successful. Therefore, the research continued to the next cycle after revising the plan based on the weaknesses identified in Cycle I. The findings of Cycle II indicated that 7 students (33%) reached the 'good' level, while 14 students (67%) achieved the 'very good' level. These results demonstrate that the implementation of the 'Gallery Walk Technique' in this study successfully enhanced students’ writing announcement text.
The differences in the results of haemoglobin level examinations in female adolescents using digital and automated hematology analyzer methods

The prevalence of anemia in Indonesian female adolescents is still high, which is around 32%. This means that 3-4 out of 10 female adolescents suffer from anemia. Anemia will greatly affect the next cycle of adolescents’ health and productivity. Therefore, it is necessary to have routine screening to determine the prevalence of anemia. Routine screening is used to identify the incidence of anemia and can be done by examination of haemoglobin (Hb). The purpose of this study is to know the difference between Hb examination using the Digital (Hb meter) and automated hematology analyzer (AHA) with the sodium lauryl sulphate (SLS) method (AHA-SLS). The research method used was descriptive comparative with female adolescents as subjects from Maros, South Sulawesi. The results showed that in the digital method of Hb examination, 106 of 505 female adolescents were categorized as anemic. Meanwhile, in the AHA-SLS method, only 33 of 106 female adolescents suffered from anemia. Hence, there was a significant difference between the results of the Hb examination with the Digital method and the AHA-SLS method (p = 0.000)
Analysis and Improvement Approach of the Impact of Data Disturbance on LSTM Prediction Algorithm

If the sampling data with noise or outliers are used to train the long-short-term memory (short as LSTM) network, whether the perturbations and outliers in sampling data affect the training performance and prediction accuracy of LSTM networks is a key problem. This paper analyzed the impact on the LSTM prediction effect when using perturbations and isolated/patchy outliers involved data for network training and prediction. The results showed that the prediction accuracy decreases as the amplitude of the perturbations and the range of outliers increase. In order to overcome this effect, an improved method of Pre-set Outliers Tolerant Filter is proposed, and an Outliers-Tolerant Multi-LSTM model, in short, the OTML model, is established. The prediction effect of the proposed model is compared with that of the LSTM model without the filter and that of the LSTM model with a mean filter. Comparison results showed that the OTML prediction model proposed in this paper can eliminate the influence of random noise and isolated outliers thoroughly. When meeting the patchy outliers whose width is smaller than the radius of the window, the OTML prediction model can filter them out too, so as to realize the high-fidelity prediction of the data.
Aspect-based Text Classification for Sentimental Analysis using Attention mechanism with RU-BiLSTM

Sentiment analysis has gained increasing attention from an educational and social perspective with the huge expansion of user interactions due to the Web’s significant improvement. The connection between an opinion target’s polarity scores and other aspects of the content is defined by aspect-based sentiment analysis. Identifying aspects and determining their different polarities is quite complicated because they are frequently implicit. To overcome these difficulties, efficient hybrid methods are used in aspect-based text classification in sentiment analysis. The existing process evaluates the aspects of polarity by using a Convolutional neural network, and it does not work with Big data. In this work, aspect-based text classification and attention mechanisms are used to assist in filtering out irrelevant information and quickly locating the essential features in big data. Initially, the data is collected, and then the data is preprocessed by using Tokenization, Stop word removal, Stemming, and Lemmatization. After preprocessing, the features are vectorized and extracted using Bag-of-Words and TF-IDF. Then, the extracted features are given into word embeddings by GloVe and Word2vec. It uses Deep Recurrent based Bidirectional Long Short Term Memory (RUBiLSTM) for aspect-based sentiment analysis. The RU-Bi-LSTM method integrates aspect-based embeddings and an attention mechanism for text classification. The attention mechanism focuses on more crucial aspects and the bidirectional LSTM to maintain context in both ways. Finally, the binary and ternary classification outcomes are obtained using the final dense softmax output layer. The proposed RU-BiLSTM uses four reviews and two Twitter datasets. The results of the studies demonstrate the efficacy of the RU-BiLSTM model, which outperformed aspect-based classifications on lengthy reviews and short tweets in terms of evaluation.
GAME ONLINE’S EFFECTS TO EMOTIONAL AND SOCIAL DEVELOPMENT ON ULA’S PUPILS

The study aims to find out how online gaming affects the emotional and social development of ULA Sekolah Setara (basic). The method in this research uses a qualitative approach. The study targeted pupils of the ULA (basic) school age who played online games with a total of five children. The data collection techniques used in this study were three-day observations and interviews. Based on the results of interviews and observations done on five (5) ULA (basic) pupils who use online gaming, researchers found both positive and negative influences. These positive influences include learning about technology, understanding the various applications of online games, and learning new vocabulary in foreign languages. As for the negative influence of online games on the emotional and social development of children, children are more irritable, more aggressive, and lack interaction with the people around them.