Reviewerly/exHarmony huggingface dataset:
Increasing Student Learning Motivation Using Keroppi Pouch Learning Media in Social Sciences Subjects for Class V at SDN 024 North Samarinda

The researcher conducted a classroom action research to improve students’ learning motivation by using Keroppi pouch as the media of learning. Forty students of class V A at SD Negeri 024 Samarinda Utara were involved as research subjects. The research was conducted in SD Negeri 024 Samarinda Utara as the research setting. At the first cycle, students’ average score was 65.62 with 47.5% students achieved the passing grade. Therefore, the cycle must be continued to the second cycle. At the second cycle, students’ average score was 83.5 with 90% students reached the passing grade. In brief, it can be concluded that the implementation of Keroppi pouch has been effective to improve students’ learning motivation, especially for social science in class V A, since students managed to achieve the passing grade with very good category.
Novel class discovery meets foundation models for 3D semantic segmentation

The task of Novel Class Discovery (NCD) in semantic segmentation entails training a model able to accurately segment unlabelled (novel) classes, relying on the available supervision from annotated (base) classes. Although extensively investigated in 2D image data, the extension of the NCD task to the domain of 3D point clouds represents a pioneering effort, characterized by assumptions and challenges that are not present in the 2D case. This paper represents an advancement in the analysis of point cloud data in four directions. Firstly, it introduces the novel task of NCD for point cloud semantic segmentation. Secondly, it demonstrates that directly transposing the only existing NCD method for 2D image semantic segmentation to 3D data yields suboptimal results. Thirdly, a new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented. Lastly, a novel evaluation protocol is proposed to rigorously assess the performance of NCD in point cloud semantic segmentation. Through comprehensive evaluations on the SemanticKITTI, SemanticPOSS, and S3DIS datasets, the paper demonstrates substantial superiority of the proposed method over the considered baselines.
Reed-Muller codes have vanishing bit-error probability below capacity: a simple tighter proof via camellia boosting

This paper shows that a class of codes such as Reed-Muller (RM) codes have vanishing bit-error probability below capacity on symmetric channels. The proof relies on the notion of `camellia codes': a class of symmetric codes decomposable into `camellias', i.e., set systems that differ from sunflowers by allowing for scattered petal overlaps. The proof then follows from a boosting argument on the camellia petals with second moment Fourier analysis. For erasure channels, this gives a self-contained proof of the bit-error result in Kudekar et al.'17, without relying on sharp thresholds for monotone properties Friedgut-Kalai'96. For error channels, this gives a shortened proof of Reeves-Pfister'23 with an exponentially tighter bound, and a proof variant of the bit-error result in Abbe-Sandon'23. The control of the full (block) error probability still requires Abbe-Sandon'23 for RM codes.
Exploratory Search Prompt Generation using n-Degree Connection in Knowledge Graph

Search engines play a vital role in retrieving in- formation, but users often struggle to express their precise information needs, resulting in less-than-optimal search results. Therefore, enhancing search query refinement is crucial to elevate the accuracy and relevance of search outcomes. One particular challenge that existing search engines face is presenting refined results for queries containing two or more unrelated entities.&#x0D; This paper presents a novel approach for efficient search prompt generation by leveraging connected nodes and attributes in the knowledge graph. We propose a comprehensive exploration technique that explores the n-degrees connections and their at- tributes to generate all possible imaginative prompts. We realized that n-degrees connection exploration is an expensive task, hence we conducted experiments to determine the effectiveness of 2- degree exploration prompts in covering all the user-asked queries within the provided dataset.&#x0D; In testing with approximately 2000 queries on a related knowl- edge graph dataset, we found out that our proposed methodol- ogy significantly improved query coverage. At n=1 depth, the coverage increased from 58% without the methodology to 84% with it. At n=2 depth, the coverage rose from 92% without the methodology to nearly 99% with it. Additionally, due to our question caching strategy at n=2, we observed faster response times for all questions.
Measuring the Effectiveness of Internship Programs in Aligning Education with Industry: A Comprehensive Analysis of Internship Outcomes in the College of Communication and Media During COVID-19

Internships offer students and graduates a chance to gain practical experience in their field of study. Over ten weeks, assessing how university education translates into a professional setting can give insights to educational leaders about the efficacy of their training modules. For the Bachelor of Science students at the College of Communication and Media Sciences, internships are integral to their studies in Media Production and Storytelling, Integrated Strategic Communication, and Tourism Communication. This study employed Kirkpatrick’s framework, focusing on the “reaction” and “learning” constructs, and to gauge the alignment of theoretical teachings with practical experiences, 64 student reports from a semester-long online internship in various UAE organizations during COVID-19 were analyzed to highlight the strengths and areas of improvement in aligning academic instruction with real-world applications.
Analysis the Role of References in Scientific Articles: Influence on Research Credibility and Impact

References are an important part of a logical article that upholds the statements, contradictions, and objectives conveyed by the author. References can also be used to avoid theft of literary works. This test is expected to dissect the reference function in logic articles, specifically its effect on the validity and influence of exploration. The research used qualitative methods with a literature review approach, and data were obtained from scientific articles published in accredited scientific journals. The results show that references play an important role in increasing the credibility and impact of research. Relevant and quality references can support the author's claims and arguments and help readers understand the research better. In this journal concludes that authors should pay attention to the quality and relevance of references used to increase the credibility and impact of research.
The Influence of the Storytelling Method on the Speaking Ability of Children Aged 5-6 Years in Kindergarten St-Antonius 1 Medan

The purpose of this study was to determine the presence or absence of the influence of the storytelling method on the ability to speak children aged 5-6 years in TK ST-Antonius 1 Medan. This research was conducted at the St-Antonius 1 Medan TK which is located at Jalan HM Joni No. 52 A Medan. The population in this study was all children in TK ST-Antonius 1 Medan with a total of 103 children. The research design used is pretest-post-test design. Sampling techniques, namely with Purposive Sampling techniques in class B3 as many as 28 children. The data obtained were analyzed using non -parametric statistics Wilcoxon signs rank test. The results of the analysis obtained the average class value using a conventional storytelling method of 16.21 with SD = 3.26 and the average class value with an interactive storytelling method of 25.78. With SD = 2.10. From the hypothesis calculation data obtained asymp. Sig. (2-tailed) worth 0,000. smaller than 0.05 (&lt;0.05), it can be concluded that there is a significant effect of the interactive storytelling method on the ability to speak in children aged 5-6 years in kindergarten. St. Antonius 1 Medan compared to the ability to speak children taught by conventional storytelling methods.
Peningkatan Ketrampilan Guru SD dalam Pembuatan Video Pembelajaran dengan Menggunakan Telepon Cerdas

Learning in digital era demands innovation of learning materials where video is one of the attractive materials. Unfortunately, many teachers don't have skills to make learning videos, including some teachers at the Muhammadiyah Elementary School in Godean. Therefore, this program aims to improve the skills of partners in making learning videos. The main equipment used is a cell phone which is owned by all teachers so there is no need to procure additional equipment. The steps carried out are preparation, implementation and evaluation. Preparations have been made by discussing with partners about the training to be held. The training was completed on April 9, 2022 at the UMY Information Technology Laboratory, attended by 37 teachers from three Muhammadiyah elementary schools in Godean District. Participants received theory and immediately practiced making videos using their respective cell phones. Training evaluation was carried out using pre and post tests as well as observation. This program has succeeded in increasing teachers' knowledge and skills in making learning videos with a significant increase in pre-test and post-test results of 54.7%.
Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Model

Language models (LMs) exhibit and amplify many types of undesirable biases learned from the training data, including gender bias. However, we lack tools for effectively and efficiently changing this behavior without hurting general language modeling performance. In this paper, we study three methods for identifying causal relations between LM components and particular output: causal mediation analysis, automated circuit discovery and our novel, efficient method called DiffMask+ based on differential masking. We apply the methods to GPT-2 small and the problem of gender bias, and use the discovered sets of components to perform parameter-efficient fine-tuning for bias mitigation. Our results show significant overlap in the identified components (despite huge differences in the computational requirements of the methods) as well as success in mitigating gender bias, with less damage to general language modeling compared to full model fine-tuning. However, our work also underscores the difficulty of defining and measuring bias, and the sensitivity of causal discovery procedures to dataset choice. We hope our work can contribute to more attention for dataset development, and lead to more effective mitigation strategies for other types of bias.
On Enhancing Fine-Tuning for Pre-trained Language Models

The remarkable capabilities of Natural Language Models to grasp language subtleties has paved the way for their widespread adoption in diverse fields. However, adapting them for specific tasks requires the time-consuming process of fine-tuning, which consumes significant computational power and energy. Therefore, optimizing the fine-tuning time is advantageous. In this study, we propose an alternate approach that limits parameter manipulation to select layers. Our exploration led to identifying layers that offer the best trade-off between time optimization and performance preservation. We further validated this approach on multiple downstream tasks, and the results demonstrated its potential to reduce fine-tuning time by up to 50% while maintaining performance within a negligible deviation of less than 5%. This research showcases a promising technique for significantly improving fine-tuning efficiency without compromising task- or domain-specific learning capabilities.
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models

Pre-trained Language Models (PLMs) are trained on vast unlabeled data, rich in world knowledge. This fact has sparked the interest of the community in quantifying the amount of factual knowledge present in PLMs, as this explains their performance on downstream tasks, and potentially justifies their use as knowledge bases. In this work, we survey methods and datasets that are used to probe PLMs for factual knowledge. Our contributions are: (1) We propose a categorization scheme for factual probing methods that is based on how their inputs, outputs and the probed PLMs are adapted; (2) We provide an overview of the datasets used for factual probing; (3) We synthesize insights about knowledge retention and prompt optimization in PLMs, analyze obstacles to adopting PLMs as knowledge bases and outline directions for future work.
Understanding the Inner-workings of Language Models Through Representation Dissimilarity

As language models are applied to an increasing number of real-world applications, understanding their inner workings has become an important issue in model trust, interpretability, and transparency. In this work we show that representation dissimilarity measures, which are functions that measure the extent to which two model’s internal representations differ, can be a valuable tool for gaining insight into the mechanics of language models. Among our insights are: (i) an apparent asymmetry in the internal representations of model using SoLU and GeLU activation functions, (ii) evidence that dissimilarity measures can identify and locate generalization properties of models that are invisible via in-distribution test set performance, and (iii) new evaluations of how language model features vary as width and depth are increased. Our results suggest that dissimilarity measures are a promising set of tools for shedding light on the inner workings of language models.
Non-Autoregressive Document-Level Machine Translation

Non-autoregressive translation (NAT) models achieve comparable performance and superior speed compared to auto-regressive translation (AT) models in the context of sentence-level machine translation (MT). However, their abilities are unexplored in document-level MT, hindering their usage in real scenarios. In this paper, we conduct a comprehensive examination of typical NAT models in the context of document-level MT and further propose a simple but effective design of sentence alignment between source and target. Experiments show that NAT models achieve high acceleration on documents, and sentence alignment significantly enhances their performance. However, current NAT models still have a significant performance gap compared to their AT counterparts. Further investigation reveals that NAT models suffer more from the multi-modality and misalignment issues in the context of document-level MT, and current NAT models struggle with exploiting document context and handling discourse phenomena. We delve into these challenges and provide our code at https://github.com/baoguangsheng/nat-on-doc.
Topic-Informed Dialogue Summarization using Topic Distribution and Prompt-based Modeling

Dealing with multiple topics should be considered an important issue in dialogue summarization, because dialogues, unlike documents, are prone to topic drift. Thus, we propose a new dialogue summarization model that reflects dialogue topic distribution to consider all topics present in the dialogue. First, the distribution of dialogue topics is estimated by an effective topic discovery model. Then topic-informed prompt transfers estimated topic distribution information to the output of encoder and decoder vectors. Finally, the topic extractor estimates the summary topic distribution from the output context vector of decoder to distinguish its difference from the dialogue topic distribution. To consider the proportion of each topic distribution appeared in the dialogue, the extractor is trained to reduce the difference between the distributions of the dialogue and the summary. The experimental results on SAMSum and DialogSum show that our model outperforms state-of-the-art methods on ROUGE scores. The human evaluation results also show that our framework well generates comprehensive summaries.
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench

We investigate the predictability of large language model (LLM) capabilities: given records of past experiments using different model families, numbers of parameters, tasks, and numbers of in-context examples, can we accurately predict LLM performance on new experiment configurations? Answering this question has practical implications for LLM users (e.g., deciding which models to try), developers (e.g., prioritizing evaluation on representative tasks), and the research community (e.g., identifying hard-to-predict capabilities that warrant further investigation). We study the performance prediction problem on experiment records from BIG-bench. On a random train-test split, an MLP-based predictor achieves an R2 score greater than 95%, indicating the presence of learnable patterns within the experiment records. We then formulate the problem of searching for “small-bench,” an informative subset of BIG-bench tasks from which the performance on the full set can be maximally recovered. We find a subset as informative as BIG-bench Hard for evaluating new model families, while being 3× smaller. Additionally, we find competitive subsets by clustering task representations learned by our MLP-based predictor and selecting tasks close to cluster centroids, highlighting the importance of task diversity in constructing “small-bench.”
In-Context Learning Creates Task Vectors

In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In particular, it is challenging to map it to the “standard’ machine learning framework, where one uses a training set S to find a best-fitting function f(x) in some hypothesis class. Here we make progress on this problem by showing that the functions learned by ICL often have a very simple structure: they correspond to the transformer LLM whose only inputs are the query x and a single “task vector’ calculated from the training set. Thus, ICL can be seen as compressing S into a single task vector 𝜃(S) and then using this task vector to modulate the transformer to produce the output. We support the above claim via comprehensive experiments across a range of models and tasks.
Task-Agnostic Low-Rank Adapters for Unseen English Dialects

Large Language Models (LLMs) are trained on corpora disproportionally weighted in favor of Standard American English. As a result, speakers of other dialects experience significantly more failures when interacting with these technologies. In practice, these speakers often accommodate their speech to be better understood. Our work shares the belief that language technologies should be designed to accommodate the diversity in English dialects and not the other way around. However, prior work on dialect struggle with generalizing to evolving and emerging dialects in a scalable manner. To fill this gap, our method, HyperLoRA, leverages expert linguistic knowledge to enable resource-efficient adaptation via hypernetworks. By disentangling dialect-specific and cross-dialectal information, HyperLoRA improves generalization to unseen dialects in a task-agnostic fashion. Not only is HyperLoRA more scalable in the number of parameters, but it also achieves the best or most competitive performance across 5 dialects in a zero-shot setting. In this way, our approach facilitates access to language technology for billions of English dialect speakers who are traditionally underrepresented.
Non-autoregressive Text Editing with Copy-aware Latent Alignments

Recent work has witnessed a paradigm shift from Seq2Seq to Seq2Edit in the field of text editing, with the aim of addressing the slow autoregressive inference problem posed by the former. Despite promising results, Seq2Edit approaches still face several challenges such as inflexibility in generation and difficulty in generalizing to other languages. In this work, we propose a novel non-autoregressive text editing method to circumvent the above issues, by modeling the edit process with latent CTC alignments. We make a crucial extension to CTC by introducing the copy operation into the edit space, thus enabling more efficient management of textual overlap in editing. We conduct extensive experiments on GEC and sentence fusion tasks, showing that our proposed method significantly outperforms existing Seq2Edit models and achieves similar or even better results than Seq2Seq with over 4× speedup. Moreover, it demonstrates good generalizability on German and Russian. In-depth analyses reveal the strengths of our method in terms of the robustness under various scenarios and generating fluent and flexible outputs.
LLM-enhanced Self-training for Cross-domain Constituency Parsing

Self-training has proven to be an effective approach for cross-domain tasks, and in this study, we explore its application to cross-domain constituency parsing. Traditional self-training methods rely on limited and potentially low-quality raw corpora. To overcome this limitation, we propose enhancing self-training with the large language model (LLM) to generate domain-specific raw corpora iteratively. For the constituency parsing, we introduce grammar rules that guide the LLM in generating raw corpora and establish criteria for selecting pseudo instances. Our experimental results demonstrate that self-training for constituency parsing, equipped with an LLM, outperforms traditional methods regardless of the LLM’s performance. Moreover, the combination of grammar rules and confidence criteria for pseudo-data selection yields the highest performance in the cross-domain constituency parsing.
UvA-MT’s Participation in the WMT 2023 General Translation Shared Task

This paper describes the UvA-MT’s submission to the WMT 2023 shared task on general machine translation. We participate in the constrained track in two directions: English ↔ Hebrew. In this competition, we show that by using one model to handle bidirectional tasks, as a minimal setting of Multilingual Machine Translation (MMT), it is possible to achieve comparable results with that of traditional bilingual translation for both directions. By including effective strategies, like back-translation, re-parameterized embedding table, and task-oriented fine-tuning, we obtained competitive final results in the automatic evaluation for both English → Hebrew and Hebrew → English directions.
Quality Estimation Using Minimum Bayes Risk

This report describes the Minimum Bayes Risk Quality Estimation (MBR-QE) submission to the Workshop on Machine Translation’s 2023 Metrics Shared Task. MBR decoding with neural utility metrics like BLEURT is known to be effective in generating high quality machine translations. We use the underlying technique of MBR decoding and develop an MBR based reference-free quality estimation metric. Our method uses an evaluator machine translation system and a reference-based utility metric (specifically BLEURT and MetricX) to calculate a quality estimation score of a model. We report results related to comparing different MBR configurations and utility metrics.