Reviewerly/exHarmony huggingface dataset:
Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation

Question generation (QGen) models are often evaluated with standardized NLG metrics that are based on n-gram overlap. In this paper, we measure whether these metric improvements translate to gains in a practical setting, focusing on the use case of helping teachers automate the generation of reading comprehension quizzes. In our study, teachers building a quiz receive question suggestions, which they can either accept or refuse with a reason. Even though we find that recent progress in QGen leads to a significant increase in question acceptance rates, there is still large room for improvement, with the best model having only 68.4% of its questions accepted by the ten teachers who participated in our study. We then leverage the annotations we collected to analyze standard NLG metrics and find that model performance has reached projected upper-bounds, suggesting new automatic metrics are needed to guide QGen research forward.
Can Rationalization Improve Robustness?

A growing line of work has investigated the development of neural NLP models that can produce rationales--subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models can also provide robustness to adversarial attacks in addition to their interpretable nature. Since these models need to first generate rationales ("rationalizer") before making predictions ("predictor"), they have the potential to ignore noise or adversarially added text by simply masking it out of the generated rationale. To this end, we systematically generate various types of 'AddText' attacks for both token and sentence-level rationalization tasks, and perform an extensive empirical evaluation of state-of-the-art rationale models across five different tasks. Our experiments reveal that the rationale models show the promise to improve robustness, while they struggle in certain scenarios--when the rationalizer is sensitive to positional bias or lexical choices of attack text. Further, leveraging human rationale as supervision does not always translate to better performance. Our study is a first step towards exploring the interplay between interpretability and robustness in the rationalize-then-predict framework.
Unified Semantic Typing with Meaningful Label Inference

Semantic typing aims at classifying tokens or spans of interest in a textual context into semantic categories such as relations, entity types, and event types. The inferred labels of semantic categories meaningfully interpret how machines understand components of text. In this paper, we present UniST, a unified framework for semantic typing that captures label semantics by projecting both inputs and labels into a joint semantic embedding space. To formulate different lexical and relational semantic typing tasks as a unified task, we incorporate task descriptions to be jointly encoded with the input, allowing UniST to be adapted to different tasks without introducing task-specific model components. UniST optimizes a margin ranking loss such that the semantic relatedness of the input and labels is reflected from their embedding similarity. Our experiments demonstrate that UniST achieves strong performance across three semantic typing tasks: entity typing, relation classification and event typing. Meanwhile, UniST effectively transfers semantic knowledge of labels and substantially improves generalizability on inferring rarely seen and unseen types. In addition, multiple semantic typing tasks can be jointly trained within the unified framework, leading to a single compact multi-tasking model that performs comparably to dedicated single-task models, while offering even better transferability.
ATDD: Fine-Grained Assured Time-Sensitive Data Deletion Scheme in Cloud Storage

With the rapid development of general cloud services, more and more individuals or collectives use cloud platforms to store data. Assured data deletion deserves investigation in cloud storage. In time-sensitive data storage scenarios, it is necessary for cloud platforms to automatically destroy data after the data owner-specified expiration time. Therefore, assured timesensitive data deletion should be sought. In this paper, a finegrained assured time-sensitive data deletion (ATDD) scheme in cloud storage is proposed by embedding the time trapdoor in Ciphertext-Policy Attribute-Based Encryption (CP-ABE). Timesensitive data is self-destructed after the data owner-specified expiration time so that the authorized users cannot get access to the related data. In addition, a credential is returned to the data owner for data deletion verification. This proposed scheme provides solutions for fine-grained access control and verifiable data self-destruction. Detailed security and performance analysis demonstrate the security and the practicability of the proposed scheme.
DESKTOP BASED LIBRARY DATA COLLECTION SYSTEM

The problems encountered during the research, namely that the library service system is not yet computerized, such as borrowing books and returning books, is still recorded in the form of bookkeeping, while searching for book data is still manual which takes a long time. Services that still use manual processes will lead to slow library work which results in student dissatisfaction with library services.
A Conceptual Model For a Decentralized Central Bank Digital Currency

Central bank digital currency (CBDC) is a digitized fiat currency. As the nature of the central bank is centralized, the CBDC is also centralized. This paper proposes a conceptual model for a decentralized CBDC controlled by many central banks or countries in the world. It is only for international transactions between member countries. While domestic transactions continue to use the national currency of each country. A decentralized CBDC could potentially explore the advantages of digital technologies more deeply than centralized ones by making macro-fundamentals reconciliations between economies in real-time. Furthermore, this concept could also potentially provide international liquidity for all (member) countries in the world sustainably, align macro-fundamental, eliminate global imbalances, and makes the international monetary system naturally more stable. This paper focuses on discussing the conceptual model and theoretical framework of how this system works in providing international liquidity and making the system balanced and stable.
Explainable deepfake and spoofing detection: an attack analysis using SHapley Additive exPlanations

Despite several years of research in deepfake and spoofing detection for automatic speaker verification, little is known about the artefacts that classifiers use to distinguish between bona fide and spoofed utterances. An understanding of these is crucial to the design of trustworthy, explainable solutions. In this paper we report an extension of our previous work to better understand classifier behaviour to the use of SHapley Additive exPlanations (SHAP) to attack analysis. Our goal is to identify the artefacts that characterise utterances generated by different attacks algorithms. Using a pair of classifiers which operate either upon raw waveforms or magnitude spectrograms, we show that visualisations of SHAP results can be used to identify attack-specific artefacts and the differences and consistencies between synthetic speech and converted voice spoofing attacks.
Using Docker virtual containers to launch services

Docker is a virtualization tool that allows developers and system administrators to run applications and services in containers isolated from each other and from the guest operating system. Docker containers, unlike other virtualization tools, work directly with the host machine hardware, which is good for system resource consumption. Containers run from prebuilt images. The ability to collect all dependencies and application data in an image allows you not to worry about compatibility with the systems in which the container will run. The independence of containers from each other allows you to run entire groups of related services, in which a change of version or data in one of the containers does not affect the functioning of the rest of the group.
Minimizing Coordination in Multi-Agent Path Finding with Dynamic Execution

Multi-agent Path Finding (MAPF) is an important problem in large games with many dynamic agents that need to follow space-time trajectories without inter-agent collisions. Modern MAPF solvers plan assuming that agents directly follow the space-time trajectories at known constant speeds without delays or speedups, resulting in rigid plans which need to be replanned if there are changes during execution. Instead we would like agents to be able to follow their computed paths with dynamic velocities while requiring minimal coordination with others to prevent collisions and deadlocks. One way to address this problem is to first produce collision free space-time paths and then compute a coordination controller that prevents collisions and deadlock during dynamic execution. This two step process prevents fully minimizing coordination as the initially planned space-time paths do not reason about coordination and can be arbitrarily bad. We introduce a novel paradigm and show how planning in space-coordination level, rather than space-time, allows us to simultaneously plan paths and a coordination controller. Our method, Space-Level Conflict-Based Search (SL-CBS), builds on the Conflict-Based Search framework and allows us to reason explicitly about coordination, producing paths as well as a coordination controller with bounded suboptimal minimal coordination. We show experimentally that this results in a 20-50% reduction in coordination compared to the closest state of the art solver.
Robust Player Plan Recognition in Digital Games with Multi-Task Multi-Label Learning

Plan recognition is a key component of player modeling. Player plan recognition focuses on modeling how and when players select goals and formulate action sequences to achieve their goals during gameplay. By occasionally asking players to describe their plans, it is possible to devise robust plan recognition models that jointly reason about player goals and action sequences in coordination with player input. In this work, we present a player plan recognition framework that leverages data from player interactions with a planning support tool embedded in an educational game for middle school science education, CRYSTAL ISLAND. Players are prompted to use the planning tool to describe their goals and planned actions in CRYSTAL ISLAND. We use this data to devise data-driven player plan recognition models using multi-label multi-task learning. Specifically, we compare single-task and multi-task learning approaches for both goal prediction and action sequence prediction. Results indicate that multi-task learning yields significant benefits for action sequence prediction. Additionally, we find that incorporating automated detectors of plan completion in plan recognition models improves predictive performance in both tasks.
The Nonsense Laboratory

We present the Nonsense Laboratory, a series of playful web-based tools for manipulating the way words sound and the way words are spelled. The Laboratory's various playful interfaces give users an opportunity to adjust, poke at, mangle, curate, rearrange and elaborate the sounds of words in written text. The project makes use of Pincelate, a code library and machine learning model for phoneme-to-grapheme and grapheme-to-phoneme tasks in English, trained on the CMU Pronouncing Dictionary. The goal of the project is twofold: first, to make possible new ways of manipulating spelling (akin to playing a musical instrument or modeling with clay); and second, to demystify machine learning by providing an intuitive, friendly interface to a machine learning model. We briefly describe the design and implementation of the Nonsense Laboratory's five interfaces, and situate the project in our own research concerning design, literary studies, linguistics, and machine learning.
Automated Play-Testing through RL Based Human-Like Play-Styles Generation

The increasing complexity of gameplay mechanisms in modern video games is leading to the emergence of a wider range of ways to play games. The variety of possible play-styles needs to be anticipated and taken into account by designers, through automated tests. Reinforcement Learning (RL) is a promising answer to the need of automating video game testing. To that effect one needs to train an agent to play the game, while ensuring this agent will generate the same play-styles as the players in order to give meaningful feedback to the designers. We present CARMI : a Configurable Agent with Relative Metrics as Input. An agent able to emulate the players play-styles, even on previously unseen levels. Unlike current methods it does not rely on having full trajectories, but only summary data. Moreover it only requires little human data, thus compatible with the constraints of modern video game production. This novel agent could be used to investigate behaviors and balancing during the production of a video game with a realistic amount of training time.
The Relevance of English Grammar Materials Used at Eed Study Program and Three-Dimensional Grammar Framework: A Case of Passive Voice

Grammar is an important language aspect in both spoken and written language uses. However, most students at English Education Department Unram found it difficult to learn. It is shown in their TOEFL achievement. The failure may come from various factors, one of which is learning materials. This study is conducted in order to find out whether the course book used by the English Education students using Three-Dimensional Grammar Framework as this framework is created to achieve balance between grammar and communication because the objectives of learning grammar in university level have not been well achieved by the students. This study is qualitative research. The source of the data in this study is the course book used by EED students at University of Mataram entitled English Grammar 2 and focused on the passive voice materials. The data is collected by using documentation. The result of this study is the course book English Grammar 2 has applied the components of Three-Dimensional Grammar Framework such as forms/structures of passive voice have been presented completely as seen from the curriculum, meaning/semantic and use/pragmatic of passive voice have been presented. However, the uses of passive voice such as (1) when the non-agent is more closely related than the agent to the theme of the text, (2) when the non-agent is participant in the immediately preceding sentence, and (3) the speaker is being tactful and evasive are not presented.
Indonesian User Perception on the Usefulness of Auto-translate Feature on Social Media

This paper focuses on the user perception of their experience in using the auto-translate feature on social media. A mixed method is employed in this study to gain quantitative and qualitative results and provide a more in-depth understanding of the analysis, with an online questionnaire as the research instrument. In order to reach a broader audience, snowball sampling is used since there are no specific criteria for the subject target. The majority of the respondents are women with bachelor's degrees aged 17-28. There are three aspects of assessment on the questionnaire following Nababan’s theory: accuracy, acceptability, and readability. The findings showed that the result of the auto-translate feature on social media has good accuracy, moderate readability, and is very acceptable. Meanwhile, the shortcomings of this feature are lack of context understanding, mistranslation due to wrong diction, and grammar updates.
Explainable CLIP-Guided 3D-Scene Generation in an AI Holodeck

This paper describes the AI Holodeck, a co-creative software prototype that creates virtual scenes from the user input text, inspired by the fictional Holodeck virtual reality device from the science fiction series Star Trek. This application collects common-sense knowledge from annotated datasets and reference images. It uses this knowledge to populate scenes with objects found in selected environments alongside those explicitly mentioned by the user. We present the system design of the AI Holodeck, and a proposed study to measure the effects of its visualizations on user perceptions of the system's creativity.
HeadSpace: Incorporating Action Failure and Character Beliefs into Narrative Planning

Previous work on story planning has lacked a knowledge representation for characters that attempt actions that fail because of the characters' misconceptions about the world state. This work describes HeadSpace, a state-space heuristic search planning system that generates stories that track and manipulate characters' beliefs about the story world. The planner produces story plans with actions that are attempted but fail. We show an example story plan that contains failed-action content that cannot be generated by typical planning-based approaches to story creation, and we provide an analytical evaluation that characterizes our planner's increased expressive range relative to other narrative planners addressing character belief and/or failed action execution.
Open-World Narrative Generation to Answer Players’ Questions

Planning-based narrative generation is effective at producing stories with a logically-sound flow of events, but it can be limiting due to the rigidity of its constraints and the high burden on the domain author to define story-world objects, initial states, and author and character goals. Giving the system the freedom to add objects and events to the story-world history arbitrarily can improve variety and reduce authorial burden, but risks leading to stories that seem jarringly contrived to the audience. I propose to use question-answering as the antidote to contrivance in a highly-generative interactive narrative system: By modeling the player's beliefs about the story world, inferring the implicit questions the player may be asking through their interactions, and answering those questions in a way consistent with the player's prior knowledge, a system could focus on creating cohesion in the ways that matter most to the player while accepting a degree of contrivance in the details that the player is likely to overlook.
Evaluating Reader Comprehension of Plan-Based Stories Containing Failed Actions

A growing number of algorithms for story planning include the ability to create stories with failed actions -- in particular failed actions that occur because of the mistaken beliefs of the characters attempting them. To date, most of these systems have been evaluated analytically, primarily by comparing their expressive range to prior story generation systems. Empirical evaluation of these systems has been preliminary. In this paper, we outline a general comprehension-based approach to the evaluation of plan-based story generation. We describe how we specialize it for use evaluating story plans containing failed actions, and we describe the design and results of an experiment using this approach to evaluate plot lines produced by HeadSpace, a system that models the beliefs of characters and uses that model to generate plot lines containing actions that are attempted but that fail.
Generosity in Indonesian Literature

This Indonesian Literary Novel contains various kinds of events, such as real human events that live in the world, both fictional and reality. Every good literary work always carries ethical and moral messages, so that good literature always voices ethical, moral appeals and is full of trust. The problem in this research, namely: what are the implications of philanthropic attitudes in character education in Indonesian literature (novels)?. The aim is to find out the description of the content of the attitude of generosity contained in Indonesian literary works (novels) related to character education. The benefits of the results of this study as reading material or additional knowledge in studying and understanding literary works, especially events related to the generosity of figures in Indonesian literature (novels). The method in this study uses a qualitative approach, research data. The data of this study are in the form of words, phrases, sentences, in the dialogues of the characters in Indonesian literary novels. The research data includes: items in the theory of philanthropy and character education as well as data sources, namely the text of Indonesian literary novels. The data collection technique in this research is text analysis in the form of language exposure in the novel. This technique is used in the form of textual descriptions. This technique is used to collect research data in the form of literary works, namely: the researcher reads carefully from the data source; the researcher identifies the units of events in each paragraph; The researcher puts a mark on the exposure of the language contained in the novel. The instrument is in the form of a data collection guide containing the data columns found and then presented and discussed. The data collection technique in this research is text analysis in the form of language exposure in the novel. This technique is used in data collection procedures carried out to obtain data. The results of this study are expected to be used as an additional basis for the concept of reference for researchers who are interested in further deepening research on literature, especially the disciplines of philanthropy and character education. This writing produces kindness to the poor, kindness in prescribing medicine, not being arrogant, caring for abandoned babies, caring for others, helping sick people, helping children whose fathers are in prison.
Foreign Teachers’ Insights Regarding Teaching “Conversation” in Boarding School

Speaking becomes one of the English skills that should be developed. One of the schools that provided a special speaking class for the students was the Islamic Boarding Based Junior High School of Amanatul Ummah. Speaking class was labeled as a conversation subject in this school. In conversation class, the teachers are required to improve the students’ speaking skills. The teachers who handled a conversation subject were foreign teachers who used full English during teaching and learning. This paper investigates the foreign teachers’ perspectives of the teaching conversation subject. This research used descriptive qualitative. The data was obtained through observation, semi-structured interviews, and documentation. The result of this research showed that the foreign teachers faced several problems related to the students’ understanding, engagement, and teaching materials. The strategies used by the teachers were using digital tools, gestures, language simplifying, early materials preparation, and discussion with senior EFL teachers. This research contributes to the literature of English language teaching related to the teaching conversation subject.
Salience Vectors for Measuring Distance between Stories

Narrative planners generate sequences of actions that represent story plots given a story domain model. This is a useful way to create branching stories for interactive narrative systems that maintain logical consistency across multiple storylines with different content. There is a need for story comparison techniques that can enable systems like experience managers and domain authoring tools to reason about similarities and differences between multiple stories or branches. We present an algorithm for summarizing narrative plans as numeric vectors based on a cognitive model of human story perception. The vectors encode important story information and can be compared using standard distance functions to quantify the overall semantic difference between two stories. We show that this distance metric is highly accurate based on human annotations of story similarity, and compare it to several alternative approaches. We also explore variations of our method in an attempt to broaden its applicability to other types of story systems.