Reviewerly/exHarmony huggingface dataset:
MRRL: Modifying the Reference via Reinforcement Learning for Non-Autoregressive Joint Multiple Intent Detection and Slot Filling

With the rise of non-autoregressive approach, some non-autoregressive models for joint multiple intent detection and slot filling have obtained the promising inference speed. However, most existing SLU models (1) suffer from the multi-modality problem that leads to reference intents and slots may not be suitable for training; (2) lack of alignment between the correct predictions of the two tasks, which extremely limits the overall accuracy. Therefore, in this paper, we propose Modifying the Reference via Reinforcement Learning (MRRL), a novel method for multiple intent detection and slot filling, which introduces a modifier and employs reinforcement learning. Specifically, we try to provide the better training target for the non-autoregressive SLU model via modifying the reference based on the output of the non-autoregressive SLU model, and propose a suitability reward to ensure that the output of the modifier module could fit well with the output of the non-autoregressive SLU model and does not deviate too far from the reference. In addition, we also propose a compromise reward to realize a flexible trade-off between the two subtasks. Experiments on two multi-intent datasets and non-autoregressive baselines demonstrate that our MRRL could consistently improve the performance of baselines. More encouragingly, our best variant achieves new state-of-the-art results, outperforming the previous best approach by 3.6 overall accuracy on MixATIS dataset.
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP

ChatGPT’s emergence heralds a transformative phase in NLP, particularly demonstrated through its excellent performance on many English benchmarks. However, the model’s efficacy across diverse linguistic contexts remains largely uncharted territory. This work aims to bridge this knowledge gap, with a primary focus on assessing ChatGPT’s capabilities on Arabic languages and dialectal varieties. Our comprehensive study conducts a large-scale automated and human evaluation of ChatGPT, encompassing 44 distinct language understanding and generation tasks on over 60 different datasets. To our knowledge, this marks the first extensive performance analysis of ChatGPT’s deployment in Arabic NLP. Our findings indicate that, despite its remarkable performance in English, ChatGPT is consistently surpassed by smaller models that have undergone finetuning on Arabic. We further undertake a meticulous comparison of ChatGPT and GPT-4’s Modern Standard Arabic (MSA) and Dialectal Arabic (DA), unveiling the relative shortcomings of both models in handling Arabic dialects compared to MSA. Although we further explore and confirm the utility of employing GPT-4 as a potential alternative for human evaluation, our work adds to a growing body of research underscoring the limitations of ChatGPT.
Towards Noise-Tolerant Speech-Referring Video Object Segmentation: Bridging Speech and Text

Linguistic communication is prevalent in Human-Computer Interaction (HCI). Speech (spoken language) serves as a convenient yet potentially ambiguous form due to noise and accents, exposing a gap compared to text. In this study, we investigate the prominent HCI task, Referring Video Object Segmentation (R-VOS), which aims to segment and track objects using linguistic references. While text input is well-investigated, speech input is under-explored. Our objective is to bridge the gap between speech and text, enabling the adaptation of existing text-input R-VOS models to accommodate noisy speech input effectively. Specifically, we propose a method to align the semantic spaces between speech and text by incorporating two key modules: 1) Noise-Aware Semantic Adjustment (NSA) for clear semantics extraction from noisy speech; and 2) Semantic Jitter Suppression (SJS) enabling R-VOS models to tolerate noisy queries. Comprehensive experiments conducted on the challenging AVOS benchmarks reveal that our proposed method outperforms state-of-the-art approaches.
Execution-Based Evaluation for Open-Domain Code Generation

To extend the scope of coding queries to more realistic settings, we propose ODEX, the first Open-Domain EXecution-based natural language (NL) to Python code generation dataset. ODEX has 945 NL-Code pairs spanning 79 diverse libraries, along with 1,707 human-written test cases for execution. Our NL-Code pairs are harvested from StackOverflow forums to encourage natural and practical coding queries. Moreover, ODEX supports four natural languages as intents, in English, Spanish, Japanese, and Russian. ODEX unveils intriguing behavioral differences among top-performing code language models (LM). While CODEX achieves better overall results, CODEGEN improves effectively via scaling – CODEGEN 6.1B performs comparably with CODEX 12B. Both models show substantial gaps between open and closed domains, but CODEGEN gaps tend to decrease with model size while CODEX gaps increase. We release ODEX to facilitate research into open-domain problems for the code generation community.
Handshape-Aware Sign Language Recognition: Extended Datasets and Exploration of Handshape-Inclusive Methods

The majority of existing work on sign language recognition encodes signed videos without explicitly acknowledging the phonological attributes of signs. Given that handshape is a vital parameter in sign languages, we explore the potential of handshape-aware sign language recognition. We augment the PHOENIX14T dataset with gloss-level handshape labels, resulting in the new PHOENIX14T-HS dataset. Two unique methods are proposed for handshape-inclusive sign language recognition: a single-encoder network and a dual-encoder network, complemented by a training strategy that simultaneously optimizes both the CTC loss and frame-level cross-entropy loss. The proposed methodology consistently outperforms the baseline performance. The dataset and code can be accessed at: www.anonymous.com.
Model-tuning Via Prompts Makes NLP Models Adversarially Robust

In recent years, NLP practitioners have converged on the following practice: (i) import an off-the-shelf pretrained (masked) language model; (ii) append a multilayer perceptron atop the CLS token’s hidden representation (with randomly initialized weights); and (iii) fine-tune the entire model on a downstream task (MLP-FT). This procedure has produced massive gains on standard NLP benchmarks, but these models remain brittle, even to mild adversarial perturbations. In this work, we demonstrate surprising gains in adversarial robustness enjoyed by Model-tuning Via Prompts (MVP), an alternative method of adapting to downstream tasks. Rather than appending an MLP head to make output prediction, MVP appends a prompt template to the input, and makes prediction via text infilling/completion. Across 5 NLP datasets, 4 adversarial attacks, and 3 different models, MVP improves performance against adversarial substitutions by an average of 8% over standard methods and even outperforms adversarial training-based state-of-art defenses by 3.5%. By combining MVP with adversarial training, we achieve further improvements in adversarial robustness while maintaining performance on unperturbed examples. Finally, we conduct ablations to investigate the mechanism underlying these gains. Notably, we find that the main causes of vulnerability of MLP-FT can be attributed to the misalignment between pre-training and fine-tuning tasks, and the randomly initialized MLP parameters.
Context-faithful Prompting for Large Language Models

Large language models (LLMs) encode parametric knowledge about world facts and have shown remarkable performance in knowledge-driven NLP tasks. However, their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks (e.g., knowledge acquisition tasks). In this paper, we seek to assess and enhance LLMs’ contextual faithfulness in two aspects: knowledge conflict and prediction with abstention. We demonstrate that LLMs’ faithfulness can be significantly improved using carefully designed prompting strategies. In particular, we identify opinion-based prompts and counterfactual demonstrations as the most effective methods. Opinion-based prompts reframe the context as a narrator’s statement and inquire about the narrator’s opinions, while counterfactual demonstrations use instances containing false facts to improve faithfulness in knowledge conflict situations. Neither technique requires additional training. We conduct experiments on three datasets of two standard NLP tasks, machine reading comprehension and relation extraction, and the results demonstrate significant improvement in faithfulness to contexts. Code and data are released at https://github.com/wzhouad/context-faithful-llm.
FinGPT: Large Generative Models for a Small Language

Risto Luukkonen, Ville Komulainen, Jouni Luoma, Anni Eskelinen, Jenna Kanerva, Hanna-Mari Kupari, Filip Ginter, Veronika Laippala, Niklas Muennighoff, Aleksandra Piktus, Thomas Wang, Nouamane Tazi, Teven Scao, Thomas Wolf, Osma Suominen, Samuli Sairanen, Mikko Merioksa, Jyrki Heinonen, Aija Vahtola, Samuel Antao, Sampo Pyysalo. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
Cross-modality Data Augmentation for End-to-End Sign Language Translation

End-to-end sign language translation (SLT) aims to directly convert sign language videos into spoken language texts without intermediate representations. It has been challenging due to the data scarcity of labeled data and the modality gap between sign videos and texts. To tackle these challenges, we propose a novel Cross-modality Data Augmentation (XmDA) framework to transfer the powerful gloss-to-text translation capabilities to end-to-end sign language translation (i.e., video-to-text). Specifically, XmDA consists of two key components: cross-modality mix-up and cross-modality knowledge distillation. The former one explicitly encourages the alignment between sign video features and gloss embeddings to bridge the modality gap. The latter one utilizes the generation knowledge from gloss-to-text teacher models to guide the spoken language text generation. Experimental results on two widely used SLT datasets, i.e., PHOENIX-2014T and CSL-Daily, demonstrate that the proposed XmDA framework significantly and consistently outperforms the baseline models. Extensive analyses confirm our claim that XmDA enhances end-to-end sign language translation by reducing the representation distance between sign videos and glosses, as well as improving the translation of low-frequency words and long sentences.
RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction

How to identify semantic relations among entities in a document when only a few labeled documents are available? Few-shot document-level relation extraction (FSDLRE) is crucial for addressing the pervasive data scarcity problem in real-world scenarios. Metric-based meta-learning is an effective framework widely adopted for FSDLRE, which constructs class prototypes for classification. However, existing works often struggle to obtain class prototypes with accurate relational semantics: 1) To build prototype for a target relation type, they aggregate the representations of all entity pairs holding that relation, while these entity pairs may also hold other relations, thus disturbing the prototype. 2) They use a set of generic NOTA (none-of-the-above) prototypes across all tasks, neglecting that the NOTA semantics differs in tasks with different target relation types. In this paper, we propose a relation-aware prototype learning method for FSDLRE to strengthen the relational semantics of prototype representations. By judiciously leveraging the relation descriptions and realistic NOTA instances as guidance, our method effectively refines the relation prototypes and generates task-specific NOTA prototypes. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches by average 2.61% F1 across various settings of two FSDLRE benchmarks.
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback

Large language models (LLMs) like ChatGPT have exhibited remarkable abilities on a wide range of natural language processing (NLP) tasks, including various machine translation abilities accomplished during chat. However, these models are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. Therefore, we propose ParroT, a framework to enhance and regulate the translation abilities during chat based on open-source LLMs (e.g., LLaMA), human-written translation and feedback data. Specifically, ParroT reformulates translation data into the instruction-following style, and introduces a “Hint” field for incorporating extra requirements to regulate the translation process. Accordingly, we propose three instruction types for finetuning ParroT models, including translation instruction, contrastive instruction, and error-guided instruction. Experiments on Flores subsets and WMT22 test sets suggest that translation instruction improves the translation performance of vanilla LLMs significantly while error-guided instruction can lead to further improvement, which demonstrates the importance of learning from low-quality translations annotated by humans. We also demonstrate the potential of automatic evaluation tools in providing quality information of translations, when constructing error-guided instructions for directions that lack human annotation data. Please refer to our Github project for more implementation details: https://github.com/wxjiao/ParroT.
The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation

Dung Nguyen Manh, Nam Le Hai, Anh T. V. Dau, Anh Minh Nguyen, Khanh Nghiem, Jin Guo, Nghi D. Q. Bui. Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023). 2023.
Analysis of Web-Based Geographic Information System Mapping of Broken Roads in Kampar Regency

Road is a very vital land transportation infrastructure and transportation route. Apart from being transportation, roads also function as a medium of socialization and accessibility for the community. Damage to the road is a big problem for some people who use the road every day. Therefore, a public complaint system is needed to the authorities in reporting the location of damaged roads, so that road repairs can be carried out quickly. The purpose of this study is to make it easier for the public to report damaged road conditions using a damaged road reporting system website that can be accessed online. The research method used in this study is the waterfall method. This waterfall method has a sequential flow of software starting from planning, analysis, design, implementation, testing and maintenance. The design of this system uses Unified Modeling Language (UML) and Object Oriented Analysis Design (OOAD). The programming language used is PHP Framework Yii and MySQL database. This research was conducted on a website reporting damaged roads in Kampar Regency which also provides information on roads that have been repaired in Kampar Regency and provides information related to the Kampar PUPR Service. With this system, PUPR's performance in repairing damaged roads that have been reported by the general public will be better.
A Deep Reinforcement Learning Control Method for a Four-Link Brachiation Robot

Brachiation is a common way for primates to move between treetops. This movement has the characteristics of adapting to complex, discontinuous environment and low energy consumption. However, traditional control methods are often difficult to complete such tasks where the support is point-contact and discrete. Reinforcement learning (RL) gives a solution to such tasks due to its strong ability to adapt to complex environments. Therefore, in this paper, we design a four-link underactuated robot model with a pair of suspended hooks and propose a deep reinforcement learning-based control method to realize its brachiation between bars. The policy for outputting control signals is trained using the proximal policy optimization (PPO) algorithm because of its strong performance and ability to handle continuous action spaces. A reward function considering energy consumption and the challenges posed by the introduction of hooks is designed to obtain an energy-optimized control policy. Through simulation, a comparison is drawn against the author's previously proposed method of generating energy-optimal offline trajectories and tracking by model predictive control (MPC), thereby substantiating the superiority of the proposed method outlined in this paper.
Deepfake Generation and Detection: Case Study and Challenges

In smart communities, social media has allowed users easy access to multimedia content. With recent advancements in computer vision and natural language processing, machine learning (ML), and deep learning (DL) models have evolved. With advancements in generative adversarial networks (GAN), it has become possible to create fake images/audio/and video streams of a person or use some person’s audio and visual details to fit other environments. Thus, deepfakes are specifically used to disseminate fake information and propaganda on social circles that tarnish the reputation of an individual or an organization. Recently, many surveys have focused on generating and detecting deepfake images, audio, and video streams. Existing surveys are mostly aligned toward detecting deepfake contents, but the generation process is not suitably discussed. To address the survey gap, the paper proposes a comprehensive review of deepfake generation and detection and the different ML/DL approaches to synthesize deepfake contents. We discuss a comparative analysis of deepfake models and public datasets present for deepfake detection purposes. We discuss the implementation challenges and future research directions regarding optimized approaches and models. A unique case study, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IBMM</i> is discussed, which presents a multi-modal overview of deepfake detection. The proposed survey would benefit researchers, industry, and academia to study deepfake generation and subsequent detection schemes.
Computation Offloading and Resource Allocation in Failure-Aware Vehicular Edge Computing

The advent of Intelligent Cyber-Physical Transportation Systems (ICTS) has not only accelerated the reformation and evolvement of smart transportation, but also ushered in a new era of vehicular applications. These applications typically impose stringent latency requirements and demand substantial computing resources. Vehicular edge computing (VEC) has emerged as an efficient solution to address these challenges, leveraging its inherent ability to provide ultra-low latency services. Existing studies primarily concentrate on either optimizing resource allocation or minimizing response latency, while ignoring the fact that the task execution in VEC is more susceptible to failures compared to cloud computing environments. Accordingly, we design a cost-efficient and failure-resistant task offloading strategy for VEC systems with the goal of minimizing the average response latency for all tasks. Specifically, our problem is modeled as a nonlinear multi-constraint continuous optimization problem, with tightly coupled optimization variables in the objective function and constraints. To tackle this issue, we initially decompose the optimization problem into per-slot optimization subproblems. Subsequently, we employ an effective algorithm with low time complexity to solve these subproblems in a slot-by-slot manner. We comprehensively evaluate the performance of our approach through extensive simulations, demonstrating that our method outperforms the baseline approaches in various aspects.
Prediction of chronic kidney disease progression using recurrent neural network and electronic health records

Chronic kidney disease (CKD) is a progressive loss in kidney function. Early detection of patients who will progress to late-stage CKD is of paramount importance for patient care. To address this, we develop a pipeline to process longitudinal electronic heath records (EHRs) and construct recurrent neural network (RNN) models to predict CKD progression from stages II/III to stages IV/V. The RNN model generates predictions based on time-series records of patients, including repeated lab tests and other clinical variables. Our investigation reveals that using a single variable, the recorded estimated glomerular filtration rate (eGFR) over time, the RNN model achieves an average area under the receiver operating characteristic curve (AUROC) of 0.957 for predicting future CKD progression. When additional clinical variables, such as demographics, vital information, lab test results, and health behaviors, are incorporated, the average AUROC increases to 0.967. In both scenarios, the standard deviation of the AUROC across cross-validation trials is less than 0.01, indicating a stable and high prediction accuracy. Our analysis results demonstrate the proposed RNN model outperforms existing standard approaches, including static and dynamic Cox proportional hazards models, random forest, and LightGBM. The utilization of the RNN model and the time-series data of previous eGFR measurements underscores its potential as a straightforward and effective tool for assessing the clinical risk of CKD patients concerning their disease progression.
Global path planning method for mobile vehicles based on Goal-Optimized JPS algorithm

The Jump Point Search algorithm (JPS) lacks goal direction guidance during search, resulting in increased redundant nodes and decreased computational efficiency. This paper proposes a Goal-Optimized JPS algorithm that constructs search direction priorities based on the direction vector of the goal point, while improving the cost function. The experimental results show that compared to the traditional JPS algorithm, the path obtained by Goal-Optimized JPS algorithm is shorter and the calculation time is reduced by more than 30%.
Blockchain and AI in Reducing Inventory Fraud and Errors

Abstract: This paper examines the integration of Blockchain and Artificial Intelligence (AI) in enhancing inventory management. It highlights how these technologies synergistically improve accuracy and efficiency, significantly reducing fraud and errors. The study explores Blockchain's secure ledger and AI's predictive analytics, emphasizing their practical applications in various industries. Challenges such as technical complexities and ethical considerations, including data privacy and regulatory compliance, are also addressed. The paper concludes by discussing the implications for businesses and researchers, underscoring the transformative impact of these technologies in inventory management and the necessity for ongoing innovation and ethical vigilance
Analisis Manajemen Risiko Aplikasi SINTESA Pada Perpustakaan XYZ

The Sintesa application is an operation support information system application that functions to assist the XYZ Library in recording loan books, a list of available books, the number of books, book loan maturities, and a list of book borrowers. In implementing this information system, there are certain risks that will hinder the process or function of the system, so a risk analysis is needed to prevent the risks that will occur. By using ISO 31000 in the XYZ library, it is hoped that it can minimize the possible risks that will occur around the SINTESA application. The results of the risk analysis with ISO 31000 in the form of documentation of possible risks that exist around the SINTESA application, classifying the possibilities - risks based on their impact. So that the results of this risk analysis can be used by the library to prevent and minimize risks and treat these risks according to the impact or risk priority that may occur in the library application