Reviewerly/exHarmony huggingface dataset:
Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models

Large language models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems. However, a major drawback of LLMs is the issue of hallucination, where they generate unfaithful or inconsistent content that deviates from the input source, leading to severe consequences. In this paper, we propose a robust discriminator named RelD to effectively detect hallucination in LLMs' generated answers. RelD is trained on the constructed RelQA, a bilingual question-answering dialogue dataset along with answers generated by LLMs and a comprehensive set of metrics. Our experimental results demonstrate that the proposed RelD successfully detects hallucination in the answers generated by diverse LLMs. Moreover, it performs well in distinguishing hallucination in LLMs' generated answers from both in-distribution and out-of-distribution datasets. Additionally, we also conduct a thorough analysis of the types of hallucinations that occur and present valuable insights. This research significantly contributes to the detection of reliable answers generated by LLMs and holds noteworthy implications for mitigating hallucination in the future work.
A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER

The objective of few-shot named entity recognition is to identify named entities with limited labeled instances. Previous works have primarily focused on optimizing the traditional token-wise classification framework, while neglecting the exploration of information based on NER data characteristics. To address this issue, we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER. Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination. These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs). In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification. Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin. Extensive analyses validate the effectiveness and generalization of MSDP.
NOVO: Learnable and Interpretable Document Identifiers for Model-Based IR

Model-based Information Retrieval (Model-based IR) has gained attention due to advancements in generative language models. Unlike traditional dense retrieval methods relying on dense vector representations of documents, model-based IR leverages language models to retrieve documents by generating their unique discrete identifiers (docids). This approach effectively reduces the requirements to store separate document representations in an index. Most existing model-based IR approaches utilize pre-defined static docids, i.e., these docids are fixed and are not learnable by training on the retrieval tasks. However, these docids are not specifically optimized for retrieval tasks, which makes it difficult to learn semantics and relationships between documents and achieve satisfactory retrieval performance. To address the above limitations, we propose Neural Optimized VOcabularial (NOVO) docids. NOVO docids are unique n-gram sets identifying each document. They can be generated in any order to retrieve the corresponding document and can be optimized through training to better learn semantics and relationships between documents. We propose to optimize NOVO docids through query denoising modeling and retrieval tasks, allowing for optimizing both semantic and token representations for such docids. Experiments on two datasets under the normal and zero-shot settings show that NOVO exhibits strong performance in more effective and interpretable model-based IR.
Node-dependent Semantic Search over Heterogeneous Graph Neural Networks

In recent years, Heterogeneous Graph Neural Networks (HGNNs) have been the state-of-the-art approaches for various tasks on Heterogeneous Graphs (HGs), e.g., recommendation and social network analysis. Despite the success of existing HGNNs, the utilization of the intricate semantic information in HGs is still insufficient. In this work, we study the problem of how to design powerful HGNNs under the guidance of node-dependent semantics. Specifically, to perform semantic search over HGNNs, we propose to develop semantic structures in terms of relation selection and connection selection, which could guide a task-relevant message flow. Furthermore, to better capture the diversified property of different node samples in HGs, we design predictors to adaptively decide the semantic structures per node. Extensive experiments on seven benchmarking datasets across different downstream tasks, i.e., node classification and recommendation, show that our method can consistently outperform various state-of-the-art baselines with shorter inference latency, which justifies its effectiveness and efficiency. The code and data are available at https://github.com/BUPT-GAMMA/NDS.
Data and Decision Fusion with Uncertainty Quantification for ML-based Healthcare Decision Systems

This paper outlines the PhD research plan to develop a comprehensive, uncertainty-aware, multimodal deep learning approach to be used in the healthcare domain. The goal is to design a multimodal deep learning framework that can leverage the complex interconnections between various modalities in order to generate highly precise predictions for intricate healthcare datasets. In addition, the framework should also incorporate methods for quantifying and dealing with uncertainty, which is an important consideration in many real-world healthcare applications. The approach will be tested on real-world multimodal datasets from Marseilles hospitals in France. We further represent some preliminary results of our early stage experiments with uncertainty quantification on multimodal datasets.
CLSPRec: Contrastive Learning of Long and Short-term Preferences for Next POI Recommendation

Next point-of-interest (POI) recommendation optimizes user travel experiences and enhances platform revenues by providing users with potentially appealing next location choices. In recent research, scholars have successfully mined users' general tastes and varying interests by modeling long-term and short-term check-in sequences. However, conventional methods for long and short-term modeling predominantly employ distinct encoders to process long and short-term interaction data independently, with disparities in encoders and data limiting the ultimate performance of these models. Instead, we propose a shared trajectory encoder and a novel Contrastive learning of Long and Short-term Preferences for next POI Recommendation (CLSPRec) model to better utilize the preference similarity among the same users and distinguish different users' travel preferences for more accurate next POI prediction. CLSPRec adopts a masking strategy in long-term sequences to enhance model robustness and further strengthens user representation through short-term sequences. Extensive experiments on three real-world datasets validate the superiority of our model. Our code is publicly available at https://github.com/Wonderdch/CLSPRec.
Leveraging Knowledge and Reinforcement Learning for Enhanced Reliability of Language Models

The Natural Language Processing (NLP) community has been using crowd-sourcing techniques to create benchmark datasets such as General Language Understanding and Evaluation (GLUE) for training modern Language Models (LMs) such as BERT. GLUE tasks measure the reliability scores using inter-annotator metrics - Cohen's Kappa (K). However, the reliability aspect of LMs has often been overlooked. To counter this problem, we explore a knowledge-guided LM ensembling approach that leverages reinforcement learning to integrate knowledge from ConceptNet and Wikipedia as knowledge graph embeddings. This approach mimics human annotators resorting to external knowledge to compensate for information deficits in the datasets. Across nine GLUE datasets, our research shows that ensembling strengthens reliability and accuracy scores, outperforming state-of-the-art.
KG4Ex: An Explainable Knowledge Graph-Based Approach for Exercise Recommendation

Effective exercise recommendation is crucial for guiding students' learning trajectories and fostering their interest in the subject matter. However, the vast exercise resource and the varying learning abilities of individual students pose a significant challenge in selecting appropriate exercise questions. Collaborative filtering-based methods often struggle with recommending suitable exercises, while deep learning-based methods lack explanation, limiting their practical adoption. To address these limitations, this paper proposes KG4Ex, a knowledge graph-based exercise recommendation method. KG4Ex facilitates the matching of diverse students with suitable exercises while providing recommendation reasons. Specifically, we introduce a feature extraction module to represent students' learning states and construct a knowledge graph for exercise recommendation. This knowledge graph comprises three key entities (knowledge concepts, students, and exercises) and their interrelationships, and can be used to recommend suitable exercises. Extensive experiments on three real-world datasets and expert interviews demonstrate the superiority of KG4Ex over existing baseline methods and highlight its strong explainability.
Towards Improving Accuracy and Computation Cost Optimization of Recommendation Systems

It is hard to avoid recommender systems (RS) these days which play a vital role in various domains, such as e-commerce, online streaming platforms, and personalized content delivery. These systems assist users in discovering relevant items based on their preferences and past interactions. A variety of methods are developed by RS communities, to address the accuracy issue. But, most of these methods are sequential and omit the item features and user side information which contains relevant and rich information that can increase the accuracy of these systems. However, enhancing the accuracy of recommendations often comes at the expense of increased computational costs. This PhD thesis aims to address the challenge of improving the accuracy of RS following a hybrid approach that allows leveraging user-item features based on some relevant state-of-the-art models and using deep learning techniques such as the contractive autoencoder (CAE) while optimizing the cost of computation using parallel/distributed paradigms.
Exploring the repair process of a 3D printer using augmented reality-based guidance

In recent years, additive manufacturing (AM) techniques have transcended their typical rapid prototyping role and become viable methods to directly manufacture end products in a highly versatile manner. Due to its low cost and relative ease of use, fused deposition modeling (FDM) has become the most universally applied AM technology. Nonetheless, skilled operators are often still required to perform maintenance, diagnostic, and repair tasks. Such operators need to be adequately trained. Here, Augmented reality (AR) technology could be used to automate this training and help to promptly provide new operators with the necessary skills to perform specific tasks as required. However, the most effective approach to designing such AR-based assistance systems has not yet been fully explored. Consequently, we address this need by reporting on how to design such guiding systems using well-known design engineering methodologies. We then further assess the applicability of our approach through a user study with domain experts. In addition, we complete our assessment with heuristical verification of system expressiveness to reason about the influence of cognitively important components of the AR interface on the operators.
Word-Graph2vec: An Efficient Word Embedding Approach on Word Co-occurrence Graph Using Random Walk Technique

Word embedding has become ubiquitous and is widely used in various natural language processing (NLP) tasks, such as web retrieval, web semantic analysis, and machine translation, and so on. Unfortunately, training the word embedding in a relatively large corpus is prohibitively expensive. We propose a graph-based word embedding algorithm, called Word-Graph2vec, which converts the large corpus into a word co-occurrence graph, then takes the word sequence samples from this graph by randomly traveling and trains the word embedding on this sampling corpus in the end. We posit that because of the limited vocabulary, huge idioms, and fixed expressions in English, the size and density of the word co-occurrence graph change slightly with the increase in the training corpus. So that Word-Graph2vec has stable runtime on the large-scale data set, and its performance advantage becomes more and more obvious with the growth of the training corpus. Extensive experiments conducted on real-world datasets show that the proposed algorithm outperforms traditional Word2vec four to five times in terms of efficiency and two to three times than FastText, while the error generated by the random walk technique is small.
A Comprehensive Survey of Explainable Artificial Intelligence (XAI) Methods: Exploring Transparency and Interpretability

Artificial Intelligence (AI) is undergoing a significant transformation. In recent years, the deployment of AI models, from Analytical to Cognitive and Generative AI, has become imminent; however, the widespread utilization of these models has prompted questions and concerns within the research and business communities regarding their transparency and interpretability. A primary challenge lies in comprehending the underlying reasoning mechanisms employed by AI-enabled systems. The absence of transparency and interpretability into the decision-making process of these systems indicates a deficiency that can have severe consequences, e.g., in domains such as medical diagnosis and financial decision-making, where valuable resources are at stake. This survey explores Explainable AI (XAI) techniques within the AI system pipeline based on existing literature. It covers tools and applications across various domains, assessing current methods and addressing challenges and opportunities, particularly in the context of Generative AI.
From nCoder to ChatGPT: From Automated Coding to Refining Human Coding

This paper investigates the potential of utilizing ChatGPT (GPT-4) as a tool for supporting coding processes for Quantitative Ethnography research. We compare the use of ChatGPT and nCoder, the most widely used automated coding tool in the QE community, on a dataset of press releases and public addresses delivered by governmental leaders from seven countries from late February to late March 2020. The study assesses the accuracy of the automated coding procedures between the two tools, and the role that ChatGPT's explanations of its coding decisions can play in improving the consistency and construct validity of human-generated codes. Results suggest that both ChatGPT and nCoder have advantages and disadvantages depending on the context, nature of the data, and researchers' goals. While nCoder is useful for straightforward coding schemes represented through regular expressions, ChatGPT can better capture a variety of language structures. ChatGPT's ability to provide explanations for its decisions can also help enhance construct validity, identify ambiguity in code definitions, and assist human coders in achieving high interrater reliability. Although we identify limitations of ChatGPT in coding constructs open to human interpretations and encompassing multiple concepts, we highlight opportunities and potential benefits provided by ChatGPT as a tool to support human researchers in their coding process.
A Novel Family of Finite Automata for Recognizing and Learning $$\omega $$-Regular Languages

Families of DFAs (FDFAs) have recently been introduced as a new representation of $$\omega $$ -regular languages. They target ultimately periodic words, with acceptors revolving around accepting some representation $$u\cdot v^\omega $$ . Three canonical FDFAs have been suggested, called periodic, syntactic, and recurrent. We propose a fourth one, limit FDFAs, which can be exponentially coarser than periodic FDFAs and are more succinct than syntactic FDFAs, while they are incomparable (and dual to) recurrent FDFAs. We show that limit FDFAs can be easily used to check not only whether $$\omega $$ -languages are regular, but also whether they are accepted by deterministic Büchi automata. We also show that canonical forms can be left behind in applications: the limit and recurrent FDFAs can complement each other nicely, and it may be a good way forward to use a combination of both. Using this observation as a starting point, we explore making more efficient use of Myhill-Nerode’s right congruences in aggressively increasing the number of don’t-care cases in order to obtain smaller progress automata. In pursuit of this goal, we gain succinctness, but pay a high price by losing constructiveness.
AUTOMATED STUDENT ATTENDANCE MONITORING SYSTEM IN CLASSROOM BASED ON CONVOLUTIONAL NEURAL NETWORKS

Attending classes by students is associated with the assimilation of educational material by students and the ability to plan and organize activities. However, at present in educational institutions, as a rule, student attendance is recorded manually. Activities are performed frequently and repeatedly, thus wasting instructors' study time. Additionally, the face is one of the most widely used biometric characteristics for personal identification so an automated attendance system using face recognition has been proposed. In recent years, convolutional neural networks (CNN) have become the dominant deep le11arning method for face recognition. In this article, the features of building an automated student attendance system by biometric face recognition using the convolution neural network modelhas been discussed. Analyzed and solved the main tasks that arise when building an automated student attendance monitoring system: creating a dataset of students' face images; building and training a biometric face recognition model; face recognition from the camera and registration in the database; extension to the face image dataset. The use of the capabilities of the Python and OpenCV libraries is shown. The conducted testing of the accuracy of the developed CNN model of biometric face recognition showed good results – the overall accuracy score is not less than 0.75. The developed automated student attendance monitoring system in classrooms can be used to determine student attendance in different forms of the educational process. Its implementation will significantly reduce the monitoring time and reduce the number of errors in maintaining attendance logs. The introduction of an automated attendance monitoring system will significantly improve the organization of the educational process to ensure its quality.
Impact of the greenfield-foreign direct investment on the economic growth of Afghanistan

Afghanistan is a least developed country; during the liberation, about three-fourths of the economy was ruined, and over the past two decades, the country has geared up a slight economic growth. The study's objective is to assess the impact of greenfield foreign direct investment on the economic growth of Afghanistan during the interval 2002-2021. The research also investigates the effect of remittances, foreign aid, and population. In this study, the stationarity of all variables is checked via the Augmented Dickey-Fuller test (ADF). The results were analysed with ARDL and ECM techniques. The results show that greenfield investment and remittances are statistically significant in the short and long run. Foreign aid is insignificant in the short and long run. The population is insignificant in the long run but significant in the short run. First, the government of Afghanistan improved its friendly investment policy to attract foreign investors. Second, the government of Afghanistan signed an agreement with neighbouring countries, especially developed countries, to send their skilled and educated labour so they will easily earn their livelihood and send remittances to their families. Third, the government of Afghanistan should avoid foreign aid that may affect its economy in the short term.
Characterizing and Early Predicting User Performance for Adaptive Search Path Recommendation

ABSTRACT User search performance is multidimensional in nature and may be better characterized by metrics that depict users' interactions with both relevant and irrelevant results. Despite previous research on one‐dimensional measures, it is still unclear how to characterize different dimensions of user performance and leverage the knowledge in developing proactive recommendations. To address this gap, we propose and empirically test a framework of search performance evaluation and build early performance prediction models to simulate proactive search path recommendations. Experimental results from four datasets of diverse types (1,482 sessions and 5,140 query segments from both controlled lab and natural settings) demonstrate that: 1) Cluster patterns characterized by cost‐gain‐based multifaceted metrics can effectively differentiate high‐performing users from other searchers, which form the empirical basis for proactive recommendations; 2) whole‐session performance can be reliably predicted at early stages of sessions (e.g., first and second queries); 3) recommendations built upon the search paths of system‐identified high‐performing searchers can significantly improve the search performance of struggling users. Experimental results demonstrate the potential of our approach for leveraging collective wisdom from automatically identified high‐performance user groups in developing and evaluating proactive in‐situ search recommendations.
A Hybrid POI Recommendation System Combining Link Analysis and Collaborative Filtering Based on Various Visiting Behaviors

Location-based recommender systems (LBRSs) have exhibited significant potential in providing personalized recommendations based on the user’s geographic location and contextual factors such as time, personal preference, and location categories. However, several challenges (such as data sparsity, the cold-start problem, and tedium problem) need to be addressed to develop more effective LBRSs. In this paper, we propose a novel POI recommendation system, called LACF-Rec3, which employs a hybrid approach of link analysis (HITS-3) and collaborative filtering (CF-3) based on three visiting behaviors: frequency, variety, and repetition. HITS-3 identifies distinctive POIs based on user- and POI-visit patterns, ranks them accordingly, and recommends them to cold-start users. For existing users, CF-3 utilizes collaborative filtering based on their previous check-in history and POI distinctive aspects. Our experimental results conducted on a Foursquare dataset demonstrate that LACF-Rec3 outperforms prior methods in terms of recommendation accuracy, ranking precision, and matching ratio. In addition, LACF-Rec3 effectively solves the challenges of data sparsity, the cold-start issue, and tedium problems for cold-start and existing users. These findings highlight the potential of LACF-Rec3 as a promising solution to the challenges encountered by LBRS.