pwc-archive/papers-with-abstracts huggingface dataset:
The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation
Authors: Zekai Qu, Ruobing Xie, Chaojun Xiao, Xingwu Sun, Zhanhui Kang
Date: 2024-04-12 00:00:00
Tasks: Language Modeling, Language Modelling, Sequential Recommendation

Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs). Some PLM-based SR models directly use PLM to encode user historical behavior's text sequences to learn user representations, while there is seldom an in-depth exploration of the capability and suitability of PLM in behavior sequence modeling. In this work, we first conduct extensive model analyses between PLMs and PLM-based SR models, discovering great underutilization and parameter redundancy of PLMs in behavior sequence modeling. Inspired by this, we explore different lightweight usages of PLMs in SR, aiming to maximally stimulate the ability of PLMs for SR while satisfying the efficiency and usability demands of practical systems. We discover that adopting behavior-tuned PLMs for item initializations of conventional ID-based SR models is the most economical framework of PLM-based SR, which would not bring in any additional inference cost but could achieve a dramatic performance boost compared with the original version. Extensive experiments on five datasets show that our simple and universal framework leads to significant improvement compared to classical SR and SOTA PLM-based SR models without additional inference costs. Our code can be found in https://github.com/777pomingzi/Rethinking-PLM-in-RS.
Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models
Authors: Feihu Jiang, Chuan Qin, Kaichun Yao, Chuyu Fang, Fuzhen Zhuang, HengShu Zhu, Hui Xiong
Date: 2024-04-10 00:00:00
Tasks: Management, Question Answering, Retrieval

Efficient knowledge management plays a pivotal role in augmenting both the operational efficiency and the innovative capacity of businesses and organizations. By indexing knowledge through vectorization, a variety of knowledge retrieval methods have emerged, significantly enhancing the efficacy of knowledge management systems. Recently, the rapid advancements in generative natural language processing technologies paved the way for generating precise and coherent answers after retrieving relevant documents tailored to user queries. However, for enterprise knowledge bases, assembling extensive training data from scratch for knowledge retrieval and generation is a formidable challenge due to the privacy and security policies of private data, frequently entailing substantial costs. To address the challenge above, in this paper, we propose EKRG, a novel Retrieval-Generation framework based on large language models (LLMs), expertly designed to enable question-answering for Enterprise Knowledge bases with limited annotation costs. Specifically, for the retrieval process, we first introduce an instruction-tuning method using an LLM to generate sufficient document-question pairs for training a knowledge retriever. This method, through carefully designed instructions, efficiently generates diverse questions for enterprise knowledge bases, encompassing both fact-oriented and solution-oriented knowledge. Additionally, we develop a relevance-aware teacher-student learning strategy to further enhance the efficiency of the training process. For the generation process, we propose a novel chain of thought (CoT) based fine-tuning method to empower the LLM-based generator to adeptly respond to user questions using retrieved documents. Finally, extensive experiments on real-world datasets have demonstrated the effectiveness of our proposed framework.
Musical Listening Qualia: A Multivariate Approach
Authors: Brendon Mizener, Mathilde Vandenberghe-Descamps, Hervé Abdi, Sylvie Chollet
Date: 2024-04-10 00:00:00

French and American participants listened to new music stimuli and evaluated the stimuli using either adjectives or quantitative musical dimensions. Results were analyzed using correspondence analysis (CA), hierarchical cluster analysis (HCA), multiple factor analysis (MFA), and partial least squares correlation (PLSC). French and American listeners differed when they described the musical stimuli using adjectives, but not when using the quantitative dimensions. The present work serves as a case study in research methodology that allows for a balance between relaxing experimental control and maintaining statistical rigor.
Apollonion: Profile-centric Dialog Agent
Authors: Shangyu Chen, Zibo Zhao, YuanYuan Zhao, Xiang Li
Date: 2024-04-10 00:00:00

The emergence of Large Language Models (LLMs) has innovated the development of dialog agents. Specially, a well-trained LLM, as a central process unit, is capable of providing fluent and reasonable response for user's request. Besides, auxiliary tools such as external knowledge retrieval, personalized character for vivid response, short/long-term memory for ultra long context management are developed, completing the usage experience for LLM-based dialog agents. However, the above-mentioned techniques does not solve the issue of \textbf{personalization from user perspective}: agents response in a same fashion to different users, without consideration of their features, such as habits, interests and past experience. In another words, current implementation of dialog agents fail in ``knowing the user''. The capacity of well-description and representation of user is under development. In this work, we proposed a framework for dialog agent to incorporate user profiling (initialization, update): user's query and response is analyzed and organized into a structural user profile, which is latter served to provide personal and more precise response. Besides, we proposed a series of evaluation protocols for personalization: to what extend the response is personal to the different users. The framework is named as \method{}, inspired by inscription of ``Know Yourself'' in the temple of Apollo (also known as \method{}) in Ancient Greek. Few works have been conducted on incorporating personalization into LLM, \method{} is a pioneer work on guiding LLM's response to meet individuation via the application of dialog agents, with a set of evaluation methods for measurement in personalization.
Enhancing Adaptive Video Streaming through Fuzzy Logic-Based Content Recommendation Systems: A Comprehensive Review and Future Directions
Authors: Koffka Khan
Date: 2024-04-10 00:00:00
Tasks: Recommendation Systems

As the demand for high-quality video content continues to rise, adaptive video streaming plays a pivotal role in delivering an optimal viewing experience. However, traditional content recommendation systems face challenges in dynamically adapting to users' preferences, content features, and contextual information. This review paper explores the integration of fuzzy logic into content recommendation systems for adaptive video streaming. Fuzzy logic, known for handling uncertainty and imprecision, provides a promising framework for modeling and accommodating the dynamic nature of user preferences and contextual factors. The paper discusses the evolution of adaptive video streaming, reviews traditional content recommendation algorithms, and introduces fuzzy logic as a solution to enhance the adaptability of these systems. Through a comprehensive exploration of case studies and applications, the effectiveness of fuzzy logic in improving user satisfaction and system performance is highlighted. The review also addresses challenges associated with the integration of fuzzy logic and suggests future research directions to further advance this approach. The proposed framework offers insights into a dynamic and context-aware content recommendation system, contributing to the evolution of adaptive video streaming technologies.
A Survey of Reasoning for Substitution Relationships: Definitions, Methods, and Directions
Authors: Anxin Yang, Zhijuan Du, Tao Sun
Date: 2024-04-09 00:00:00
Tasks: Recommendation Systems

Substitute relationships are fundamental to people's daily lives across various domains. This study aims to comprehend and predict substitute relationships among products in diverse fields, extensively analyzing the application of machine learning algorithms, natural language processing, and other technologies. By comparing model methodologies across different domains, such as defining substitutes, representing and learning substitute relationships, and substitute reasoning, this study offers a methodological foundation for delving deeper into substitute relationships. Through ongoing research and innovation, we can further refine the personalization and accuracy of substitute recommendation systems, thus advancing the development and application of this field.
Access to Library Information Resources by University Students during COVID-19 Pandemic in Africa: A Systematic Literature Review
Authors: Joyce Charles Shikali, Paul Samwel Muneja
Date: 2024-04-08 00:00:00
Tasks: Systematic Literature Review

The study examined access to library information resources by university students during the outbreak of the COVID-19 pandemic. The study investigated measures that were adopted by academic libraries for smooth delivery of library information resources to their patrons. It also identified technological tools that were employed by libraries to facilitate access to library information resources. We also investigated the challenges faced by students in accessing library information resources. A systematic literature review approach using PRISMA guidelines was employed to investigate the relevant literature on the subject. The keyword search strategy was employed to search for relevant literature from four scholarly databases Scopus, emerald, Research4life, and Google Scholar. In this study, 23 studies that fulfilled the criteria were included. The findings revealed that the majority of the reviewed studies indicate that, during the COVID-19 pandemic many academic libraries in Africa adopted different approaches to facilitate access to library information resources by university students including expanding access to electronic resources off-campus, virtual reference services, circulation and lending services. To support access to different library services and information resources academic libraries in Africa used various digital technological tools like social media, library websites, email and video conferencing. Moreover, the study revealed that limited access to internet services and ICT devices, inadequate electronic library collection and inadequate digital and information literacy were the major challenges faced by patrons during the pandemic. This study recommends investment in ICT infrastructures and expanding electronic resource collections which are vital resources in the digital era.
Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning
Authors: Teo Susnjak, Peter Hwang, Napoleon H. Reyes, Andre L. C. Barczak, Timothy R. McIntosh, Surangika Ranathunga
Date: 2024-04-08 00:00:00
Tasks: Hallucination, Language Modeling, Language Modelling, Large Language Model

This research pioneers the use of fine-tuned Large Language Models (LLMs) to automate Systematic Literature Reviews (SLRs), presenting a significant and novel contribution in integrating AI to enhance academic research methodologies. Our study employed the latest fine-tuning methodologies together with open-sourced LLMs, and demonstrated a practical and efficient approach to automating the final execution stages of an SLR process that involves knowledge synthesis. The results maintained high fidelity in factual accuracy in LLM responses, and were validated through the replication of an existing PRISMA-conforming SLR. Our research proposed solutions for mitigating LLM hallucination and proposed mechanisms for tracking LLM responses to their sources of information, thus demonstrating how this approach can meet the rigorous demands of scholarly research. The findings ultimately confirmed the potential of fine-tuned LLMs in streamlining various labor-intensive processes of conducting literature reviews. Given the potential of this approach and its applicability across all research domains, this foundational study also advocated for updating PRISMA reporting guidelines to incorporate AI-driven processes, ensuring methodological transparency and reliability in future SLRs. This study broadens the appeal of AI-enhanced tools across various academic and research fields, setting a new standard for conducting comprehensive and accurate literature reviews with more efficiency in the face of ever-increasing volumes of academic studies.
Information Retrieval with Entity Linking
Authors: Dahlia Shehata
Date: 2024-04-07 00:00:00
Tasks: Entity Linking, Information Retrieval, Retrieval

Despite the advantages of their low-resource settings, traditional sparse retrievers depend on exact matching approaches between high-dimensional bag-of-words (BoW) representations of both the queries and the collection. As a result, retrieval performance is restricted by semantic discrepancies and vocabulary gaps. On the other hand, transformer-based dense retrievers introduce significant improvements in information retrieval tasks by exploiting low-dimensional contextualized representations of the corpus. While dense retrievers are known for their relative effectiveness, they suffer from lower efficiency and lack of generalization issues, when compared to sparse retrievers. For a lightweight retrieval task, high computational resources and time consumption are major barriers encouraging the renunciation of dense models despite potential gains. In this work, I propose boosting the performance of sparse retrievers by expanding both the queries and the documents with linked entities in two formats for the entity names: 1) explicit and 2) hashed. A zero-shot end-to-end dense entity linking system is employed for entity recognition and disambiguation to augment the corpus. By leveraging the advanced entity linking methods, I believe that the effectiveness gap between sparse and dense retrievers can be narrowed. Experiments are conducted on the MS MARCO passage dataset using the original qrel set, the re-ranked qrels favoured by MonoT5 and the latter set further re-ranked by DuoT5. Since I am concerned with the early stage retrieval in cascaded ranking architectures of large information retrieval systems, the results are evaluated using recall@1000. The suggested approach is also capable of retrieving documents for query subsets judged to be particularly difficult in prior work.
PMG : Personalized Multimodal Generation with Large Language Models
Authors: Xiaoteng Shen, Rui Zhang, Xiaoyan Zhao, Jieming Zhu, Xi Xiao
Date: 2024-04-07 00:00:00
Tasks: multimodal generation, Reading Comprehension, Recommendation Systems

The emergence of large language models (LLMs) has revolutionized the capabilities of text comprehension and generation. Multi-modal generation attracts great attention from both the industry and academia, but there is little work on personalized generation, which has important applications such as recommender systems. This paper proposes the first method for personalized multimodal generation using LLMs, showcases its applications and validates its performance via an extensive experimental study on two datasets. The proposed method, Personalized Multimodal Generation (PMG for short) first converts user behaviors (e.g., clicks in recommender systems or conversations with a virtual assistant) into natural language to facilitate LLM understanding and extract user preference descriptions. Such user preferences are then fed into a generator, such as a multimodal LLM or diffusion model, to produce personalized content. To capture user preferences comprehensively and accurately, we propose to let the LLM output a combination of explicit keywords and implicit embeddings to represent user preferences. Then the combination of keywords and embeddings are used as prompts to condition the generator. We optimize a weighted sum of the accuracy and preference scores so that the generated content has a good balance between them. Compared to a baseline method without personalization, PMG has a significant improvement on personalization for up to 8% in terms of LPIPS while retaining the accuracy of generation.
RecGPT: Generative Personalized Prompts for Sequential Recommendation via ChatGPT Training Paradigm
Authors: Yabin Zhang, Wenhui Yu, Erhan Zhang, Xu Chen, Lantao Hu, Peng Jiang, Kun Gai
Date: 2024-04-06 00:00:00
Tasks: Natural Language Understanding, Sequential Recommendation

ChatGPT has achieved remarkable success in natural language understanding. Considering that recommendation is indeed a conversation between users and the system with items as words, which has similar underlying pattern with ChatGPT, we design a new chat framework in item index level for the recommendation task. Our novelty mainly contains three parts: model, training and inference. For the model part, we adopt Generative Pre-training Transformer (GPT) as the sequential recommendation model and design a user modular to capture personalized information. For the training part, we adopt the two-stage paradigm of ChatGPT, including pre-training and fine-tuning. In the pre-training stage, we train GPT model by auto-regression. In the fine-tuning stage, we train the model with prompts, which include both the newly-generated results from the model and the user's feedback. For the inference part, we predict several user interests as user representations in an autoregressive manner. For each interest vector, we recall several items with the highest similarity and merge the items recalled by all interest vectors into the final result. We conduct experiments with both offline public datasets and online A/B test to demonstrate the effectiveness of our proposed method.
Taxonomy and Analysis of Sensitive User Queries in Generative AI Search
Authors: Hwiyeol Jo, Taiwoo Park, Hyunwoo Lee, Nayoung Choi, Changbong Kim, Ohjoon Kwon, Donghyeon Jeon, Eui-Hyeon Lee, Kyoungho Shin, Sun Suk Lim, Kyungmi Kim, Jihye Lee, Sun Kim
Date: 2024-04-05 00:00:00

Although there has been a growing interest among industries in integrating generative LLMs into their services, limited experience and scarcity of resources act as a barrier in launching and servicing large-scale LLM-based services. In this paper, we share our experiences in developing and operating generative AI models within a national-scale search engine, with a specific focus on the sensitiveness of user queries. We propose a taxonomy for sensitive search queries, outline our approaches, and present a comprehensive analysis report on sensitive queries from actual users. We believe that our experiences in launching generative AI search systems can contribute to reducing the barrier in building generative LLM-based services.
Navigating the Evaluation Funnel to Optimize Iteration Speed for Recommender Systems
Authors: Claire Schultzberg, Brammert Ottens
Date: 2024-04-03 00:00:00
Tasks: counterfactual, Recommendation Systems

Over the last decades has emerged a rich literature on the evaluation of recommendation systems. However, less is written about how to efficiently combine different evaluation methods from this rich field into a single efficient evaluation funnel. In this paper we aim to build intuition for how to choose evaluation methods, by presenting a novel framework that simplifies the reasoning around the evaluation funnel for a recommendation system. Our contribution is twofold. First we present our framework for how to decompose the definition of success to construct efficient evaluation funnels, focusing on how to identify and discard non-successful iterations quickly. We show that decomposing the definition of success into smaller necessary criteria for success enables early identification of non-successful ideas. Second, we give an overview of the most common and useful evaluation methods, discuss their pros and cons, and how they fit into, and complement each other in, the evaluation process. We go through so-called offline and online evaluation methods such as counterfactual logging, validation, verification, A/B testing, and interleaving. The paper concludes with some general discussion and advice on how to design an efficient evaluation process for recommender systems.
Combining PatternRank with Huffman Coding: A Novel Compression Algorithm
Authors: Jasurbek Shukurov
Date: 2024-04-02 00:00:00

The escalating volume of data involved in Android backup packages necessitates an innovative approach to compression beyond traditional methods like GZIP, which may not fully exploit the redundancy inherent in Android backups, particularly those containing extensive XML data. This paper introduces the PatternRank algorithm, a novel compression strategy specifically designed for Android backups. PatternRank leverages pattern recognition and ranking, combined with Huffman coding, to efficiently compress data by identifying and replacing frequent, longer patterns with shorter codes. We detail two versions of the PatternRank algorithm: the original version focuses on dynamic pattern extraction and ranking, while the second version incorporates a pre-defined dictionary optimized for the common patterns found in Android backups, particularly within XML files. This tailored approach ensures that PatternRank not only outperforms traditional compression methods in terms of compression ratio and speed but also remains highly effective when dealing with the specific challenges posed by Android backup data. Our analysis includes a comparative study of compression performance across GZIP, PatternRank v1, PatternRank v2, and a combined PatternRank-Huffman method, highlighting the superior efficiency and potential of PatternRank in managing the growing data demands of Android backup packages. Through this exploration, we underscore the significance of adopting pattern-based compression algorithms in optimizing data storage and transmission in the mobile domain.
A Comprehensive Survey on AI-based Methods for Patents
Authors: Homaira Huda Shomee, Zhu Wang, Sathya N. Ravi, Sourav Medya
Date: 2024-04-02 00:00:00
Tasks: Retrieval, Survey

Recent advancements in Artificial Intelligence (AI) and machine learning have demonstrated transformative capabilities across diverse domains. This progress extends to the field of patent analysis and innovation, where AI-based tools present opportunities to streamline and enhance important tasks in the patent cycle such as classification, retrieval, and valuation prediction. This not only accelerates the efficiency of patent researchers and applicants but also opens new avenues for technological innovation and discovery. Our survey provides a comprehensive summary of recent AI tools in patent analysis from more than 40 papers from 26 venues between 2017 and 2023. Unlike existing surveys, we include methods that work for patent image and text data. Furthermore, we introduce a novel taxonomy for the categorization based on the tasks in the patent life cycle as well as the specifics of the AI methods. This interdisciplinary survey aims to serve as a resource for researchers and practitioners who are working at the intersection of AI and patent analysis as well as the patent offices that are aiming to build efficient patent systems.