pwc-archive/papers-with-abstracts huggingface dataset:
Collaborative Large Language Model for Recommender Systems
Authors: Yaochen Zhu, Liang Wu, Qi Guo, Liangjie Hong, Jundong Li
Date: 2023-11-02 00:00:00
Tasks: Hallucination, Language Modeling, Language Modelling, Large Language Model, model, Recommendation Systems

Recently, there has been growing interest in developing the next-generation recommender systems (RSs) based on pretrained large language models (LLMs). However, the semantic gap between natural language and recommendation tasks is still not well addressed, leading to multiple issues such as spuriously correlated user/item descriptors, ineffective language modeling on user/item data, inefficient recommendations via auto-regression, etc. In this paper, we propose CLLM4Rec, the first generative RS that tightly integrates the LLM paradigm and ID paradigm of RSs, aiming to address the above challenges simultaneously. We first extend the vocabulary of pretrained LLMs with user/item ID tokens to faithfully model user/item collaborative and content semantics. Accordingly, a novel soft+hard prompting strategy is proposed to effectively learn user/item collaborative/content token embeddings via language modeling on RS-specific corpora, where each document is split into a prompt consisting of heterogeneous soft (user/item) tokens and hard (vocab) tokens and a main text consisting of homogeneous item tokens or vocab tokens to facilitate stable and effective language modeling. In addition, a novel mutual regularization strategy is introduced to encourage CLLM4Rec to capture recommendation-related information from noisy user/item content. Finally, we propose a novel recommendation-oriented finetuning strategy for CLLM4Rec, where an item prediction head with multinomial likelihood is added to the pretrained CLLM4Rec backbone to predict hold-out items based on soft+hard prompts established from masked user-item interaction history, where recommendations of multiple items can be generated efficiently without hallucination. Codes are released at https://github.com/yaochenzhu/llm4rec.
Recommendations by Concise User Profiles from Review Text
Authors: Ghazaleh Haratinezhad Torbati, Anna Tigunova, Andrew Yates, Gerhard Weikum
Date: 2023-11-02 00:00:00
Tasks: Language Modeling, Language Modelling, Recommendation Systems, Representation Learning, Small Language Model

Recommender systems perform well for popular items and users with ample interactions (likes, ratings etc.). This work addresses the difficult and underexplored case of users who have very sparse interactions but post informative review texts. This setting naturally calls for encoding user-specific text with large language models (LLM). However, feeding the full text of all reviews through an LLM has a weak signal-to-noise ratio and incurs high costs of processed tokens. This paper addresses these two issues. It presents a light-weight framework, called CUP, which first computes concise user profiles and feeds only these into the training of transformer-based recommenders. For user profiles, we devise various techniques to select the most informative cues from noisy reviews. Experiments, with book reviews data, show that fine-tuning a small language model with judiciously constructed profiles achieves the best performance, even in comparison to LLM-generated rankings.
VM-Rec: A Variational Mapping Approach for Cold-start User Recommendation
Authors: Linan Zheng, Jiale Chen, Pengsheng Liu, Guangfa Zhang, Jinyun Fang
Date: 2023-11-02 00:00:00
Tasks: Recommendation Systems

The cold-start problem is a common challenge for most recommender systems. The practical application of most cold-start methods is hindered by the deficiency in auxiliary content information for users. Moreover, most methods necessitate simultaneous updates to the extensive parameters of recommender models, leading to significant training costs, particularly in large-scale industrial scenarios. We observe that the model can generate expressive embeddings for warm users with relatively more interactions. Initially, these users were cold-start users, and after transitioning to warm users, they exhibit clustering patterns in their embeddings with consistent initial interactions. Based on this motivation, we propose a Variational Mapping approach for cold-start user Recommendation (VM-Rec), mapping from few initial interactions to expressive embeddings for cold-start users. Specifically, we encode the initial interactions into a latent representation, where each dimension disentangledly signifies the degree of association with each warm user. Subsequently, we utilize this latent representation as the parameters for the mapping function, mapping (decoding) it into an expressive embedding, which can be integrated into a pre-trained recommender model directly. Our method is evaluated on three datasets using the same base model, demonstrating superior performance compared to other popular cold-start methods.
Efficient Neural Ranking using Forward Indexes and Lightweight Encoders
Authors: Jurek Leonhardt, Henrik Müller, Koustav Rudra, Megha Khosla, Abhijit Anand, Avishek Anand
Date: 2023-11-02 00:00:00
Tasks: Computational Efficiency, Re-Ranking, Retrieval

Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency. We propose Fast-Forward indexes -- vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially. We perform evaluation to show the effectiveness and efficiency of Fast-Forward indexes -- our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.
Bi-Preference Learning Heterogeneous Hypergraph Networks for Session-based Recommendation
Authors: Xiaokun Zhang, Bo Xu, Fenglong Ma, Chenliang Li, Yuan Lin, Hongfei Lin
Date: 2023-11-02 00:00:00
Tasks: Multi-Task Learning, Session-Based Recommendations

Session-based recommendation intends to predict next purchased items based on anonymous behavior sequences. Numerous economic studies have revealed that item price is a key factor influencing user purchase decisions. Unfortunately, existing methods for session-based recommendation only aim at capturing user interest preference, while ignoring user price preference. Actually, there are primarily two challenges preventing us from accessing price preference. Firstly, the price preference is highly associated to various item features (i.e., category and brand), which asks us to mine price preference from heterogeneous information. Secondly, price preference and interest preference are interdependent and collectively determine user choice, necessitating that we jointly consider both price and interest preference for intent modeling. To handle above challenges, we propose a novel approach Bi-Preference Learning Heterogeneous Hypergraph Networks (BiPNet) for session-based recommendation. Specifically, the customized heterogeneous hypergraph networks with a triple-level convolution are devised to capture user price and interest preference from heterogeneous features of items. Besides, we develop a Bi-Preference Learning schema to explore mutual relations between price and interest preference and collectively learn these two preferences under the multi-task learning architecture. Extensive experiments on multiple public datasets confirm the superiority of BiPNet over competitive baselines. Additional research also supports the notion that the price is crucial for the task.
Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation
Authors: Tianyu Zhu, Yansong Shi, Yuan Zhang, Yihong Wu, Fengran Mo, Jian-Yun Nie
Date: 2023-11-02 00:00:00
Tasks: Recommendation Systems, Sequential Recommendation

Modern recommender systems employ various sequential modules such as self-attention to learn dynamic user interests. However, these methods are less effective in capturing collaborative and transitional signals within user interaction sequences. First, the self-attention architecture uses the embedding of a single item as the attention query, making it challenging to capture collaborative signals. Second, these methods typically follow an auto-regressive framework, which is unable to learn global item transition patterns. To overcome these limitations, we propose a new method called Multi-Query Self-Attention with Transition-Aware Embedding Distillation (MQSA-TED). First, we propose an $L$-query self-attention module that employs flexible window sizes for attention queries to capture collaborative signals. In addition, we introduce a multi-query self-attention method that balances the bias-variance trade-off in modeling user preferences by combining long and short-query self-attentions. Second, we develop a transition-aware embedding distillation module that distills global item-to-item transition patterns into item embeddings, which enables the model to memorize and leverage transitional signals and serves as a calibrator for collaborative signals. Experimental results on four real-world datasets demonstrate the effectiveness of the proposed modules.
Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study
Authors: Theresia Veronika Rampisela, Maria Maistro, Tuukka Ruotsalo, Christina Lioma
Date: 2023-11-02 00:00:00
Tasks: Exposure Fairness, Fairness, Recommendation Systems

Fairness is an emerging and challenging topic in recommender systems. In recent years, various ways of evaluating and therefore improving fairness have emerged. In this study, we examine existing evaluation measures of fairness in recommender systems. Specifically, we focus solely on exposure-based fairness measures of individual items that aim to quantify the disparity in how individual items are recommended to users, separate from item relevance to users. We gather all such measures and we critically analyse their theoretical properties. We identify a series of limitations in each of them, which collectively may render the affected measures hard or impossible to interpret, to compute, or to use for comparing recommendations. We resolve these limitations by redefining or correcting the affected measures, or we argue why certain limitations cannot be resolved. We further perform a comprehensive empirical analysis of both the original and our corrected versions of these fairness measures, using real-world and synthetic datasets. Our analysis provides novel insights into the relationship between measures based on different fairness concepts, and different levels of measure sensitivity and strictness. We conclude with practical suggestions of which fairness measures should be used and when. Our code is publicly available. To our knowledge, this is the first critical comparison of individual item fairness measures in recommender systems.
Research Team Identification Based on Representation Learning of Academic Heterogeneous Information Network
Authors: Junfu Wang, Yawen Li, Zhe Xue, Ang Li
Date: 2023-11-02 00:00:00
Tasks: Representation Learning

Academic networks in the real world can usually be described by heterogeneous information networks composed of multi-type nodes and relationships. Some existing research on representation learning for homogeneous information networks lacks the ability to explore heterogeneous information networks in heterogeneous information networks. It cannot be applied to heterogeneous information networks. Aiming at the practical needs of effectively identifying and discovering scientific research teams from the academic heterogeneous information network composed of massive and complex scientific and technological big data, this paper proposes a scientific research team identification method based on representation learning of academic heterogeneous information networks. The attention mechanism at node level and meta-path level learns low-dimensional, dense and real-valued vector representations on the basis of retaining the rich topological information of nodes in the network and the semantic information based on meta-paths, and realizes effective identification and discovery of scientific research teams and important team members in academic heterogeneous information networks based on maximizing node influence. Experimental results show that our proposed method outperforms the comparative methods.
Is Human Culture Locked by Evolution?
Authors: Hao Wang
Date: 2023-10-28 00:00:00
Tasks: Recommendation Systems

Human culture has evolved for thousands of years and thrived in the era of Internet. Due to the availability of big data, we could do research on human culture by analyzing its representation such as user item rating values on websites like MovieLens and Douban. Industrial workers have applied recommender systems in big data to predict user behavior and promote web traffic. In this paper, we analyze the social impact of an algorithm named ZeroMat to show that human culture is locked into a state where individual's cultural taste is predictable at high precision without historic data. We also provide solutions to this problem and interpretation of current Chinese government's regulations and policies.
Autoencoders with Variable Sized Latent Vector for Image Compression
Authors: Alekh Karkada Ashok, Nagaraju Palani
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Compression

Learning to compress images is an interesting and challenging task. Autoencoders have long been used to compress images into a code of small but fixed size. As different images need different sized code based on their complexity, we propose an autoencoder architecture with a variable sized latent vector. We propose an attention based model which attends over the image and summarizes it into a small code. This summarization is repeated many times depending on the complexity of the image, producing a new code each time to encode new information so as to get a better reconstruction. These small codes then form sub-units of the final code. Our approach is quality progressive and has flexible quality setting which are desirable properties in compression. We show that the proposed model shows better performance compared to JPEG
AMGNET: multi-scale graph neural networks for flow field prediction
Authors: Zhishuang Yang, Yidao Dong, Xiaogang Deng &Laiping Zhang
Date: 2022-10-13 00:00:00
Proceeding: journal 2022 10
Tasks: Decoder, Deep Learning, Graph Neural Network, Prediction, Representation Learning

Solving partial differential equations of complex physical systems is a computationally expensive task, especially in Computational Fluid Dynamics(CFD). This drives the application of deep learning methods in solving physical systems. There exist a few deep learning models that are very successful in predicting flow fields of complex physical models, yet most of these still exhibit large errors compared to simulation. Here we introduce AMGNET, a multi-scale graph neural network model based on Encoder-Process-Decoder structure for flow field prediction. Our model employs message passing of graph neural networks at different mesh graph scales. Our method has significantly lower prediction errors than the GCN baseline on several complex fluid prediction tasks, such as airfoil flow and cylinder flow. Our results show that multi-scale representation learning at the graph level is more effective in improving the prediction accuracy of flow field.
Comparison of Different Segmentations in Automated Detection of Hypertension Using Electrocardiography with Empirical Mode Decomposition
Authors: Yunus Emre Erdoğan, Ali Narin, Walid Hariri
Date: 2023-11-02 00:00:00

Hypertension (HPT) refers to a condition where the pressure exerted on the walls of arteries by blood pumped from the heart to the body reaches levels that can lead to various ailments. Annually, a significant number of lives are lost globally due to diseases linked to HPT. Therefore, the early and accurate diagnosis of HPT is of utmost importance. This study aimed to automatically and with minimal error detect patients suffering from HPT by utilizing electrocardiogram (ECG) signals. The research involved the collection of ECG signals from two distinct groups. These groups consisted of ECG data of both five thousand and ten thousand data points in length, respectively. The performance in HPT detection was evaluated using entropy measurements derived from the 5-layer Intrinsic Mode Function(IMF) signals through the application of the Empirical Mode Decomposition method. The resulting performances were compared based on the nine features extracted from each IMF. To summarize, employing the 5-fold cross-validation technique, the most exceptional accuracy rates achieved were 99.9991% and 99.9989% for ECG data of lengths five thousand and ten thousand,respectively, using decision tree algorithms. These remarkable performance results indicate the potential usefulness of this method in assisting medical professionals to identify individuals with HPT.
Deep Convolutional Spiking Neural Networks for Keyword Spotting
Authors: Emre Yilmaz, Özgür Bora Gevrek, Jibin Wu, Yuxiang Chen, Xuanbo Meng, Haizhou Li
Date: 2020-10-25 00:00:00
Proceeding: Interspeech 2020 10
Tasks: Keyword Spotting

This paper investigates the use of deep convolutional spiking neural networks (SNN) for keyword spotting (KWS) and wakeword detection tasks. The brain-inspired SNN mimic the spike-based information processing of biological neural networks and they can operate on the emerging ultra-low power neuromorphic chips. Unlike conventional artificial neural networks (ANN), SNN process input information asynchronously in an event-driven manner. With temporally sparse input information, this event-driven processing substantially reduces the computational requirements compared to the synchronous computation performed in ANN-based KWS approaches. To explore the effectiveness and computational complexity of SNN on KWS and wakeword detection, we compare the performance and computational costs of spiking fully-connected and convolutional neural networks with ANN counterparts under clean and noisy testing conditions. The results obtained on the Speech Commands and Hey Snips corpora have shown the effectiveness of the convolutional SNN model compared to a conventional CNN with comparable performance on KWS and better performance on the wakeword detection task. With its competitive performance and reduced computational complexity, convolutional SNN models running on energy-efficient neuromorphic hardware offer a low-power and effective solution for mobile KWS applications.
Using Multi-Output Learning to Diagnose Plant Disease and Stress Severity
Authors: Gianni Fenu; Francesca Maridina Malloci
Date: 2021-02-10 00:00:00
Proceeding: Complexity, Hindawi 2021 2

Early diagnosis of leaf diseases is a fundamental tool in precision agriculture, thanks to its high correlation with food safety and environmental sustainability. It is proven that plant diseases are responsible for serious economic losses every year. The aim of this work is to study an efficient network capable of assisting farmers in recognizing pear leaf symptoms and providing targeted information for rational use of pesticides. The proposed model consists of a multioutput system based on convolutional neural networks. The deep learning approach considers five pretrained CNN architectures, namely, VGG-16, VGG-19, ResNet50, InceptionV3, MobileNetV2, and EfficientNetB0, as feature extractors to classify three diseases and six severity levels. Computational experiments are conducted to evaluate the model on the DiaMOS Plant dataset, a self-collected dataset in the field. The results obtained confirm the robustness of the proposed model in automatically extracting the discriminating features of diseased leaves by adopting the multitasking learning paradigm.
DiaMOS Plant: A Dataset for Diagnosis and Monitoring Plant Disease
Authors: Gianni Fenu; Francesca Maridina Malloci
Date: 2021-10-21 00:00:00
Proceeding: Agronomy 2021 10

The classification and recognition of foliar diseases is an increasingly developing field of research, where the concepts of machine and deep learning are used to support agricultural stakeholders. Datasets are the fuel for the development of these technologies. In this paper, we release and make publicly available the field dataset collected to diagnose and monitor plant symptoms, called DiaMOS Plant, consisting of 3505 images of pear fruit and leaves affected by four diseases. In addition, we perform a comparative analysis of existing literature datasets designed for the classification and recognition of leaf diseases, highlighting the main features that maximize the value and information content of the collected data. This study provides guidelines that will be useful to the research community in the context of the selection and construction of datasets.
Design, modeling and control of a hybrid grid-connected photovoltaic-wind system for the region of Adrar, Algeria
Authors: Yassine Kebbati
Date: 2022-08-01 00:00:00
Proceeding: International Journal of Environmental Science and Technology 2022 8

The use of fossil energy for electricity production is an evident source of pollution, global warming and climate change. Consequently, researchers have been working to shift toward sustainable and clean energy by exploiting renewable an environmentally friendly resources such as wind and solar energies. On the other hand, energy security can only be achieved by considering multiple resources. Large-scale renewable energy power plants are a key solution for diversifying the total energy mix and ensuring energy security. This paper presents a contribution to diversify the energy mix in Algeria and help mitigate power shortages and improve grid performance. In particular, the paper aims at designing and modeling a large-scale hybrid photovoltaic-wind system that is grid connected. An innovative control approach using improved particle swarm optimized PI controllers is proposed to control the hybrid system and generate the maximum power from the available wind and solar energy resources. Furthermore, economic, environmental and feasibility studies of the project were conducted using HOMER software to assess the viability of the system and its contribution to reduce greenhouse gas emissions.
Neural Network and ANFIS based auto-adaptive MPC for path tracking in autonomous vehicles
Authors: Yassine Kebbati, Naima Ait-Oufroukh, Vincent Vigneron, Dalil Ichalal
Date: 2021-12-05 00:00:00
Proceeding: IEEE International Conference on Networking, Sensing and Control 2021 12
Tasks: Autonomous Vehicles, Self-Driving Cars

Self-driving cars operate in constantly changing environments and are exposed to a variety of uncertainties and disturbances. These factors render classical controllers ineffective, especially for the lateral control in such vehicles. Therefore, an adaptive MPC controller is designed in this paper for the path tracking task, the developed controller is tuned by an improved particle swarm optimization algorithm. Furthermore, online parameter adaption is performed using Neural Networks and ANFIS. The designed controller showed promising results and adaptation capability against the standard MPC in a triple lane change scenario and a general trajectory test.