pwc-archive/papers-with-abstracts huggingface dataset:
Foundation Models for Recommender Systems: A Survey and New Perspectives
Authors: Chengkai Huang, Tong Yu, Kaige Xie, Shuai Zhang, Lina Yao, Julian McAuley
Date: 2024-02-17 00:00:00
Tasks: Recommendation Systems, Representation Learning

Recently, Foundation Models (FMs), with their extensive knowledge bases and complex architectures, have offered unique opportunities within the realm of recommender systems (RSs). In this paper, we attempt to thoroughly examine FM-based recommendation systems (FM4RecSys). We start by reviewing the research background of FM4RecSys. Then, we provide a systematic taxonomy of existing FM4RecSys research works, which can be divided into four different parts including data characteristics, representation learning, model type, and downstream tasks. Within each part, we review the key recent research developments, outlining the representative models and discussing their characteristics. Moreover, we elaborate on the open problems and opportunities of FM4RecSys aiming to shed light on future research directions in this area. In conclusion, we recap our findings and discuss the emerging trends in this field.
Analyzing Reward Dynamics and Decentralization in Ethereum 2.0: An Advanced Data Engineering Workflow and Comprehensive Datasets for Proof-of-Stake Incentives
Authors: Tao Yan, Shengnan Li, Benjamin Kraner, Luyao Zhang, Claudio J. Tessone
Date: 2024-02-17 00:00:00
Tasks: POS

Ethereum 2.0, as the preeminent smart contract blockchain platform, guarantees the precise execution of applications without third-party intervention. At its core, this system leverages the Proof-of-Stake (PoS) consensus mechanism, which utilizes a stochastic process to select validators for block proposal and validation, consequently rewarding them for their contributions. However, the implementation of blockchain technology often diverges from its central tenet of decentralized consensus, presenting significant analytical challenges. Our study collects consensus reward data from the Ethereum Beacon chain and conducts a comprehensive analysis of reward distribution and evolution, categorizing them into attestation, proposer and sync committee rewards. To evaluate the degree of decentralization in PoS Ethereum, we apply several inequality indices, including the Shannon entropy, the Gini Index, the Nakamoto Coefficient, and the Herfindahl-Hirschman Index (HHI). Our comprehensive dataset is publicly available on Harvard Dataverse, and our analytical methodologies are accessible via GitHub, promoting open-access research. Additionally, we provide insights on utilizing our data for future investigations focused on assessing, augmenting, and refining the decentralization, security, and efficiency of blockchain systems.
Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts
Authors: Yueqin Yin, Zhendong Wang, Yi Gu, Hai Huang, Weizhu Chen, Mingyuan Zhou
Date: 2024-02-12 00:00:00

In the field of large language models (LLMs), aligning models with the diverse preferences of users is a critical challenge. Direct Preference Optimization (DPO) has played a key role in this area. It works by using pairs of preferences derived from the same prompts, and it functions without needing an additional reward model. However, DPO does not fully reflect the complex nature of human learning, which often involves understanding contrasting responses to not only identical but also similar questions. To overcome this shortfall, we propose Relative Preference Optimization (RPO). RPO is designed to discern between more and less preferred responses derived from both identical and related prompts. It introduces a contrastive weighting mechanism, enabling the tuning of LLMs using a broader range of preference data, including both paired and unpaired sets. This approach expands the learning capabilities of the model, allowing it to leverage insights from a more varied set of prompts. Through empirical tests, including dialogue and summarization tasks, and evaluations using the AlpacaEval2.0 leaderboard, RPO has demonstrated a superior ability to align LLMs with user preferences and to improve their adaptability during the training process. Our code can be viewed at https://github.com/yinyueqin/relative-preference-optimization
Assessment of low-carbon tourism development from multi-aspect analysis: A case study of the Yellow River Basin, China
Authors: Xiaopeng Si, Zi Tang
Date: 2024-02-18 00:00:00

Climate change has become an unavoidable problem in achieving sustainable development. As one of the major industries worldwide, tourism can make a significant contribution to mitigating climate change. The main objective of the paper is to assess the development level of low-carbon tourism from multi-aspect, using the Yellow River Basin as an example. Firstly, this study quantified tourism carbon dioxide emissions and tourism economy, and analyzed their evolution characteristics. The interaction and coordination degree between tourism carbon dioxide emissions and tourism economy were then analyzed using the improved coupling coordination degree model. Finally, this study analyzed the change in total factor productivity of low-carbon tourism by calculating the Malmquist-Luenberger productivity index. The results showed that: (1) The tourism industry in the Yellow River Basin has the characteristics of the initial environmental Kuznets curve. (2) There was a strong interaction between tourism carbon dioxide emissions and tourism economy, which was manifested as mutual promotion. (3) The total factor productivity of low-carbon tourism was increasing. Based on the above results, it could be concluded that the development level of low-carbon tourism in the Yellow River Basin has been continuously improved from 2000 to 2019, but it is still in the early development stage with the continuous growth of carbon dioxide emissions.
Graph neural topic model with commonsense knowledge
Authors: Bingshan Zhu, Yi Cai, Haopeng Ren
Date: 2023-03-02 00:00:00
Proceeding: journal 2023 3
Tasks: Graph Neural Network, model, Topic Models

Traditional topic models are based on the bag-of-words assumption, which states that the topic assignment of each word is independent of the others. However, this assumption ignores the relationship between words, which may hinder the quality of extracted topics. To address this issue, some recent works formulate documents as graphs based on word co-occurrence patterns. It assumes that if two words co-occur frequently, they should have the same topic. Nevertheless, it introduces noise edges into the model and thus hinders topic quality since two words co-occur frequently do not mean that they are on the same topic. In this paper, we use the commonsense relationship between words as a bridge to connect the words in each document. Compared to word co-occurrence, the commonsense relationship can explicitly imply the semantic relevance between words, which can be utilized to filter out noise edges. We use a relational graph neural network to capture the relation information in the graph. Moreover, manifold regularization is utilized to constrain the documents’ topic distributions. Experimental results on a public dataset show that our method is effective at extracting topics compared to baseline methods.
Generating Natural Language from Logic Expressions with Structural Representation
Authors: Xin Wu; Yi Cai; Zetao Lian; Ho-fung Leung; Tao Wang
Date: 2023-04-04 00:00:00
Proceeding: journal 2023 4
Tasks: Sentence

Incorporating logic reasoning with deep neural networks (DNNs) is an important challenge in machine learning. In this article, we study the problem of converting logical expressions into natural language. In particular, given a sequential logic expression, the goal is to generate its corresponding natural sentence. Since the information in a logic expression often has a hierarchical structure, a sequence-to-sequence baseline struggles to capture the full dependencies between words, and hence it often generates incorrect sentences. To alleviate this problem, we propose a model to convert Structural Logic Expressions into Natural Language (SLEtoNL). SLEtoNL converts sequential logic expressions into structural representation and leverages structural encoders to capture the dependencies between nodes. The quantitative and qualitative analyses demonstrate that our proposed method outperforms the seq2seq model, which is based on the sequential representation, and outperforms strong pretrained language models (e.g., T5, BART, GPT3) with a large margin (28.6 in BLEU3) in out-of-distribution evaluation.
StreamE: Learning to Update Representations for Temporal Knowledge Graphs in Streaming Scenarios
Authors: Jiasheng Zhang, Jie Shao, Bin Cui
Date: 2023-07-01 00:00:00
Proceeding: journal 2023 7
Tasks: Knowledge Graphs

Learning representations for temporal knowledge graphs (TKGs) is a fundamental task. Most existing methods regard TKG as a sequence of static snapshots and recurrently learn representations by retracing the previous snapshots. However, new knowledge can be continuously accrued to TKGs as streams. These methods either cannot handle new entities or fail to update representations in real time, making them unfeasible to adapt to the streaming scenarios. In this paper, we propose a lightweight framework called StreamE towards the efficient generation of TKG representations in streaming scenarios. To reduce the parameter size, entity representations in StreamE are decoupled from the model training to serve as the memory module to store the historical information of entities. To achieve efficient update and generation, the process of generating representations is decoupled as two functions in StreamE. An update function is learned to incrementally update entity representations based on the newly-arrived knowledge and a read function is learned to predict the future semantics of entity representations. The update function avoids the recurrent modeling paradigm and thus gains high efficiency while the read function considers multiple semantic change properties. We further propose a joint training strategy with two temporal regularizations to effectively optimize the framework. Experimental results show that StreamE can achieve better performance than baseline methods with 100x faster in inference, 25x faster in training, and only 1/5 parameter size, which demonstrates its superiority. Code is available at https://github.com/zjs123/StreamE.
A Self-Attention Causal LSTM Model for Precipitation Nowcasting
Authors: Lei She; Chenghong Zhang; Xin Man; Xuewei Luo; Jie Shao
Date: 2023-07-10 00:00:00
Proceeding: journal 2023 7
Tasks: Generative Adversarial Network

Precipitation nowcasting is a critical task that can facilitate multiple applications including urban warnings and traffic. Deep learning methods combining convolutional neural networks and recurrent neural networks have been investigated for this problem. However, they always capture local and ineffective spatial dependencies through convolutional layers, while long-range spatial dependencies are critical for spatial applications. To improve long-range representations, we propose a spatiotemporal prediction model called Self-Attention Causal LSTM (SAC-LSTM), which utilizes a self-attention mechanism to model channel correlations. SAC-LSTM aggregates sequence features by extracting spatial features with global and local dependencies so that the visual details of each frame can be significantly preserved. Furthermore, SAC-LSTM is trained by a generative adversarial network with a learned perceptual loss, which improves the perceptual quality of predictions. Experimental results show that the proposed SAC-LSTM outperforms other models. Code is available at https://github.com/LeiShel/SAC-LSTM-MindSpore.
Artificial intelligence models to evaluate the impact of climate change on groundwater resources
Authors: Daniele Secci, *, Maria Giovanna Tanda a, Marco D’Oria, Valeria Todaro
Date: 2023-10-09 00:00:00
Proceeding: Journal of Hydrology 627 (2023) 130359 2023 10

This study develops three different artificial intelligence (AI) models in order to investigate the effects of climate 
change on groundwater resources using historical records of precipitation, temperature and groundwater levels 
together with regional climate projections. In particular, the Non-linear Autoregressive Neural Network (NARX), 
the Long-Short Term Memory Neural Network (LSTM) and the Convolutional Neural Network (CNN) were 
compared. Considering an aquifer located in northern Italy as a case study, the neural networks were trained to 
replicate observed groundwater levels by taking as input precipitation and temperature records, and in the case 
of the NARX also antecedent groundwater levels, on a monthly scale. The trained networks were used to infer 
groundwater levels until the end of the century based on precipitation and temperature projections provided by 
an ensemble of 13 Regional Climate Models (RCMs) from the EURO-CORDEX initiative. Two emission pathways 
were considered: the RCP4.5 and RCP8.5. All the AI models show good performance metrics during the training 
phase, but NARXs perform poorly compared to the other models during validation and testing. For the future, the 
NARX and LSTM models predict a decline in groundwater levels, especially for the RCP8.5 scenario, while slight 
changes are expected using the CNN. As NARXs are not deep learning techniques and CNNs may not be able to 
extrapolate values outside the training range, LSTMs appear to be better suited for climate change impact 
evaluations.
Automated Detection of Welding Defects without Segmentation
Authors: Domingo Mery
Date: 2019-08-01 00:00:00
Proceeding: NDT & E International 2019 8
Tasks: Classification, Human Detection, object-detection, Object Detection

Abstract
Substantial research has been performed on automated detection and classification of
welding defects in continuous welds using X-ray imaging. Typically, the detection
follows a pattern recognition schema (segmentation, feature extraction and
classification). In computer vision community, however, many object detection and
classification problems, like face and human detection, have been recently solved -
without segmentation- using sliding-windows and novel features like local binary
patterns extracted from saliency maps. For this reason, we propose in this paper the
use of sliding-windows with the mentioned features to perform automatically the
automated detection of welding defects. In the experiments, we analyzed 5000
detection windows (24x24 pixels) and 572 intensity features from 10 representative
X-ray images. Cross validation yielded a detection performance of 94% using a
support vector machine classifier with only 14 selected features. The method was
implemented and tested on real X-ray images showing high effectiveness. We believe
that the proposed approach opens new possibilities in the field of automated detection
of welding defects
Flexible Robust Optimal Bidding of Renewable Virtual Power Plants in Sequential Markets
Authors: Hadi Nemati, Pedro Sánchez-Martín, Álvaro Ortega, Lukas Sigrist, Enrique Lobato, Luis Rouco
Date: 2024-02-19 00:00:00
Tasks: Computational Efficiency

In this paper, a novel approach to define the optimal bidding of renewable-only virtual power plants (RVPPs) in the day-ahead, secondary reserve, and intra-day markets is proposed. To this aim, a robust optimization algorithm is developed to account for the asymmetric nature of the uncertainties that characterize the market prices, as well as the energy production of the RVPP stochastic sources and flexible demand consumption. Simulation results show increased RVPP benefits compared to other existing solutions and demonstrate the potential of renewable sources to further increase their economic competitiveness. The simplicity of the implementation, the computational efficiency, and the flexible robustness are also verified.
H²O: Heatmap by Hierarchical Occlusion
Authors: Luc-Etienne Pommé, Romain Bourqui, Romain Giot
Date: 2023-09-20 00:00:00
Proceeding: CBMI 2023 2023 9
Tasks: image-classification, Image Classification, Image Segmentation, object-detection, Object Detection, Semantic Segmentation

The rise of Deep Learning (DL) has led to a breakthrough in the research field of content-based multimedia indexing. Newly de- veloped systems based on complex models outperform classic ma- chine learning algorithms in object detection, image segmentation or classification tasks. However, despite their high performance, these systems still make mistakes. To be used in industrial condi- tions, these systems must be able to provide trustworthy decisions with guarantees or justifications. Therefore, it is crucial to provide means to analyze and comprehend the decision process that leads a model to its decision. Image classification implies tracking and understanding which input features the model relies on to make its prediction. This paper focuses on features attribution techniques and proposes Heatmaps by Hierarchical Occlusion (H²O), a novel method for detecting pattern-relevant features in an image. We also propose two new pairs of metrics that overcome some evaluation issues: (a) Insertion and Deletion Spearman correlation coefficients which both estimate a correlation between the computed scores in a saliency map and the importance for the model of the associated pixels in the image. (b) Insertion Positive and Deletion Negative Gradient Sums both estimate the coherence of the scores in the saliency maps. Both visual inspection and evaluation on 7 metrics show that H²O is competitive against state-of-the-art methods.
From parcels to people: development of a spatially explicit risk indicator to monitor residential pesticide exposure in agricultural areas
Authors: Francesco Galimberti, Stephanie Bopp, Alessandro Carletti, Rui Catarino, Martin Claverie, Pietro Florio, Alessio Ippolito, Arwyn Jones, Flavio Marchetto, Michael Olvedy, Alberto Pistocchi, Astrid Verhegghen, Marijn van der Velde, Diana Vieira, Raphael d'Andrimont
Date: 2024-02-16 00:00:00

The increase in global pesticide use has mirrored the rising demand for food over the last decades, resulting in a boost in crop yields. However, concerns about the impact of pesticides on biodiversity, ecosystems, and human health, especially for populations residing close to cultivated areas, are growing. This study investigates how exposure and possible risks to residents can be estimated at high spatial granularity based on plant protection product data. The complexities of such analysis were explored in France, where relevant data with good granularity are publicly available. Integrating sets of spatial datasets and exposure assessment methodologies, we have developed an indicator to monitor the levels of pesticide risk faced by residents. By spatialising pesticide sales data according to their authorization on specific crops, we developed a detailed map depicting potential pesticide loads at parcel level across France. This spatial distribution served as the basis for an exposure and risk assessment, modelled following the European Food Safety Authority's guidelines. Combining the risk map with population distribution data, we have developed an indicator that allows to monitor patterns in non-dietary exposure to pesticides. Our results show that in France, on average, 13% of people might be exposed to pesticides due to living in the proximity to treated crops. This exposure is in the lower range for 34%, moderate range for 40% and higher range for 25% of the exposed population. The risk evaluation is based on worst case assumptions and values should not be taken as a regulatory risk assessment but as indicator to use, for example, for monitoring time trends. The purpose of this indicator is to demonstrate that more granular pesticide data can improve risk reduction strategies. Harmonized and high-resolution data can help in identifying regions where to focus on sustainable farming.
ACRE: Actor-Critic with Reward-Preserving Exploration
Authors: A. Ch. Kapoutsis, D. I. Koutras, Ch. D. Korkas, E. B. Kosmatopoulos
Date: 2023-08-14 00:00:00
Proceeding: Neural Computing and Applications 2023 8
Tasks: continuous-control, Continuous Control, Reinforcement Learning (RL)

While reinforcement learning (RL) algorithms have generated impressive strategies for a wide range of tasks, the performance improvements in continuous-domain, real-world problems do not follow the same trend. Poor exploration and quick convergence to locally optimal solutions play a dominant role. Advanced RL algorithms attempt to mitigate this issue by introducing exploration signals during the training procedure. This successful integration has paved the way to introduce signals from the intrinsic exploration branch. ACRE algorithm is a framework that concretely describes the conditions for such an integration, avoiding transforming the Markov decision process into time varying, and as a result, making the whole optimization scheme brittle and susceptible to instability. The key distinction of ACRE lies in the way of handling and storing both extrinsic and intrinsic rewards. ACRE is an off-policy, actor-critic style RL algorithm that separately approximates the forward novelty return. ACRE is shipped with a Gaussian mixture model to calculate the instantaneous novelty; however, different options could also be integrated. Using such an effective early exploration, ACRE results in substantial improvements over alternative RL methods, in a range of continuous control RL environments, such as learning from policy-misleading reward signals. Open-source implementation is available here: https://github.com/athakapo/ACRE.
Distributionally Robust Graph-based Recommendation System
Authors: Bohao Wang, Jiawei Chen, Changdong Li, Sheng Zhou, Qihao Shi, Yang Gao, Yan Feng, Chun Chen, Can Wang
Date: 2024-02-20 00:00:00
Tasks: Recommendation Systems

With the capacity to capture high-order collaborative signals, Graph Neural Networks (GNNs) have emerged as powerful methods in Recommender Systems (RS). However, their efficacy often hinges on the assumption that training and testing data share the same distribution (a.k.a. IID assumption), and exhibits significant declines under distribution shifts. Distribution shifts commonly arises in RS, often attributed to the dynamic nature of user preferences or ubiquitous biases during data collection in RS. Despite its significance, researches on GNN-based recommendation against distribution shift are still sparse. To bridge this gap, we propose Distributionally Robust GNN (DR-GNN) that incorporates Distributional Robust Optimization (DRO) into the GNN-based recommendation. DR-GNN addresses two core challenges: 1) To enable DRO to cater to graph data intertwined with GNN, we reinterpret GNN as a graph smoothing regularizer, thereby facilitating the nuanced application of DRO; 2) Given the typically sparse nature of recommendation data, which might impede robust optimization, we introduce slight perturbations in the training distribution to expand its support. Notably, while DR-GNN involves complex optimization, it can be implemented easily and efficiently. Our extensive experiments validate the effectiveness of DR-GNN against three typical distribution shifts. The code is available at https://github.com/WANGBohaO-jpg/DR-GNN.