pwc-archive/papers-with-abstracts huggingface dataset:
Contrastive Pre-training for Deep Session Data Understanding
Authors: Zixuan Li, Lizi Liao, Yunshan Ma, Tat-Seng Chua
Date: 2024-03-05 00:00:00
Tasks: Contrastive Learning

Session data has been widely used for understanding user's behavior in e-commerce. Researchers are trying to leverage session data for different tasks, such as purchase intention prediction, remaining length prediction, recommendation, etc., as it provides context clues about the user's dynamic interests. However, online shopping session data is semi-structured and complex in nature, which contains both unstructured textual data about the products, search queries, and structured user action sequences. Most existing works focus on leveraging the coarse-grained item sequences for specific tasks, while largely ignore the fine-grained information from text and user action details. In this work, we delve into deep session data understanding via scrutinizing the various clues inside the rich information in user sessions. Specifically, we propose to pre-train a general-purpose User Behavior Model (UBM) over large-scale session data with rich details, such as product title, attributes and various kinds of user actions. A two-stage pre-training scheme is introduced to encourage the model to self-learn from various augmentations with contrastive learning objectives, which spans different granularity levels of session data. Then the well-trained session understanding model can be easily fine-tuned for various downstream tasks. Extensive experiments show that UBM better captures the complex intra-item semantic relations, inter-item connections and inter-interaction dependencies, leading to large performance gains as compared to the baselines on several downstream tasks. And it also demonstrates strong robustness when data is sparse.
Learning to Ask Critical Questions for Assisting Product Search
Authors: Zixuan Li, Lizi Liao, Tat-Seng Chua
Date: 2024-03-05 00:00:00
Tasks: Information Retrieval

Product search plays an essential role in eCommerce. It was treated as a special type of information retrieval problem. Most existing works make use of historical data to improve the search performance, which do not take the opportunity to ask for user's current interest directly. Some session-aware methods take the user's clicks within the session as implicit feedback, but it is still just a guess on user's preference. To address this problem, recent conversational or question-based search models interact with users directly for understanding the user's interest explicitly. However, most users do not have a clear picture on what to buy at the initial stage. Asking critical attributes that the user is looking for after they explored for a while should be a more efficient way to help them searching for the target items. In this paper, we propose a dual-learning model that hybrids the best from both implicit session feedback and proactively clarifying with users on the most critical questions. We first establish a novel utility score to measure whether a clicked item provides useful information for finding the target. Then we develop the dual Selection Net and Ranking Net for choosing the critical questions and ranking the items. It innovatively links traditional click-stream data and text-based questions together. To verify our proposal, we did extensive experiments on a public dataset, and our model largely outperformed other state-of-the-art methods.
Uplift Modeling for Target User Attacks on Recommender Systems
Authors: Wenjie Wang, Changsheng Wang, Fuli Feng, Wentao Shi, Daizong Ding, Tat-Seng Chua
Date: 2024-03-05 00:00:00
Tasks: Recommendation Systems

Recommender systems are vulnerable to injective attacks, which inject limited fake users into the platforms to manipulate the exposure of target items to all users. In this work, we identify that conventional injective attackers overlook the fact that each item has its unique potential audience, and meanwhile, the attack difficulty across different users varies. Blindly attacking all users will result in a waste of fake user budgets and inferior attack performance. To address these issues, we focus on an under-explored attack task called target user attacks, aiming at promoting target items to a particular user group. In addition, we formulate the varying attack difficulty as heterogeneous treatment effects through a causal lens and propose an Uplift-guided Budget Allocation (UBA) framework. UBA estimates the treatment effect on each target user and optimizes the allocation of fake user budgets to maximize the attack performance. Theoretical and empirical analysis demonstrates the rationality of treatment effect estimation methods of UBA. By instantiating UBA on multiple attackers, we conduct extensive experiments on three datasets under various settings with different target items, target users, fake user budgets, victim models, and defense models, validating the effectiveness and robustness of UBA.
Magnetic Localization for In-body Nano-communication Medical Systems
Authors: Krzysztof Skos, Josep Miquel Jornet, Pawel Kulakowski
Date: 2024-03-04 00:00:00
Tasks: Diagnostic

Nano-machines circulating inside the human body, collecting data on tissue conditions, represent a vital part of next-generation medical diagnostic systems. However, for these devices to operate effectively, they need to relay not only their medical measurements but also their positions. This paper introduces a novel localization method for in-body nano-machines based on the magnetic field, leveraging the advantageous magnetic permeability of all human tissues. The entire proposed localization system is described, starting from 10x10 ${\mu}m^2$ magnetometers to be integrated into the nano-machines, to a set of external wires generating the magnetic field. Mathematical equations for the localization algorithm are also provided, assuming the nano-machines do not execute the computations themselves, but transmit their magnetic field measurements together with medical data outside of the body. The whole system is validated with computer simulations that capture the measurement error of the magnetometers, the error induced by the Earth magnetic field, and a human body model assuming different possible positions of nano-machines. The results show a very high system accuracy with localization errors even below 1 cm.
CODE-ACCORD: A Corpus of Building Regulatory Data for Rule Generation towards Automatic Compliance Checking
Authors: Hansi Hettiarachchi, Amna Dridi, Mohamed Medhat Gaber, Pouyan Parsafard, Nicoleta Bocaneala, Katja Breitenfelder, Gonçal Costa, Maria Hedblom, Mihaela Juganaru-Mathieu, Thamer Mecharnia, Sumee Park, He Tan, Abdel-Rahman H. Tawil, Edlira Vakaj
Date: 2024-03-04 00:00:00
Tasks: Relation Extraction, Sentence, text-classification, Text Classification

Automatic Compliance Checking (ACC) within the Architecture, Engineering, and Construction (AEC) sector necessitates automating the interpretation of building regulations to achieve its full potential. Converting textual rules into machine-readable formats is challenging due to the complexities of natural language and the scarcity of resources for advanced Machine Learning (ML). Addressing these challenges, we introduce CODE-ACCORD, a dataset of 862 sentences from the building regulations of England and Finland. Only the self-contained sentences, which express complete rules without needing additional context, were considered as they are essential for ACC. Each sentence was manually annotated with entities and relations by a team of 12 annotators to facilitate machine-readable rule generation, followed by careful curation to ensure accuracy. The final dataset comprises 4,297 entities and 4,329 relations across various categories, serving as a robust ground truth. CODE-ACCORD supports a range of ML and Natural Language Processing (NLP) tasks, including text classification, entity recognition, and relation extraction. It enables applying recent trends, such as deep neural networks and large language models, to ACC.
Evaluating the Explainability of Neural Rankers
Authors: Saran Pandian, Debasis Ganguly, Sean MacAvaney
Date: 2024-03-04 00:00:00
Tasks: Information Retrieval, Sentence

Information retrieval models have witnessed a paradigm shift from unsupervised statistical approaches to feature-based supervised approaches to completely data-driven ones that make use of the pre-training of large language models. While the increasing complexity of the search models have been able to demonstrate improvements in effectiveness (measured in terms of relevance of top-retrieved results), a question worthy of a thorough inspection is - "how explainable are these models?", which is what this paper aims to evaluate. In particular, we propose a common evaluation platform to systematically evaluate the explainability of any ranking model (the explanation algorithm being identical for all the models that are to be evaluated). In our proposed framework, each model, in addition to returning a ranked list of documents, also requires to return a list of explanation units or rationales for each document. This meta-information from each document is then used to measure how locally consistent these rationales are as an intrinsic measure of interpretability - one that does not require manual relevance assessments. Additionally, as an extrinsic measure, we compute how relevant these rationales are by leveraging sub-document level relevance assessments. Our findings show a number of interesting observations, such as sentence-level rationales are more consistent, an increase in complexity mostly leads to less consistent explanations, and that interpretability measures offer a complementary dimension of evaluation of IR systems because consistency is not well-correlated with nDCG at top ranks.
Recommending Missed Citations Identified by Reviewers: A New Task, Dataset and Baselines
Authors: Kehan Long, Shasha Li, Pancheng Wang, Chenlong Bao, Jintao Tang, Ting Wang
Date: 2024-03-04 00:00:00
Tasks: Citation Recommendation, Recommendation Systems

Citing comprehensively and appropriately has become a challenging task with the explosive growth of scientific publications. Current citation recommendation systems aim to recommend a list of scientific papers for a given text context or a draft paper. However, none of the existing work focuses on already included citations of full papers, which are imperfect and still have much room for improvement. In the scenario of peer reviewing, it is a common phenomenon that submissions are identified as missing vital citations by reviewers. This may lead to a negative impact on the credibility and validity of the research presented. To help improve citations of full papers, we first define a novel task of Recommending Missed Citations Identified by Reviewers (RMC) and construct a corresponding expert-labeled dataset called CitationR. We conduct an extensive evaluation of several state-of-the-art methods on CitationR. Furthermore, we propose a new framework RMCNet with an Attentive Reference Encoder module mining the relevance between papers, already-made citations, and missed citations. Empirical results prove that RMC is challenging, with the proposed architecture outperforming previous methods in all metrics. We release our dataset and benchmark models to motivate future research on this challenging new task.
Unleashing Graph Partitioning for Large-Scale Nearest Neighbor Search
Authors: Lars Gottesbüren, Laxman Dhulipala, Rajesh Jayaram, Jakub Lacki
Date: 2024-03-04 00:00:00
Tasks: graph partitioning

We consider the fundamental problem of decomposing a large-scale approximate nearest neighbor search (ANNS) problem into smaller sub-problems. The goal is to partition the input points into neighborhood-preserving shards, so that the nearest neighbors of any point are contained in only a few shards. When a query arrives, a routing algorithm is used to identify the shards which should be searched for its nearest neighbors. This approach forms the backbone of distributed ANNS, where the dataset is so large that it must be split across multiple machines. In this paper, we design simple and highly efficient routing methods, and prove strong theoretical guarantees on their performance. A crucial characteristic of our routing algorithms is that they are inherently modular, and can be used with any partitioning method. This addresses a key drawback of prior approaches, where the routing algorithms are inextricably linked to their associated partitioning method. In particular, our new routing methods enable the use of balanced graph partitioning, which is a high-quality partitioning method without a naturally associated routing algorithm. Thus, we provide the first methods for routing using balanced graph partitioning that are extremely fast to train, admit low latency, and achieve high recall. We provide a comprehensive evaluation of our full partitioning and routing pipeline on billion-scale datasets, where it outperforms existing scalable partitioning methods by significant margins, achieving up to 2.14x higher QPS at 90% recall$@10$ than the best competitor.
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Authors: Chao Zhang, Shiwei Wu, Haoxin Zhang, Tong Xu, Yan Gao, Yao Hu, Di wu, Enhong Chen
Date: 2024-03-04 00:00:00
Tasks: Contrastive Learning, Language Modeling, Language Modelling, Large Language Model

People enjoy sharing "notes" including their experiences within online communities. Therefore, recommending notes aligned with user interests has become a crucial task. Existing online methods only input notes into BERT-based models to generate note embeddings for assessing similarity. However, they may underutilize some important cues, e.g., hashtags or categories, which represent the key concepts of notes. Indeed, learning to generate hashtags/categories can potentially enhance note embeddings, both of which compress key note information into limited content. Besides, Large Language Models (LLMs) have significantly outperformed BERT in understanding natural languages. It is promising to introduce LLMs into note recommendation. In this paper, we propose a novel unified framework called NoteLLM, which leverages LLMs to address the item-to-item (I2I) note recommendation. Specifically, we utilize Note Compression Prompt to compress a note into a single special token, and further learn the potentially related notes' embeddings via a contrastive learning approach. Moreover, we use NoteLLM to summarize the note and generate the hashtag/category automatically through instruction tuning. Extensive validations on real scenarios demonstrate the effectiveness of our proposed method compared with the online baseline and show major improvements in the recommendation system of Xiaohongshu.
TweetInfo: An Interactive System to Mitigate Online Harm
Authors: Gautam Kishore Shahi
Date: 2024-03-03 00:00:00
Tasks: Misinformation

The increase in active users on social networking sites (SNSs) has also observed an increase in harmful content on social media sites. Harmful content is described as an inappropriate activity to harm or deceive an individual or a group of users. Alongside existing methods to detect misinformation and hate speech, users still need to be well-informed about the harmfulness of the content on SNSs. This study proposes a user-interactive system TweetInfo for mitigating the consumption of harmful content by providing metainformation about the posts. It focuses on two types of harmful content: hate speech and misinformation. TweetInfo provides insights into tweets by doing content analysis. Based on previous research, we have selected a list of metainformation. We offer the option to filter content based on metainformation Bot, Hate Speech, Misinformation, Verified Account, Sentiment, Tweet Category, Language. The proposed user interface allows customising the user's timeline to mitigate harmful content. This study present the demo version of the propose user interface of TweetInfo.
Measuring Technological Convergence in Encryption Technologies with Proximity Indices: A Text Mining and Bibliometric Analysis using OpenAlex
Authors: Alessandro Tavazzi, Dimitri Percia David, Julian Jang-Jaccard, Alain Mermoud
Date: 2024-03-03 00:00:00

Identifying technological convergence among emerging technologies in cybersecurity is crucial for advancing science and fostering innovation. Unlike previous studies focusing on the binary relationship between a paper and the concept it attributes to technology, our approach utilizes attribution scores to enhance the relationships between research papers, combining keywords, citation rates, and collaboration status with specific technological concepts. The proposed method integrates text mining and bibliometric analyses to formulate and predict technological proximity indices for encryption technologies using the "OpenAlex" catalog. Our case study findings highlight a significant convergence between blockchain and public-key cryptography, evidenced by the increasing proximity indices. These results offer valuable strategic insights for those contemplating investments in these domains.
TFPred: Learning Discriminative Representations from Unlabeled Data for Few-Label Rotating Machinery Fault Diagnosis
Authors: Xiaohan Chen, Rui Yang, Yihao Xue, Baoye Song, Zidong Wang
Date: 2024-05-01 00:00:00
Proceeding: Control Engineering Practice 2024 5
Tasks: Fault Detection, Fault Diagnosis, Self-Supervised Learning, Time Series Classification

Recent advances in intelligent rotating machinery fault diagnosis have been enabled by the availability of massive labeled training data. However, in practical industrial applications, it is often challenging and costly to annotate a large amount of data. To address the few-label fault diagnosis problem, a time–frequency prediction (TFPred) self-supervised learning framework is proposed to extract latent fault representations from unlabeled fault data. Specifically, the TFPred framework consists of a time encoder and a frequency encoder, with the frequency encoder to predict the low-dimensional representations of time domain signals generated by the time encoder with randomly augmented data. Subsequently, the pre-trained network is hooked with a classification head and fine-tuned with limited labeled data. Finally, the proposed framework is evaluated on a run-to-failure bearing dataset and a hardware-in-the-loop high-speed train simulation platform. The experiments demonstrate that the self-supervised learning framework TFPred achieved competitive performance with only 1% and 5% labeled data. Code is available at https://github.com/Xiaohan-Chen/TFPred.
Metric Nearness Made Practical
Authors: Wenye Li, Fangchen Yu, Zichen Ma
Date: 2023-06-26 00:00:00
Proceeding: The Thirty-Seventh AAAI Conference on Artificial Intelligence 2023 6
Tasks: valid

Given a square matrix with noisy dissimilarity measures between pairs of data samples, the metric nearness model computes the best approximation of the matrix from a set of valid distance metrics. Despite its wide applications in machine learning and data processing tasks, the model faces non-trivial computational requirements in seeking the solution due to the large number of metric constraints associated with the feasible region. Our work designed a practical approach in two stages to tackle the challenge and improve the model’s scalability and applicability. The first stage computes a fast yet high-quality approximate solution from a set of isometrically embeddable metrics, further improved by an effective heuristic. The second stage refines the approximate solution with the Halpern-Lions-Wittmann-Bauschke projection algorithm, which converges quickly to the optimal solution. In empirical evaluations, the proposed approach runs at least an order of magnitude faster than the state-of-the-art solutions, with significantly improved scalability, complete conformity to constraints, less memory consumption, and other desirable features in real applications.
Equilibrium Model with Anisotropy for Model-Based Reconstruction in Magnetic Particle Imaging
Authors: Marco Maass, Tobias Kluth, Christine Droigk, Hannes Albers, Konrad Scheffler, Alfred Mertins, Tobias Knopp
Date: 2024-03-01 00:00:00
Tasks: Image Reconstruction, model

Magnetic particle imaging is a tracer-based tomographic imaging technique that allows the concentration of magnetic nanoparticles to be determined with high spatio-temporal resolution. To reconstruct an image of the tracer concentration, the magnetization dynamics of the particles must be accurately modeled. A popular ensemble model is based on solving the Fokker-Plank equation, taking into account either Brownian or N\'eel dynamics. The disadvantage of this model is that it is computationally expensive due to an underlying stiff differential equation. A simplified model is the equilibrium model, which can be evaluated directly but in most relevant cases it suffers from a non-negligible modeling error. In the present work, we investigate an extended version of the equilibrium model that can account for particle anisotropy. We show that this model can be expressed as a series of Bessel functions, which can be truncated based on a predefined accuracy, leading to very short computation times, which are about three orders of magnitude lower than equivalent Fokker-Planck computation times. We investigate the accuracy of the model for 2D Lissajous magnetic particle imaging sequences and show that the difference between the Fokker-Planck and the equilibrium model with anisotropy is sufficiently small so that the latter model can be used for image reconstruction on experimental data with only marginal loss of image quality, even compared to a system matrix-based reconstruction.
Spatial Variation Generation Algorithm for Motor Imagery Data Augmentation: Increasing the Density of Sample Vicinity
Authors: Chengxuan Qin, Rui Yang, Mengjie Huang, Weibo Liu, Zidong Wang
Date: 2023-09-12 00:00:00
Proceeding: IEEE Transactions on Neural Systems and Rehabilitation Engineering 2023 9
Tasks: Data Augmentation, EEG, Eeg Decoding, EEG Left/Right hand, Motor Imagery

The imbalanced development between deep learning-based model design and motor imagery (MI) data acquisition raises concerns about the potential overfitting issue—models can identify training data well but fail to generalize test data. In this study, a Spatial Variation Generation (SVG) algorithm for MI data augmentation is proposed to alleviate the overfitting issue. In essence, SVG generates MI data using variations of electrode placement and brain spatial pattern, ultimately elevating the density of the raw sample vicinity. The proposed SVG prevents models from memorizing the training data by replacing the raw samples with the proper vicinal distribution. Moreover, SVG generates a uniform distribution and stabilizes the training process of models. In comparison studies involving five deep learning-based models across eight datasets, the proposed SVG algorithm exhibited a notable improvement of 0.021 in the area under the receiver operating characteristic curve (AUC). The improvement achieved by SVG outperforms other data augmentation algorithms. Further results from the ablation study verify the effectiveness of each component of SVG. Finally, the studies in the control group with varying numbers of samples show that the SVG algorithm consistently improves the AUC, with improvements ranging from approximately 0.02 to 0.15.