pwc-archive/papers-with-abstracts huggingface dataset:
MelodyVis: Visual Analytics for Melodic Patterns in Sheet Music
Authors: Matthias Miller, Daniel Fürst, Maximilian T. Fischer, Hanna Hauptmann, Daniel Keim, Mennatallah El-Assady
Date: 2024-07-07 00:00:00

Manual melody detection is a tedious task requiring high expertise level, while automatic detection is often not expressive or powerful enough. Thus, we present MelodyVis, a visual application designed in collaboration with musicology experts to explore melodic patterns in digital sheet music. MelodyVis features five connected views, including a Melody Operator Graph and a Voicing Timeline. The system utilizes eight atomic operators, such as transposition and mirroring, to capture melody repetitions and variations. Users can start their analysis by manually selecting patterns in the sheet view, and then identifying other patterns based on the selected samples through an interactive exploration process. We conducted a user study to investigate the effectiveness and usefulness of our approach and its integrated melodic operators, including usability and mental load questions. We compared the analysis executed by 25 participants with and without the operators. The study results indicate that the participants could identify at least twice as many patterns with activated operators. MelodyVis allows analysts to steer the analysis process and interpret results. Our study also confirms the usefulness of MelodyVis in supporting common analytical tasks in melodic analysis, with participants reporting improved pattern identification and interpretation. Thus, MelodyVis addresses the limitations of fully-automated approaches, enabling music analysts to step into the analysis process and uncover and understand intricate melodic patterns and transformations in sheet music.
Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation
Authors: Xinglong Wu, Anfeng Huang, HongWei Yang, Hui He, Yu Tai, Weizhe Zhang
Date: 2024-07-07 00:00:00
Tasks: cross-modal alignment, Multi-modal Recommendation, Recommendation Systems, Semantic Similarity, Semantic Textual Similarity

Multi-modal recommendation greatly enhances the performance of recommender systems by modeling the auxiliary information from multi-modality contents. Most existing multi-modal recommendation models primarily exploit multimedia information propagation processes to enrich item representations and directly utilize modal-specific embedding vectors independently obtained from upstream pre-trained models. However, this might be inappropriate since the abundant task-specific semantics remain unexplored, and the cross-modality semantic gap hinders the recommendation performance. Inspired by the recent progress of the cross-modal alignment model CLIP, in this paper, we propose a novel \textbf{CLIP} \textbf{E}nhanced \textbf{R}ecommender (\textbf{CLIPER}) framework to bridge the semantic gap between modalities and extract fine-grained multi-view semantic information. Specifically, we introduce a multi-view modality-alignment approach for representation extraction and measure the semantic similarity between modalities. Furthermore, we integrate the multi-view multimedia representations into downstream recommendation models. Extensive experiments conducted on three public datasets demonstrate the consistent superiority of our model over state-of-the-art multi-modal recommendation models.
Redefining POI Popularity: Integrating User Preferences and Recency for Enhanced Recommendations
Authors: Alif Al Hasan, Md. Musfique Anwar, M. Arifur Rahman
Date: 2024-07-07 00:00:00

The task of point-of-interest (POI) recommendation is to predict users' immediate future movements based on their previous records and present circumstances. Popularity is considered as one of the primary deciding factors for selecting the next place to visit. Existing approaches mainly focused on the number of check-ins to model the popularity of a POI. However, not enough attention is paid to the temporal impact or number of people check-ins for a particular POI. Thus, to prioritize more on recent check-ins, we propose recency-oriented definition of POI's popularity by considering the temporal effect of the POIs, the number of check-ins, as well as the number of users who registered in those check-ins. Our experimental results on real dataset show the efficacy of the proposed approach.
Understanding and Addressing Gender Bias in Expert Finding Task
Authors: Maddalena Amendola, Carlos Castillo, Andrea Passarella, Raffaele Perego
Date: 2024-07-07 00:00:00

The Expert Finding (EF) task is critical in community Question&Answer (CQ&A) platforms, significantly enhancing user engagement by improving answer quality and reducing response times. However, biases, especially gender biases, have been identified in these platforms. This study investigates gender bias in state-of-the-art EF models and explores methods to mitigate it. Utilizing a comprehensive dataset from StackOverflow, the largest community in the StackExchange network, we conduct extensive experiments to analyze how EF models' candidate identification processes influence gender representation. Our findings reveal that models relying on reputation metrics and activity levels disproportionately favor male users, who are more active on the platform. This bias results in the underrepresentation of female experts in the ranking process. We propose adjustments to EF models that incorporate a more balanced preprocessing strategy and leverage content-based and social network-based information, with the aim to provide a fairer representation of genders among identified experts. Our analysis shows that integrating these methods can significantly enhance gender balance without compromising model accuracy. To the best of our knowledge, this study is the first to focus on detecting and mitigating gender bias in EF methods.
Ensemble Boost: Greedy Selection for Superior Recommender Systems
Authors: Zainil Mehta, Tobias Vente
Date: 2024-07-07 00:00:00
Tasks: Recommendation Systems

Ensemble techniques have demonstrated remarkable success in improving predictive performance across various domains by aggregating predictions from multiple models [1]. In the realm of recommender systems, this research explores the application of ensemble technique to enhance recommendation quality. Specifically, we propose a novel approach to combine top-k recommendations from ten diverse recommendation models resulting in superior top-n recommendations using this novel ensemble technique. Our method leverages a Greedy Ensemble Selection(GES) strategy, effectively harnessing the collective intelligence of multiple models. We conduct experiments on five distinct datasets to evaluate the effectiveness of our approach. Evaluation across five folds using the NDCG metric reveals significant improvements in recommendation accuracy across all datasets compared to single best performing model. Furthermore, comprehensive comparisons against existing models underscore the efficacy of our ensemble approach in enhancing recommendation quality. Our ensemble approach yielded an average improvement of 21.67% across different NDCG@N metrics and the five datasets, compared to single best model. The popularity recommendation model serves as the baseline for comparison. This research contributes to the advancement of ensemble-based recommender systems, offering insights into the potential of combining diverse recommendation strategies to enhance user experience and satisfaction. By presenting a novel approach and demonstrating its superiority over existing methods, we aim to inspire further exploration and innovation in this domain.
A Survey of Datasets for Information Diffusion Tasks
Authors: Fuxia Guo, Xiaowen Wang, Yanwei Xie, Zehao Wang, Jingqiu Li, Lanjun Wang
Date: 2024-07-06 00:00:00
Tasks: Misinformation, Survey

Information diffusion across various new media platforms gradually influences perceptions, decisions, and social behaviors of individual users. In communication studies, the famous Five W's of Communication model (5W Model) has displayed the process of information diffusion clearly. At present, although plenty of studies and corresponding datasets about information diffusion have emerged, a systematic categorization of tasks and an integration of datasets are still lacking. To address this gap, we survey a systematic taxonomy of information diffusion tasks and datasets based on the "5W Model" framework. We first categorize the information diffusion tasks into ten subtasks with definitions and datasets analysis, from three main tasks of information diffusion prediction, social bot detection, and misinformation detection. We also collect the publicly available dataset repository of information diffusion tasks with the available links and compare them based on six attributes affiliated to users and content: user information, social network, bot label, propagation content, propagation network, and veracity label. In addition, we discuss the limitations and future directions of current datasets and research topics to advance the future development of information diffusion. The dataset repository can be accessed at our website https://github.com/fuxiaG/Information-Diffusion-Datasets.
Consistency and Discrepancy-Based Contrastive Tripartite Graph Learning for Recommendations
Authors: Linxin Guo, Yaochen Zhu, Min Gao, Yinghui Tao, Junliang Yu, Chen Chen
Date: 2024-07-06 00:00:00
Tasks: Contrastive Learning, Graph Learning, Recommendation Systems

Tripartite graph-based recommender systems markedly diverge from traditional models by recommending unique combinations such as user groups and item bundles. Despite their effectiveness, these systems exacerbate the longstanding cold-start problem in traditional recommender systems, because any number of user groups or item bundles can be formed among users or items. To address this issue, we introduce a Consistency and Discrepancy-based graph contrastive learning method for tripartite graph-based Recommendation. This approach leverages two novel meta-path-based metrics consistency and discrepancy to capture nuanced, implicit associations between the recommended objects and the recommendees. These metrics, indicative of high-order similarities, can be efficiently calculated with infinite graph convolutional networks layers under a multi-objective optimization framework, using the limit theory of GCN.
Preference Distillation for Personalized Generative Recommendation
Authors: Jerome Ramos, Bin Wu, Aldo Lipani
Date: 2024-07-06 00:00:00
Tasks: Explanation Generation, Recommendation Systems, Sequential Recommendation

Recently, researchers have investigated the capabilities of Large Language Models (LLMs) for generative recommender systems. Existing LLM-based recommender models are trained by adding user and item IDs to a discrete prompt template. However, the disconnect between IDs and natural language makes it difficult for the LLM to learn the relationship between users. To address this issue, we propose a PErsonAlized PrOmpt Distillation (PeaPOD) approach, to distill user preferences as personalized soft prompts. Considering the complexities of user preferences in the real world, we maintain a shared set of learnable prompts that are dynamically weighted based on the user's interests to construct the user-personalized prompt in a compositional manner. Experimental results on three real-world datasets demonstrate the effectiveness of our PeaPOD model on sequential recommendation, top-n recommendation, and explanation generation tasks.
MemoCRS: Memory-enhanced Sequential Conversational Recommender Systems with Large Language Models
Authors: Yunjia Xi, Weiwen Liu, Jianghao Lin, Bo Chen, Ruiming Tang, Weinan Zhang, Yong Yu
Date: 2024-07-06 00:00:00
Tasks: Recommendation Systems

Conversational recommender systems (CRSs) aim to capture user preferences and provide personalized recommendations through multi-round natural language dialogues. However, most existing CRS models mainly focus on dialogue comprehension and preferences mining from the current dialogue session, overlooking user preferences in historical dialogue sessions. The preferences embedded in the user's historical dialogue sessions and the current session exhibit continuity and sequentiality, and we refer to CRSs with this characteristic as sequential CRSs. In this work, we leverage memory-enhanced LLMs to model the preference continuity, primarily focusing on addressing two key issues: (1) redundancy and noise in historical dialogue sessions, and (2) the cold-start users problem. To this end, we propose a Memory-enhanced Conversational Recommender System Framework with Large Language Models (dubbed MemoCRS) consisting of user-specific memory and general memory. User-specific memory is tailored to each user for their personalized interests and implemented by an entity-based memory bank to refine preferences and retrieve relevant memory, thereby reducing the redundancy and noise of historical sessions. The general memory, encapsulating collaborative knowledge and reasoning guidelines, can provide shared knowledge for users, especially cold-start users. With the two kinds of memory, LLMs are empowered to deliver more precise and tailored recommendations for each user. Extensive experiments on both Chinese and English datasets demonstrate the effectiveness of MemoCRS.
A Survey of Controllable Learning: Methods and Applications in Information Retrieval
Authors: Chenglei Shen, Xiao Zhang, Teng Shi, Changshuo Zhang, Guofu Xie, Jun Xu
Date: 2024-07-04 00:00:00
Tasks: Information Retrieval, Retrieval

Controllability has become a crucial aspect of trustworthy machine learning, enabling learners to meet predefined targets and adapt dynamically at test time without requiring retraining as the targets shift. We provide a formal definition of controllable learning (CL), and discuss its applications in information retrieval (IR) where information needs are often complex and dynamic. The survey categorizes CL according to what is controllable (e.g., multiple objectives, user portrait, scenario adaptation), who controls (users or platforms), how control is implemented (e.g., rule-based method, Pareto optimization, hypernetwork and others), and where to implement control (e.g., pre-processing, in-processing, post-processing methods). Then, we identify challenges faced by CL across training, evaluation, task setting, and deployment in online environments. Additionally, we outline promising directions for CL in theoretical analysis, efficient computation, empowering large language models, application scenarios and evaluation frameworks.
On Evaluation Validity in Music Autotagging
Authors: Fabien Gouyon, Bob L. Sturm, Joao Lobato Oliveira, Nuno Hespanhol, Thibault Langlois
Date: 2014-09-30 00:00:00
Tasks: Information Retrieval, Music Information Retrieval

Music autotagging, an established problem in Music Information Retrieval, aims to alleviate the human cost required to manually annotate collections of recorded music with textual labels by automating the process. Many autotagging systems have been proposed and evaluated by procedures and datasets that are now standard (used in MIREX, for instance). Very little work, however, has been dedicated to determine what these evaluations really mean about an autotagging system, or the comparison of two systems, for the problem of annotating music in the real world. In this article, we are concerned with explaining the figure of merit of an autotagging system evaluated with a standard approach. Specifically, does the figure of merit, or a comparison of figures of merit, warrant a conclusion about how well autotagging systems have learned to describe music with a specific vocabulary? The main contributions of this paper are a formalization of the notion of validity in autotagging evaluation, and a method to test it in general. We demonstrate the practical use of our method in experiments with three specific state-of-the-art autotagging systems --all of which are reproducible using the linked code and data. Our experiments show for these specific systems in a simple and objective two-class task that the standard evaluation approach does not provide valid indicators of their performance.
Large-Scale Music Annotation and Retrieval: Learning to Rank in Joint Semantic Spaces
Authors: Jason Weston, Samy Bengio, Philippe Hamel
Date: 2011-05-26 00:00:00
Tasks: Learning-To-Rank, Multi-Task Learning, Retrieval, TAG

Music prediction tasks range from predicting tags given a song or clip of audio, predicting the name of the artist, or predicting related songs given a song, clip, artist name or tag. That is, we are interested in every semantic relationship between the different musical concepts in our database. In realistically sized databases, the number of songs is measured in the hundreds of thousands or more, and the number of artists in the tens of thousands or more, providing a considerable challenge to standard machine learning techniques. In this work, we propose a method that scales to such datasets which attempts to capture the semantic similarities between the database items by modeling audio, artist names, and tags in a single low-dimensional semantic space. This choice of space is learnt by optimizing the set of prediction tasks of interest jointly using multi-task learning. Our method both outperforms baseline methods and, in comparison to them, is faster and consumes less memory. We then demonstrate how our method learns an interpretable model, where the semantic space captures well the similarities of interest.
Implicit Neural Representation for Videos Based on Residual Connection
Authors: Taiga Hayami, Hiroshi Watanabe
Date: 2024-06-15 00:00:00
Tasks: Image Reconstruction, Model Compression, Video Compression

Video compression technology is essential for transmitting and storing videos. Many video compression methods reduce information in videos by removing high-frequency components and utilizing similarities between frames. Alternatively, the implicit neural representations (INRs) for videos, which use networks to represent and compress videos through model compression. A conventional method improves the quality of reconstruction by using frame features. However, the detailed representation of the frames can be improved. To improve the quality of reconstructed frames, we propose a method that uses low-resolution frames as residual connection that is considered effective for image reconstruction. Experimental results show that our method outperforms the existing method, HNeRV, in PSNR for 46 of the 49 videos.
Meme Analysis using LLM-based Contextual Information and U-net Encapsulated Transformer
Authors: Marvin John Ignacio, Thanh Tin Nguyen, Hulin Jin, and Yong-guk Kim
Date: 2024-07-08 00:00:00
Proceeding: IEEE Access 2024 7
Tasks: Sentiment Analysis

A meme is social media content with which the creator tries to convey a certain idea in public via the internet. Each meme consists of typically an image and supporting text. Its message can be humorous and inspiring, but hilarious and offensive often targeting a specific audience. To address the potential harm such memes can cause, Artificial Intelligence researchers have proposed solutions to classify a meme automatically according to the sentiment, emotion, and intensity felt by the users. Recent models for meme analysis often adopt the Transformer architecture, which is known to perform well but computationally expensive. The present study aims to introduce a novel method by providing (1) deep contextual information and (2) reducing resource utilization while keeping its efficiency. For the former, GPT-4 has been utilized to provide meaningful insights regarding the context behind the meme. For the latter, we extract Keyphrases and forward them to a U-net Encapsulated Transformer, called UET, to process the information. Extensive evaluation with ablation study using three standard meme datasets, i.e. Memotions, suggests that it outperforms state-of-the-art models on sentiment analysis, while it shows comparable performance on the emotion and intensity task. As the proposed model is more lightweight than a standard one and yet shows high performance, it provides new insights into meme analysis and could be useful for other Natural Language Processing tasks.
High Fidelity Text-Guided Music Editing via Single-Stage Flow Matching
Authors: Gael Le Lan, Bowen Shi, Zhaoheng Ni, Sidd Srinivasan, Anurag Kumar, Brian Ellis, David Kant, Varun Nagaraja, Ernie Chang, Wei-Ning Hsu, Yangyang Shi, Vikas Chandra
Date: 2024-07-04 00:00:00
Tasks: Denoising, Music Generation

We introduce MelodyFlow, an efficient text-controllable high-fidelity music generation and editing model. It operates on continuous latent representations from a low frame rate 48 kHz stereo variational auto encoder codec. Based on a diffusion transformer architecture trained on a flow-matching objective the model can edit diverse high quality stereo samples of variable duration, with simple text descriptions. We adapt the ReNoise latent inversion method to flow matching and compare it with the original implementation and naive denoising diffusion implicit model (DDIM) inversion on a variety of music editing prompts. Our results indicate that our latent inversion outperforms both ReNoise and DDIM for zero-shot test-time text-guided editing on several objective metrics. Subjective evaluations exhibit a substantial improvement over previous state of the art for music editing. Code and model weights will be publicly made available. Samples are available at https://melodyflow.github.io.
VisioBlend: Sketch and Stroke-Guided Denoising Diffusion Probabilistic Model for Realistic Image Generation
Authors: Harshkumar Devmurari, Gautham Kuckian, Prajjwal Vishwakarma, Krunali Vartak
Date: 2024-05-15 00:00:00
Tasks: Denoising, Efficient Diffusion Personalization, Image Generation

Generating images from hand-drawings is a crucial and fundamental task in content creation. The translation is challenging due to the infinite possibilities and the diverse expectations of users. However, traditional methods are often limited by the availability of training data. Therefore, VisioBlend, a unified framework supporting three-dimensional control over image synthesis from sketches and strokes based on diffusion models, is proposed. It enables users to decide the level of faithfulness to the input strokes and sketches. VisioBlend achieves state-of-the-art performance in terms of realism and flexibility, enabling various applications in image synthesis from sketches and strokes. It solves the problem of data availability by synthesizing new data points from hand-drawn sketches and strokes, enriching the dataset and enabling more robust and diverse image synthesis. This work showcases the power of diffusion models in image creation, offering a user-friendly and versatile approach for turning artistic visions into reality.
Neural Garment Dynamics via Manifold-Aware Transformers
Authors: Peizhuo Li, Tuanfeng Y. Wang, Timur Levent Kesdogan, Duygu Ceylan, Olga Sorkine-Hornung
Date: 2024-05-13 00:00:00

Data driven and learning based solutions for modeling dynamic garments have significantly advanced, especially in the context of digital humans. However, existing approaches often focus on modeling garments with respect to a fixed parametric human body model and are limited to garment geometries that were seen during training. In this work, we take a different approach and model the dynamics of a garment by exploiting its local interactions with the underlying human body. Specifically, as the body moves, we detect local garment-body collisions, which drive the deformation of the garment. At the core of our approach is a mesh-agnostic garment representation and a manifold-aware transformer network design, which together enable our method to generalize to unseen garment and body geometries. We evaluate our approach on a wide variety of garment types and motion sequences and provide competitive qualitative and quantitative results with respect to the state of the art.