GenAI4ELab/papercli-papers huggingface dataset:
PyTorch-Lifestream: Learning Embeddings on Discrete Event Sequences
Authors: Artem Sakhno; Ivan Kireev; Dmitrii Babaev; Maxim Savchenko; Gleb Gusev; Andrey Savchenko
Venue: IJCAI 2025 (main)

Abstract: The domain of event sequences is widely applied in various industrial tasks in banking, healthcare, etc., where temporal tabular data processing is required. This paper introduces PyTorch-Lifestream, the first open-source library specially designed to handle event sequences. It supports scenarios with multimodal data and offers a variety of techniques for learning embeddings of event sequences and end-to-end model training. Furthermore, PyTorch-Lifestream efficiently implements state-of-the-art methods for event sequence analysis and adapts approaches from similar domains, thus enhancing the versatility and performance of sequence-based models for a wide range of applications, including financial risk scoring, campaigning, user ID matching, churn prediction, fraud detection, medical diagnostics, and recommender systems.
Aletheia: Detect, Discuss, and Stay Informed on Fake News
Authors: Dorsaf Sallami; Esma Aïmeur
Venue: IJCAI 2025 (main)

Abstract: In today's digital era, the rapid spread of fake news undermines both social unity and democratic institutions, demanding effective countermeasures. Current browser extensions to counter fake news have significant limitations, such as opaque models, dependency on traditional Machine Learning (ML) techniques, lack of explanatory features, and limited focus on detection without user engagement support. This paper introduces Aletheia, a novel browser extension that addresses these shortcomings by leveraging Retrieval Augmented Generation (RAG) and Large Language Models (LLMs) to enhance fake news detection and provide evidence-based explanations. Additionally, Aletheia incorporates two key components: a Discussion Hub, enabling users to discuss instances of fake news, and a Stay Informed feature, which displays the latest fact-checks. Aletheia surpasses state-of-the-art methods according to experimental results.
SAFE: Structured Argumentation for Fact-checking with Explanations
Authors: Xiaoou Wang; Elena Cabrio; Serena Villata
Venue: IJCAI 2025 (main)

Abstract: Explainable fact-checking plays a vital role in the fight against disinformation in today’s digital landscape. With the increasing volume of unverified content online, providing justifications for fact-checking has become essential to help users make informed decisions. While recent studies provide user-friendly explanations through abstractive or extractive summarization, they often assume the availability of human-written fact-checking articles, which is not always the case. This demo introduces SAFE, an argument-based framework designed to enhance both fact-checking and its justification. Specifically, SAFE offers three key features: i) producing argument-structured summaries of human-written fact-checking articles, ii) in the absence of human-written articles, generating structured summaries based on evidence retrieved from a corpus through a jointly trained summarization and evidence retrieval system, and iii) assessing the truthfulness of a claim by analyzing the structured summary.
OpenIAI-SNIO: A Systematic AR-Based Assembly Guidance System for Small-Scale, High-Density Industrial Components
Authors: Yuntao Wang; Yu Cheng; Junhao Geng
Venue: IJCAI 2025 (main)

Abstract: This paper develops an AR-based assembly guidance system, OpenIAI-SNIO, for small-scale, high-density industrial components (SHIC), which addresses the challenge of existing AR technology's inability to achieve complete, accurate, and stable visual cognition and assembly operation guidance for SHIC. OpenIAI-SNIO combines artificial intelligence methods such as computer vision and deep learning with rule-based reasoning and augmented reality to achieve adaptive, whole process, and precise guidance of SHIC assembly in situations where visual information is insufficient. The application case shows that OpenIAI-SNIO can effectively improve the efficiency and quality of SHIC assembly, and reduce the workload of operators, realizing the systematic and practical application of AR technology in SHIC assembly.
HealthLens: A Natural Language Querying System for Interactive Visualization of Electronic Health Records
Authors: Haodi Zhang; Siqi Ning; Qiyong Zheng; Yuanfeng Song; Liang-Jie Zhang
Venue: IJCAI 2025 (main)

Abstract: As an essential part of modern healthcare systems, extracting valuable insights from electronic medical records (EMRs) remains challenging due to the complexity of structured and unstructured data. Data visualization is essential for transforming complex data into comprehensible visuals that enable professionals to identify patterns and trends. This process involves selecting data attributes, transforming the data, choosing appropriate visual encoding methods, and rendering graphical representations using declarative visualization languages (DVLs). However, achieving proficiency in DVLs requires a deep understanding of domain-specific data and expertise in these languages, which poses a significant barrier for beginners and non-technical users. To address these challenges, we present HealthLens, the first user-friendly visualization tool in the EMR domain that eliminates the need for prior knowledge of DVLs. Built on the MedCodeT5 model developed by us and leveraging a large language model with a bilevel optimization approach, HealthLens enables the generation of EMR visualizations from natural language queries. This demonstrates the feasibility of creating sophisticated visualizations with minimal technical expertise, advancing accessibility in the EMR field.
PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models
Authors: Zheng Zhang; Jinyi Li; Yihuai Lan; Xiang Wang; Hao Wang
Venue: IJCAI 2025 (main)

Abstract: Prompt engineering enables Large Language Models (LLMs) to perform a variety of tasks. However, lengthy prompts significantly increase computational complexity and economic costs. To address this issue, prompt compression reduces prompt length while maintaining LLM response quality. To support rapid implementation and standardization, we present the Prompt Compression Toolkit (PCToolkit), a unified plug-and-play framework for LLM prompt compression. PCToolkit integrates state-of-the-art compression algorithms, benchmark datasets, and evaluation metrics, enabling systematic performance analysis. Its modular architecture simplifies customization, offering portable interfaces for seamless incorporation of new datasets, metrics, and compression methods. Our code is available at https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression. Our demo is at https://huggingface.co/spaces/CjangCjengh/Prompt-Compression-Toolbox.
A Smart Multimodal Healthcare Copilot with Powerful LLM Reasoning
Authors: Xuejiao Zhao; Siyan Liu; Su-Yin Yang; Chunyan Miao
Venue: IJCAI 2025 (main)

Abstract: Misdiagnosis causes significant harm to healthcare systems worldwide, leading to increased costs and patient risks. MedRAG is a smart multimodal healthcare copilot equipped with powerful large language model (LLM) reasoning, designed to enhance medical decision-making. It supports multiple input modalities, including non-intrusive voice monitoring, general medical queries, and electronic health records. MedRAG provides recommendations on diagnosis, treatment, medication, and follow-up questioning. Leveraging retrieval-augmented generation enhanced by knowledge graph-elicited reasoning, MedRAG retrieves and integrates critical diagnostic insights, reducing the risk of misdiagnosis. It has been evaluated on both public and private datasets, outperforming existing models and offering more specific and accurate healthcare assistance. A demonstration video of MedRAG is available at: https://www.youtube.com/watch?v=PNIBDMYRfDM. The source code is available at: https://github.com/SNOWTEAM2023/MedRAG.
Automated Decision-Making on Networks with LLMs through Knowledge-Guided Evolution
Authors: Xiaohan Zheng; Lanning Wei; Yong Li; Quanming Yao
Venue: IJCAI 2025 (main)

Abstract: Effective decision-making on networks often relies on learning from graph-structured data, where Graph Neural Networks (GNNs) play a central role, but they take efforts to configure and tune. In this demo, we propose LLMNet, showing how to design GNN automated through Large Language Models. Our system develops a set of agents that construct graph-related knowlege bases and then leverages Retrieval-Augmented Generation (RAG) to support automated configuration and refinement of GNN models through a knowledge-guided evolution process. These agents, equipped with specialized knowledge bases, extract insights into tasks and graph structures by interacting with the knowledge bases.Empirical results show LLMNet excels in twelve datasets across three graph learning tasks, validating its effectiveness of GNN model designing.
What If LLMs Can Smell: A Prototype
Authors: Xueyi Zhou; Qi Lu; Dong-Kyu Chae
Venue: IJCAI 2025 (main)

Abstract: The olfaction is hardly mentioned in the studies of multi-modal Large Language Models (LLMs). This demo presents a prototypical framework to embody prevalent LLMs with smelling ability using a plug-and-play olfactory signal processing service. To this end, we collect a dataset on Korean beers by self-developed electronic noses (e-noses) and an open-source dataset. An olfaction-related question-answering corpus is also generated to fine-tune LLMs. A gas classification model is applied to identify the smelling liquor upon the e-nose data. We then adopt and fine-tune LLMs on the generated datasets. The results show that LLMs under this framework can interact with the environment by its `nose' and provide olfaction-related answers augmented by our dataset. To the best of our knowledge, this is the first work on embodying LLMs with artificial olfaction. We additionally deployed the gas classification model and the trained LLM in a simple web-based system to show the feasibility of our prototype. Our demo video can be found at: https://bit.ly/4j8x6ZY.
Bridging Speech Science and Technology â Now and Into the Future
Authors: Shrikanth Narayanan
Venue: Interspeech 2023 (main)

Abstract: Speech research is remarkable in so many ways â in its essential human-centeredness, the rich interconnections between the science and technology, and its wide-ranging impact that is both fundamental and applied. Crucial advances in speech science research catalyze and leverage technological advances across the machine intelligence ecosystem, from sensing and imaging to signal processing and machine learning. Likewise, creation of speech-centric societal applications benefits from an understanding of how humans produce, process and use speech in communication. In these complementary endeavors, two intertwined lines of inquiry endure: illuminating the rich information tapestry and inherent variability in speech and creating trustworthy speech technologies.This talk will highlight some advances and possibilities in this multifaceted speech research realm. The first is capturing and modeling the human vocal instrument during speaking and how related technological and clinical applications leverage this technology. The second focuses on speech-based informatics tools to support research and clinical translation related to human health and wellbeing. Finally, the talk will highlight the critical goal of designing trustworthy speech and spoken language machine intelligence tools that are inclusive, equitable, robust, safe, and secure.
Emotional Talking Head Generation based on Memory-Sharing and Attention-Augmented Networks
Authors: Jianrong Wang; Yaxin Zhao; Li Liu; Tianyi Xu; Qi Li; Sen Li
Venue: Interspeech 2023 (main)

Abstract: Given an audio clip and a reference face image, the goal of the talking head generation is to generate a high-fidelity talking head video. Although some audio-driven methods of generating talking head videos have made some achievements in the past, most of them only focused on lip and audio synchronization and lack the ability to reproduce the facial expressions of the target person. To this end, we propose a talking head generation model consisting of a Memory-Sharing Emotion Feature extractor (MSEF) and an Attention-Augmented Translator based on U-net (AATU). Firstly, MSEF can extract implicit emotional auxiliary features from audio to estimate more accurate emotional face landmarks. Secondly, AATU acts as a translator between the estimated landmarks and the photo-realistic video frames. Extensive qualitative and quantitative experiments have shown the superiority of the proposed method to the previous works. Codes will be made publicly available.
Speech Synthesis with Self-Supervisedly Learnt Prosodic Representations
Authors: Zhao-Ci Liu; Zhen-Hua Ling; Ya-Jun Hu; Jia Pan; Jin-Wei Wang; Yun-Di Wu
Venue: Interspeech 2023 (main)

Abstract: This paper presents S4LPR, a Speech Synthesis model conditioned on Self-Supervisedly Learnt Prosodic Representations. Instead of using raw acoustic features, such as F0 and energy, as intermediate prosodic variables, three self-supervised speech models are designed for comparison and are pre-trained on large-scale unlabeled data to extract frame-level prosodic representations. In addition to vanilla wav2vec 2.0, the other two pre-trained models learn representations from LPC residuals or adopt a multi-task learning strategy to focus on the prosodic information in speech. Based on FastSpeech2 and PnGBERT, our acoustic model is built with the learned prosodic representations as intermediate variables. Experimental results demonstrate that the naturalness of speech synthesized using S4LPR is significantly better than the FastSpeech2 baseline.
EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesis
Authors: Haobin Tang; Xulong Zhang; Jianzong Wang; Ning Cheng; Jing Xiao
Venue: Interspeech 2023 (main)

Abstract: There has been significant progress in emotional Text-To-Speech (TTS) synthesis technology in recent years. However, existing methods primarily focus on the synthesis of a limited number of emotion types and have achieved unsatisfactory performance in intensity control. To address these limitations, we propose EmoMix, which can generate emotional speech with specified intensity or a mixture of emotions. Specifically, EmoMix is a controllable emotional TTS model based on a diffusion probabilistic model and a pre-trained speech emotion recognition (SER) model used to extract emotion embedding. Mixed emotion synthesis is achieved by combining the noises predicted by diffusion model conditioned on different emotions during only one sampling process at the run-time. We further apply the Neutral and specific primary emotion mixed in varying degrees to control intensity. Experimental results validate the effectiveness of EmoMix for synthesizing mixed emotion and intensity control.
Laughter Synthesis using Pseudo Phonetic Tokens with a Large-scale In-the-wild Laughter Corpus
Authors: Detai Xin; Shinnosuke Takamichi; Ai Morimatsu; Hiroshi Saruwatari
Venue: Interspeech 2023 (main)

Abstract: We present a large-scale in-the-wild Japanese laughter corpus and a laughter synthesis method. Previous work on laughter synthesis lacks not only data but also proper ways to represent laughter. To solve these problems, we first propose an in-the-wild corpus comprising 3.5 hours of laughter, which is to our best knowledge the largest laughter corpus designed for laughter synthesis. We then propose pseudo phonetic tokens (PPTs) to represent laughter by a sequence of discrete tokens, which are obtained by training a clustering model on features extracted from laughter by a pretrained self-supervised model. Laughter can then be synthesized by feeding PPTs into a text-to-speech system. We further show PPTs can be used to train a language model for unconditional laughter generation. Results of comprehensive subjective and objective evaluations demonstrate that the proposed method significantly outperforms a baseline method, and can generate natural laughter unconditionally.
Explicit Intensity Control for Accented Text-to-speech
Authors: Rui Liu; Haolin Zuo; De Hu; Guanglai Gao; Haizhou Li
Venue: Interspeech 2023 (main)

Abstract: Accented text-to-speech (TTS) synthesis seeks to generate speech with an accent (L2) as a variant of the standard version (L1). How to control the intensity of accent is a very interesting research direction. Recent works design a speaker-adversarial loss to disentangle the speaker and accent information, and then adjust the loss weight to control the accent intensity. However, there is no direct correlation between the disentanglement factor and natural accent intensity. To this end, this paper proposes a new intuitive and explicit accent intensity control scheme for accented TTS. Specifically, we first extract the posterior probability from the L1 speech recognition model to quantify the phoneme accent intensity for accented speech, then design a FastSpeech2 based TTS model, named Ai-TTS, to take the accent intensity expression into account during speech generation. Experiments show that our method outperforms the baseline model in terms of accent rendering and intensity control.
Comparing normalizing flows and diffusion models for prosody and acoustic modelling in text-to-speech
Authors: Guangyan Zhang; Thomas Merritt; Sam Ribeiro; Biel Tura-Vecino; Kayoko Yanagisawa; Kamil Pokora; Abdelhamid Ezzerg; Sebastian Cygert; Ammar Abbas; Piotr Bilinski; Roberto Barra-Chicote; Daniel Korzekwa; Jaime Lorenzo-Trueba
Venue: Interspeech 2023 (main)

Abstract: Neural text-to-speech systems are often optimized on L1/L2 losses, which make strong assumptions about the distributions of the target data space. Aiming to improve those assumptions, Normalizing Flows and Diffusion Probabilistic Models were recently proposed as alternatives. In this paper, we compare traditional L1/L2-based approaches to diffusion and flow-based approaches for the tasks of prosody and mel-spectrogram prediction for text-to-speech synthesis. We use a prosody model to generate log-f0 and duration features, which are used to condition an acoustic model that generates mel-spectrograms. Experimental results demonstrate that the flow-based model achieves the best performance for spectrogram prediction, improving over equivalent diffusion and L1 models. Meanwhile, both diffusion and flow-based prosody predictors result in significant improvements over a typical L2-trained prosody models.
Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer
Authors: Paul-Ambroise Duquenne; Holger Schwenk; BenoÃ®t Sagot
Venue: Interspeech 2023 (main)

Abstract: Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation. In this work, we show that this type of approach can be further improved with multilingual training. We observe significant improvements in zero-shot cross-modal speech translation, even outperforming a supervised approach based on XLSR for several languages.
Improving Isochronous Machine Translation with Target Factors and Auxiliary Counters
Authors: Proyag Pal; Brian Thompson; Yogesh Virkar; Prashant Mathur; Alexandra Chronopoulou; Marcello Federico
Venue: Interspeech 2023 (main)

Abstract: To translate speech for automatic dubbing, machine translation needs to be isochronous, i.e. translated speech needs to be aligned with the source in terms of speech durations. We introduce target factors in a transformer model to predict durations jointly with target language phoneme sequences. We also introduce auxiliary counters to help the decoder to keep track of the timing information while generating target phonemes. We show that our model improves translation quality and isochrony compared to previous work where the translation model is instead trained to predict interleaved sequences of phonemes and durations.
StyleS2ST: Zero-shot Style Transfer for Direct Speech-to-speech Translation
Authors: Kun Song; Yi Ren; Yi Lei; Chunfeng Wang; Kun Wei; Lei Xie; Xiang Yin; Zejun Ma
Venue: Interspeech 2023 (main)

Abstract: Direct speech-to-speech translation (S2ST) has gradually become popular as it has many advantages compared with cascade S2ST. However, current research mainly focuses on the accuracy of semantic translation and ignores the speech style transfer from a source language to a target language. The lack of high-fidelity expressive parallel data makes such style transfer challenging, especially in more practical zero-shot scenarios. To solve this problem, we first build a parallel corpus using a multi-lingual multi-speaker text-to-speech synthesis (TTS) system and then propose the StyleS2ST model with cross-lingual speech style transfer ability based on a style adaptor on a direct S2ST system framework. Enabling continuous style space modeling of an acoustic model through parallel corpus training and non-parallel TTS data augmentation, StyleS2ST captures cross-lingual acoustic feature mapping from the source to the target language. Experiments show that StyleS2ST achieves good style similarity and naturalness in both in-set and out-of-set zero-shot scenarios.
Joint Speech Translation and Named Entity Recognition
Authors: Marco Gaido; Sara Papi; Matteo Negri; Marco Turchi
Venue: Interspeech 2023 (main)

Abstract: Modern automatic translation systems aim at supporting the users by providing contextual knowledge. In this framework, a critical task is the output enrichment with information regarding the mentioned entities. This is currently achieved by processing the generated translations with named entity recognition (NER) tools and retrieving their description from knowledge bases. In light of the recent promising results shown by direct speech translation (ST) models and the known weaknesses of cascades (error propagation and additional latency), in this paper we propose multitask models that jointly perform ST and NER, and compare them with a cascade baseline. Experimental results on three language pairs (en-es/fr/it) show that our models significantly outperform the cascade on the NER task (by 0.4-1.0 F1), without degradation in terms of translation quality, and with the same computational efficiency of a plain direct ST model.
Analysis of Acoustic information in End-to-End Spoken Language Translation
Authors: Gerard Sant; Carlos Escolano
Venue: Interspeech 2023 (main)

Abstract: End-to-End Transformer-based models are the most popular approach for Spoken Language Translation (SLT). While obtaining state-of-the-art results, we are still far from understanding how these models extract acoustic information from the data and how they are transformed into semantic representations. In this paper, we seek to provide a better understanding of the flow of acoustic information along speech-to-text translation models. By means of the Speaker Classification and Spectrogram Reconstruction tasks, this study (i) interprets the main role of the encoder with respect to the acoustic features, (ii) highlights the importance of the acoustic information throughout the model and its transfer between encoder and decoder, and (iii) reveals the significant effect of downsampling convolutional layers for learning acoustic features. (iv) Finally, we also observe the existence of a strong correlation between the semantic domain and the speakers' labels in MuST-C.