Reviewerly/exHarmony huggingface dataset:
Continual Domain Adversarial Adaptation via Double-Head Discriminators

Authors:
raw_author_name: Shen, Yan
raw_author_name: Ji, Zhanghexuan
raw_author_name: Ma, Chunwei
raw_author_name: Gao, Mingchen

Abstract: Domain adversarial adaptation in a continual setting poses a significant challenge due to the limitations on accessing previous source domain data. Despite extensive research in continual learning, the task of adversarial adaptation cannot be effectively accomplished using only a small number of stored source domain data, which is a standard setting in memory replay approaches. This limitation arises from the erroneous empirical estimation of $\gH$-divergence with few source domain samples. To tackle this problem, we propose a double-head discriminator algorithm, by introducing an addition source-only domain discriminator that are trained solely on source learning phase. We prove that with the introduction of a pre-trained source-only domain discriminator, the empirical estimation error of $\gH$-divergence related adversarial loss is reduced from the source domain side. Further experiments on existing domain adaptation benchmark show that our proposed algorithm achieves more than 2$\%$ improvement on all categories of target domain adaptation task while significantly mitigating the forgetting on source domain.
A Survey of Privacy Threats and Defense in Vertical Federated Learning:  From Model Life Cycle Perspective

Authors:
raw_author_name: Yu, Lei
raw_author_name: Han, Meng
raw_author_name: Li, Yiming
raw_author_name: Lin, Changting
raw_author_name: Zhang, Mingyang
raw_author_name: Liu, Yan
raw_author_name: Weng, Haiqin
raw_author_name: Jeon, Yuseok
raw_author_name: Chow, Ka-Ho
raw_author_name: Patterson, Stacy

Abstract: Vertical Federated Learning (VFL) is a federated learning paradigm where multiple participants, who share the same set of samples but hold different features, jointly train machine learning models. Although VFL enables collaborative machine learning without sharing raw data, it is still susceptible to various privacy threats. In this paper, we conduct the first comprehensive survey of the state-of-the-art in privacy attacks and defenses in VFL. We provide taxonomies for both attacks and defenses, based on their characterizations, and discuss open challenges and future research directions. Specifically, our discussion is structured around the model's life cycle, by delving into the privacy threats encountered during different stages of machine learning and their corresponding countermeasures. This survey not only serves as a resource for the research community but also offers clear guidance and actionable insights for practitioners to safeguard data privacy throughout the model's life cycle.
Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue

Authors:
raw_author_name: Ouyang, Kun
raw_author_name: Jing, Liqiang
raw_author_name: Song, Xuemeng
raw_author_name: Liu, Meng
raw_author_name: Hu, Yupeng
raw_author_name: Nie, Liqiang

Abstract: Sarcasm Explanation in Dialogue (SED) is a new yet challenging task, which aims to generate a natural language explanation for the given sarcastic dialogue that involves multiple modalities (i.e., utterance, video, and audio). Although existing studies have achieved great success based on the generative pretrained language model BART, they overlook exploiting the sentiments residing in the utterance, video and audio, which are vital clues for sarcasm explanation. In fact, it is non-trivial to incorporate sentiments for boosting SED performance, due to three main challenges: 1) diverse effects of utterance tokens on sentiments; 2) gap between video-audio sentiment signals and the embedding space of BART; and 3) various relations among utterances, utterance sentiments, and video-audio sentiments. To tackle these challenges, we propose a novel sEntiment-enhanceD Graph-based multimodal sarcasm Explanation framework, named EDGE. In particular, we first propose a lexicon-guided utterance sentiment inference module, where a heuristic utterance sentiment refinement strategy is devised. We then develop a module named Joint Cross Attention-based Sentiment Inference (JCA-SI) by extending the multimodal sentiment analysis model JCA to derive the joint sentiment label for each video-audio clip. Thereafter, we devise a context-sentiment graph to comprehensively model the semantic relations among the utterances, utterance sentiments, and video-audio sentiments, to facilitate sarcasm explanation generation. Extensive experiments on the publicly released dataset WITS verify the superiority of our model over cutting-edge methods.
Deep MSFOP: Multiple Spectral filter Operators Preservation in Deep  Functional Maps for Unsupervised Shape Matching

Authors:
raw_author_name: Luo, Feifan
raw_author_name: Li, Qingsong
raw_author_name: Hu, Ling
raw_author_name: Liu, Xinru
raw_author_name: Xu, Haojun
raw_author_name: Wang, Haibo
raw_author_name: Li, Ting
raw_author_name: Liu, Shengjun

Abstract: We propose a novel constraint called Multiple Spectral filter Operators Preservation (MSFOR) to compute functional maps and based on it, develop an efficient deep functional map architecture called Deep MSFOP for shape matching. The core idea is that, instead of using the general descriptor preservation constraint, we require our maps to preserve multiple spectral filter operators. This allows us to incorporate more informative geometrical information, contained in different frequency bands of functions, into the functional map computing. This can be confirmed by that some previous techniques like wavelet preservation and LBO commutativity are actually our special cases. Moreover, we also develop a very efficient way to compute the maps with MSFOP constraint, which can be conveniently embedded into the deep learning, especially having learnable filter operators. Utilizing the above results, we finally design our Deep MSFOP pipeline, equipped with a suitable unsupervised loss jointly penalizing the functional map and the underlying pointwise map. Our deep functional map has notable advantages, including that the functional map is more geometrically informative and guaranteed to be proper, and the computing is numerically stable. Extensive experimental results on different datasets demonstrate that our approach outperforms the existing state-of-the-art methods, especially in challenging settings like non-isometric and inconsistent topology datasets.
Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation

Authors:
raw_author_name: Wang, Ziyang
raw_author_name: Zheng, Jian-Qing
raw_author_name: Zhang, Yichi
raw_author_name: Li, Lei

Abstract: In recent advancements in medical image analysis, Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have set significant benchmarks. While the former excels in capturing local features through its convolution operations, the latter achieves remarkable global context understanding by leveraging self-attention mechanisms. However, both architectures exhibit limitations in efficiently modeling long-range dependencies within medical images, which is a critical aspect for precise segmentation. Inspired by the Mamba architecture, known for its proficiency in handling long sequences and global contextual information with enhanced computational efficiency as a State Space Model (SSM), we propose Mamba-UNet, a novel architecture that synergizes the U-Net in medical image segmentation with Mamba's capability. Mamba-UNet adopts a pure Visual Mamba (VMamba)-based encoder-decoder structure, infused with skip connections to preserve spatial information across different scales of the network. This design facilitates a comprehensive feature learning process, capturing intricate details and broader semantic contexts within medical images. We introduce a novel integration mechanism within the VMamba blocks to ensure seamless connectivity and information flow between the encoder and decoder paths, enhancing the segmentation performance. We conducted experiments on publicly available MRI cardiac multi-structures segmentation dataset. The results show that Mamba-UNet outperforms UNet, Swin-UNet in medical image segmentation under the same hyper-parameter setting. The source code and baseline implementations are available.
Democratizing Large Language Models via Personalized Parameter-Efficient  Fine-tuning

Authors:
raw_author_name: Tan, Zhaoxuan
raw_author_name: Zeng, Qingkai
raw_author_name: Tian, Yijun
raw_author_name: Liu, Zheyuan
raw_author_name: Yin, Bing
raw_author_name: Jiang, Meng

Abstract: Personalization in large language models (LLMs) is increasingly important, aiming to align LLM's interactions, content, and recommendations with individual user preferences. Recent advances in LLM personalization have spotlighted effective prompt design, by enriching user queries with non-parametric knowledge through behavior history retrieval and textual profiles. However, these approaches were limited due to a lack of model ownership, resulting in constrained customization and privacy issues. Moreover, they often failed to accurately capture user behavior patterns, especially in cases where user data were complex and dynamic. To address these shortcomings, we introduce One PEFT Per User (OPPU), which employs personalized parameter-efficient fine-tuning (PEFT) modules, to store user-specific behavior patterns and preferences. By plugging in users' personal PEFT parameters, they can own and use their LLMs personally. OPPU integrates parametric user knowledge in the personal PEFT parameters with the non-parametric knowledge acquired through retrieval and profile. This integration adapts individual LLMs to user behavior shifts. Experimental results demonstrate that OPPU significantly outperforms existing prompt-based methods across seven diverse tasks in the LaMP benchmark. Further in-depth studies reveal OPPU's enhanced capabilities in handling user behavior shifts, modeling users at different active levels, maintaining robustness across various user history formats, and displaying versatility with different PEFT methods.
De-amplifying Bias from Differential Privacy in Language Model  Fine-tuning

Authors:
raw_author_name: Srivastava, Sanjari
raw_author_name: Mardziel, Piotr
raw_author_name: Datta, Anupam
raw_author_name: Mitchell, John C

Abstract: Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy.
V2VSSC: A 3D Semantic Scene Completion Benchmark for Perception with  Vehicle to Vehicle Communication

Authors:
raw_author_name: Zhang, Yuanfang
raw_author_name: Li, Junxuan
raw_author_name: Luo, Kaiqing
raw_author_name: Han, Jiayi
raw_author_name: Liu, Nian
raw_author_name: Han, Peng
raw_author_name: Xu, Chengpei

Abstract: Semantic scene completion (SSC) has recently gained popularity because it can provide both semantic and geometric information that can be used directly for autonomous vehicle navigation. However, there are still challenges to overcome. SSC is often hampered by occlusion and short-range perception due to sensor limitations, which can pose safety risks. This paper proposes a fundamental solution to this problem by leveraging vehicle-to-vehicle (V2V) communication. We propose the first generalized collaborative SSC framework that allows autonomous vehicles to share sensing information from different sensor views to jointly perform SSC tasks. To validate the proposed framework, we further build V2VSSC, the first V2V SSC benchmark, on top of the large-scale V2V perception dataset OPV2V. Extensive experiments demonstrate that by leveraging V2V communication, the SSC performance can be increased by 8.3% on geometric metric IoU and 6.0% mIOU.
Open-Vocabulary Calibration for Vision-Language Models

Authors:
raw_author_name: Wang, Shuoyuan
raw_author_name: Wang, Jindong
raw_author_name: Wang, Guoqing
raw_author_name: Zhang, Bob
raw_author_name: Zhou, Kaiyang
raw_author_name: Wei, Hongxin

Abstract: Vision-language models (VLMs) have emerged as formidable tools, showing their strong capability in handling various open-vocabulary tasks in image recognition, text-driven visual content generation, and visual chatbots, to name a few. In recent years, considerable efforts and resources have been devoted to adaptation methods for improving downstream performance of VLMs, particularly on parameter-efficient fine-tuning methods like prompt learning. However, a crucial aspect that has been largely overlooked is the confidence calibration problem in fine-tuned VLMs, which could greatly reduce reliability when deploying such models in the real world. This paper bridges the gap by systematically investigating the confidence calibration problem in the context of prompt learning and reveals that existing calibration methods are insufficient to address the problem, especially in the open-vocabulary setting. To solve the problem, we present a simple and effective approach called Distance-Aware Calibration (DAC), which is based on scaling the temperature using as guidance the distance between predicted text labels and base classes. The experiments with 7 distinct prompt learning methods applied across 11 diverse downstream datasets demonstrate the effectiveness of DAC, which achieves high efficacy without sacrificing the inference speed.
Mesh-based Gaussian Splatting for Real-time Large-scale Deformation

Authors:
raw_author_name: Yang, Jie
raw_author_name: Zhang, Bo-Tao
raw_author_name: Sun, Jia-Mu
raw_author_name: Yuan, Yu-Jie
raw_author_name: Lai, Yu-Kun

Abstract: Neural implicit representations, including Neural Distance Fields and Neural Radiance Fields, have demonstrated significant capabilities for reconstructing surfaces with complicated geometry and topology, and generating novel views of a scene. Nevertheless, it is challenging for users to directly deform or manipulate these implicit representations with large deformations in the real-time fashion. Gaussian Splatting(GS) has recently become a promising method with explicit geometry for representing static scenes and facilitating high-quality and real-time synthesis of novel views. However,it cannot be easily deformed due to the use of discrete Gaussians and lack of explicit topology. To address this, we develop a novel GS-based method that enables interactive deformation. Our key idea is to design an innovative mesh-based GS representation, which is integrated into Gaussian learning and manipulation. 3D Gaussians are defined over an explicit mesh, and they are bound with each other: the rendering of 3D Gaussians guides the mesh face split for adaptive refinement, and the mesh face split directs the splitting of 3D Gaussians. Moreover, the explicit mesh constraints help regularize the Gaussian distribution, suppressing poor-quality Gaussians(e.g. misaligned Gaussians,long-narrow shaped Gaussians), thus enhancing visual quality and avoiding artifacts during deformation. Based on this representation, we further introduce a large-scale Gaussian deformation technique to enable deformable GS, which alters the parameters of 3D Gaussians according to the manipulation of the associated mesh. Our method benefits from existing mesh deformation datasets for more realistic data-driven Gaussian deformation. Extensive experiments show that our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate(65 FPS on average).
Experimental implementation of quantum-walk-based portfolio optimisation

Authors:
raw_author_name: Dengke Qu
raw_affiliation_string: Beijing Computational Science Research Center, Beijing, Beijing, 100193, CHINA
raw_author_name: Edric Matwiejew
raw_affiliation_string: The University of Western Australia, The University of Western Australia, Perth, Western Australia, 6009, AUSTRALIA
raw_author_name: Kunkun Wang
raw_affiliation_string: Beijing Computational Science Research Center, Beijing, Beijing, 100193, CHINA
raw_author_name: Jingbo B Wang
raw_affiliation_string: School of Physics, The University of Western Australia, Crawley, Western Australia 6907, Perth, Western Australia, 6009, AUSTRALIA
raw_author_name: Peng Xue
raw_affiliation_string: Beijing Computational Science Research Center, Beijing 100084, China, Beijing, 100193, CHINA

Abstract: Abstract The application of quantum algorithms has attracted much attention as it holds the promise of solving practical problems that are intractable to classical algorithms. One such application is the recent development of a quantum-walk-based optimisation algorithm approach to portfolio optimisation under the modern portfolio theory framework. In this paper, we demonstrate an experimental realisation of the alternating phase-shift and continuous-time quantum walk unitaries that underpin this quantum algorithm using optical networks and single photons. The experimental analysis conﬁrms that the probability of states corresponding to high-quality solutions is eﬃciently ampliﬁed by increasing the number of phase-shift and quantum walk iterations. This work provides strong evidence for practical applications of quantum-walk-based algorithms such as ﬁnancial portfolio optimisation.&amp;#xD;
Cluster-weighted modeling with measurement error in covariates

Authors:
raw_author_name: Shaho Zarei
raw_affiliation_string: Department of Statistics, Faculty of Science, University of Kurdistan, Sanandaj, Iran

Abstract: The cluster-weighted model (CWM) is a model-based clustering approach that utilizes a mixture of regression models to cluster data points based on both a response variable Y and covariates 𝑿, where the covariates are assumed to be random. The Gaussian CWM (GCWM) is the most commonly used member of the CWM family, where the Gaussian distribution is adopted for both the covariates and the response given the covariates. In mixture of regression, assignment of data points to the clusters is based on the conditional distribution of the response variable given covariates and is independent of the covariates’ distribution. In CWM, to increase clustering performance, the covariates’ distribution is also used to assign data points to the clusters. Existing researches on CWMs are limited to the directly observed covariates, which may not reflect real-world scenarios where measurement errors (MEs) occur. The measurement error can lead to inconsistent estimates, consequently, produce spurious or obscure clusters. In this article, we assume that random covariates 𝑿 are latent, observed with an independent ME that has the Gaussian distribution. A new generalized expectation maximization algorithm is defined for estimating model parameters. The performance of the proposal is illustrated and compared with the GCWM using both simulated and real data.
Use of Chat GPT in English for Engineering Classes: Are Students’ and Teachers’ Views on Its Opportunities and Challenges Similar?

Authors:
raw_author_name: Oksana Synekop
raw_affiliation_string: National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
raw_author_name: Iryna Lytovchenko
raw_affiliation_string: National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
raw_author_name: Yuliana Lavrysh
raw_affiliation_string: National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"
raw_author_name: Valentyna Lukianenko
raw_affiliation_string: National Technical University of Ukraine Igor Sikorsky Kyiv Polytechnic Institute

Abstract: Mobile technologies have become increasingly important in the field of education, providing innovative ways to engage students and enrich their learning experiences. ChatGPT is an innovative tool that can be used in English for Specific Purposes (ESP) classes to engage students, enhance their knowledge and skills, and add variety and interest to the traditional process of learning engineering English, whether accessed through computers or mobile devices. In light of this, the study aims to investigate and compare the attitudes of technical university students and teachers towards using ChatGPT in English classes, as well as their perspectives on academic integrity issues and the strategies they employ when using ChatGPT. The study involved 22 teachers and 60 students from a Ukrainian university. Quantitative research, which involved collecting data through surveys, showed that students exhibited positive attitudes, while teachers took a more neutral approach. Both groups agree that ChatGPT plays a supplementary role in enhancing both hard and soft skills, as well as English language communication skills. However, they emphasized its impact on the development of research skills. The study highlights critical issues related to academic integrity, specifically the differing perceptions of possible violations between students and teachers. The authors present ideas on integrating ChatGPT into ESP classes, enabling teachers to leverage artificial intelligence (AI) tools for language learning and adapt to changing educational landscapes. The study emphasizes the importance of integrating ChatGPT effectively into engineering English classes to support students’ learning. While emphasizing the importance of guidelines for academic integrity, it also advocates for the creative use of AI tools by teachers and students. Addressing the disparity in attitudes suggests providing teacher professional development to ensure proficient use of AI tools.
History of generative Artificial Intelligence (AI) chatbots: past,  present, and future development

Authors:
raw_author_name: Al-Amin, Md.
raw_author_name: Ali, Mohammad Shazed
raw_author_name: Salam, Abdus
raw_author_name: Khan, Arif
raw_author_name: Ullah, Ahsan
raw_author_name: Alam, Md Nur
raw_author_name: Chowdhury, Shamsul Kabir

Abstract: This research provides an in-depth comprehensive review of the progress of chatbot technology over time, from the initial basic systems relying on rules to today's advanced conversational bots powered by artificial intelligence. Spanning many decades, the paper explores the major milestones, innovations, and paradigm shifts that have driven the evolution of chatbots. Looking back at the very basic statistical model in 1906 via the early chatbots, such as ELIZA and ALICE in the 1960s and 1970s, the study traces key innovations leading to today's advanced conversational agents, such as ChatGPT and Google Bard. The study synthesizes insights from academic literature and industry sources to highlight crucial milestones, including the introduction of Turing tests, influential projects such as CALO, and recent transformer-based models. Tracing the path forward, the paper highlights how natural language processing and machine learning have been integrated into modern chatbots for more sophisticated capabilities. This chronological survey of the chatbot landscape provides a holistic reference to understand the technological and historical factors propelling conversational AI. By synthesizing learnings from this historical analysis, the research offers important context about the developmental trajectory of chatbots and their immense future potential across various field of application which could be the potential take ways for the respective research community and stakeholders.