pwc-archive/papers-with-abstracts huggingface dataset:
Personalized Large Language Model Assistant with Evolving Conditional Memory
Authors: Ruifeng Yuan, Shichao Sun, Yongqi Li, Zili Wang, Ziqiang Cao, Wenjie Li
Date: 2023-12-22 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Retrieval

With the rapid development of large language models, AI assistants like ChatGPT have become increasingly integrated into people's works and lives but are limited in personalized services. In this paper, we present a plug-and-play framework that could facilitate personalized large language model assistants with evolving conditional memory. The personalized assistant focuses on intelligently preserving the knowledge and experience from the history dialogue with the user, which can be applied to future tailored responses that better align with the user's preferences. Generally, the assistant generates a set of records from the dialogue dialogue, stores them in a memory bank, and retrieves related memory to improve the quality of the response. For the crucial memory design, we explore different ways of constructing the memory and propose a new memorizing mechanism named conditional memory. We also investigate the retrieval and usage of memory in the generation process. We build the first benchmark to evaluate personalized assistants' ability from three aspects. The experimental results illustrate the effectiveness of our method.
From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models
Authors: Wolfgang Messner, Tatum Greene, Josephine Matalone
Date: 2023-12-21 00:00:00
Tasks: Information Retrieval, Retrieval

Large language models (LLMs) are able to engage in natural-sounding conversations with humans, showcasing unprecedented capabilities for information retrieval and automated decision support. They have disrupted human-technology interaction and the way businesses operate. However, technologies based on generative artificial intelligence (GenAI) are known to hallucinate, misinform, and display biases introduced by the massive datasets on which they are trained. Existing research indicates that humans may unconsciously internalize these biases, which can persist even after they stop using the programs. This study explores the cultural self-perception of LLMs by prompting ChatGPT (OpenAI) and Bard (Google) with value questions derived from the GLOBE project. The findings reveal that their cultural self-perception is most closely aligned with the values of English-speaking countries and countries characterized by sustained economic competitiveness. Recognizing the cultural biases of LLMs and understanding how they work is crucial for all members of society because one does not want the black box of artificial intelligence to perpetuate bias in humans, who might, in turn, inadvertently create and train even more biased algorithms.
Faithful Model Evaluation for Model-Based Metrics
Authors: Palash Goyal, Qian Hu, Rahul Gupta
Date: 2023-12-19 00:00:00
Tasks: model

Statistical significance testing is used in natural language processing (NLP) to determine whether the results of a study or experiment are likely to be due to chance or if they reflect a genuine relationship. A key step in significance testing is the estimation of confidence interval which is a function of sample variance. Sample variance calculation is straightforward when evaluating against ground truth. However, in many cases, a metric model is often used for evaluation. For example, to compare toxicity of two large language models, a toxicity classifier is used for evaluation. Existing works usually do not consider the variance change due to metric model errors, which can lead to wrong conclusions. In this work, we establish the mathematical foundation of significance testing for model-based metrics. With experiments on public benchmark datasets and a production system, we show that considering metric model errors to calculate sample variances for model-based metrics changes the conclusions in certain experiments.
Darwin3: A large-scale neuromorphic chip with a Novel ISA and On-Chip Learning
Authors: De Ma, Xiaofei Jin, Shichun Sun, Yitao Li, Xundong Wu, Youneng Hu, Fangchao Yang, Huajin Tang, Xiaolei Zhu, Peng Lin, Gang Pan
Date: 2023-12-29 00:00:00
Tasks: Computational Efficiency

Spiking Neural Networks (SNNs) are gaining increasing attention for their biological plausibility and potential for improved computational efficiency. To match the high spatial-temporal dynamics in SNNs, neuromorphic chips are highly desired to execute SNNs in hardware-based neuron and synapse circuits directly. This paper presents a large-scale neuromorphic chip named Darwin3 with a novel instruction set architecture(ISA), which comprises 10 primary instructions and a few extended instructions. It supports flexible neuron model programming and local learning rule designs. The Darwin3 chip architecture is designed in a mesh of computing nodes with an innovative routing algorithm. We used a compression mechanism to represent synaptic connections, significantly reducing memory usage. The Darwin3 chip supports up to 2.35 million neurons, making it the largest of its kind in neuron scale. The experimental results showed that code density was improved up to 28.3x in Darwin3, and neuron core fan-in and fan-out were improved up to 4096x and 3072x by connection compression compared to the physical memory depth. Our Darwin3 chip also provided memory saving between 6.8X and 200.8X when mapping convolutional spiking neural networks (CSNN) onto the chip, demonstrating state-of-the-art performance in accuracy and latency compared to other neuromorphic chips.
K-PERM: Personalized Response Generation Using Dynamic Knowledge Retrieval and Persona-Adaptive Queries
Authors: Kanak Raj, Kaushik Roy, Vamshi Bonagiri, Priyanshul Govil, Krishnaprasad Thirunarayanan, Manas Gaur
Date: 2023-12-29 00:00:00
Tasks: Nutrition, Response Generation, Retrieval

Personalizing conversational agents can enhance the quality of conversations and increase user engagement. However, they often lack external knowledge to appropriately tend to a user's persona. This is particularly crucial for practical applications like mental health support, nutrition planning, culturally sensitive conversations, or reducing toxic behavior in conversational agents. To enhance the relevance and comprehensiveness of personalized responses, we propose using a two-step approach that involves (1) selectively integrating user personas and (2) contextualizing the response with supplementing information from a background knowledge source. We develop K-PERM (Knowledge-guided PErsonalization with Reward Modulation), a dynamic conversational agent that combines these elements. K-PERM achieves state-of-the-art performance on the popular FoCus dataset, containing real-world personalized conversations concerning global landmarks. We show that using responses from K-PERM can improve performance in state-of-the-art LLMs (GPT 3.5) by 10.5%, highlighting the impact of K-PERM for personalizing chatbots.
Investigating the Effects of Sparse Attention on Cross-Encoders
Authors: Ferdinand Schlatt, Maik Fröbe, Matthias Hagen
Date: 2023-12-29 00:00:00
Tasks: Re-Ranking, Retrieval

Cross-encoders are effective passage and document re-rankers but less efficient than other neural or classic retrieval models. A few previous studies have applied windowed self-attention to make cross-encoders more efficient. However, these studies did not investigate the potential and limits of different attention patterns or window sizes. We close this gap and systematically analyze how token interactions can be reduced without harming the re-ranking effectiveness. Experimenting with asymmetric attention and different window sizes, we find that the query tokens do not need to attend to the passage or document tokens for effective re-ranking and that very small window sizes suffice. In our experiments, even windows of 4 tokens still yield effectiveness on par with previous cross-encoders while reducing the memory requirements by at least 22% / 59% and being 1% / 43% faster at inference time for passages / documents.
Uncertain research country rankings. Should we continue producing uncertain rankings?
Authors: Alonso Rodriguez-Navarro
Date: 2023-12-29 00:00:00

Purpose: Citation-based assessments of countries' research capabilities often misrepresent their ability to achieve breakthrough advancements. These assessments commonly classify Japan as a developing country, which contradicts its prominent scientific standing. The purpose of this study is to investigate the underlying causes of such inaccurate assessments and to propose methods for conducting more reliable evaluations. Design/methodology/approach: The study evaluates the effectiveness of top-percentile citation metrics as indicators of breakthrough research. Using case studies of selected countries and research topics, the study examines how deviations from lognormal citation distributions impact the accuracy of these percentile indicators. A similar analysis is conducted using university data from the Leiden Ranking to investigate citation distribution deviations at the institutional level. Findings: The study finds that inflated lower tails in citation distributions lead to undervaluation of research capabilities in advanced technological countries, as captured by some percentile indicators. Conversely, research-intensive universities exhibit the opposite trend: a reduced lower tail relative to the upper tail, which causes percentile indicators to overestimate their actual research capacity. Research limitations: The descriptions are mathematical facts that are self-evident. Practical implications: Due to variations in citation patterns across countries and institutions, the Ptop 10%/P and Ptop 1%/P ratios are not universal predictors of breakthrough research. Evaluations should move away from these metrics. Relying on inappropriate citation-based measures could lead to poor decision-making in research policy, undermining the effectiveness of research strategies and their outcomes.
Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering
Authors: Huiyuan Chen, Vivian Lai, Hongye Jin, Zhimeng Jiang, Mahashweta Das, Xia Hu
Date: 2023-12-29 00:00:00
Tasks: Collaborative Filtering, Contrastive Learning, Data Augmentation

Contrastive Learning (CL) has shown promising performance in collaborative filtering. The key idea is to generate augmentation-invariant embeddings by maximizing the Mutual Information between different augmented views of the same instance. However, we empirically observe that existing CL models suffer from the \textsl{dimensional collapse} issue, where user/item embeddings only span a low-dimension subspace of the entire feature space. This suppresses other dimensional information and weakens the distinguishability of embeddings. Here we propose a non-contrastive learning objective, named nCL, which explicitly mitigates dimensional collapse of representations in collaborative filtering. Our nCL aims to achieve geometric properties of \textsl{Alignment} and \textsl{Compactness} on the embedding space. In particular, the alignment tries to push together representations of positive-related user-item pairs, while compactness tends to find the optimal coding length of user/item embeddings, subject to a given distortion. More importantly, our nCL does not require data augmentation nor negative sampling during training, making it scalable to large datasets. Experimental results demonstrate the superiority of our nCL.
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
Authors: Sihan Chen, Handong Li, Qunbo Wang, Zijia Zhao, Mingzhen Sun, Xinxin Zhu, Jing Liu
Date: 2023-05-29 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Audio captioning, Audio-Visual Captioning, Audio-visual Question Answering, Cross-Modal Retrieval, Image Captioning, Language Modelling, Large Language Model, Text to Audio Retrieval, text-to-audiovisual retrieval, TGIF-Frame, Video Captioning, Video Question Answering, Video Retrieval, Visual Question Answering (VQA), Zero-Shot Cross-Modal Retrieval, Zero-Shot Video Retrieval

Vision and text have been fully explored in contemporary video-text foundational models, while other modalities such as audio and subtitles in videos have not received sufficient attention. In this paper, we resort to establish connections between multi-modality video tracks, including Vision, Audio, and Subtitle, and Text by exploring an automatically generated large-scale omni-modality video caption dataset called VAST-27M. Specifically, we first collect 27 million open-domain video clips and separately train a vision and an audio captioner to generate vision and audio captions. Then, we employ an off-the-shelf Large Language Model (LLM) to integrate the generated captions, together with subtitles and instructional prompts into omni-modality captions. Based on the proposed VAST-27M dataset, we train an omni-modality video-text foundational model named VAST, which can perceive and process vision, audio, and subtitle modalities from video, and better support various tasks including vision-text, audio-text, and multi-modal video-text tasks (retrieval, captioning and QA). Extensive experiments have been conducted to demonstrate the effectiveness of our proposed VAST-27M corpus and VAST foundation model. VAST achieves 22 new state-of-the-art results on various cross-modality benchmarks. Code, model and dataset will be released at https://github.com/TXH-mercury/VAST.
Deep Learning-based Sentiment Classification: A Comparative Survey
Authors: Mohamed Kayed, Rebeca P. Díaz-Redondo, Alhassan Mabrouk
Date: 2023-12-12 00:00:00
Tasks: Classification, Deep Learning, Sentiment Analysis, Sentiment Classification, Survey

Recently, Deep Learning (DL) approaches have been applied to solve the Sentiment Classification (SC) problem, which is a core task in reviews mining or Sentiment Analysis (SA). The performances of these approaches are affected by different factors. This paper addresses these factors and classifies them into three categories: data preparation based factors, feature representation based factors and the classification techniques based factors. The paper is a comprehensive literature-based survey that compares the performance of more than 100 DL-based SC approaches by using 21 public datasets of reviews given by customers within three specific application domains (products, movies and restaurants). These 21 datasets have different characteristics (balanced/imbalanced, size, etc.) to give a global vision for our study. The comparison explains how the proposed factors quantitatively affect the performance of the studied DL-based SC approaches.
Semantic segmentation of SEM images of lower bainitic and tempered martensitic steels
Authors: Xiaohan Bie, Manoj Arthanari, Evelin Barbosa de Melo, Juancheng Li, Stephen Yue, Salim Brahimi, Jun Song
Date: 2023-12-02 00:00:00
Tasks: Deep Learning, Semantic Segmentation

This study employs deep learning techniques to segment scanning electron microscope images, enabling a quantitative analysis of carbide precipitates in lower bainite and tempered martensite steels with comparable strength. Following segmentation, carbides are investigated, and their volume percentage, size distribution, and orientations are probed within the image dataset. Our findings reveal that lower bainite and tempered martensite exhibit comparable volume percentages of carbides, albeit with a more uniform distribution of carbides in tempered martensite. Carbides in lower bainite demonstrate a tendency for better alignment than those in tempered martensite, aligning with the observations of other researchers. However, both microstructures display a scattered carbide orientation, devoid of any discernible pattern. Comparative analysis of aspect ratios and sizes of carbides in lower bainite and tempered martensite unveils striking similarities. The deep learning model achieves an impressive pixelwise accuracy of 98.0% in classifying carbide/iron matrix at the individual pixel level. The semantic segmentation derived from deep learning extends its applicability to the analysis of secondary phases in various materials, offering a time-efficient, versatile AI-powered workflow for quantitative microstructure analysis.
Flying By ML -- CNN Inversion of Affine Transforms
Authors: L. Van Warren
Date: 2023-12-22 00:00:00
Tasks: Hyperparameter Optimization

This paper describes a machine learning method to automate reading of cockpit gauges, using a CNN to invert affine transformations and deduce aircraft states from instrument images. Validated with synthetic images of a turn-and-bank indicator, this research introduces methods such as generating datasets from a single image, the 'Clean Training Principle' for optimal noise-free training, and CNN interpolation for continuous value predictions from categorical data. It also offers insights into hyperparameter optimization and ML system software engineering.
TimePillars: Temporally-Recurrent 3D LiDAR Object Detection
Authors: Ernesto Lozano Calvo, Bernardo Taveira, Fredrik Kahl, Niklas Gustafsson, Jonathan Larsson, Adam Tonderski
Date: 2023-12-22 00:00:00
Tasks: Autonomous Driving, Diversity, Object, object-detection, Object Detection

Object detection applied to LiDAR point clouds is a relevant task in robotics, and particularly in autonomous driving. Single frame methods, predominant in the field, exploit information from individual sensor scans. Recent approaches achieve good performance, at relatively low inference time. Nevertheless, given the inherent high sparsity of LiDAR data, these methods struggle in long-range detection (e.g. 200m) which we deem to be critical in achieving safe automation. Aggregating multiple scans not only leads to a denser point cloud representation, but it also brings time-awareness to the system, and provides information about how the environment is changing. Solutions of this kind, however, are often highly problem-specific, demand careful data processing, and tend not to fulfil runtime requirements. In this context we propose TimePillars, a temporally-recurrent object detection pipeline which leverages the pillar representation of LiDAR data across time, respecting hardware integration efficiency constraints, and exploiting the diversity and long-range information of the novel Zenseact Open Dataset (ZOD). Through experimentation, we prove the benefits of having recurrency, and show how basic building blocks are enough to achieve robust and efficient results.
Transformer-Based Multi-Object Smoothing with Decoupled Data Association and Smoothing
Authors: Juliano Pinto, Georg Hess, Yuxuan Xia, Henk Wymeersch, Lennart Svensson
Date: 2023-12-22 00:00:00
Tasks: Multi-Object Tracking, Object, Object Tracking

Multi-object tracking (MOT) is the task of estimating the state trajectories of an unknown and time-varying number of objects over a certain time window. Several algorithms have been proposed to tackle the multi-object smoothing task, where object detections can be conditioned on all the measurements in the time window. However, the best-performing methods suffer from intractable computational complexity and require approximations, performing suboptimally in complex settings. Deep learning based algorithms are a possible venue for tackling this issue but have not been applied extensively in settings where accurate multi-object models are available and measurements are low-dimensional. We propose a novel DL architecture specifically tailored for this setting that decouples the data association task from the smoothing task. We compare the performance of the proposed smoother to the state-of-the-art in different tasks of varying difficulty and provide, to the best of our knowledge, the first comparison between traditional Bayesian trackers and DL trackers in the smoothing problem setting.
Single-channel speech enhancement using learnable loss mixup
Authors: Oscar Chang, Dung N. Tran, Kazuhito Koishida
Date: 2023-12-20 00:00:00
Tasks: Speech Enhancement

Generalization remains a major problem in supervised learning of single-channel speech enhancement. In this work, we propose learnable loss mixup (LLM), a simple and effortless training diagram, to improve the generalization of deep learning-based speech enhancement models. Loss mixup, of which learnable loss mixup is a special variant, optimizes a mixture of the loss functions of random sample pairs to train a model on virtual training data constructed from these pairs of samples. In learnable loss mixup, by conditioning on the mixed data, the loss functions are mixed using a non-linear mixing function automatically learned via neural parameterization. Our experimental results on the VCTK benchmark show that learnable loss mixup achieves 3.26 PESQ, outperforming the state-of-the-art.
Distance approximation to support customer selection in vehicle routing problems
Authors: Fabian Akkerman, Martijn Mes
Date: 2022-04-07 00:00:00
Proceeding: Annals of Operations Research 2022 4
Tasks: regression, Traveling Salesman Problem

Estimating the solution value of transportation problems can be useful to assign customers to days for multi-period vehicle routing problems, or to make customer selection decisions very fast (e.g., within an online environment). In this paper, we apply several regression methods to predict the total distance of the traveling salesman problem (TSP) and vehicle routing problem (VRP). We show that distance can be estimated fairly accurate using simple regression models and only a limited number of features. Besides using features found in the scientific literature, we also introduce new classes of spatial features. The model is validated on a fictional case with different spatial instances considering both a backordering and lost sales configuration, and on a realistic case that involves dynamic waste collection in the city of Amsterdam, The Netherlands. For the fictional case, we show differences in performance per instance type and configuration, and we show that our model can save up to 25.3% in distance compared with a heuristic approximation. For the waste collection case, we introduce a cost function that combines the travel distance and service level, and show that our model can reduce distances up to 17% compared to a well-known heuristic approximation while maintaining the same service level. Furthermore, we show the benefits of using approximations for combining offline learning with online or frequent optimization.
ShennongAlpha: an AI-driven sharing and collaboration platform for intelligent curation, acquisition, and translation of natural medicinal material knowledge
Authors: Zijie Yang, Yongjing Yin, Chaojun Kong, Tiange Chi, Wufan Tao, Yue Zhang, Tian Xu
Date: 2023-12-27 00:00:00
Tasks: Drug Discovery, Machine Translation, Management, Translation

Natural Medicinal Materials (NMMs) have a long history of global clinical applications and a wealth of records and knowledge. Although NMMs are a major source for drug discovery and clinical application, the utilization and sharing of NMM knowledge face crucial challenges, including the standardized description of critical information, efficient curation and acquisition, and language barriers. To address these, we developed ShennongAlpha, an AI-driven sharing and collaboration platform for intelligent knowledge curation, acquisition, and translation. For standardized knowledge curation, the platform introduced a Systematic Nomenclature to enable accurate differentiation and identification of NMMs. More than fourteen thousand Chinese NMMs have been curated into the platform along with their knowledge. Furthermore, the platform pioneered chat-based knowledge acquisition, standardized machine translation, and collaborative knowledge updating. Together, our study represents the first major advance in leveraging AI to empower NMM knowledge sharing, which not only marks a novel application of AI for Science, but also will significantly benefit the global biomedical, pharmaceutical, physician, and patient communities.