pwc-archive/papers-with-abstracts huggingface dataset:
Rational Kriging
Authors: V. Roshan Joseph
Date: 2023-12-08 00:00:00
Tasks: parameter estimation, Uncertainty Quantification

This article proposes a new kriging that has a rational form. It is shown that the generalized least squares estimate of the mean from rational kriging is much more well behaved than that from ordinary kriging. Parameter estimation and uncertainty quantification for rational kriging are proposed using a Gaussian process framework. Its potential applications in emulation and calibration of computer models are also discussed.
Cross Domain LifeLong Sequential Modeling for Online Click-Through Rate Prediction
Authors: Ruijie Hou, Zhaoyang Yang, Yu Ming, Hongyu Lu, Zhuobin Zheng, Yu Chen, Qinsong Zeng, Ming Chen
Date: 2023-12-11 00:00:00
Tasks: Click-Through Rate Prediction, Prediction, Recommendation Systems

Deep neural networks (DNNs) that incorporated lifelong sequential modeling (LSM) have brought great success to recommendation systems in various social media platforms. While continuous improvements have been made in domain-specific LSM, limited work has been done in cross-domain LSM, which considers modeling of lifelong sequences of both target domain and source domain. In this paper, we propose Lifelong Cross Network (LCN) to incorporate cross-domain LSM to improve the click-through rate (CTR) prediction in the target domain. The proposed LCN contains a LifeLong Attention Pyramid (LAP) module that comprises of three levels of cascaded attentions to effectively extract interest representations with respect to the candidate item from lifelong sequences. We also propose Cross Representation Production (CRP) module to enforce additional supervision on the learning and alignment of cross-domain representations so that they can be better reused on learning of the CTR prediction in the target domain. We conducted extensive experiments on WeChat Channels industrial dataset as well as on benchmark dataset. Results have revealed that the proposed LCN outperforms existing work in terms of both prediction accuracy and online performance.
MUST: An Effective and Scalable Framework for Multimodal Search of Target Modality
Authors: Mengzhao Wang, Xiangyu Ke, Xiaoliang Xu, Lu Chen, Yunjun Gao, Pinpin Huang, Runkai Zhu
Date: 2023-12-11 00:00:00
Tasks: Information Retrieval

We investigate the problem of multimodal search of target modality, where the task involves enhancing a query in a specific target modality by integrating information from auxiliary modalities. The goal is to retrieve relevant objects whose contents in the target modality match the specified multimodal query. The paper first introduces two baseline approaches that integrate techniques from the Database, Information Retrieval, and Computer Vision communities. These baselines either merge the results of separate vector searches for each modality or perform a single-channel vector search by fusing all modalities. However, both baselines have limitations in terms of efficiency and accuracy as they fail to adequately consider the varying importance of fusing information across modalities. To overcome these limitations, the paper proposes a novel framework, called MUST. Our framework employs a hybrid fusion mechanism, combining different modalities at multiple stages. Notably, we leverage vector weight learning to determine the importance of each modality, thereby enhancing the accuracy of joint similarity measurement. Additionally, the proposed framework utilizes a fused proximity graph index, enabling efficient joint search for multimodal queries. MUST offers several other advantageous properties, including pluggable design to integrate any advanced embedding techniques, user flexibility to customize weight preferences, and modularized index construction. Extensive experiments on real-world datasets demonstrate the superiority of MUST over the baselines in terms of both search accuracy and efficiency. Our framework achieves over 10x faster search times while attaining an average of 93% higher accuracy. Furthermore, MUST exhibits scalability to datasets containing more than 10 million data elements.
RecJPQ: Training Large-Catalogue Sequential Recommenders
Authors: Aleksandr V. Petrov, Craig Macdonald
Date: 2023-12-11 00:00:00
Tasks: Passage Retrieval, Recommendation Systems, Retrieval, Sequential Recommendation

Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec and SASRec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size makes training these models costly. The Joint Product Quantisation (JPQ) method, originally proposed for passage retrieval, markedly reduces the size of the retrieval index with minimal effect on model effectiveness, by replacing passage embeddings with a limited number of shared sub-embeddings. This paper introduces RecJPQ, a novel adaptation of JPQ for sequential recommendations, which takes the place of item embeddings tensor and replaces item embeddings with a concatenation of a limited number of shared sub-embeddings and, therefore, limits the number of learnable model parameters. The main idea of RecJPQ is to split items into sub-item entities before training the main recommendation model, which is inspired by splitting words into tokens and training tokenisers in language models. We apply RecJPQ to SASRec, BERT4Rec, and GRU4rec models on three large-scale sequential datasets. Our results showed that RecJPQ could notably reduce the model size (e.g., 48% reduction for the Gowalla dataset with no effectiveness degradation). RecJPQ can also improve model performance through a regularisation effect (e.g. +0.96% NDCG@10 improvement on the Booking.com dataset). Overall, RecJPQ allows the training of state-of-the-art transformer recommenders in industrial applications, where datasets with millions of items are common.
mir_ref: A Representation Evaluation Framework for Music Information Retrieval Tasks
Authors: Christos Plachouras, Pablo Alonso-Jiménez, Dmitry Bogdanov
Date: 2023-12-10 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Representation Learning, Retrieval

Music Information Retrieval (MIR) research is increasingly leveraging representation learning to obtain more compact, powerful music audio representations for various downstream MIR tasks. However, current representation evaluation methods are fragmented due to discrepancies in audio and label preprocessing, downstream model and metric implementations, data availability, and computational resources, often leading to inconsistent and limited results. In this work, we introduce mir_ref, an MIR Representation Evaluation Framework focused on seamless, transparent, local-first experiment orchestration to support representation development. It features implementations of a variety of components such as MIR datasets, tasks, embedding models, and tools for result analysis and visualization, while facilitating the implementation of custom components. To demonstrate its utility, we use it to conduct an extensive evaluation of several embedding models across various tasks and datasets, including evaluating their robustness to various audio perturbations and the ease of extracting relevant information from them.
Towards Global, Socio-Economic, and Culturally Aware Recommender Systems
Authors: Kelley Ann Yohe
Date: 2023-12-10 00:00:00
Tasks: Recommendation Systems

Recommender systems have gained increasing attention to personalise consumer preferences. While these systems have primarily focused on applications such as advertisement recommendations (e.g., Google), personalized suggestions (e.g., Netflix and Spotify), and retail selection (e.g., Amazon), there is potential for these systems to benefit from a more global, socio-economic, and culturally aware approach, particularly as companies seek to expand into diverse markets. This paper aims to investigate the potential of a recommender system that considers cultural identity and socio-economic factors. We review the most recent developments in recommender systems and explore the impact of cultural identity and socio-economic factors on consumer preferences. We then propose an ontology and approach for incorporating these factors into recommender systems. To illustrate the potential of our approach, we present a scenario in consumer subscription plan selection within the entertainment industry. We argue that existing recommender systems have limited ability to precisely understand user preferences due to a lack of awareness of socio-economic factors and cultural identity. They also fail to update recommendations in response to changing socio-economic conditions. We explore various machine learning models and develop a final artificial neural network model (ANN) that addresses this gap. We evaluate the effectiveness of socio-economic and culturally aware recommender systems across four dimensions: Precision, Accuracy, F1, and Recall. We find that a highly tuned ANN model incorporating domain-specific data, select cultural indices and relevant socio-economic factors predicts user preference in subscriptions with an accuracy of 95%, a precision of 94%, a F1 Score of 92\%, and a Recall of 90\%.
EEND-DEMUX: End-to-End Neural Speaker Diarization via Demultiplexed Speaker Embeddings
Authors: Sung Hwan Mun, Min Hyun Han, Canyeong Moon, Nam Soo Kim
Date: 2023-12-11 00:00:00
Tasks: speaker-diarization, Speaker Diarization

In recent years, there have been studies to further improve the end-to-end neural speaker diarization (EEND) systems. This letter proposes the EEND-DEMUX model, a novel framework utilizing demultiplexed speaker embeddings. In this work, we focus on disentangling speaker-relevant information in the latent space and then transform each separated latent variable into its corresponding speech activity. EEND-DEMUX can directly obtain separated speaker embeddings through the demultiplexing operation in the inference phase without an external speaker diarization system, an embedding extractor, or a heuristic decoding technique. Furthermore, we employ a multi-head cross-attention mechanism to capture the correlation between mixture and separated speaker embeddings effectively. We formulate three loss functions based on matching, orthogonality, and sparsity constraints to learn robust demultiplexed speaker embeddings. The experimental results on the LibriMix dataset show consistently improved performance in both a fixed and flexible number of speakers scenarios.
https://dl.acm.org/doi/abs/10.1145/3447993.3448618
Authors: Hao Wu1, Xuejin Tian1, Minghao Li1 2, Yunxin Liu3 Ganesh Ananthanarayanan4, Fengyuan Xu1*, and Sheng Zhong
Date: 2021-10-21 00:00:00
Proceeding: Proceedings of the 27th Annual International Conference on Mobile Computing and Networking 2021 10
Tasks: Generative Adversarial Network

As Video Streaming and Analytics (VSA) systems become increasingly popular, serious privacy concerns have risen on exposing too much unnecessary private information to the VSA providers. Yet, it is challenging to protect privacy while still preserving desired VSA features, i.e., effective analytics, forensic support, resource efficiency, and real-time execution. In this paper, we present a VSA privacy enhancement system (PECAM), which addresses the above challenge with no change in the VSA back-end. PECAM leverages a novel Generative Adversarial Network to perform the privacy-enhanced securely-reversible video transformation. PECAM also incorporates a couple of system optimizations into its VSA workflow to reduce network bandwidth usage and enable real-time processing on cameras. We implement our PECAM prototype on commodity hardware and evaluate its performance via both security study and extensive experiments. Results demonstrate that PECAM can effectively enhance the visual privacy of VSA in the presence of an adversary, and its transformed videos, when taken as input for various VSA back-end tasks, maintain a 96% accuracy of corresponding original videos. Additionally, it performs 12.3× and 1.8× better than baseline methods in terms of the computing cost and network bandwidth usage, respectively.
RACE-IT: A Reconfigurable Analog CAM-Crossbar Engine for In-Memory Transformer Acceleration
Authors: Lei Zhao, Luca Buonanno, Ron M. Roth, Sergey Serebryakov, Archit Gajjar, John Moon, Jim Ignowski, Giacomo Pedretti
Date: 2023-11-29 00:00:00

Transformer models represent the cutting edge of Deep Neural Networks (DNNs) and excel in a wide range of machine learning tasks. However, processing these models demands significant computational resources and results in a substantial memory footprint. While In-memory Computing (IMC) offers promise for accelerating Matrix-Vector Multiplications (MVMs) with high computational parallelism and minimal data movement, employing it for implementing other crucial operators within DNNs remains a formidable task. This challenge is exacerbated by the extensive use of Softmax and data-dependent matrix multiplications within the attention mechanism. Furthermore, existing IMC designs encounter difficulties in fully harnessing the benefits of analog MVM acceleration due to the area and energy-intensive nature of Analog-to-Digital Converters (ADCs). To tackle these challenges, we introduce a novel Compute Analog Content Addressable Memory (Compute-ACAM) structure capable of performing various non-MVM operations within Transformers. Together with the crossbar structure, our proposed RACE-IT accelerator enables efficient execution of all operations within Transformer models in the analog domain. Given the flexibility of our proposed Compute-ACAMs to perform arbitrary operations, RACE-IT exhibits adaptability to diverse non-traditional and future DNN architectures without necessitating hardware modifications. Leveraging the capability of Compute-ACAMs to process analog input and produce digital output, we also replace ADCs, thereby reducing the overall area and energy costs. By evaluating various Transformer models against state-of-the-art GPUs and existing IMC accelerators, RACE-IT increases performance by 10.7x and 5.9x, and reduces energy by 1193x, and 3.9x, respectively
Gluing GAP to RAS Mutants: A New Approach to an Old Problem in Cancer Drug Development
Authors: Ivan Ranđelović, Kinga Nyíri, Gergely Koppány, Marcel Baranyi, József Tóvári, Attila Kigyós, József Timár, Beáta G. Vértessy, Vince Grolmusz
Date: 2023-12-10 00:00:00
Tasks: Specificity

Mutated genes may lead to cancer development in numerous tissues. While more than 600 cancer-causing genes are known today, some of the most widespread mutations are connected to the RAS gene: RAS mutations are found in approximately 25% of all human tumors. Specifically, KRAS mutations are involved in the three most lethal cancers in U.S.: pancreatic ductal adenocarcinoma, colorectal adenocarcinoma, and lung adenocarcinoma. These cancers are among the most difficult to treat, and they are frequently excluded from chemotherapeutic attacks as hopeless cases. The mutated KRAS proteins have specific 3-dimensional conformations, which perturb functional interaction with the GAP protein on the GAP:RAS complex surface leading to a signaling cascade and uncontrolled cell growth. Here we describe a gluing docking method for finding small molecules that bind to both the GAP and the mutated KRAS molecules. These small molecules glue together the GAP and the mutated KRAS molecules and may serve as new cancer drugs for the most lethal, most difficult-to-treat carcinomas. As a proof of concept, we identify two new, drug-like small molecules with the new method: these compounds specifically inhibit the growth of PANC-1 cell line with KRAS mutation G12D in vitro and in vivo. Importantly, the two new compounds show significantly lower IC50 and higher specificity against the G12D KRAS mutant as compared to the recently described MRTX-1133 inhibitor against the G12D KRAS mutant.
On the Impact of CDL and TDL Augmentation for RF Fingerprinting under Impaired Channels
Authors: Omer Melih Gul, Michel Kulhandjian, Burak Kantarci, Claude D'Amours, Azzedine Touazi, Cliff Ellement
Date: 2023-12-11 00:00:00
Tasks: Autonomous Vehicles

Cyber-physical systems have recently been used in several areas (such as connected and autonomous vehicles) due to their high maneuverability. On the other hand, they are susceptible to cyber-attacks. Radio frequency (RF) fingerprinting emerges as a promising approach. This work aims to analyze the impact of decoupling tapped delay line and clustered delay line (TDL+CDL) augmentation-driven deep learning (DL) on transmitter-specific fingerprints to discriminate malicious users from legitimate ones. This work also considers 5G-only-CDL, WiFi-only-TDL augmentation approaches. RF fingerprinting models are sensitive to changing channels and environmental conditions. For this reason, they should be considered during the deployment of a DL model. Data acquisition can be another option. Nonetheless, gathering samples under various conditions for a train set formation may be quite hard. Consequently, data acquisition may not be feasible. This work uses a dataset that includes 5G, 4G, and WiFi samples, and it empowers a CDL+TDL-based augmentation technique in order to boost the learning performance of the DL model. Numerical results show that CDL+TDL, 5G-only-CDL, and WiFi-only-TDL augmentation approaches achieve 87.59%, 81.63%, 79.21% accuracy on unobserved data while TDL/CDL augmentation technique and no augmentation approach result in 77.81% and 74.84% accuracy on unobserved data, respectively.
Multimodal Group Emotion Recognition In-the-wild Using Privacy-Compliant Features
Authors: Anderson Augusma, Dominique Vaufreydaz, Frédérique Letué
Date: 2023-12-06 00:00:00
Tasks: Emotion Recognition

This paper explores privacy-compliant group-level emotion recognition ''in-the-wild'' within the EmotiW Challenge 2023. Group-level emotion recognition can be useful in many fields including social robotics, conversational agents, e-coaching and learning analytics. This research imposes itself using only global features avoiding individual ones, i.e. all features that can be used to identify or track people in videos (facial landmarks, body poses, audio diarization, etc.). The proposed multimodal model is composed of a video and an audio branches with a cross-attention between modalities. The video branch is based on a fine-tuned ViT architecture. The audio branch extracts Mel-spectrograms and feed them through CNN blocks into a transformer encoder. Our training paradigm includes a generated synthetic dataset to increase the sensitivity of our model on facial expression within the image in a data-driven way. The extensive experiments show the significance of our methodology. Our privacy-compliant proposal performs fairly on the EmotiW challenge, with 79.24% and 75.13% of accuracy respectively on validation and test set for the best models. Noticeably, our findings highlight that it is possible to reach this accuracy level with privacy-compliant features using only 5 frames uniformly distributed on the video.
All Rivers Run to the Sea: Private Learning with Asymmetric Flows
Authors: Yue Niu, Ramy E. Ali, Saurav Prakash, Salman Avestimehr
Date: 2023-12-05 00:00:00
Proceeding: CVPR 2024 1
Tasks: All, Quantization

Data privacy is of great concern in cloud machine-learning service platforms, when sensitive data are exposed to service providers. While private computing environments (e.g., secure enclaves), and cryptographic approaches (e.g., homomorphic encryption) provide strong privacy protection, their computing performance still falls short compared to cloud GPUs. To achieve privacy protection with high computing performance, we propose Delta, a new private training and inference framework, with comparable model performance as non-private centralized training. Delta features two asymmetric data flows: the main information-sensitive flow and the residual flow. The main part flows into a small model while the residuals are offloaded to a large model. Specifically, Delta embeds the information-sensitive representations into a low-dimensional space while pushing the information-insensitive part into high-dimension residuals. To ensure privacy protection, the low-dimensional information-sensitive part is secured and fed to a small model in a private environment. On the other hand, the residual part is sent to fast cloud GPUs, and processed by a large model. To further enhance privacy and reduce the communication cost, Delta applies a random binary quantization technique along with a DP-based technique to the residuals before sharing them with the public platform. We theoretically show that Delta guarantees differential privacy in the public environment and greatly reduces the complexity in the private environment. We conduct empirical analyses on CIFAR-10, CIFAR-100 and ImageNet datasets and ResNet-18 and ResNet-34, showing that Delta achieves strong privacy protection, fast training, and inference without significantly compromising the model utility.
Model Copyright Protection in Buyer-seller Environment
Authors: Yusheng Guo, Nan Zhong, Zhenxing Qian, Xinpeng Zhang
Date: 2023-12-05 00:00:00
Tasks: model, valid

Training a deep neural network (DNN) requires a high computational cost. Buying models from sellers with a large number of computing resources has become prevailing. However, the buyer-seller environment is not always trusted. To protect the neural network models from leaking in an untrusted environment, we propose a novel copyright protection scheme for DNN using an input-sensitive neural network (ISNN). The main idea of ISNN is to make a DNN sensitive to the key and copyright information. Therefore, only the buyer with a correct key can utilize the ISNN. During the training phase, we add a specific perturbation to the clean images and mark them as legal inputs, while the other inputs are treated as illegal input. We design a loss function to make the outputs of legal inputs close to the true ones, while the illegal inputs are far away from true results. Experimental results demonstrate that the proposed scheme is effective, valid, and secure.
Deep Learning for Joint Design of Pilot, Channel Feedback, and Hybrid Beamforming in FDD Massive MIMO-OFDM Systems
Authors: Junyi Yang, Weifeng Zhu, Shu Sun, Xiaofeng Li, Xingqin Lin, Meixia Tao
Date: 2023-12-10 00:00:00
Tasks: Graph Neural Network

This letter considers the transceiver design in frequency division duplex (FDD) massive multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) systems for high-quality data transmission. We propose a novel deep learning based framework where the procedures of pilot design, channel feedback, and hybrid beamforming are realized by carefully crafted deep neural networks. All the considered modules are jointly learned in an end-to-end manner, and a graph neural network is adopted to effectively capture interactions between beamformers based on the built graphical representation. Numerical results validate the effectiveness of our method.