pwc-archive/papers-with-abstracts huggingface dataset:
DiffATR: Diffusion-based Generative Modeling for Audio-Text Retrieval
Authors: Yifei Xin, Xuxin Cheng, Zhihong Zhu, Xusheng Yang, Yuexian Zou
Date: 2024-09-16 00:00:00
Tasks: AudioCaps, Retrieval, Text Retrieval

Existing audio-text retrieval (ATR) methods are essentially discriminative models that aim to maximize the conditional likelihood, represented as p(candidates|query). Nevertheless, this methodology fails to consider the intrinsic data distribution p(query), leading to difficulties in discerning out-of-distribution data. In this work, we attempt to tackle this constraint through a generative perspective and model the relationship between audio and text as their joint probability p(candidates,query). To this end, we present a diffusion-based ATR framework (DiffATR), which models ATR as an iterative procedure that progressively generates joint distribution from noise. Throughout its training phase, DiffATR is optimized from both generative and discriminative viewpoints: the generator is refined through a generation loss, while the feature extractor benefits from a contrastive loss, thus combining the merits of both methodologies. Experiments on the AudioCaps and Clotho datasets with superior performances, verify the effectiveness of our approach. Notably, without any alterations, our DiffATR consistently exhibits strong performance in out-of-domain retrieval settings.
Practical and Asymptotically Optimal Quantization of High-Dimensional Vectors in Euclidean Space for Approximate Nearest Neighbor Search
Authors: Jianyang Gao, Yutong Gou, Yuexuan Xu, Yongyi Yang, Cheng Long, Raymond Chi-Wing Wong
Date: 2024-09-16 00:00:00
Tasks: Quantization

Approximate nearest neighbor (ANN) query in high-dimensional Euclidean space is a key operator in database systems. For this query, quantization is a popular family of methods developed for compressing vectors and reducing memory consumption. Recently, a method called RaBitQ achieves the state-of-the-art performance among these methods. It produces better empirical performance in both accuracy and efficiency when using the same compression rate and provides rigorous theoretical guarantees. However, the method is only designed for compressing vectors at high compression rates (32x) and lacks support for achieving higher accuracy by using more space. In this paper, we introduce a new quantization method to address this limitation by extending RaBitQ. The new method inherits the theoretical guarantees of RaBitQ and achieves the asymptotic optimality in terms of the trade-off between space and error bounds as to be proven in this study. Additionally, we present efficient implementations of the method, enabling its application to ANN queries to reduce both space and time consumption. Extensive experiments on real-world datasets confirm that our method consistently outperforms the state-of-the-art baselines in both accuracy and efficiency when using the same amount of memory.
CROSS-JEM: Accurate and Efficient Cross-encoders for Short-text Ranking Tasks
Authors: Bhawna Paliwal, Deepak Saini, Mudit Dhawan, Siddarth Asokan, Nagarajan Natarajan, Surbhi Aggarwal, Pankaj Malhotra, Jian Jiao, Manik Varma
Date: 2024-09-15 00:00:00

Ranking a set of items based on their relevance to a given query is a core problem in search and recommendation. Transformer-based ranking models are the state-of-the-art approaches for such tasks, but they score each query-item independently, ignoring the joint context of other relevant items. This leads to sub-optimal ranking accuracy and high computational costs. In response, we propose Cross-encoders with Joint Efficient Modeling (CROSS-JEM), a novel ranking approach that enables transformer-based models to jointly score multiple items for a query, maximizing parameter utilization. CROSS-JEM leverages (a) redundancies and token overlaps to jointly score multiple items, that are typically short-text phrases arising in search and recommendations, and (b) a novel training objective that models ranking probabilities. CROSS-JEM achieves state-of-the-art accuracy and over 4x lower ranking latency over standard cross-encoders. Our contributions are threefold: (i) we highlight the gap between the ranking application's need for scoring thousands of items per query and the limited capabilities of current cross-encoders; (ii) we introduce CROSS-JEM for joint efficient scoring of multiple items per query; and (iii) we demonstrate state-of-the-art accuracy on standard public datasets and a proprietary dataset. CROSS-JEM opens up new directions for designing tailored early-attention-based ranking models that incorporate strict production constraints such as item multiplicity and latency.
HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applications
Authors: Rishi Kalra, Zekun Wu, Ayesha Gulley, Airlie Hilliard, Xin Guan, Adriano Koshiyama, Philip Treleaven
Date: 2024-08-29 00:00:00
Tasks: Question Answering, RAG, Retrieval, Retrieval-augmented Generation, Text Generation

Large Language Models (LLMs) face limitations in AI legal and policy applications due to outdated knowledge, hallucinations, and poor reasoning in complex contexts. Retrieval-Augmented Generation (RAG) systems address these issues by incorporating external knowledge, but suffer from retrieval errors, ineffective context integration, and high operational costs. This paper presents the Hybrid Parameter-Adaptive RAG (HyPA-RAG) system, designed for the AI legal domain, with NYC Local Law 144 (LL144) as the test case. HyPA-RAG integrates a query complexity classifier for adaptive parameter tuning, a hybrid retrieval approach combining dense, sparse, and knowledge graph methods, and a comprehensive evaluation framework with tailored question types and metrics. Testing on LL144 demonstrates that HyPA-RAG enhances retrieval accuracy, response fidelity, and contextual precision, offering a robust and adaptable solution for high-stakes legal and policy applications.
MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases
Authors: Rithesh Murthy, Liangwei Yang, Juntao Tan, Tulika Manoj Awalgaonkar, Yilun Zhou, Shelby Heinecke, Sachin Desai, Jason Wu, ran Xu, Sarah Tan, JianGuo Zhang, Zhiwei Liu, Shirley Kokane, Zuxin Liu, Ming Zhu, Huan Wang, Caiming Xiong, Silvio Savarese
Date: 2024-06-12 00:00:00
Tasks: Benchmarking, Model Compression, Quantization

The deployment of Large Language Models (LLMs) and Large Multimodal Models (LMMs) on mobile devices has gained significant attention due to the benefits of enhanced privacy, stability, and personalization. However, the hardware constraints of mobile devices necessitate the use of models with fewer parameters and model compression techniques like quantization. Currently, there is limited understanding of quantization's impact on various task performances, including LLM tasks, LMM tasks, and, critically, trust and safety. There is a lack of adequate tools for systematically testing these models on mobile devices. To address these gaps, we introduce MobileAIBench, a comprehensive benchmarking framework for evaluating mobile-optimized LLMs and LMMs. MobileAIBench assesses models across different sizes, quantization levels, and tasks, measuring latency and resource consumption on real devices. Our two-part open-source framework includes a library for running evaluations on desktops and an iOS app for on-device latency and hardware utilization measurements. Our thorough analysis aims to accelerate mobile AI research and deployment by providing insights into the performance and feasibility of deploying LLMs and LMMs on mobile platforms.
Anomaly Detection in Dynamic Graphs: A Comprehensive Survey
Authors: Ocheme Anthony Ekle, William Eberle
Date: 2024-05-31 00:00:00
Tasks: Anomaly Detection, Graph Mining, Survey

This survey paper presents a comprehensive and conceptual overview of anomaly detection using dynamic graphs. We focus on existing graph-based anomaly detection (AD) techniques and their applications to dynamic networks. The contributions of this survey paper include the following: i) a comparative study of existing surveys on anomaly detection; ii) a Dynamic Graph-based Anomaly Detection (DGAD) review framework in which approaches for detecting anomalies in dynamic graphs are grouped based on traditional machine-learning models, matrix transformations, probabilistic approaches, and deep-learning approaches; iii) a discussion of graphically representing both discrete and dynamic networks; and iv) a discussion of the advantages of graph-based techniques for capturing the relational structure and complex interactions in dynamic graph data. Finally, this work identifies the potential challenges and future directions for detecting anomalies in dynamic networks. This DGAD survey approach aims to provide a valuable resource for researchers and practitioners by summarizing the strengths and limitations of each approach, highlighting current research trends, and identifying open challenges. In doing so, it can guide future research efforts and promote advancements in anomaly detection in dynamic graphs. Keywords: Graphs, Anomaly Detection, dynamic networks,Graph Neural Networks (GNN), Node anomaly, Graph mining.
Facial Expressions Recognition System Using FPGA-Based Convolutional Neural Network
Authors: Tai Nguyen Duong Phuc, Nam Nguyen Nhut, Nguyen Trinh & Linh Tran
Date: 2021-01-05 00:00:00
Proceeding: Research in Intelligent and Computing in Engineering 2021 1
Tasks: Emotion Recognition

Emotion detection has become one of the most significant aspects to consider in any project related to Affective Computing. There are several researches in emotion recognition,where Convolutional Neural Network is considered as an efficient algorithm that achieves the state-of-the-art performance in image recognition. There have been plenty of methods applying CNN model and conducting in software. However, traditional software based computation is not fast enough to meet the demand of real-time image processing. Therefore, we propose an efficient method for expressions recognition using FPGA-based. This model is used for classifying seven elementary types of human emotions: angry, fear, disgust, happy, sad and neutral by extracting features through those layers respectively. We also tested the functions of this model on FPGA simulation and achieved over 65% of accuracy using FER2013 dataset.
Developing a Chatbot system using Deep Learning based for Universities consultancy
Authors: Le-Tien Thuong, Nguyen-DP Tai, Huynh-Y Vy
Date: 2022-02-28 00:00:00
Proceeding: International Conference on Ubiquitous Information Management and Communication (IMCOM) 2022 2
Tasks: Chatbot, Deep Learning, Deep Reinforcement Learning, Dialogue Management, Intent Recognition, Management, text-classification, Text Classification

Inspired by the recent successes of Deep Learning on Natural Language Processing (NLP), we propose a chatbot system using Deep Learning for Vietnamese Universities consultancy which can be implemented for any university. The system has three important tasks: User’s Intent Recognition, Dialogue Management, and Reply Channels. For the User’s Intent Recognition task, the Pattern Matching method is combined with the Text Classification model using Bidirectional-LSTM which has the Attention mechanism. Besides, we use the Deep Reinforcement Learning architecture to train an Agent for Dialogue Management task. In this paper, we conduct a Proof of Concept to the Ho Chi Minh City University of Technology (HCMUT). The experimental results achieve an average 89% F1-score of 3 classes of the Text Classification task using Deep Learning, the evaluation result of Dialogue Management that the rate of success achieves by 86%. Our demo version of a production web application is available at https://hcmutbot.herokuapp.com/.
Hierarchical Uncertainty Aggregation and Emphasis Loss for Active Learning in Object Detection
Authors: Nguyen Tai, Nguyen Khoa, Nguyen, Thanh, Nguyen Tri, Nguyen Anh, Kim Karrman
Date: 2023-12-15 00:00:00
Proceeding: IEEE International Conference on Big Data (BigData) special session Machine Learning 2023 12
Tasks: Active Learning, object-detection, Object Detection

Object detection has achieved remarkable advancements through the utilization of deep neural networks; however, its heavy reliance on extensive labeled data remains a significant challenge. We first delve into active learning strategies that leverage the power of uncertainty estimation and detector resilience. Departing from approaches biased toward high performing classes, we present an innovative solution known as Hierarchical Uncertainty Aggregation and Emphasis Loss (HUALE). This approach introduces hierarchical uncertainty aggregation, which effectively retrieves and ranks unlabeled images for precise image selection. Secondly, we introduce the Emphasis loss, a novel loss augmentation to object detection training and combine it with the EIoU loss and Synchronic IoU-Localization loss to elevate the accuracy of bounding box localization and address non-convexity challenges. Our method, built upon the Faster R-CNN framework, consistently outperforms the baseline (random selection) with average mAP (%) improvements of 4.53/3.44/1.57 across the PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO datasets.
Constructing Adversarial Examples for Vertical Federated Learning: Optimal Client Corruption through Multi-Armed Bandit
Authors: Duanyi Yao (HKUST), Songze Li (Southeast University), Ye Xue (Shenzhen Research Institute of Big Data, CUHK(SZ)), Jin Liu (HKUST(GZ))
Date: 2024-05-07 00:00:00
Proceeding: The International Conference on Learning Representations (ICLR) 2024 5
Tasks: Federated Learning, Thompson Sampling, Vertical Federated Learning

Vertical federated learning (VFL), where each participating client holds a subset of data features, has found numerous applications in finance, healthcare, and IoT systems. However, adversarial attacks, particularly through the injection of adversarial examples (AEs), pose serious challenges to the security of VFL models. In this paper, we investigate such vulnerabilities through developing a novel attack to disrupt the VFL inference process, under a practical scenario where the adversary is able to adaptively corrupt a subset of clients. We formulate the problem of finding optimal attack strategies as an online optimization problem, which is decomposed into an inner problem of adversarial example generation (AEG) and an outer problem of corruption pattern selection (CPS). Specifically, we establish the equivalence between the formulated CPS problem and a multiarmed bandit (MAB) problem, and propose the Thompson sampling with Empirical maximum reward (E-TS) algorithm for the adversary to efficiently identify the optimal subset of clients for corruption. The key idea of E-TS is to introduce an estimation of the expected maximum reward for each arm, which helps to specify a small set of competitive arms, on which the exploration for the optimal arm is performed. This significantly reduces the exploration space, which otherwise can quickly become prohibitively large as the number of clients increases. We analytically characterize the regret bound of E-TS, and empirically demonstrate its capability of efficiently revealing the optimal corruption pattern with the highest attack success rate, under various datasets of popular VFL tasks
Response Surface Methodology coupled with desirability functions for multi-objective optimization: minimizing indoor overheating hours and maximizing useful daylight illuminance
Authors: Juan Gamero-Salinas, Jesús López-Fidalgo
Date: 2024-09-12 00:00:00

Response Surface Methodology (RSM) and desirability functions were employed in a case study to optimize the thermal and daylight performance of a computational model of a tropical housing typology. Specifically, this approach simultaneously optimized Indoor Overheating Hours (IOH) and Useful Daylight Illuminance (UDI) metrics through an Overall Desirability (D). The lack of significant association between IOH and other annual daylight metrics enabled a focused optimization of IOH and UDI. Each response required only 138 simulation runs (~30 hours for 276 runs) to determine the optimal values for passive strategies: window-to-wall ratio (WWR) and roof overhang depth across four orientations, totalling eight factors. First, initial screening based on $2_V^{8-2}$ fractional factorial design, identified four key factors using stepwise and Lasso regression, narrowed down to three: roof overhang depth on the south and west, WWR on the west, and WWR on the south. Then, RSM optimization yielded an optimal solution (roof overhang: 3.78 meters, west WWR: 3.76%, south WWR: 29.3%) with a D of 0.625 (IOH: 8.33%, UDI: 79.67%). Finally, robustness analysis with 1,000 bootstrap replications provided 95% confidence intervals for the optimal values. This study optimally balances thermal comfort and daylight with few experiments using a computationally-efficient multi-objective approach.
Compositional Video Generation as Flow Equalization
Authors: Xingyi Yang, Xinchao Wang
Date: 2024-06-10 00:00:00
Tasks: Video Editing, Video Generation

Large-scale Text-to-Video (T2V) diffusion models have recently demonstrated unprecedented capability to transform natural language descriptions into stunning and photorealistic videos. Despite the promising results, a significant challenge remains: these models struggle to fully grasp complex compositional interactions between multiple concepts and actions. This issue arises when some words dominantly influence the final video, overshadowing other concepts.To tackle this problem, we introduce \textbf{Vico}, a generic framework for compositional video generation that explicitly ensures all concepts are represented properly. At its core, Vico analyzes how input tokens influence the generated video, and adjusts the model to prevent any single concept from dominating. Specifically, Vico extracts attention weights from all layers to build a spatial-temporal attention graph, and then estimates the influence as the \emph{max-flow} from the source text token to the video target token. Although the direct computation of attention flow in diffusion models is typically infeasible, we devise an efficient approximation based on subgraph flows and employ a fast and vectorized implementation, which in turn makes the flow computation manageable and differentiable. By updating the noisy latent to balance these flows, Vico captures complex interactions and consequently produces videos that closely adhere to textual descriptions. We apply our method to multiple diffusion-based video models for compositional T2V and video editing. Empirical results demonstrate that our framework significantly enhances the compositional richness and accuracy of the generated videos. Visit our website at~\href{https://adamdad.github.io/vico/}{\url{https://adamdad.github.io/vico/}}.
ElasticAI: Creating and Deploying Energy-Efficient Deep Learning Accelerator for Pervasive Computing
Authors: Chao Qian, Tianheng Ling, Gregor Schiele
Date: 2024-08-29 00:00:00

Deploying Deep Learning (DL) on embedded end devices is a scorching trend in pervasive computing. Since most Microcontrollers on embedded devices have limited computing power, it is necessary to add a DL accelerator. Embedded Field Programmable Gate Arrays (FPGAs) are suitable for deploying DL accelerators for embedded devices, but developing an energy-efficient DL accelerator on an FPGA is not easy. Therefore, we propose the ElasticAI-Workflow that aims to help DL developers to create and deploy DL models as hardware accelerators on embedded FPGAs. This workflow consists of two key components: the ElasticAI-Creator and the Elastic Node. The former is a toolchain for automatically generating DL accelerators on FPGAs. The latter is a hardware platform for verifying the performance of the generated accelerators. With this combination, the performance of the accelerator can be sufficiently guaranteed. We will demonstrate the potential of our approach through a case study.
Gene and RNA Editing: Methods, Enabling Technologies, Applications, and Future Directions
Authors: Mohammed Aledhari, Mohamed Rahouti
Date: 2024-09-01 00:00:00

Gene and RNA editing methods, technologies, and applications are emerging as innovative forms of therapy and medicine, offering more efficient implementation compared to traditional pharmaceutical treatments. Current trends emphasize the urgent need for advanced methods and technologies to detect public health threats, including diseases and viral agents. Gene and RNA editing techniques enhance the ability to identify, modify, and ameliorate the effects of genetic diseases, disorders, and disabilities. Viral detection and identification methods present numerous opportunities for enabling technologies, such as CRISPR, applicable to both RNA and gene editing through the use of specific Cas proteins. This article explores the distinctions and benefits of RNA and gene editing processes, emphasizing their contributions to the future of medical treatment. CRISPR technology, particularly its adaptation via the Cas13 protein for RNA editing, is a significant advancement in gene editing. The article will delve into RNA and gene editing methodologies, focusing on techniques that alter and modify genetic coding. A-to-I and C-to-U editing are currently the most predominant methods of RNA modification. CRISPR stands out as the most cost-effective and customizable technology for both RNA and gene editing. Unlike permanent changes induced by cutting an individual's DNA genetic code, RNA editing offers temporary modifications by altering nucleoside bases in RNA strands, which can then attach to DNA strands as temporary modifiers.
LSR-IGRU: Stock Trend Prediction Based on Long Short-Term Relationships and Improved GRU
Authors: Peng Zhu, Yuante Li, Yifan Hu, Qinyuan Liu, Dawei Cheng, Yuqi Liang
Date: 2024-08-26 00:00:00
Tasks: Algorithmic Trading, Stock Price Prediction, Stock Trend Prediction

Stock price prediction is a challenging problem in the field of finance and receives widespread attention. In recent years, with the rapid development of technologies such as deep learning and graph neural networks, more research methods have begun to focus on exploring the interrelationships between stocks. However, existing methods mostly focus on the short-term dynamic relationships of stocks and directly integrating relationship information with temporal information. They often overlook the complex nonlinear dynamic characteristics and potential higher-order interaction relationships among stocks in the stock market. Therefore, we propose a stock price trend prediction model named LSR-IGRU in this paper, which is based on long short-term stock relationships and an improved GRU input. Firstly, we construct a long short-term relationship matrix between stocks, where secondary industry information is employed for the first time to capture long-term relationships of stocks, and overnight price information is utilized to establish short-term relationships. Next, we improve the inputs of the GRU model at each step, enabling the model to more effectively integrate temporal information and long short-term relationship information, thereby significantly improving the accuracy of predicting stock trend changes. Finally, through extensive experiments on multiple datasets from stock markets in China and the United States, we validate the superiority of the proposed LSR-IGRU model over the current state-of-the-art baseline models. We also apply the proposed model to the algorithmic trading system of a financial company, achieving significantly higher cumulative portfolio returns compared to other baseline methods. Our sources are released at https://github.com/ZP1481616577/Baselines_LSR-IGRU.