pwc-archive/papers-with-abstracts huggingface dataset:
High-Precision Bootstrapping for Approximate Homomorphic Encryption by Error Variance Minimization
Authors: Yongwoo Lee, Joon-Woo Lee, Young-Sik Kim, Yongjune Kim, Jong-Seon No ， HyungChul Kang
Date: 2022-05-25 00:00:00
Proceeding: Annual International Conference on the Theory and Applications of Cryptographic Techniques  EUROCRYPT 2022: Advances in Cryptology – EUROCRYPT 2022 2022 5
Tasks: Privacy Preserving

The Cheon-Kim-Kim-Song (CKKS) scheme (Asiacrypt’17) is one of the most promising homomorphic encryption (HE) schemes as it enables privacy-preserving computing over real (or complex) numbers. It is known that bootstrapping is the most challenging part of the CKKS scheme. Further, homomorphic evaluation of modular reduction is the core of the CKKS bootstrapping. As modular reduction is not represented by the addition and multiplication of complex numbers, approximate polynomials for modular reduction should be used. The best-known techniques (Eurocrypt’21) use a polynomial approximation for trigonometric functions and their composition. However, all the previous methods are based on an indirect approximation, and thus it requires lots of multiplicative depth to achieve high accuracy. This paper proposes a direct polynomial approximation of modular reduction for CKKS bootstrapping, which is optimal in error variance and depth. Further, we propose an efficient algorithm, namely the lazy baby-step giant-step (BSGS) algorithm, to homomorphically evaluate the approximate polynomial, utilizing the lazy relinearization/rescaling technique. The lazy-BSGS reduces the computational complexity by half compared to the ordinary BSGS algorithm. The performance improvement for the CKKS scheme by the proposed algorithm is verified by implementation using HE libraries. The implementation results show that the proposed method has a multiplicative depth of 10 for modular reduction to achieve the state-of-the-art accuracy, while the previous methods have depths of 11 to 12. Moreover, we achieve higher accuracy within a small multiplicative depth, for example, 93-bit within multiplicative depth 11.
VALD-GAN: video anomaly detection using latent discriminator augmented GAN
Authors: Rituraj Singh, Anikeit Sethi, Krishanu Saini, Sumeet Saurav, Aruna Tiwari, Sanjay Singh
Date: 2023-10-18 00:00:00
Proceeding: Signal, Image and Video Processing 2023 10
Tasks: Anomaly Detection, Anomaly Detection In Surveillance Videos, Video Anomaly Detection

The most crucial and difficult challenge for intelligent video surveillance is to identify anomalies in a video that comprises anomalous behavior or occurrences. The ambiguous definition of the anomaly makes the detection of it a challenging task. Inspired by the wide adoption of generative adversarial networks (GANs), we proposed video anomaly detection using latent discriminator augmented GAN (VALD-GAN), which combines the representation power of GANs with a novel latent discriminator framework to make the latent space follow a pre-defined distribution. We show through our experimental results that the proposed method significantly increases the anomaly discrimination capability of the model. VALD-GAN achieves an AUC and EER score of 97.98, 6.0% on UCSD Peds1, 97.74, 7.01% on UCSD Peds2, and 91.03, 9.04% on CUHK Avenue dataset, respectively. Also, it is able to detect 62 out of a total of 66 anomalous events with 4 as false alarms and 19 out of a total of 19 with 1 false alarm from Subway Entrance and Exit video datasets, respectively.
A Toolkit for Reliable Benchmarking and Research in Multi-Objective Reinforcement Learning
Authors: Florian Felten, Lucas N. Alegre, Ann Nowé, Ana L. C. Bazzan, El-Ghazali Talbi, Grégoire Danoy, Bruno C. da Silva
Date: 2023-09-26 00:00:00
Proceeding: Conference on Neural Information Processing Systems Datasets and Benchmarks Track 2023 9
Tasks: Benchmarking, Multi-Objective Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Multi-objective reinforcement learning algorithms (MORL) extend standard reinforcement learning (RL) to scenarios where agents must optimize multiple---potentially conflicting---objectives, each represented by a distinct reward function. To facilitate and accelerate research and benchmarking in multi-objective RL problems, we introduce a comprehensive collection of software libraries that includes: (i) MO-Gymnasium, an easy-to-use and flexible API enabling the rapid construction of novel MORL environments. It also includes more than 20 environments under this API. This allows researchers to effortlessly evaluate any algorithms on any existing domains; (ii) MORL-Baselines, a collection of reliable and efficient implementations of state-of-the-art MORL algorithms, designed to provide a solid foundation for advancing research. Notably, all algorithms are inherently compatible with MO-Gymnasium; and (iii) a thorough and robust set of benchmark results and comparisons of MORL-Baselines algorithms, tested across various challenging MO-Gymnasium environments. These benchmarks were constructed to serve as guidelines for the research community, underscoring the properties, advantages, and limitations of each particular state-of-the-art method.
SPOT! Revisiting Video-Language Models for Event Understanding
Authors: Gengyuan Zhang, Jinhe Bi, Jindong Gu, Yanyu Chen, Volker Tresp
Date: 2023-11-21 00:00:00
Tasks: Attribute, Video Understanding

Understanding videos is an important research topic for multimodal learning. Leveraging large-scale datasets of web-crawled video-text pairs as weak supervision has become a pre-training paradigm for learning joint representations and showcased remarkable potential in video understanding tasks. However, videos can be multi-event and multi-grained, while these video-text pairs usually contain only broad-level video captions. This raises a question: with such weak supervision, can video representation in video-language models gain the ability to distinguish even factual discrepancies in textual description and understand fine-grained events? To address this, we introduce SPOT Prober, to benchmark existing video-language models's capacities of distinguishing event-level discrepancies as an indicator of models' event understanding ability. Our approach involves extracting events as tuples (<Subject, Predicate, Object, Attribute, Timestamps>) from videos and generating false event tuples by manipulating tuple components systematically. We reevaluate the existing video-language models with these positive and negative captions and find they fail to distinguish most of the manipulated events. Based on our findings, we propose to plug in these manipulated event captions as hard negative samples and find them effective in enhancing models for event understanding.
Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentation
Authors: Xiao Song, Jiafan Liu, Yun Li, Yan Liu, Wenbin Lei, Ruxin Wang
Date: 2023-11-22 00:00:00
Tasks: counterfactual, Sentence, Text Generation

Radiology Report Generation (RRG) draws attention as a vision-and-language interaction of biomedical fields. Previous works inherited the ideology of traditional language generation tasks, aiming to generate paragraphs with high readability as reports. Despite significant progress, the independence between diseases-a specific property of RRG-was neglected, yielding the models being confused by the co-occurrence of diseases brought on by the biased data distribution, thus generating inaccurate reports. In this paper, to rethink this issue, we first model the causal effects between the variables from a causal perspective, through which we prove that the co-occurrence relationships between diseases on the biased distribution function as confounders, confusing the accuracy through two backdoor paths, i.e. the Joint Vision Coupling and the Conditional Sequential Coupling. Then, we proposed a novel model-agnostic counterfactual augmentation method that contains two strategies, i.e. the Prototype-based Counterfactual Sample Synthesis (P-CSS) and the Magic-Cube-like Counterfactual Report Reconstruction (Cube), to intervene the backdoor paths, thus enhancing the accuracy and generalization of RRG models. Experimental results on the widely used MIMIC-CXR dataset demonstrate the effectiveness of our proposed method. Additionally, a generalization performance is evaluated on IU X-Ray dataset, which verifies our work can effectively reduce the impact of co-occurrences caused by different distributions on the results.
An Embodied Generalist Agent in 3D World
Authors: Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, Siyuan Huang
Date: 2023-11-18 00:00:00
Tasks: 3D dense captioning, 3D Question Answering (3D-QA), Question Answering, Robot Manipulation, Scene-Aware Dialogue, Vision-Language-Action, Vision-Language Navigation

Leveraging massive knowledge from large language models (LLMs), recent machine learning models show notable successes in general-purpose task solving in diverse domains such as computer vision and robotics. However, several significant challenges remain: (i) most of these models rely on 2D images yet exhibit a limited capacity for 3D input; (ii) these models rarely explore the tasks inherently defined in 3D world, e.g., 3D grounding, embodied reasoning and acting. We argue these limitations significantly hinder current models from performing real-world tasks and approaching general intelligence. To this end, we introduce LEO, an embodied multi-modal generalist agent that excels in perceiving, grounding, reasoning, planning, and acting in the 3D world. LEO is trained with a unified task interface, model architecture, and objective in two stages: (i) 3D vision-language (VL) alignment and (ii) 3D vision-language-action (VLA) instruction tuning. We collect large-scale datasets comprising diverse object-level and scene-level tasks, which require considerable understanding of and interaction with the 3D world. Moreover, we meticulously design an LLM-assisted pipeline to produce high-quality 3D VL data. Through extensive experiments, we demonstrate LEO's remarkable proficiency across a wide spectrum of tasks, including 3D captioning, question answering, embodied reasoning, navigation and manipulation. Our ablative studies and scaling analyses further provide valuable insights for developing future embodied generalist agents. Code and data are available on project page.
Adversarial sample generation and training using geometric masks for accurate and resilient license plate character recognition
Authors: Bishal Shrestha, Griwan Khakurel, Kritika Simkhada, Badri Adhikari
Date: 2023-10-25 00:00:00
Tasks: Adversarial Attack, License Plate Detection, License Plate Recognition

Reading dirty license plates accurately in moving vehicles is challenging for automatic license plate recognition systems. Moreover, license plates are often intentionally tampered with a malicious intent to avoid police apprehension. Usually, such groups and individuals know how to fool the existing recognition systems by making minor unnoticeable plate changes. Designing and developing deep learning methods resilient to such real-world 'attack' practices remains an active research problem. As a solution, this work develops a resilient method to recognize license plate characters. Extracting 1057 character images from 160 Nepalese vehicles, as the first step, we trained several standard deep convolutional neural networks to obtain 99.5% character classification accuracy. On adversarial images generated to simulate malicious tampering, however, our model's accuracy dropped to 25%. Next, we enriched our dataset by generating and adding geometrically masked images, retrained our models, and investigated the models' predictions. The proposed approach of training with generated adversarial images helped our adversarial attack-aware license plate character recognition (AA-LPCR) model achieves an accuracy of 99.7%. This near-perfect accuracy demonstrates that the proposed idea of random geometric masking is highly effective for improving the accuracy of license plate recognition models. Furthermore, by performing interpretability studies to understand why our models work, we identify and highlight attack-prone regions in the input character images. In sum, although Nepal's embossed license plate detection systems are vulnerable to malicious attacks, our findings suggest that these systems can be upgraded to close to 100% resilience.
Decrypting Nonlinearity: Koopman Interpretation and Analysis of Cryptosystems
Authors: Robin Strässer, Sebastian Schlor, Frank Allgöwer
Date: 2023-11-21 00:00:00

Public-key cryptosystems rely on computationally difficult problems for security, traditionally analyzed using number theory methods. In this paper, we introduce a novel perspective on cryptosystems by viewing the Diffie-Hellman key exchange and the Rivest-Shamir-Adleman cryptosystem as nonlinear dynamical systems. By applying Koopman theory, we transform these dynamical systems into higher-dimensional spaces and analytically derive equivalent purely linear systems. This formulation allows us to reconstruct the secret integers of the cryptosystems through straightforward manipulations, leveraging the tools available for linear systems analysis. Additionally, we establish an upper bound on the minimum lifting dimension required to achieve perfect accuracy. Our results on the required lifting dimension are in line with the intractability of brute-force attacks. To showcase the potential of our approach, we establish connections between our findings and existing results on algorithmic complexity. Furthermore, we extend this methodology to a data-driven context, where the Koopman representation is learned from data samples of the cryptosystems.
Adapting pretrained speech model for Mandarin lyrics transcription and alignment
Authors: Jun-You Wang, Chon-In Leong, Yu-Chen Lin, Li Su, Jyh-Shing Roger Jang
Date: 2023-11-21 00:00:00
Tasks: Automatic Lyrics Transcription, Data Augmentation

The tasks of automatic lyrics transcription and lyrics alignment have witnessed significant performance improvements in the past few years. However, most of the previous works only focus on English in which large-scale datasets are available. In this paper, we address lyrics transcription and alignment of polyphonic Mandarin pop music in a low-resource setting. To deal with the data scarcity issue, we adapt pretrained Whisper model and fine-tune it on a monophonic Mandarin singing dataset. With the use of data augmentation and source separation model, results show that the proposed method achieves a character error rate of less than 18% on a Mandarin polyphonic dataset for lyrics transcription, and a mean absolute error of 0.071 seconds for lyrics alignment. Our results demonstrate the potential of adapting a pretrained speech model for lyrics transcription and alignment in low-resource scenarios.
HPCNeuroNet: Advancing Neuromorphic Audio Signal Processing with Transformer-Enhanced Spiking Neural Networks
Authors: Murat Isik, Hiruna Vishwamith, Kayode Inadagbo, I. Can Dikmen
Date: 2023-11-21 00:00:00
Tasks: Audio Signal Processing, CPU, GPU

This paper presents a novel approach to neuromorphic audio processing by integrating the strengths of Spiking Neural Networks (SNNs), Transformers, and high-performance computing (HPC) into the HPCNeuroNet architecture. Utilizing the Intel N-DNS dataset, we demonstrate the system's capability to process diverse human vocal recordings across multiple languages and noise backgrounds. The core of our approach lies in the fusion of the temporal dynamics of SNNs with the attention mechanisms of Transformers, enabling the model to capture intricate audio patterns and relationships. Our architecture, HPCNeuroNet, employs the Short-Time Fourier Transform (STFT) for time-frequency representation, Transformer embeddings for dense vector generation, and SNN encoding/decoding mechanisms for spike train conversions. The system's performance is further enhanced by leveraging the computational capabilities of NVIDIA's GeForce RTX 3060 GPU and Intel's Core i9 12900H CPU. Additionally, we introduce a hardware implementation on the Xilinx VU37P HBM FPGA platform, optimizing for energy efficiency and real-time processing. The proposed accelerator achieves a throughput of 71.11 Giga-Operations Per Second (GOP/s) with a 3.55 W on-chip power consumption at 100 MHz. The comparison results with off-the-shelf devices and recent state-of-the-art implementations illustrate that the proposed accelerator has obvious advantages in terms of energy efficiency and design flexibility. Through design-space exploration, we provide insights into optimizing core capacities for audio tasks. Our findings underscore the transformative potential of integrating SNNs, Transformers, and HPC for neuromorphic audio processing, setting a new benchmark for future research and applications.
Formal Verification of Long Short-Term Memory based Audio Classifiers: A Star based Approach
Authors: Neelanjana Pal, Taylor T Johnson
Date: 2023-11-16 00:00:00
Tasks: Audio Classification, Classification, Management

Formally verifying audio classification systems is essential to ensure accurate signal classification across real-world applications like surveillance, automotive voice commands, and multimedia content management, preventing potential errors with serious consequences. Drawing from recent research, this study advances the utilization of star-set-based formal verification, extended through reachability analysis, tailored explicitly for Long Short-Term Memory architectures and their Convolutional variations within the audio classification domain. By conceptualizing the classification process as a sequence of set operations, the star set-based reachability approach streamlines the exploration of potential operational states attainable by the system. The paper serves as an encompassing case study, validating and verifying sequence audio classification analytics within real-world contexts. It accentuates the necessity for robustness verification to ensure precise and dependable predictions, particularly in light of the impact of noise on the accuracy of output classifications.
Estimating time of arrival of vehicle fleets with GCN based traffic prediction
Authors: Shivika Sharma, Nandini Mawane, Dhruthick Gowda M, Mayur Taware, Chetan Kumar, Yash Chandrashekhar Dixit, Rakshit Ramesh
Date: 2023-11-21 00:00:00
Tasks: Management, Position, Traffic Prediction, valid

This paper presents an effective framework for estimating time of arrival of vehicles (buses) in an Intelligent Transit Management System (ITMS) having sparse position updates. Our contributions towards this is firstly in implementing a constrained optimization based road linestring segmenting framework ensuring ideal segment lengths and segments with sufficient density of vehicle position measurements which will result in valid statistics for scenarios involving sparse position measurements. Over this we propose a comprehensive approach for predicting traffic delays and estimated time of vehicle arrival addressing both the spatial and temporal dependencies of traffic. The traffic delay model is built on top of the T-GCN architecture on which we optimally augment an adjacency matrix which models a complexly connected road network considering the degree of influence between road segments, enabling the traffic delay model to look beyond physical road connectivity in predicting traffic delays and therefore producing better estimates of arrival times to points along the designated route of the vehicles.
A Data-Driven Integrated Framework for Fast-Charging Facility Planning using Multi-Period Bi-Objective Optimization
Authors: Mingjia He, Panchamy Krishnakumari, Ding Luo, Jiaqi Chen
Date: 2023-11-21 00:00:00

With the electrification in freight transportation, the availability of fast-charging facilities becomes essential to facilitate en-route charging for freight electric vehicles. Most studies focus on planning charging facilities based on mathematical modeling and hypothetical scenarios. This study aims to develop a data-driven integrated framework for fast-charging facility planning. By leveraging the highway traffic data, we extracted, analyzed, and compared spatial and temporal flow patterns of general traffic and freight traffic. Furthermore, graph theory-based network evaluation methods are employed to identify traffic nodes within the highway network that play a significant role in accommodating charging infrastructure. A candidate selection method is proposed to obtain potential deployment locations for charging stations and to-go chargers. Based on this, we present a multi-period bi-objective optimization model to provide optimal solutions for the placement of charging facilities, with the objectives of minimizing investment cost and maximizing demand coverage. The case study on the Amsterdam highway network shows how existing traffic data can be used to generate more realistic charging demand scenarios and how it can be integrated and evaluated within the optimization framework for facility planning. The study also shows that the proposed model can leverage the potential of early investment in improving the charging demand coverage.
Data-driven System Interconnections and a Novel Data-enabled Internal Model Control
Authors: Yasaman Pedari, Jaeho Lee, Yongsoon Eun, Hamid Ossareh
Date: 2023-11-21 00:00:00
Tasks: LEMMA

Over the past two decades, there has been a growing interest in control systems research to transition from model-based methods to data-driven approaches. In this study, we aim to bridge a divide between conventional model-based control and emerging data-driven paradigms grounded in Willem's fundamental lemma. Specifically, we study how input/output data from two separate systems can be manipulated to represent the behavior of interconnected systems, either connected in series or through feedback. Using these results, this paper introduces the Internal Behavior Control (IBC), a new control strategy based on the well-known Internal Model Control (IMC) but viewed under the lens of Behavioral System Theory. Similar to IMC, the IBC is easy to tune and results in perfect tracking and disturbance rejection but, unlike IMC, does not require a parametric model of the dynamics. We present two approaches for IBC implementation: a component-by-component one and a unified one. We compare the two approaches in terms of filter design, computations, and memory requirements.
Convexifying Regulation Market Clearing of State-of-Charge Dependent Bid
Authors: Siying Li, Cong Chen, Lang Tong
Date: 2023-11-21 00:00:00
Tasks: Bilevel Optimization

We consider the problem of merchant storage participating in the regulation market with state-of-charge (SoC) dependent bids. Because storage can simultaneously provide regulation up and regulation down capacities, the market-clearing engine faces the computation challenge of evaluating storage costs under different regulation scenarios. One approach is to employ a bilevel optimization that minimizes the worst-case storage cost among all potential regulation events. However, subproblems of such a bilevel optimization are nonconvex, resulting in prohibitive computation challenges for the real-time clearing of the regulation market. We show that the complex nonconvex market clearing problem can be convexified by a simple restriction on the SoC-dependent bid, rendering the intractable market clearing computation to standard linear programs. Numerical simulations demonstrate that SoC-dependent bids satisfying the convexification conditions increase the profits of merchant storage owners by 12.32-77.38% compared with SoC-independent bids.