pwc-archive/papers-with-abstracts huggingface dataset:
A Hybrid CDG-CSF Based Approach for Enhancing Lifetime of Wireless Sensor Network
Authors: Ghodke Vijay Kumar, Parth Sharma, Pyari Mohan Pradhan
Date: 2024-03-05 00:00:00
Proceeding: 7th International Conference on Computer Applications in Electrical Engineering-Recent Advances (CERA) 2024 3
Tasks: compressed sensing

A wireless sensor network (WSN) is constrained by the energy of sensors, communication distance, limited resources, etc. Increasing the lifetime of WSNs with optimal energy consumption methods is a challenging problem. Two essential techniques to increase the lifetime of a WSN are efficient data acquisition and forwarding based on compressed sensing and optimal sensor activation. The compressed sensing (CS) technique provides a new solution for gathering data efficiently. By exploiting the compressibility of signals, they are efficiently reconstructed from fewer samples. This article analyzes the performance of Compressive Data Gathering (CDG) and Compressive Sparse Function (CSF) with practical data in the presence of abnormalities. Thereafter, it proposes a hybrid CDGCSF-based approach that increases the lifetime of the WSN using modified sensing matrices. The performance of the proposed hybrid approach is found to be better than that of individual schemes in the presence of abnormalities in the data.
Robust Adaptive System Identification and Noise Filtering Using Bregman Divergence Based Algorithms
Authors: Parth Sharma, Pyari Mohan Pradhan
Date: 2024-01-01 00:00:00
Proceeding: International Conference on Electrical, Electronics, Communication and Computers (ELEXCOM) 2024 1

This article proffers a new set of least mean square (LMS) algorithms based on the Bregman divergence class of information-theoretic divergence, which are named as Kullback-Leibler LMS (KLLMS), Itakura-Saito LMS (ISLMS), and ßLMS. To achieve a tractable analysis, a bound on the learning rate of proffered algorithms is derived. Steady-state mean-square deviation (MSD) and computational cost provide insight into the performance and efficiency of the proffered algorithms in the presence of Gaussian and impulsive noise. Simulation results also show the effectiveness of the proffered algorithms in adaptive noise filtering.
Bregman Divergence Based Approach for Adaptive System Identification and Line Enhancement
Authors: Parth Sharma, Pyari Mohan Pradhan
Date: 2024-03-25 00:00:00
Proceeding: Third International Conference on Power, Control and Computing Technologies (ICPC2T) 2024 3

This article puts forth a novel class of least mean square (LMS) techniques, including beta divergence-inspired LMS (BLMS), Itakura-Saito divergence-inspired LMS (ISBLMS), and Kullback-Leibler divergence-inspired LMS (KLLMS). The mentioned divergence measures are part of the Bregman divergence category of information-theoretic divergence. Mean and mean-square analysis for the proffered class of algorithms is derived to find the bound on the learning-rate for stable convergence. The effectiveness of the introduced class of algorithms is showcased for the application of time-varying system identification and adaptive line enhancement.
Development of Amari Alpha Divergence-Based Gradient-Descent Least Mean Square Algorithm
Authors: Parth Sharma, Pyari Mohan Pradhan
Date: 2023-03-15 00:00:00
Proceeding: IEEE Transactions on Circuits and Systems II: Express Briefs 2023 3

In this brief, a new least mean square (LMS) adaptive filtering algorithm based on Amari-Alpha information theoretic divergence is proposed which is named as Amari-Alpha LMS (AALMS). The local convergence and stability analysis show the upper bound of step size for tractable analysis. The steady-state performance of the proposed algorithm is analyzed, and the mean-square deviation (MSD) at steady-state is derived. Error-in-variable (EIV) model where input and desired signals both are corrupted with Gaussian noise is considered in this brief. Simulation results and computational complexity analysis show that the proposed AALMS algorithm performs better in comparison to well-known algorithms in terms of MSD in stationary and non-stationary scenarios. Comparison of different distance measures that can be derived from Amari-Alpha divergence is also carried out in terms of the MSD.
A Novel Family of Robust Incremental Adaptive Algorithms for Distributed Estimation Based on Bregman Divergence
Authors: Parth Sharma, Pyari Mohan Pradhan
Date: 2023-07-17 00:00:00
Proceeding: IEEE Sensors Letters 2023 7

Distributed estimation in the wireless sensor network (WSN) faces severe challenges when outliers are present in the desired data. Conventional estimation techniques such as the block-least-mean-square (BLMS) algorithm have been designed to perform well in the presence of Gaussian noise. However, the performance of these conventional techniques degrades significantly in the presence of outliers in the desired data. In this letter, a novel family of incremental distributed estimation techniques for WSN based on the Bregman divergence measure is proposed, which are named as incremental Kullback–Leibler-divergence-based BLMS algorithm, incremental Itakura–Saito-divergence-based BLMS algorithm, incremental Amari-alpha-divergence-based BLMS algorithm, and incremental beta-divergence-based BLMS algorithm. The local stability analysis shows the upper bound of the step size for the tractable analysis. Simulation results show that the proposed Bregman-divergence-based incremental algorithms perform better compared to the incremental BLMS algorithm in terms of the mean-square deviation.
Modelling Behaviour of Sensors using a Novel β-divergence based Adaptive Filter
Authors: Parth Sharma, Pyari Mohan Pradhan
Date: 2024-08-29 00:00:00
Proceeding: IEEE Sensors Journal 2024 8
Tasks: Computational Efficiency, Event Detection

Sensors are widely used in many applications, acting as a link between the physical and digital domains. However, the sensor output is often affected by external factors such as noise, interference, drifts, and non-linearities, making it challenging for accurate signal and event detection. To address this issue, a novel approach to model a sensor using an adaptive filter based on the β-divergence inspired cost function is proposed to make the sensor robust against changing environmental conditions over time. The proposed β-divergence based least mean square (βLMS) algorithm can effectively analyze and compensate for external influences in sensor output, leading to a more dependable and precise representation of sensor behavior while maintaining computational efficiency. Additionally, the mean and mean-square stability bounds are determined to establish the acceptable range of learning rates for the βLMS algorithm. Furthermore, this paper examines the transient, steady-state, tracking performance, and robustness of the proposed βLMS algorithm. Moreover, the performance of the proposed βLMS algorithm is assessed through simulation study, demonstrating its efficacy in various real-world applications of sensors such as system identification, missile tracking, and active noise cancellation. The hardware implementation is also carried out to validate the performance of the proposed βLMS algorithm in real-time scenario to model high-cost sensor behavior using low-cost sensors. The results indicate that the proposed βLMS algorithm significantly enhances the performance of the sensors in practical applications.
Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms
Authors: Melkamu Abay Mersha, Mesay Gemeda Yigezu, Jugal Kalita
Date: 2024-09-30 00:00:00
Tasks: Clustering, Semantic Similarity, Semantic Textual Similarity

Topic modeling is a powerful technique to discover hidden topics and patterns within a collection of documents without prior knowledge. Traditional topic modeling and clustering-based techniques encounter challenges in capturing contextual semantic information. This study introduces an innovative end-to-end semantic-driven topic modeling technique for the topic extraction process, utilizing advanced word and document embeddings combined with a powerful clustering algorithm. This semantic-driven approach represents a significant advancement in topic modeling methodologies. It leverages contextual semantic information to extract coherent and meaningful topics. Specifically, our model generates document embeddings using pre-trained transformer-based language models, reduces the dimensions of the embeddings, clusters the embeddings based on semantic similarity, and generates coherent topics for each cluster. Compared to ChatGPT and traditional topic modeling algorithms, our model provides more coherent and meaningful topics.
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models
Authors: Seanie Lee, Haebin Seong, Dong Bok Lee, Minki Kang, Xiaoyin Chen, Dominik Wagner, Yoshua Bengio, Juho Lee, Sung Ju Hwang
Date: 2024-10-02 00:00:00
Tasks: Data Augmentation, Knowledge Distillation

Safety guard models that detect malicious queries aimed at large language models (LLMs) are essential for ensuring the secure and responsible deployment of LLMs in real-world applications. However, deploying existing safety guard models with billions of parameters alongside LLMs on mobile devices is impractical due to substantial memory requirements and latency. To reduce this cost, we distill a large teacher safety guard model into a smaller one using a labeled dataset of instruction-response pairs with binary harmfulness labels. Due to the limited diversity of harmful instructions in the existing labeled dataset, naively distilled models tend to underperform compared to larger models. To bridge the gap between small and large models, we propose HarmAug, a simple yet effective data augmentation method that involves jailbreaking an LLM and prompting it to generate harmful instructions. Given a prompt such as, "Make a single harmful instruction prompt that would elicit offensive content", we add an affirmative prefix (e.g., "I have an idea for a prompt:") to the LLM's response. This encourages the LLM to continue generating the rest of the response, leading to sampling harmful instructions. Another LLM generates a response to the harmful instruction, and the teacher model labels the instruction-response pair. We empirically show that our HarmAug outperforms other relevant baselines. Moreover, a 435-million-parameter safety guard model trained with HarmAug achieves an F1 score comparable to larger models with over 7 billion parameters, and even outperforms them in AUPRC, while operating at less than 25% of their computational cost.
Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion?
Authors: Zhenyu Pan, Rongyu Cao, Yongchang Cao, Yingwei Ma, Binhua Li, Fei Huang, Han Liu, Yongbin Li
Date: 2024-10-02 00:00:00
Tasks: Code Completion, Code Generation

Code completion, a key downstream task in code generation, is one of the most frequent and impactful methods for enhancing developer productivity in software development. As intelligent completion tools evolve, we need a robust evaluation benchmark that enables meaningful comparisons between products and guides future advancements. However, existing benchmarks focus more on coarse-grained tasks without industrial analysis resembling general code generation rather than the real-world scenarios developers encounter. Moreover, these benchmarks often rely on costly and time-consuming human annotation, and the standalone test cases fail to leverage minimal tests for maximum repository-level understanding and code coverage. To address these limitations, we first analyze business data from an industrial code completion tool and redefine the evaluation criteria to better align with the developer's intent and desired completion behavior throughout the coding process. Based on these insights, we introduce Codev-Agent, an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage, ensuring fair and effective comparisons. Using Codev-Agent, we present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Bench assesses whether a code completion tool can capture a developer's immediate intent and suggest appropriate code across diverse contexts, providing a more realistic benchmark for code completion in modern software development.
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding
Authors: Kevin Xu, Issei Sato
Date: 2024-10-02 00:00:00

Looped Transformers provide advantages in parameter efficiency, computational capabilities, and generalization for reasoning tasks. However, their expressive power regarding function approximation remains underexplored. In this paper, we establish the approximation rate of Looped Transformers by defining the modulus of continuity for sequence-to-sequence functions. This reveals a limitation specific to the looped architecture. That is, the analysis prompts the incorporation of scaling parameters for each loop, conditioned on timestep encoding. Experiments validate the theoretical results, showing that increasing the number of loops enhances performance, with further gains achieved through the timestep encoding.
KnobGen: Controlling the Sophistication of Artwork in Sketch-Based Diffusion Models
Authors: Pouyan Navard, Amin Karimi Monsefi, Mengxi Zhou, Wei-Lun Chao, Alper Yilmaz, Rajiv Ramnath
Date: 2024-10-02 00:00:00
Tasks: Image Generation

Recent advances in diffusion models have significantly improved text-to-image (T2I) generation, but they often struggle to balance fine-grained precision with high-level control. Methods like ControlNet and T2I-Adapter excel at following sketches by seasoned artists but tend to be overly rigid, replicating unintentional flaws in sketches from novice users. Meanwhile, coarse-grained methods, such as sketch-based abstraction frameworks, offer more accessible input handling but lack the precise control needed for detailed, professional use. To address these limitations, we propose KnobGen, a dual-pathway framework that democratizes sketch-based image generation by seamlessly adapting to varying levels of sketch complexity and user skill. KnobGen uses a Coarse-Grained Controller (CGC) module for high-level semantics and a Fine-Grained Controller (FGC) module for detailed refinement. The relative strength of these two modules can be adjusted through our knob inference mechanism to align with the user's specific needs. These mechanisms ensure that KnobGen can flexibly generate images from both novice sketches and those drawn by seasoned artists. This maintains control over the final output while preserving the natural appearance of the image, as evidenced on the MultiGen-20M dataset and a newly collected sketch dataset.
MetaFormer and CNN Hybrid Model for Polyp Image Segmentation
Authors: Hyunnam Lee, Joohan Yoo
Date: 2024-09-16 00:00:00
Proceeding: IEEE Access 2024 9
Tasks: Benchmarking, Image Segmentation, Medical Image Segmentation, Semantic Segmentation

Transformer-based methods have become dominant in the medical image research field since the Vision Transformer achieved superior performance. Although transformer-based approaches have resolved long-range dependency problems inherent in Convolutional Neural Network (CNN) methods, they struggle to capture local detail information. Recent research focuses on the robust combination of local detail and semantic information. To address this problem, we propose a novel transformer-CNN hybrid network named RAPUNet. The proposed approach employs MetaFormer as the transformer backbone and introduces a custom convolutional block, RAPU (Residual and Atrous Convolution in Parallel Unit), to enhance local features and alleviate the combination problem of local and global features. We evaluate the segmentation performance of RAPUNet on popular benchmarking datasets for polyp segmentation, including Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, EndoScene-CVC300, and ETIS-LaribPolypDB. Experimental results show that our model achieves competitive performance in terms of mean Dice and mean IoU. Particularly, RAPUNet outperforms state-of-the-art methods on the CVC-ClinicDB dataset. Code available: https://github.com/hyunnamlee/RAPUNet .
Gaussian-Det: Learning Closed-Surface Gaussians for 3D Object Detection
Authors: Hongru Yan, Yu Zheng, Yueqi Duan
Date: 2024-10-02 00:00:00
Tasks: 3D Object Detection, NeRF, object-detection, Object Detection

Skins wrapping around our bodies, leathers covering over the sofa, sheet metal coating the car - it suggests that objects are enclosed by a series of continuous surfaces, which provides us with informative geometry prior for objectness deduction. In this paper, we propose Gaussian-Det which leverages Gaussian Splatting as surface representation for multi-view based 3D object detection. Unlike existing monocular or NeRF-based methods which depict the objects via discrete positional data, Gaussian-Det models the objects in a continuous manner by formulating the input Gaussians as feature descriptors on a mass of partial surfaces. Furthermore, to address the numerous outliers inherently introduced by Gaussian splatting, we accordingly devise a Closure Inferring Module (CIM) for the comprehensive surface-based objectness deduction. CIM firstly estimates the probabilistic feature residuals for partial surfaces given the underdetermined nature of Gaussian Splatting, which are then coalesced into a holistic representation on the overall surface closure of the object proposal. In this way, the surface information Gaussian-Det exploits serves as the prior on the quality and reliability of objectness and the information basis of proposal refinement. Experiments on both synthetic and real-world datasets demonstrate that Gaussian-Det outperforms various existing approaches, in terms of both average precision and recall.
Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking
Authors: Mattia Segu, Luigi Piccinelli, Siyuan Li, Yung-Hsu Yang, Bernt Schiele, Luc van Gool
Date: 2024-10-02 00:00:00
Tasks: Multiple Object Tracking, Object Tracking

Multiple object tracking in complex scenarios - such as coordinated dance performances, team sports, or dynamic animal groups - presents unique challenges. In these settings, objects frequently move in coordinated patterns, occlude each other, and exhibit long-term dependencies in their trajectories. However, it remains a key open research question on how to model long-range dependencies within tracklets, interdependencies among tracklets, and the associated temporal occlusions. To this end, we introduce Samba, a novel linear-time set-of-sequences model designed to jointly process multiple tracklets by synchronizing the multiple selective state-spaces used to model each tracklet. Samba autoregressively predicts the future track query for each sequence while maintaining synchronized long-term memory representations across tracklets. By integrating Samba into a tracking-by-propagation framework, we propose SambaMOTR, the first tracker effectively addressing the aforementioned issues, including long-range dependencies, tracklet interdependencies, and temporal occlusions. Additionally, we introduce an effective technique for dealing with uncertain observations (MaskObs) and an efficient training recipe to scale SambaMOTR to longer sequences. By modeling long-range dependencies and interactions among tracked objects, SambaMOTR implicitly learns to track objects accurately through occlusions without any hand-crafted heuristics. Our approach significantly surpasses prior state-of-the-art on the DanceTrack, BFT, and SportsMOT datasets.
FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images
Authors: Cheng Zhang, Yuanhao Wang, Francisco Vicente Carrasco, Chenglei Wu, Jinlong Yang, Thabo Beeler, Fernando de la Torre
Date: 2024-10-02 00:00:00
Tasks: Denoising, Image to 3D

We introduce FabricDiffusion, a method for transferring fabric textures from a single clothing image to 3D garments of arbitrary shapes. Existing approaches typically synthesize textures on the garment surface through 2D-to-3D texture mapping or depth-aware inpainting via generative models. Unfortunately, these methods often struggle to capture and preserve texture details, particularly due to challenging occlusions, distortions, or poses in the input image. Inspired by the observation that in the fashion industry, most garments are constructed by stitching sewing patterns with flat, repeatable textures, we cast the task of clothing texture transfer as extracting distortion-free, tileable texture materials that are subsequently mapped onto the UV space of the garment. Building upon this insight, we train a denoising diffusion model with a large-scale synthetic dataset to rectify distortions in the input texture image. This process yields a flat texture map that enables a tight coupling with existing Physically-Based Rendering (PBR) material generation pipelines, allowing for realistic relighting of the garment under various lighting conditions. We show that FabricDiffusion can transfer various features from a single clothing image including texture patterns, material properties, and detailed prints and logos. Extensive experiments demonstrate that our model significantly outperforms state-to-the-art methods on both synthetic data and real-world, in-the-wild clothing images while generalizing to unseen textures and garment shapes.
Windowed MAPF with Completeness Guarantees
Authors: Rishi Veerapaneni, Muhammad Suhail Saleem, Jiaoyang Li, Maxim Likhachev
Date: 2024-10-02 00:00:00
Tasks: Heuristic Search, Multi-Agent Path Finding

Traditional multi-agent path finding (MAPF) methods try to compute entire start-goal paths which are collision free. However, computing an entire path can take too long for MAPF systems where agents need to replan fast. Methods that address this typically employ a "windowed" approach and only try to find collision free paths for a small windowed timestep horizon. This adaptation comes at the cost of incompleteness; all current windowed approaches can become stuck in deadlock or livelock. Our main contribution is to introduce our framework, WinC-MAPF, for Windowed MAPF that enables completeness. Our framework uses heuristic update insights from single-agent real-time heuristic search algorithms as well as agent independence ideas from MAPF algorithms. We also develop Single-Step CBS (SS-CBS), an instantiation of this framework using a novel modification to CBS. We show how SS-CBS, which only plans a single step and updates heuristics, can effectively solve tough scenarios where existing windowed approaches fail.
When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1
Authors: R. Thomas McCoy, Shunyu Yao, Dan Friedman, Mathew D. Hardy, Thomas L. Griffiths
Date: 2024-10-02 00:00:00
Tasks: Language Modeling, Language Modelling

In "Embers of Autoregression" (McCoy et al., 2023), we showed that several large language models (LLMs) have some important limitations that are attributable to their origins in next-word prediction. Here we investigate whether these issues persist with o1, a new system from OpenAI that differs from previous LLMs in that it is optimized for reasoning. We find that o1 substantially outperforms previous LLMs in many cases, with particularly large improvements on rare variants of common tasks (e.g., forming acronyms from the second letter of each word in a list, rather than the first letter). Despite these quantitative improvements, however, o1 still displays the same qualitative trends that we observed in previous systems. Specifically, o1 -- like previous LLMs -- is sensitive to the probability of examples and tasks, performing better and requiring fewer "thinking tokens" in high-probability settings than in low-probability ones. These results show that optimizing a language model for reasoning can mitigate but might not fully overcome the language model's probability sensitivity.