OpenRaiser/Intern-Atlas huggingface dataset:
A prony method variant which surpasses the Adaptive LMS filter in the output signal's representation of input
Venue: arXiv cs.MS 2024
Year: 2024

The Prony method for approximating signals comprising sinusoidal/exponential components is known through the pioneering work of Prony in his seminal dissertation in the year 1795. However, the Prony method saw the light of real world application only upon the advent of the computational era, which made feasible the extensive numerical intricacies and labor which the method demands inherently. The Adaptive LMS Filter which has been the most pervasive method for signal filtration and approximation since its inception in 1965 does not provide a consistently assured level of highly precise results as the extended experiment in this work proves. As a remedy this study improvises upon the Prony method by observing that a better (more precise) computational approximation can be obtained under the premise that adjustment can be made for computational error , in the autoregressive model setup in the initial step of the Prony computation itself. This adjustment is in proportion to the deviation of the coefficients in the same autoregressive model. The results obtained by this improvisation live up to the expectations of obtaining consistency and higher value in the precision of the output (recovered signal) approximations as shown in this current work and as compared with the results obtained using the Adaptive LMS Filter.
DAVIDE: Depth-Aware Video Deblurring
Venue: arXiv cs.CV 2024
Year: 2024

Video deblurring aims at recovering sharp details from a sequence of blurry frames. Despite the proliferation of depth sensors in mobile phones and the potential of depth information to guide deblurring, depth-aware deblurring has received only limited attention. In this work, we introduce the 'Depth-Aware VIdeo DEblurring' (DAVIDE) dataset to study the impact of depth information in video deblurring. The dataset comprises synchronized blurred, sharp, and depth videos. We investigate how the depth information should be injected into the existing deep RGB video deblurring models, and propose a strong baseline for depth-aware video deblurring. Our findings reveal the significance of depth information in video deblurring and provide insights into the use cases where depth cues are beneficial. In addition, our results demonstrate that while the depth improves deblurring performance, this effect diminishes when models are provided with a longer temporal context. Project page: https://germanftv.github.io/DAVIDE.github.io/ .
A Systematic Review of Business Process Improvement: Achievements and Potentials in Combining Concepts from Operations Research and Business Process Management
Venue: arXiv cs.SE 2024
Year: 2024

Business Process Management and Operations Research are two research fields that both aim to enhance value creation in organizations. While Business Process Management has historically emphasized on providing precise models, Operations Research has focused on constructing tractable models and their solutions. This systematic literature review identifies and analyzes work that uses combined concepts from both disciplines. In particular, it analyzes how business process models have been conceptualized as mathematical models and which optimization techniques have been applied to these models. Results indicate a strong focus on resource allocation and scheduling problems. Current approaches often lack support of the stochastic nature of many problems, and do only sparsely use information from process models or from event logs, such as resource-related information or information from the data perspective.
Comprehensive up-to-date impact of the IoMT in healthcare and patients
Venue: arXiv cs.CY 2024
Year: 2024

The Internet of Medical Things (IoMT) is a quickly expanding field that intends to develop the features, effectiveness, and availability of healthcare services by applying numerous technologies to gather and diffuse medical data. IoMT devices incorporate wearable sensors, implantable devices, smart home methods, telemedicine policies, and mobile applications. IoMT applications range from chronic disease administration, remote patient monitoring, emergency response, and clinical decision support to health promotion and wellness. This paper aligns on the advantages, defies, and outlook directions of this developing domain. The paper also examines the ethical, legal, and social implications of IoMT, as well as the possible risks and vulnerabilities of the IoMT environment
Disentangling Mean Embeddings for Better Diagnostics of Image Generators
Venue: arXiv cs.CV 2024
Year: 2024

The evaluation of image generators remains a challenge due to the limitations of traditional metrics in providing nuanced insights into specific image regions. This is a critical problem as not all regions of an image may be learned with similar ease. In this work, we propose a novel approach to disentangle the cosine similarity of mean embeddings into the product of cosine similarities for individual pixel clusters via central kernel alignment. Consequently, we can quantify the contribution of the cluster-wise performance to the overall image generation performance. We demonstrate how this enhances the explainability and the likelihood of identifying pixel regions of model misbehavior across various real-world use cases.
Membership Inference Attacks Against In-Context Learning
Venue: arXiv cs.CR 2024
Year: 2024

Adapting Large Language Models (LLMs) to specific tasks introduces concerns about computational efficiency, prompting an exploration of efficient methods such as In-Context Learning (ICL). However, the vulnerability of ICL to privacy attacks under realistic assumptions remains largely unexplored. In this work, we present the first membership inference attack tailored for ICL, relying solely on generated texts without their associated probabilities. We propose four attack strategies tailored to various constrained scenarios and conduct extensive experiments on four popular large language models. Empirical results show that our attacks can accurately determine membership status in most cases, e.g., 95\% accuracy advantage against LLaMA, indicating that the associated risks are much higher than those shown by existing probability-based attacks. Additionally, we propose a hybrid attack that synthesizes the strengths of the aforementioned strategies, achieving an accuracy advantage of over 95\% in most cases. Furthermore, we investigate three potential defenses targeting data, instruction, and output. Results demonstrate combining defenses from orthogonal dimensions significantly reduces privacy leakage and offers enhanced privacy assurances.
VLSI Hypergraph Partitioning with Deep Learning
Venue: arXiv cs.AR 2024
Year: 2024

Partitioning is a known problem in computer science and is critical in chip design workflows, as advancements in this area can significantly influence design quality and efficiency. Deep Learning (DL) techniques, particularly those involving Graph Neural Networks (GNNs), have demonstrated strong performance in various node, edge, and graph prediction tasks using both inductive and transductive learning methods. A notable area of recent interest within GNNs are pooling layers and their application to graph partitioning. While these methods have yielded promising results across social, computational, and other random graphs, their effectiveness has not yet been explored in the context of VLSI hypergraph netlists. In this study, we introduce a new set of synthetic partitioning benchmarks that emulate real-world netlist characteristics and possess a known upper bound for solution cut quality. We distinguish these benchmarks with the prior work and evaluate existing state-of-the-art partitioning algorithms alongside GNN-based approaches, highlighting their respective advantages and disadvantages.
DiffCSG: Differentiable CSG via Rasterization
Venue: arXiv cs.GR 2024
Year: 2024

Differentiable rendering is a key ingredient for inverse rendering and machine learning, as it allows to optimize scene parameters (shape, materials, lighting) to best fit target images. Differentiable rendering requires that each scene parameter relates to pixel values through differentiable operations. While 3D mesh rendering algorithms have been implemented in a differentiable way, these algorithms do not directly extend to Constructive-Solid-Geometry (CSG), a popular parametric representation of shapes, because the underlying boolean operations are typically performed with complex black-box mesh-processing libraries. We present an algorithm, DiffCSG, to render CSG models in a differentiable manner. Our algorithm builds upon CSG rasterization, which displays the result of boolean operations between primitives without explicitly computing the resulting mesh and, as such, bypasses black-box mesh processing. We describe how to implement CSG rasterization within a differentiable rendering pipeline, taking special care to apply antialiasing along primitive intersections to obtain gradients in such critical areas. Our algorithm is simple and fast, can be easily incorporated into modern machine learning setups, and enables a range of applications for computer-aided design, including direct and image-based editing of CSG primitives. Code and data: https://yyyyyhc.github.io/DiffCSG/.
Last-Iterate Convergence of Payoff-Based Independent Learning in Zero-Sum Stochastic Games
Venue: arXiv cs.LG 2024
Year: 2024

In this paper, we consider two-player zero-sum matrix and stochastic games and develop learning dynamics that are payoff-based, convergent, rational, and symmetric between the two players. Specifically, the learning dynamics for matrix games are based on the smoothed best-response dynamics, while the learning dynamics for stochastic games build upon those for matrix games, with additional incorporation of the minimax value iteration. To our knowledge, our theoretical results present the first finite-sample analysis of such learning dynamics with last-iterate guarantees. In the matrix game setting, the results imply a sample complexity of $O(ε^{-1})$ to find the Nash distribution and a sample complexity of $O(ε^{-8})$ to find a Nash equilibrium. In the stochastic game setting, the results also imply a sample complexity of $O(ε^{-8})$ to find a Nash equilibrium. To establish these results, the main challenge is to handle stochastic approximation algorithms with multiple sets of coupled and stochastic iterates that evolve on (possibly) different time scales. To overcome this challenge, we developed a coupled Lyapunov-based approach, which may be of independent interest to the broader community studying the convergence behavior of stochastic approximation algorithms.
Swarm Systems as a Platform for Open-Ended Evolutionary Dynamics
Venue: arXiv cs.NE 2024
Year: 2024

Artificial swarm systems have been extensively studied and used in computer science, robotics, engineering and other technological fields, primarily as a platform for implementing robust distributed systems to achieve pre-defined objectives. However, such swarm systems, especially heterogeneous ones, can also be utilized as an ideal platform for creating *open-ended evolutionary dynamics* that do not converge toward pre-defined goals but keep exploring diverse possibilities and generating novel outputs indefinitely. In this article, we review Swarm Chemistry and its variants as concrete sample cases to illustrate beneficial characteristics of heterogeneous swarm systems, including the cardinality leap of design spaces, multiscale structures/behaviors and their diversity, and robust self-organization, self-repair and ecological interactions of emergent patterns, all of which serve as the driving forces for open-ended evolutionary processes. Applications to science, engineering, and art/entertainment as well as the directions of further research are also discussed.
Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions
Venue: arXiv cs.CV 2024
Year: 2024

The visible-light camera, which is capable of environment perception and navigation assistance, has emerged as an essential imaging sensor for marine surface vessels in intelligent waterborne transportation systems (IWTS). However, the visual imaging quality inevitably suffers from several kinds of degradations (e.g., limited visibility, low contrast, color distortion, etc.) under complex weather conditions (e.g., haze, rain, and low-lightness). The degraded visual information will accordingly result in inaccurate environment perception and delayed operations for navigational risk. To promote the navigational safety of vessels, many computational methods have been presented to perform visual quality enhancement under poor weather conditions. However, most of these methods are essentially specific-purpose implementation strategies, only available for one specific weather type. To overcome this limitation, we propose to develop a general-purpose multi-scene visibility enhancement method, i.e., edge reparameterization- and attention-guided neural network (ERANet), to adaptively restore the degraded images captured under different weather conditions. In particular, our ERANet simultaneously exploits the channel attention, spatial attention, and reparameterization technology to enhance the visual quality while maintaining low computational cost. Extensive experiments conducted on standard and IWTS-related datasets have demonstrated that our ERANet could outperform several representative visibility enhancement methods in terms of both imaging quality and computational efficiency. The superior performance of IWTS-related object detection and scene segmentation could also be steadily obtained after ERANet-based visibility enhancement under complex weather conditions.
Improving Robustness of Spectrogram Classifiers with Neural Stochastic Differential Equations
Venue: arXiv cs.LG 2024
Year: 2024

Signal analysis and classification is fraught with high levels of noise and perturbation. Computer-vision-based deep learning models applied to spectrograms have proven useful in the field of signal classification and detection; however, these methods aren't designed to handle the low signal-to-noise ratios inherent within non-vision signal processing tasks. While they are powerful, they are currently not the method of choice in the inherently noisy and dynamic critical infrastructure domain, such as smart-grid sensing, anomaly detection, and non-intrusive load monitoring.
EA-RAS: Towards Efficient and Accurate End-to-End Reconstruction of Anatomical Skeleton
Venue: arXiv cs.CV 2024
Year: 2024

Efficient, accurate and low-cost estimation of human skeletal information is crucial for a range of applications such as biology education and human-computer interaction. However, current simple skeleton models, which are typically based on 2D-3D joint points, fall short in terms of anatomical fidelity, restricting their utility in fields. On the other hand, more complex models while anatomically precise, are hindered by sophisticate multi-stage processing and the need for extra data like skin meshes, making them unsuitable for real-time applications. To this end, we propose the EA-RAS (Towards Efficient and Accurate End-to-End Reconstruction of Anatomical Skeleton), a single-stage, lightweight, and plug-and-play anatomical skeleton estimator that can provide real-time, accurate anatomically realistic skeletons with arbitrary pose using only a single RGB image input. Additionally, EA-RAS estimates the conventional human-mesh model explicitly, which not only enhances the functionality but also leverages the outside skin information by integrating features into the inside skeleton modeling process. In this work, we also develop a progressive training strategy and integrated it with an enhanced optimization process, enabling the network to obtain initial weights using only a small skin dataset and achieve self-supervision in skeleton reconstruction. Besides, we also provide an optional lightweight post-processing optimization strategy to further improve accuracy for scenarios that prioritize precision over real-time processing. The experiments demonstrated that our regression method is over 800 times faster than existing methods, meeting real-time requirements. Additionally, the post-processing optimization strategy provided can enhance reconstruction accuracy by over 50% and achieve a speed increase of more than 7 times.
Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models
Venue: arXiv cs.CV 2024
Year: 2024

Categorization, a core cognitive ability in humans that organizes objects based on common features, is essential to cognitive science as well as computer vision. To evaluate the categorization ability of visual AI models, various proxy tasks on recognition from datasets to open world scenarios have been proposed. Recent development of Large Multimodal Models (LMMs) has demonstrated impressive results in high-level visual tasks, such as visual question answering, video temporal reasoning, etc., utilizing the advanced architectures and large-scale multimodal instruction tuning. Previous researchers have developed holistic benchmarks to measure the high-level visual capability of LMMs, but there is still a lack of pure and in-depth quantitative evaluation of the most fundamental categorization ability. According to the research on human cognitive process, categorization can be seen as including two parts: category learning and category use. Inspired by this, we propose a novel, challenging, and efficient benchmark based on composite blocks, called ComBo, which provides a disentangled evaluation framework and covers the entire categorization process from learning to use. By analyzing the results of multiple evaluation tasks, we find that although LMMs exhibit acceptable generalization ability in learning new categories, there are still gaps compared to humans in many ways, such as fine-grained perception of spatial relationship and abstract category understanding. Through the study of categorization, we can provide inspiration for the further development of LMMs in terms of interpretability and generalization.
ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition
Venue: arXiv cs.CV 2024
Year: 2024

Spiking Neural Networks (SNNs) have emerged as a compelling, energy-efficient alternative to traditional Artificial Neural Networks (ANNs) for static image tasks such as image classification and segmentation. However, in the more complex video classification domain, SNN-based methods fall considerably short of ANN-based benchmarks due to the challenges in processing dense frame sequences. To bridge this gap, we propose ReSpike, a hybrid framework that synergizes the strengths of ANNs and SNNs to tackle action recognition tasks with high accuracy and low energy cost. By decomposing film clips into spatial and temporal components, i.e., RGB image Key Frames and event-like Residual Frames, ReSpike leverages ANN for learning spatial information and SNN for learning temporal information. In addition, we propose a multi-scale cross-attention mechanism for effective feature fusion. Compared to state-of-the-art SNN baselines, our ReSpike hybrid architecture demonstrates significant performance improvements (e.g., >30% absolute accuracy improvement on HMDB-51, UCF-101, and Kinetics-400). Furthermore, ReSpike achieves comparable performance with prior ANN approaches while bringing better accuracy-energy tradeoff.
Exploring Hannan Limitation for 3D Antenna Array
Venue: arXiv cs.IT 2024
Year: 2024

Hannan Limitation successfully links the directivity characteristics of 2D arrays with the aperture gain limit, providing the radiation efficiency upper limit for large 2D planar antenna arrays. This demonstrates the inevitable radiation efficiency degradation caused by mutual coupling effects between array elements. However, this limitation is derived based on the assumption of infinitely large 2D arrays, which means that it is not an accurate law for small-size arrays. In this paper, we extend this theory and propose an estimation formula for the radiation efficiency upper limit of finite-sized 2D arrays. Furthermore, we analyze a 3D array structure consisting of two parallel 2D arrays. Specifically, we provide evaluation formulas for the mutual coupling strengths for both infinite and finite size arrays and derive the fundamental efficiency limit of 3D arrays. Moreover, based on the established gain limit of antenna arrays with fixed aperture sizes, we derive the achievable gain limit of finite size 3D arrays. Besides the performance analyses, we also investigate the spatial radiation characteristics of the considered 3D array structure, offering a feasible region for 2D phase settings under a given energy attenuation threshold. Through simulations, we demonstrate the effectiveness of our proposed theories and gain advantages of 3D arrays for better spatial coverage under various scenarios.
EDCSSM: Edge Detection with Convolutional State Space Model
Venue: arXiv cs.CV 2024
Year: 2024

Edge detection in images is the foundation of many complex tasks in computer graphics. Due to the feature loss caused by multi-layer convolution and pooling architectures, learning-based edge detection models often produce thick edges and struggle to detect the edges of small objects in images. Inspired by state space models, this paper presents an edge detection algorithm which effectively addresses the aforementioned issues. The presented algorithm obtains state space variables of the image from dual-input channels with minimal down-sampling processes and utilizes these state variables for real-time learning and memorization of image text. Additionally, to achieve precise edges while filtering out false edges, a post-processing algorithm called wind erosion has been designed to handle the binary edge map. To further enhance the processing speed of the algorithm, we have designed parallel computing circuits for the most computationally intensive parts of presented algorithm, significantly improving computational speed and efficiency. Experimental results demonstrate that the proposed algorithm achieves precise thin edge localization and exhibits noise suppression capabilities across various types of images. With the parallel computing circuits, the algorithm to achieve processing speeds exceeds 30 FPS on 5K images.