OpenRaiser/Intern-Atlas huggingface dataset:
Probabilistic Parameter Estimators and Calibration Metrics for Pose Estimation from Image Features
Venue: arXiv cs.RO 2024
Year: 2024

This paper addresses the challenge of probabilistic parameter estimation given measurement uncertainty in real-time. We provide a general formulation and apply this to pose estimation for an autonomous visual landing system. We present three probabilistic parameter estimators: a least-squares sampling approach, a linear approximation method, and a probabilistic programming estimator. To evaluate these estimators, we introduce novel closed-form expressions for measuring calibration and sharpness specifically for multivariate normal distributions. Our experimental study compares the three estimators under various noise conditions. We demonstrate that the linear approximation estimator can produce sharp and well-calibrated pose predictions significantly faster than the other methods but may yield overconfident predictions in certain scenarios. Additionally, we demonstrate that these estimators can be integrated with a Kalman filter for continuous pose estimation during a runway approach where we observe a 50\% improvement in sharpness while maintaining marginal calibration. This work contributes to the integration of data-driven computer vision models into complex safety-critical aircraft systems and provides a foundation for developing rigorous certification guidelines for such systems.
Hidden Web Caches Discovery
Venue: arXiv cs.CR 2024
Year: 2024

Web caches play a crucial role in web performance and scalability. However, detecting cached responses is challenging when web servers do not reliably communicate the cache status through standardized headers. This paper presents a novel methodology for cache detection using timing analysis. Our approach eliminates the dependency on cache status headers, making it applicable to any web server. The methodology relies on sending paired requests using HTTP multiplexing functionality and makes heavy use of cache-busting to control the origin of the responses. By measuring the time it takes to receive responses from paired requests, we can determine if a response is cached or not. In each pair, one request is cache-busted to force retrieval from the origin server, while the other request is not and might be served from the cache, if present. A faster response time for the non-cache-busted request compared to the cache-busted one suggests the first one is coming from the cache. We implemented this approach in a tool and achieved an estimated accuracy of 89.6% compared to state-of-the-art methods based on cache status headers. Leveraging our cache detection approach, we conducted a large-scale experiment on the Tranco Top 50k websites. We identified a significant presence of hidden caches (5.8%) that do not advertise themselves through headers. Additionally, we employed our methodology to detect Web Cache Deception (WCD) vulnerabilities in these hidden caches. We discovered that 1.020 of them are susceptible to WCD vulnerabilities, potentially leaking sensitive data. Our findings demonstrate the effectiveness of our timing analysis methodology for cache discovery and highlight the importance of a tool that does not rely on cache-communicated cache status headers.
Capital as Artificial Intelligence
Venue: arXiv cs.CY 2024
Year: 2024

We gather many perspectives on Capital and synthesize their commonalities. We provide a characterization of Capital as a historical agential system and propose a model of Capital using tools from computer science. Our model consists of propositions which, if satisfied by a specific grounding, constitute a valid model of Capital. We clarify the manners in which Capital can evolve. We claim that, when its evolution is driven by quantitative optimization processes, Capital can possess qualities of Artificial Intelligence. We find that Capital may not uniquely represent meaning, in the same way that optimization is not intentionally meaningful. We find that Artificial Intelligences like modern day Large Language Models are a part of Capital. We link our readers to a web-interface where they can interact with a part of Capital.
PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing
Venue: arXiv cs.AI 2024
Year: 2024

Deploying language models (LMs) necessitates outputs to be both high-quality and compliant with safety guidelines. Although Inference-Time Guardrails (ITG) offer solutions that shift model output distributions towards compliance, we find that current methods struggle in balancing safety with helpfulness. ITG Methods that safely address non-compliant queries exhibit lower helpfulness while those that prioritize helpfulness compromise on safety. We refer to this trade-off as the guardrail tax, analogous to the alignment tax. To address this, we propose PrimeGuard, a novel ITG method that utilizes structured control flow. PrimeGuard routes requests to different self-instantiations of the LM with varying instructions, leveraging its inherent instruction-following capabilities and in-context learning. Our tuning-free approach dynamically compiles system-designer guidelines for each query. We construct and release safe-eval, a diverse red-team safety benchmark. Extensive evaluations demonstrate that PrimeGuard, without fine-tuning, overcomes the guardrail tax by (1) significantly increasing resistance to iterative jailbreak attacks and (2) achieving state-of-the-art results in safety guardrailing while (3) matching helpfulness scores of alignment-tuned models. Extensive evaluations demonstrate that PrimeGuard, without fine-tuning, outperforms all competing baselines and overcomes the guardrail tax by improving the fraction of safe responses from 61% to 97% and increasing average helpfulness scores from 4.17 to 4.29 on the largest models, while reducing attack success rate from 100% to 8%. PrimeGuard implementation is available at https://github.com/dynamofl/PrimeGuard and safe-eval dataset is available at https://huggingface.co/datasets/dynamoai/safe_eval.
Understanding Impacts of Electromagnetic Signal Injection Attacks on Object Detection
Venue: arXiv cs.CR 2024
Year: 2024

Object detection can localize and identify objects in images, and it is extensively employed in critical multimedia applications such as security surveillance and autonomous driving. Despite the success of existing object detection models, they are often evaluated in ideal scenarios where captured images guarantee the accurate and complete representation of the detecting scenes. However, images captured by image sensors may be affected by different factors in real applications, including cyber-physical attacks. In particular, attackers can exploit hardware properties within the systems to inject electromagnetic interference so as to manipulate the images. Such attacks can cause noisy or incomplete information about the captured scene, leading to incorrect detection results, potentially granting attackers malicious control over critical functions of the systems. This paper presents a research work that comprehensively quantifies and analyzes the impacts of such attacks on state-of-the-art object detection models in practice. It also sheds light on the underlying reasons for the incorrect detection outcomes.
Improving multidimensional projection quality with user-specific metrics and optimal scaling
Venue: arXiv cs.GR 2024
Year: 2024

The growing prevalence of high-dimensional data has fostered the development of multidimensional projection (MP) techniques, such as t-SNE, UMAP, and LAMP, for data visualization and exploration. However, conventional MP methods typically employ generic quality metrics, neglecting individual user preferences. This study proposes a new framework that tailors MP techniques based on user-specific quality criteria, enhancing projection interpretability. Our approach combines three visual quality metrics, stress, neighborhood preservation, and silhouette score, to create a composite metric for a precise MP evaluation. We then optimize the projection scale by maximizing the composite metric value. We conducted an experiment involving two users with different projection preferences, generating projections using t-SNE, UMAP, and LAMP. Users rate projections according to their criteria, producing two training sets. We derive optimal weights for each set and apply them to other datasets to determine the best projections per user. Our findings demonstrate that personalized projections effectively capture user preferences, fostering better data exploration and enabling more informed decision-making. This user-centric approach promotes advancements in multidimensional projection techniques that accommodate diverse user preferences and enhance interpretability.
Nudging Using Autonomous Agents: Risks and Ethical Considerations
Venue: arXiv cs.CY 2024
Year: 2024

This position paper briefly discusses nudging, its use by autonomous agents, potential risks and ethical considerations while creating such systems. Instead of taking a normative approach, which guides all situations, the paper proposes a risk-driven questions-and-answer approach. The paper takes the position that this is a pragmatic method, that is transparent about beneficial intentions, foreseeable risks, and mitigations. Given the uncertainty in AI and autonomous agent capabilities, we believe that such pragmatic methods offer a plausibly safe path, without sacrificing flexibility in domain and technology.
Navigating Uncertainty in Medical Image Segmentation
Venue: arXiv cs.CV 2024
Year: 2024

We address the selection and evaluation of uncertain segmentation methods in medical imaging and present two case studies: prostate segmentation, illustrating that for minimal annotator variation simple deterministic models can suffice, and lung lesion segmentation, highlighting the limitations of the Generalized Energy Distance (GED) in model selection. Our findings lead to guidelines for accurately choosing and developing uncertain segmentation models, that integrate aleatoric and epistemic components. These guidelines are designed to aid researchers and practitioners in better developing, selecting, and evaluating uncertain segmentation methods, thereby facilitating enhanced adoption and effective application of segmentation uncertainty in practice.
FCNR: Fast Compressive Neural Representation of Visualization Images
Venue: arXiv cs.CV 2024
Year: 2024

We present FCNR, a fast compressive neural representation for tens of thousands of visualization images under varying viewpoints and timesteps. The existing NeRVI solution, albeit enjoying a high compression ratio, incurs slow speeds in encoding and decoding. Built on the recent advances in stereo image compression, FCNR assimilates stereo context modules and joint context transfer modules to compress image pairs. Our solution significantly improves encoding and decoding speed while maintaining high reconstruction quality and satisfying compression ratio. To demonstrate its effectiveness, we compare FCNR with state-of-the-art neural compression methods, including E-NeRV, HNeRV, NeRVI, and ECSIC. The source code can be found at https://github.com/YunfeiLu0112/FCNR.
Deep Learning Assisted Inertial Dead Reckoning and Fusion
Venue: arXiv cs.RO 2024
Year: 2024

The interest in mobile platforms across a variety of applications has increased significantly in recent years. One of the reasons is the ability to achieve accurate navigation by using low-cost sensors. To this end, inertial sensors are fused with global navigation satellite systems (GNSS) signals. GNSS outages during platform operation can result in pure inertial navigation, causing the navigation solution to drift. In such situations, periodic trajectories with dedicated algorithms were suggested to mitigate the drift. With periodic dynamics, inertial deep learning approaches can capture the motion more accurately and provide accurate dead-reckoning for drones and mobile robots. In this paper, we propose approaches to extend deep learning-assisted inertial sensing and fusion capabilities during periodic motion. We begin by demonstrating that fusion between GNSS and inertial sensors in periodic trajectories achieves better accuracy compared to straight-line trajectories. Next, we propose an empowered network architecture to accurately regress the change in distance of the platform. Utilizing this network, we drive a hybrid approach for a neural-inertial fusion filter. Finally, we utilize this approach for situations when GNSS is available and show its benefits. A dataset of 337 minutes of data collected from inertial sensors mounted on a mobile robot and a quadrotor is used to evaluate our approaches.
Prisec II -- A Comprehensive Model for IoT Security: Cryptographic Algorithms and Cloud Integration
Venue: arXiv cs.CR 2024
Year: 2024

This study addresses the critical issue of ensuring data security and efficiency in interconnected devices, especially in IoT environments. The objective is to design and implement a model using cryptographic algorithms to enhance data security in 5G networks. Challenges arise from the limited computational capabilities of IoT devices, which require the analysis and selection of cryptographic algorithms to achieve efficient data transmission. This study proposes a model that includes four levels of security, each employing different levels of encryption to provide better data security. Finally, cloud computing optimizes processing efficiency and resource utilization to improve data transmission.
Lymphoid Infiltration Assessment of the Tumor Margins in H&E Slides
Venue: arXiv cs.CV 2024
Year: 2024

Lymphoid infiltration at tumor margins is a key prognostic marker in solid tumors, playing a crucial role in guiding immunotherapy decisions. Current assessment methods, heavily reliant on immunohistochemistry (IHC), face challenges in tumor margin delineation and are affected by tissue preservation conditions. In contrast, we propose a Hematoxylin and Eosin (H&E) staining-based approach, underpinned by an advanced lymphocyte segmentation model trained on a public dataset for the precise detection of CD3+ and CD20+ lymphocytes. In our colorectal cancer study, we demonstrate that our H&E-based method offers a compelling alternative to traditional IHC, achieving comparable results in many cases. Our method's validity is further explored through a Turing test, involving blinded assessments by a pathologist of anonymized curves from H&E and IHC slides. This approach invites the medical community to consider Turing tests as a standard for evaluating medical applications involving expert human evaluation, thereby opening new avenues for enhancing cancer management and immunotherapy planning.
Side-Channel Analysis of OpenVINO-based Neural Network Models
Venue: arXiv cs.CR 2024
Year: 2024

Embedded devices with neural network accelerators offer great versatility for their users, reducing the need to use cloud-based services. At the same time, they introduce new security challenges in the area of hardware attacks, the most prominent being side-channel analysis (SCA). It was shown that SCA can recover model parameters with a high accuracy, posing a threat to entities that wish to keep their models confidential. In this paper, we explore the susceptibility of quantized models implemented in OpenVINO, an embedded framework for deploying neural networks on embedded and Edge devices. We show that it is possible to recover model parameters with high precision, allowing the recovered model to perform very close to the original one. Our experiments on GoogleNet v1 show only a 1% difference in the Top 1 and a 0.64% difference in the Top 5 accuracies.
BONES: a Benchmark fOr Neural Estimation of Shapley values
Venue: arXiv cs.LG 2024
Year: 2024

Shapley Values are concepts established for eXplainable AI. They are used to explain black-box predictive models by quantifying the features' contributions to the model's outcomes. Since computing the exact Shapley Values is known to be computationally intractable on real-world datasets, neural estimators have emerged as alternative, more scalable approaches to get approximated Shapley Values estimates. However, experiments with neural estimators are currently hard to replicate as algorithm implementations, explainer evaluators, and results visualizations are neither standardized nor promptly usable. To bridge this gap, we present BONES, a new benchmark focused on neural estimation of Shapley Value. It provides researchers with a suite of state-of-the-art neural and traditional estimators, a set of commonly used benchmark datasets, ad hoc modules for training black-box models, as well as specific functions to easily compute the most popular evaluation metrics and visualize results. The purpose is to simplify XAI model usage, evaluation, and comparison. In this paper, we showcase BONES results and visualizations for XAI model benchmarking on both tabular and image data. The open-source library is available at the following link: https://github.com/DavideNapolitano/BONES.
Joint Resource-Power Allocation and UE Rank Selection in Multi-User MIMO Systems with Linear Transceivers
Venue: arXiv cs.IT 2024
Year: 2024

Next-generation wireless networks aim to deliver data speeds much faster than 5G. This requires base stations with lots of antennas and a large operating bandwidth. These advanced base stations are expected to serve several multiantenna user-equipment (UEs) simultaneously on the same time-frequency resources on both the uplink and the downlink. The UE data rates are affected by the following three main factors: UE rank, which refers to the number of data layers used by each UE, UE frequency allocation, which refers to the assignment of slices of the overall frequency band to use for each UE in an orthogonal frequency-division multiplexing (OFDM) system, and UE power allocation/control, which refers to the allocation of power by the base station for data transmission to each UE on the downlink or the power used by each UE to send data on the uplink. Since multiple UEs are to be simultaneously served, the type of precoder used for downlink transmission and the type of receiver used for uplink reception predominantly influence these three aforementioned factors and the resulting overall UE throughput. This paper addresses the problem of jointly selecting these three parameters specifically when zero-forcing (ZF) precoders are used for downlink transmission and linear minimum mean square error (LMMSE) receivers are employed for uplink reception.
HDRSplat: Gaussian Splatting for High Dynamic Range 3D Scene Reconstruction from Raw Images
Venue: arXiv cs.CV 2024
Year: 2024

The recent advent of 3D Gaussian Splatting (3DGS) has revolutionized the 3D scene reconstruction space enabling high-fidelity novel view synthesis in real-time. However, with the exception of RawNeRF, all prior 3DGS and NeRF-based methods rely on 8-bit tone-mapped Low Dynamic Range (LDR) images for scene reconstruction. Such methods struggle to achieve accurate reconstructions in scenes that require a higher dynamic range. Examples include scenes captured in nighttime or poorly lit indoor spaces having a low signal-to-noise ratio, as well as daylight scenes with shadow regions exhibiting extreme contrast. Our proposed method HDRSplat tailors 3DGS to train directly on 14-bit linear raw images in near darkness which preserves the scenes' full dynamic range and content. Our key contributions are two-fold: Firstly, we propose a linear HDR space-suited loss that effectively extracts scene information from noisy dark regions and nearly saturated bright regions simultaneously, while also handling view-dependent colors without increasing the degree of spherical harmonics. Secondly, through careful rasterization tuning, we implicitly overcome the heavy reliance and sensitivity of 3DGS on point cloud initialization. This is critical for accurate reconstruction in regions of low texture, high depth of field, and low illumination. HDRSplat is the fastest method to date that does 14-bit (HDR) 3D scene reconstruction in $\le$15 minutes/scene ($\sim$30x faster than prior state-of-the-art RawNeRF). It also boasts the fastest inference speed at $\ge$120fps. We further demonstrate the applicability of our HDR scene reconstruction by showcasing various applications like synthetic defocus, dense depth map extraction, and post-capture control of exposure, tone-mapping and view-point.
Language-Based Security for Low-Level MPC
Venue: arXiv cs.CR 2024
Year: 2024

Secure Multi-Party Computation (MPC) is an important enabling technology for data privacy in modern distributed applications. Currently, proof methods for low-level MPC protocols are primarily manual and thus tedious and error-prone, and are also non-standardized and unfamiliar to most PL theorists. As a step towards better language support and language-based enforcement, we develop a new staged PL for defining a variety of low-level probabilistic MPC protocols. We also formulate a collection of confidentiality and integrity hyperproperties for our language model that are familiar from information flow, including conditional noninterference, gradual release, and robust declassification. We demonstrate their relation to standard MPC threat models of passive and malicious security, and how they can be leveraged in security verification of protocols. To prove these properties we develop automated tactics in $\mathbb{F}_2$ that can be integrated with separation logic-style reasoning.
AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game
Venue: arXiv cs.CL 2024
Year: 2024

Strategic social deduction games serve as valuable testbeds for evaluating the understanding and inference skills of language models, offering crucial insights into social science, artificial intelligence, and strategic gaming. This paper focuses on creating proxies of human behavior in simulated environments, with Among Us utilized as a tool for studying simulated human behavior. The study introduces a text-based game environment, named AmongAgents, that mirrors the dynamics of Among Us. Players act as crew members aboard a spaceship, tasked with identifying impostors who are sabotaging the ship and eliminating the crew. Within this environment, the behavior of simulated language agents is analyzed. The experiments involve diverse game sequences featuring different configurations of Crewmates and Impostor personality archetypes. Our work demonstrates that state-of-the-art large language models (LLMs) can effectively grasp the game rules and make decisions based on the current context. This work aims to promote further exploration of LLMs in goal-oriented games with incomplete information and complex action spaces, as these settings offer valuable opportunities to assess language model performance in socially driven scenarios.
Imperfect Vision Encoders: Efficient and Robust Tuning for Vision-Language Models
Venue: arXiv cs.CV 2024
Year: 2024

Vision language models (VLMs) demonstrate impressive capabilities in visual question answering and image captioning, acting as a crucial link between visual and language models. However, existing open-source VLMs heavily rely on pretrained and frozen vision encoders (such as CLIP). Despite CLIP's robustness across diverse domains, it still exhibits non-negligible image understanding errors. These errors propagate to the VLM responses, resulting in sub-optimal performance. In our work, we propose an efficient and robust method for updating vision encoders within VLMs. Our approach selectively and locally updates encoders, leading to substantial performance improvements on data where previous mistakes occurred, while maintaining overall robustness. Furthermore, we demonstrate the effectiveness of our method during continual few-shot updates. Theoretical grounding, generality, and computational efficiency characterize our approach.