OpenRaiser/Intern-Atlas huggingface dataset:
Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations
Venue: arXiv cs.CL 2024
Year: 2024

The robustness of recent Large Language Models (LLMs) has become increasingly crucial as their applicability expands across various domains and real-world applications. Retrieval-Augmented Generation (RAG) is a promising solution for addressing the limitations of LLMs, yet existing studies on the robustness of RAG often overlook the interconnected relationships between RAG components or the potential threats prevalent in real-world databases, such as minor textual errors. In this work, we investigate two underexplored aspects when assessing the robustness of RAG: 1) vulnerability to noisy documents through low-level perturbations and 2) a holistic evaluation of RAG robustness. Furthermore, we introduce a novel attack method, the Genetic Attack on RAG (\textit{GARAG}), which targets these aspects. Specifically, GARAG is designed to reveal vulnerabilities within each component and test the overall system functionality against noisy documents. We validate RAG robustness by applying our \textit{GARAG} to standard QA datasets, incorporating diverse retrievers and LLMs. The experimental results show that GARAG consistently achieves high attack success rates. Also, it significantly devastates the performance of each component and their synergy, highlighting the substantial risk that minor textual inaccuracies pose in disrupting RAG systems in the real world.
Non-Uniform Exposure Imaging via Neuromorphic Shutter Control
Venue: arXiv cs.CV 2024
Year: 2024

By leveraging the blur-noise trade-off, imaging with non-uniform exposures largely extends the image acquisition flexibility in harsh environments. However, the limitation of conventional cameras in perceiving intra-frame dynamic information prevents existing methods from being implemented in the real-world frame acquisition for real-time adaptive camera shutter control. To address this challenge, we propose a novel Neuromorphic Shutter Control (NSC) system to avoid motion blurs and alleviate instant noises, where the extremely low latency of events is leveraged to monitor the real-time motion and facilitate the scene-adaptive exposure. Furthermore, to stabilize the inconsistent Signal-to-Noise Ratio (SNR) caused by the non-uniform exposure times, we propose an event-based image denoising network within a self-supervised learning paradigm, i.e., SEID, exploring the statistics of image noises and inter-frame motion information of events to obtain artificial supervision signals for high-quality imaging in real-world scenes. To illustrate the effectiveness of the proposed NSC, we implement it in hardware by building a hybrid-camera imaging prototype system, with which we collect a real-world dataset containing well-synchronized frames and events in diverse scenarios with different target scenes and motion patterns. Experiments on the synthetic and real-world datasets demonstrate the superiority of our method over state-of-the-art approaches.
DEQ-MCL: Discrete-Event Queue-based Monte-Carlo Localization
Venue: arXiv cs.AI 2024
Year: 2024

Spatial cognition in hippocampal formation is posited to play a crucial role in the development of self-localization techniques for robots. In this paper, we propose a self-localization approach, DEQ-MCL, based on the discrete event queue hypothesis associated with phase precession within the hippocampal formation. Our method effectively estimates the posterior distribution of states, encompassing both past, present, and future states that are organized as a queue. This approach enables the smoothing of the posterior distribution of past states using current observations and the weighting of the joint distribution by considering the feasibility of future states. Our findings indicate that the proposed method holds promise for augmenting self-localization performance in indoor environments.
QCore: Data-Efficient, On-Device Continual Calibration for Quantized Models -- Extended Version
Venue: arXiv cs.LG 2024
Year: 2024

We are witnessing an increasing availability of streaming data that may contain valuable information on the underlying processes. It is thus attractive to be able to deploy machine learning models on edge devices near sensors such that decisions can be made instantaneously, rather than first having to transmit incoming data to servers. To enable deployment on edge devices with limited storage and computational capabilities, the full-precision parameters in standard models can be quantized to use fewer bits. The resulting quantized models are then calibrated using back-propagation and full training data to ensure accuracy. This one-time calibration works for deployments in static environments. However, model deployment in dynamic edge environments call for continual calibration to adaptively adjust quantized models to fit new incoming data, which may have different distributions. The first difficulty in enabling continual calibration on the edge is that the full training data may be too large and thus not always available on edge devices. The second difficulty is that the use of back-propagation on the edge for repeated calibration is too expensive. We propose QCore to enable continual calibration on the edge. First, it compresses the full training data into a small subset to enable effective calibration of quantized models with different bit-widths. We also propose means of updating the subset when new streaming data arrives to reflect changes in the environment, while not forgetting earlier training data. Second, we propose a small bit-flipping network that works with the subset to update quantized model parameters, thus enabling efficient continual calibration without back-propagation. An experimental study, conducted with real-world data in a continual learning setting, offers insight into the properties of QCore and shows that it is capable of outperforming strong baseline methods.
SIGY: Breaking Intel SGX Enclaves with Malicious Exceptions & Signals
Venue: arXiv cs.CR 2024
Year: 2024

User programs recover from hardware exceptions and respond to signals by executing custom handlers that they register specifically for such events. We present SIGY attack, which abuses this programming model on Intel SGX to break the confidentiality and integrity guarantees of enclaves. SIGY uses the untrusted OS to deliver fake hardware events and injects fake signals in an enclave at any point. Such unintended execution of benign program-defined handlers in an enclave corrupts its state and violates execution integrity. 7 runtimes and library OSes (OpenEnclave, Gramine, Scone, Asylo, Teaclave, Occlum, EnclaveOS) are vulnerable to SIGY. 8 languages supported in Intel SGX have programming constructs that are vulnerable to SIGY. We use SIGY to demonstrate 4 proof of concept exploits on webservers (Nginx, Node.js) to leak secrets and data analytics workloads in different languages (C and Java) to break execution integrity.
OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks
Venue: arXiv cs.CV 2024
Year: 2024

We introduce a self-supervised pretraining method, called OccFeat, for camera-only Bird's-Eye-View (BEV) segmentation networks. With OccFeat, we pretrain a BEV network via occupancy prediction and feature distillation tasks. Occupancy prediction provides a 3D geometric understanding of the scene to the model. However, the geometry learned is class-agnostic. Hence, we add semantic information to the model in the 3D space through distillation from a self-supervised pretrained image foundation model. Models pretrained with our method exhibit improved BEV semantic segmentation performance, particularly in low-data scenarios. Moreover, empirical results affirm the efficacy of integrating feature distillation with 3D occupancy prediction in our pretraining approach. Repository: https://github.com/valeoai/Occfeat
Teaching Scrum with a focus on compliance assessment
Venue: arXiv cs.SE 2024
Year: 2024

The Scrum framework has gained widespread adoption in the industry for its emphasis on collaboration and continuous improvement. However, it has not reached a similar relevance in Software Engineering (SE) curricula. This work reports the experience of five editions of a SE course within an MSc. Degree in Computer Engineering. The course primary educational objective is to provide students with the skills to manage software development projects with Scrum. The course is based on the execution of a team project and on the definition of qualitative and quantitative means of assessment of the application of Scrum. The conduction of five editions of the course allowed us to identify several lessons learned about time budgeting and team compositions in agile student projects and its evidence of the applicability of the framework to software development courses.
Covert Multi-Access Communication with a Non-Covert User
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we characterize the fundamental limits of a communication system with three users (i.e., three transmitters) and a single receiver where communication from two covert users must remain undetectable to an external warden. Our results show a tradeoff between the highest rates that are simultaneously achievable for the three users. They further show that the presence of a non-covert user in the system can enhance the capacities of the covert users under stringent secret-key constraints. To derive our fundamental limits, we provide an information-theoretic converse proof and present a coding scheme that achieves the performance of our converse result. Our coding scheme is based on multiplexing different code phases, which seems to be essential to exhaust the entire tradeoff region between the rates at the covert and the two non-covert users. This property is reminiscent of the setup with multiple non-covert users, where multiplexing is also required to exhaust the entire rate-region.
From Rigid to Soft Robotic Approaches for Minimally Invasive Neurosurgery
Venue: arXiv cs.RO 2024
Year: 2024

Robotic assistance has significantly improved the outcomes of open microsurgery and rigid endoscopic surgery, however is yet to make an impact in flexible endoscopic neurosurgery. Some of the most common intracranial procedures for treatment of hydrocephalus and tumors stand to benefit from increased dexterity and reduced invasiveness offered by robotic systems that can navigate in the deep ventricular system of the brain. We review a spectrum of flexible robotic devices, from the traditional highly actuated approach, to more novel and bio-inspired mechanisms for safe navigation. For each technology, we identify the operating principle and are able to evaluate the potential for minimally invasive surgical applications. Overall, rigid-type continuum robots have seen the most development, however, approaches combining rigid and soft robotic principles into innovative devices, are ideally situated to address safety and complexity limitations after future design evolution. We also observe a number of related challenges in the field, from surgeon-robot interfaces to robot evaluation procedures. Fundamentally, the challenges revolve around a guarantee of safety in robotic devices with the prerequisites to assist and improve upon surgical tasks. With innovative designs, materials and evaluation techniques emerging, we see potential impacts in the next 5--10 years.
TWIMP: Two-Wheel Inverted Musculoskeletal Pendulum as a Learning Control Platform in the Real World with Environmental Physical Contact
Venue: arXiv cs.RO 2024
Year: 2024

By the recent spread of machine learning in the robotics field, a humanoid that can act, perceive, and learn in the real world through contact with the environment needs to be developed. In this study, as one of the choices, we propose a novel humanoid TWIMP, which combines a human mimetic musculoskeletal upper limb with a two-wheel inverted pendulum. By combining the benefit of a musculoskeletal humanoid, which can achieve soft contact with the external environment, and the benefit of a two-wheel inverted pendulum with a small footprint and high mobility, we can easily investigate learning control systems in environments with contact and sudden impact. We reveal our whole concept and system details of TWIMP, and execute several preliminary experiments to show its potential ability.
A Method of Joint Angle Estimation Using Only Relative Changes in Muscle Lengths for Tendon-driven Humanoids with Complex Musculoskeletal Structures
Venue: arXiv cs.RO 2024
Year: 2024

Tendon-driven musculoskeletal humanoids typically have complex structures similar to those of human beings, such as ball joints and the scapula, in which encoders cannot be installed. Therefore, joint angles cannot be directly obtained and need to be estimated using the changes in muscle lengths. In previous studies, methods using table-search and extended kalman filter have been developed. These methods express the joint-muscle mapping, which is the nonlinear relationship between joint angles and muscle lengths, by using a data table, polynomials, or a neural network. However, due to computational complexity, these methods cannot consider the effects of polyarticular muscles. In this study, considering the limitation of the computational cost, we reduce unnecessary degrees of freedom, divide joints and muscles into several groups, and formulate a joint angle estimation method that takes into account polyarticular muscles. Also, we extend the estimation method to propose a joint angle estimation method using only the relative changes in muscle lengths. By this extension, which does not use absolute muscle lengths, we do not need to execute a difficult calibration of muscle lengths for tendon-driven musculoskeletal humanoids. Finally, we conduct experiments in simulation and actual environments, and verify the effectiveness of this study.
PGNAA Spectral Classification of Aluminium and Copper Alloys with Machine Learning
Venue: arXiv cs.LG 2024
Year: 2024

In this paper, we explore the optimization of metal recycling with a focus on real-time differentiation between alloys of copper and aluminium. Spectral data, obtained through Prompt Gamma Neutron Activation Analysis (PGNAA), is utilized for classification. The study compares data from two detectors, cerium bromide (CeBr$_{3}$) and high purity germanium (HPGe), considering their energy resolution and sensitivity. We test various data generation, preprocessing, and classification methods, with Maximum Likelihood Classifier (MLC) and Conditional Variational Autoencoder (CVAE) yielding the best results. The study also highlights the impact of different detector types on classification accuracy, with CeBr$_{3}$ excelling in short measurement times and HPGe performing better in longer durations. The findings suggest the importance of selecting the appropriate detector and methodology based on specific application requirements.
Dismantling Common Internet Services for Ad-Malware Detection
Venue: arXiv cs.NI 2024
Year: 2024

Online advertising represents a main instrument for publishers to fund content on the World Wide Web. Unfortunately, a significant number of online advertisements often accommodates potentially malicious content, such as cryptojacking hidden in web banners - even on reputable websites. In order to protect Internet users from such online threats, the thorough detection of ad-malware campaigns plays a crucial role for a safe Web. Today, common Internet services like VirusTotal can label suspicious content based on feedback from contributors and from the entire Web community. However, it is open to which extent ad-malware is actually taken into account and whether the results of these services are consistent. In this pre-study, we evaluate who defines ad-malware on the Internet. In a first step, we crawl a vast set of websites and fetch all HTTP requests (particularly to online advertisements) within these websites. Then we query these requests both against popular filtered DNS providers and VirusTotal. The idea is to validate, how much content is labeled as a potential threat. The results show that up to 0.47% of the domains found during crawling are labeled as suspicious by DNS providers and up to 8.8% by VirusTotal. Moreover, only about 0.7% to 3.2% of these domains are categorized as ad-malware. The overall responses from the used Internet services paint a divergent picture: All considered services have different understandings to the definition of suspicious content. Thus, we outline potential research efforts to the automated detection of ad-malware. We further bring up the open question of a common definition of ad-malware to the Web community.
Generalizable Neural Human Renderer
Venue: arXiv cs.CV 2024
Year: 2024

While recent advancements in animatable human rendering have achieved remarkable results, they require test-time optimization for each subject which can be a significant limitation for real-world applications. To address this, we tackle the challenging task of learning a Generalizable Neural Human Renderer (GNH), a novel method for rendering animatable humans from monocular video without any test-time optimization. Our core method focuses on transferring appearance information from the input video to the output image plane by utilizing explicit body priors and multi-view geometry. To render the subject in the intended pose, we utilize a straightforward CNN-based image renderer, foregoing the more common ray-sampling or rasterizing-based rendering modules. Our GNH achieves remarkable generalizable, photorealistic rendering with unseen subjects with a three-stage process. We quantitatively and qualitatively demonstrate that GNH significantly surpasses current state-of-the-art methods, notably achieving a 31.3% improvement in LPIPS.
NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results
Venue: arXiv cs.CV 2024
Year: 2024

This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results. The aim of this challenge is to discover an effective network design or solution capable of generating brighter, clearer, and visually appealing results when dealing with a variety of conditions, including ultra-high resolution (4K and beyond), non-uniform illumination, backlighting, extreme darkness, and night scenes. A notable total of 428 participants registered for the challenge, with 22 teams ultimately making valid submissions. This paper meticulously evaluates the state-of-the-art advancements in enhancing low-light images, reflecting the significant progress and creativity in this field.
Linear Search for an Escaping Target with Unknown Speed
Venue: arXiv cs.DM 2024
Year: 2024

We consider linear search for an escaping target whose speed and initial position are unknown to the searcher. A searcher (an autonomous mobile agent) is initially placed at the origin of the real line and can move with maximum speed $1$ in either direction along the line. An oblivious mobile target that is moving away from the origin with an unknown constant speed $v<1$ is initially placed by an adversary on the infinite line at distance $d$ from the origin in an unknown direction. We consider two cases, depending on whether $d$ is known or unknown. The main contribution of this paper is to prove a new lower bound and give algorithms leading to new upper bounds for search in these settings. This results in an optimal (up to lower order terms in the exponent) competitive ratio in the case where $d$ is known and improved upper and lower bounds for the case where $d$ is unknown. Our results solve an open problem proposed in [Coleman et al., Proc. OPODIS 2022].
On-the-Fly Point Annotation for Fast Medical Video Labeling
Venue: arXiv cs.CV 2024
Year: 2024

Purpose: In medical research, deep learning models rely on high-quality annotated data, a process often laborious and timeconsuming. This is particularly true for detection tasks where bounding box annotations are required. The need to adjust two corners makes the process inherently frame-by-frame. Given the scarcity of experts' time, efficient annotation methods suitable for clinicians are needed. Methods: We propose an on-the-fly method for live video annotation to enhance the annotation efficiency. In this approach, a continuous single-point annotation is maintained by keeping the cursor on the object in a live video, mitigating the need for tedious pausing and repetitive navigation inherent in traditional annotation methods. This novel annotation paradigm inherits the point annotation's ability to generate pseudo-labels using a point-to-box teacher model. We empirically evaluate this approach by developing a dataset and comparing on-the-fly annotation time against traditional annotation method. Results: Using our method, annotation speed was 3.2x faster than the traditional annotation technique. We achieved a mean improvement of 6.51 +- 0.98 AP@50 over conventional method at equivalent annotation budgets on the developed dataset. Conclusion: Without bells and whistles, our approach offers a significant speed-up in annotation tasks. It can be easily implemented on any annotation platform to accelerate the integration of deep learning in video-based medical research.
Rethinking Legal Compliance Automation: Opportunities with Large Language Models
Venue: arXiv cs.SE 2024
Year: 2024

As software-intensive systems face growing pressure to comply with laws and regulations, providing automated support for compliance analysis has become paramount. Despite advances in the Requirements Engineering (RE) community on legal compliance analysis, important obstacles remain in developing accurate and generalizable compliance automation solutions. This paper highlights some observed limitations of current approaches and examines how adopting new automation strategies that leverage Large Language Models (LLMs) can help address these shortcomings and open up fresh opportunities. Specifically, we argue that the examination of (textual) legal artifacts should, first, employ a broader context than sentences, which have widely been used as the units of analysis in past research. Second, the mode of analysis with legal artifacts needs to shift from classification and information extraction to more end-to-end strategies that are not only accurate but also capable of providing explanation and justification. We present a compliance analysis approach designed to address these limitations. We further outline our evaluation plan for the approach and provide preliminary evaluation results based on data processing agreements (DPAs) that must comply with the General Data Protection Regulation (GDPR). Our initial findings suggest that our approach yields substantial accuracy improvements and, at the same time, provides justification for compliance decisions.
A Stochastic Geo-spatiotemporal Bipartite Network to Optimize GCOOS Sensor Placement Strategies
Venue: arXiv cs.MA 2024
Year: 2024

This paper proposes two new measures applicable in a spatial bipartite network model: coverage and coverage robustness. The bipartite network must consist of observer nodes, observable nodes, and edges that connect observer nodes to observable nodes. The coverage and coverage robustness scores evaluate the effectiveness of the observer node placements. This measure is beneficial for stochastic data as it may be coupled with Monte Carlo simulations to identify optimal placements for new observer nodes. In this paper, we construct a Geo-SpatioTemporal Bipartite Network (GSTBN) within the stochastic and dynamical environment of the Gulf of Mexico. This GSTBN consists of GCOOS sensor nodes and HYCOM Region of Interest (RoI) event nodes. The goal is to identify optimal placements to expand GCOOS to improve the forecasting outcomes by the HYCOM ocean prediction model.