OpenRaiser/Intern-Atlas huggingface dataset:
Diffusion-based Light Field Synthesis
Venue: arXiv cs.CV 2024
Year: 2024

Light fields (LFs), conducive to comprehensive scene radiance recorded across angular dimensions, find wide applications in 3D reconstruction, virtual reality, and computational photography.However, the LF acquisition is inevitably time-consuming and resource-intensive due to the mainstream acquisition strategy involving manual capture or laborious software synthesis.Given such a challenge, we introduce LFdiff, a straightforward yet effective diffusion-based generative framework tailored for LF synthesis, which adopts only a single RGB image as input.LFdiff leverages disparity estimated by a monocular depth estimation network and incorporates two distinctive components: a novel condition scheme and a noise estimation network tailored for LF data.Specifically, we design a position-aware warping condition scheme, enhancing inter-view geometry learning via a robust conditional signal.We then propose DistgUnet, a disentanglement-based noise estimation network, to harness comprehensive LF representations.Extensive experiments demonstrate that LFdiff excels in synthesizing visually pleasing and disparity-controllable light fields with enhanced generalization capability.Additionally, comprehensive results affirm the broad applicability of the generated LF data, spanning applications like LF super-resolution and refocusing.
Tropical Decision Boundaries for Neural Networks Are Robust Against Adversarial Attacks
Venue: arXiv cs.LG 2024
Year: 2024

We introduce a simple, easy to implement, and computationally efficient tropical convolutional neural network architecture that is robust against adversarial attacks. We exploit the tropical nature of piece-wise linear neural networks by embedding the data in the tropical projective torus in a single hidden layer which can be added to any model. We study the geometry of its decision boundary theoretically and show its robustness against adversarial attacks on image datasets using computational experiments.
BrainSLAM: SLAM on Neural Population Activity Data
Venue: arXiv cs.RO 2024
Year: 2024

Simultaneous localisation and mapping (SLAM) algorithms are commonly used in robotic systems for learning maps of novel environments. Brains also appear to learn maps, but the mechanisms are not known and it is unclear how to infer these maps from neural activity data. We present BrainSLAM; a method for performing SLAM using only population activity (local field potential, LFP) data simultaneously recorded from three brain regions in rats: hippocampus, prefrontal cortex, and parietal cortex. This system uses a convolutional neural network (CNN) to decode velocity and familiarity information from wavelet scalograms of neural local field potential data recorded from rats as they navigate a 2D maze. The CNN's output drives a RatSLAM-inspired architecture, powering an attractor network which performs path integration plus a separate system which performs `loop closure' (detecting previously visited locations and correcting map aliasing errors). Together, these three components can construct faithful representations of the environment while simultaneously tracking the animal's location. This is the first demonstration of inference of a spatial map from brain recordings. Our findings expand SLAM to a new modality, enabling a new method of mapping environments and facilitating a better understanding of the role of cognitive maps in navigation and decision making.
Are Synthetic Time-series Data Really not as Good as Real Data?
Venue: arXiv cs.LG 2024
Year: 2024

Time-series data presents limitations stemming from data quality issues, bias and vulnerabilities, and generalization problem. Integrating universal data synthesis methods holds promise in improving generalization. However, current methods cannot guarantee that the generator's output covers all unseen real data. In this paper, we introduce InfoBoost -- a highly versatile cross-domain data synthesizing framework with time series representation learning capability. We have developed a method based on synthetic data that enables model training without the need for real data, surpassing the performance of models trained with real data. Additionally, we have trained a universal feature extractor based on our synthetic data that is applicable to all time-series data. Our approach overcomes interference from multiple sources rhythmic signal, noise interference, and long-period features that exceed sampling window capabilities. Through experiments, our non-deep-learning synthetic data enables models to achieve superior reconstruction performance and universal explicit representation extraction without the need for real data.
Rhoban Football Club: RoboCup Humanoid Kid-Size 2023 Champion Team Paper
Venue: arXiv cs.RO 2024
Year: 2024

In 2023, Rhoban Football Club reached the first place of the KidSize soccer competition for the fifth time, and received the best humanoid award. This paper presents and reviews important points in robots architecture and workflow, with hindsights from the competition.
Bialgebraic Reasoning on Higher-Order Program Equivalence
Venue: arXiv cs.LO 2024
Year: 2024

Logical relations constitute a key method for reasoning about contextual equivalence of programs in higher-order languages. They are usually developed on a per-case basis, with a new theory required for each variation of the language or of the desired notion of equivalence. In the present paper we introduce a general construction of (step-indexed) logical relations at the level of Higher-Order Mathematical Operational Semantics, a highly parametric categorical framework for modeling the operational semantics of higher-order languages. Our main result asserts that for languages whose weak operational model forms a lax bialgebra, the logical relation is automatically sound for contextual equivalence. Our abstract theory is shown to instantiate to combinatory logics and $λ$-calculi with recursive types, and to different flavours of contextual equivalence.
Deep Robot Sketching: An application of Deep Q-Learning Networks for human-like sketching
Venue: arXiv cs.RO 2024
Year: 2024

The current success of Reinforcement Learning algorithms for its performance in complex environments has inspired many recent theoretical approaches to cognitive science. Artistic environments are studied within the cognitive science community as rich, natural, multi-sensory, multi-cultural environments. In this work, we propose the introduction of Reinforcement Learning for improving the control of artistic robot applications. Deep Q-learning Neural Networks (DQN) is one of the most successful algorithms for the implementation of Reinforcement Learning in robotics. DQN methods generate complex control policies for the execution of complex robot applications in a wide set of environments. Current art painting robot applications use simple control laws that limits the adaptability of the frameworks to a set of simple environments. In this work, the introduction of DQN within an art painting robot application is proposed. The goal is to study how the introduction of a complex control policy impacts the performance of a basic art painting robot application. The main expected contribution of this work is to serve as a first baseline for future works introducing DQN methods for complex art painting robot frameworks. Experiments consist of real world executions of human drawn sketches using the DQN generated policy and TEO, the humanoid robot. Results are compared in terms of similarity and obtained reward with respect to the reference inputs
Comparative Study of Large Language Model Architectures on Frontier
Venue: arXiv cs.DC 2024
Year: 2024

Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world's first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.
Approximating Optimal Morphing Attacks using Template Inversion
Venue: arXiv cs.CV 2024
Year: 2024

Recent works have demonstrated the feasibility of inverting face recognition systems, enabling to recover convincing face images using only their embeddings. We leverage such template inversion models to develop a novel type ofdeep morphing attack based on inverting a theoretical optimal morph embedding, which is obtained as an average of the face embeddings of source images. We experiment with two variants of this approach: the first one exploits a fully self-contained embedding-to-image inversion model, while the second leverages the synthesis network of a pretrained StyleGAN network for increased morph realism. We generate morphing attacks from several source datasets and study the effectiveness of those attacks against several face recognition networks. We showcase that our method can compete with and regularly beat the previous state of the art for deep-learning based morph generation in terms of effectiveness, both in white-box and black-box attack scenarios, and is additionally much faster to run. We hope this might facilitate the development of large scale deep morph datasets for training detection models.
BIOMERO: BioImage analysis in OMERO
Venue: arXiv cs.SE 2024
Year: 2024

In the rapidly evolving field of bioimaging, the integration and orchestration of Findable, Accessible, Interoperable, and Reusable (FAIR) image analysis workflows remains a challenge. We introduce BIOMERO, a bridge connecting OMERO, a renowned bioimaging data management platform, FAIR workflows and high-performance computing (HPC) environments. BIOMERO, featuring our opensource Python library "OMERO Slurm Client", facilitates seamless execution of FAIR workflows, particularly for large datasets from High Content or High Throughput Screening. BIOMERO empowers researchers by eliminating the need for specialized knowledge, enabling scalable image processing directly from OMERO. BIOMERO notably supports the sharing and utilization of FAIR workflows between OMERO, Cytomine/BIAFLOWS, and other bioimaging communities. BIOMERO will promote the widespread adoption of FAIR workflows, emphasizing reusability, across the realm of bioimaging research. Its user-friendly interface will empower users, including those without technical expertise, to seamlessly apply these workflows to their datasets, democratizing the utilization of AI by the broader research community.
Control-Theoretic Techniques for Online Adaptation of Deep Neural Networks in Dynamical Systems
Venue: arXiv cs.LG 2024
Year: 2024

Deep neural networks (DNNs), trained with gradient-based optimization and backpropagation, are currently the primary tool in modern artificial intelligence, machine learning, and data science. In many applications, DNNs are trained offline, through supervised learning or reinforcement learning, and deployed online for inference. However, training DNNs with standard backpropagation and gradient-based optimization gives no intrinsic performance guarantees or bounds on the DNN, which is essential for applications such as controls. Additionally, many offline-training and online-inference problems, such as sim2real transfer of reinforcement learning policies, experience domain shift from the training distribution to the real-world distribution. To address these stability and transfer learning issues, we propose using techniques from control theory to update DNN parameters online. We formulate the fully-connected feedforward DNN as a continuous-time dynamical system, and we propose novel last-layer update laws that guarantee desirable error convergence under various conditions on the time derivative of the DNN input vector. We further show that training the DNN under spectral normalization controls the upper bound of the error trajectories of the online DNN predictions, which is desirable when numerically differentiated quantities or noisy state measurements are input to the DNN. The proposed online DNN adaptation laws are validated in simulation to learn the dynamics of the Van der Pol system under domain shift, where parameters are varied in inference from the training dataset. The simulations demonstrate the effectiveness of using control-theoretic techniques to derive performance improvements and guarantees in DNN-based learning systems.
To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming
Venue: arXiv cs.HC 2024
Year: 2024

The convergence of generative AI and web search is reshaping problem-solving for programmers. However, the lack of understanding regarding their interplay in the information-seeking process often leads programmers to perceive them as alternatives rather than complementary tools. To analyze this interaction and explore their synergy, we conducted an interview study with eight experienced programmers. Drawing from the results and literature, we have identified three major challenges and proposed three decision-making stages, each with its own relevant factors. Additionally, we present a comprehensive process model that captures programmers' interaction patterns. This model encompasses decision-making stages, the information-foraging loop, and cognitive activities during system interaction, offering a holistic framework to comprehend and optimize the use of these convergent tools in programming.
Hausdorff Reductions and the Exponential Hierarchies
Venue: arXiv cs.CC 2024
Year: 2024

The Strong Exponential Hierarchy $SEH$ was shown to collapse to $P^{NExp}$ by Hemachandra by proving $P^{NExp} = NP^{NExp}$ via a census argument. Nonetheless, Hemachandra also asked for certificate-based and alternating Turing machine characterizations of the $SEH$ levels, in the hope that these might have revealed deeper structural reasons behind the collapse. These open questions have thus far remained unanswered. To close them, by building upon the notion of Hausdorff reductions, we investigate a natural normal form for the intermediate levels of the (generalized) exponential hierarchies, i.e., the single-, the double-Exponential Hierarchy, and so on. Although the two characterizations asked for derive from our Hausdorff characterization, it is nevertheless from the latter that a surprising structural reason behind the collapse of $SEH$ is uncovered as a consequence of a very general result: the intermediate levels of the exponential hierarchies are precisely characterized by specific "Hausdorff classes", which define these levels without resorting to oracle machines. By this, contrarily to oracle classes, which may have different shapes for a same class (e.g., $P^{NP}_{||} = P^{NP[Log]} = LogSpace^{NP}$), hierarchy intermediate levels are univocally identified by Hausdorff classes (under the hypothesis of no hierarchy collapse). In fact, we show that the rather simple reason behind many equivalences of oracle classes is that they just refer to different ways of deciding the languages of a same Hausdorff class, and this happens also for $P^{NExp}$ and $NP^{NExp}$. In addition, via Hausdorff classes, we define complete problems for various intermediate levels of the exponential hierarchies. Through these, we obtain matching lower-bounds for problems known to be in $P^{NExp[Log]}$, but whose hardness was left open due to the lack of known $P^{NExp[Log]}$-complete problems.
Data On the Go: Seamless Data Routing for Intermittently-Powered Battery-Free Sensing
Venue: arXiv cs.NI 2024
Year: 2024

The rising demand for sustainable IoT has promoted the adoption of battery-free devices intermittently powered by ambient energy for sensing. However, the intermittency poses significant challenges in sensing data collection. Despite recent efforts to enable one-to-one communication, routing data across multiple intermittently-powered battery-free devices, a crucial requirement for a sensing system, remains a formidable challenge. This paper fills this gap by introducing Swift, which enables seamless data routing in intermittently-powered battery-free sensing systems. Swift overcomes the challenges posed by device intermittency and heterogeneous energy conditions through three major innovative designs. First, Swift incorporates a reliable node synchronization protocol backed by number theory, ensuring successful synchronization regardless of energy conditions. Second, Swift adopts a low-latency message forwarding protocol, allowing continuous message forwarding without repeated synchronization. Finally, Swift features a simple yet effective mechanism for routing path construction, enabling nodes to obtain the optimal path to the sink node with minimum hops. We implement Swift and perform large-scale experiments representing diverse realworld scenarios. The results demonstrate that Swift achieves an order of magnitude reduction in end-to-end message delivery time compared with the state-of-the-art approaches for intermittentlypowered battery-free sensing systems.
dRG-MEC: Decentralized Reinforced Green Offloading for MEC-enabled Cloud Network
Venue: arXiv cs.NI 2024
Year: 2024

Multi-access-Mobile Edge Computing (MEC) is a promising solution for computationally demanding rigorous applications, that can meet 6G network service requirements. However, edge servers incur high computation costs during task processing. In this paper, we proposed a technique to minimize the total computation and communication overhead for optimal resource utilization with joint computational offloading that enables a green environment. Our optimization problem is NP-hard; thus, we proposed a decentralized Reinforcement Learning (dRL) approach where we eliminate the problem of dimensionality and over-estimation of the value functions. Compared to baseline schemes our technique achieves a 37.03% reduction in total system costs.
Utilizing Large Language Models to Translate RFC Protocol Specifications to CPSA Definitions
Venue: arXiv cs.CR 2024
Year: 2024

This paper proposes the use of Large Language Models (LLMs) for translating Request for Comments (RFC) protocol specifications into a format compatible with the Cryptographic Protocol Shapes Analyzer (CPSA). This novel approach aims to reduce the complexities and efforts involved in protocol analysis, by offering an automated method for translating protocol specifications into structured models suitable for CPSA. In this paper we discuss the implementation of an RFC Protocol Translator, its impact on enhancing the accessibility of formal methods analysis, and its potential for improving the security of internet protocols.
Weakly Supervised Learners for Correction of AI Errors with Provable Performance Guarantees
Venue: arXiv cs.LG 2024
Year: 2024

We present a new methodology for handling AI errors by introducing weakly supervised AI error correctors with a priori performance guarantees. These AI correctors are auxiliary maps whose role is to moderate the decisions of some previously constructed underlying classifier by either approving or rejecting its decisions. The rejection of a decision can be used as a signal to suggest abstaining from making a decision. A key technical focus of the work is in providing performance guarantees for these new AI correctors through bounds on the probabilities of incorrect decisions. These bounds are distribution agnostic and do not rely on assumptions on the data dimension. Our empirical example illustrates how the framework can be applied to improve the performance of an image classifier in a challenging real-world task where training data are scarce.
Graph Domain Adaptation: Challenges, Progress and Prospects
Venue: arXiv cs.LG 2024
Year: 2024

As graph representation learning often suffers from label scarcity problems in real-world applications, researchers have proposed graph domain adaptation (GDA) as an effective knowledge-transfer paradigm across graphs. In particular, to enhance model performance on target graphs with specific tasks, GDA introduces a bunch of task-related graphs as source graphs and adapts the knowledge learnt from source graphs to the target graphs. Since GDA combines the advantages of graph representation learning and domain adaptation, it has become a promising direction of transfer learning on graphs and has attracted an increasing amount of research interest in recent years. In this paper, we comprehensively overview the studies of GDA and present a detailed survey of recent advances. Specifically, we outline the research status and challenges, propose a taxonomy, introduce the details of representative works, and discuss the prospects. To the best of our knowledge, this paper is the first survey for graph domain adaptation. A detailed paper list is available at https://github.com/Skyorca/Awesome-Graph-Domain-Adaptation-Papers.
A Practical Evaluation of Commercial Industrial Augmented Reality Systems in an Industry 4.0 Shipyard
Venue: arXiv cs.HC 2024
Year: 2024

The principles of the Industry 4.0 are guiding manufacturing companies towards more automated and computerized factories. Such principles are also applied in shipbuilding, which usually involves numerous complex processes whose automation will improve its efficiency and performance. Navantia, a company that has been building ships for 300 years, is modernizing its shipyards according to the Industry 4.0 principles with the help of the latest technologies. Augmented Reality (AR), which when utilized in an industrial environment is called Industrial AR (IAR), is one of such technologies, since it can be applied in numerous situations in order to provide useful and attractive interfaces that allow shipyard operators to obtain information on their tasks and to interact with certain elements that surround them. This article first reviews the state of the art on IAR applications for shipbuilding and smart manufacturing. Then, the most relevant IAR hardware and software tools are detailed, as well as the main use cases for the application of IAR in a shipyard. Next, it is described Navantia's IAR system, which is based on a fog-computing architecture. Such a system is evaluated when making use of three IAR devices (a smartphone, a tablet and a pair of smart glasses), two AR SDKs (ARToolKit and Vuforia) and multiple IAR markers, with the objective of determining their performance in a shipyard workshop and inside a ship under construction. The results obtained show remarkable performance differences among the different IAR tools and the impact of factors like lighting, pointing out the best combinations of markers, hardware and software to be used depending on the characteristics of the shipyard scenario.