OpenRaiser/Intern-Atlas huggingface dataset:
Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis
Venue: arXiv cs.LG 2023
Year: 2023

This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL), such that the safety constraint violations are bounded at any point during learning. In a variety of RL applications the safety of the agent is particularly important, e.g. autonomous platforms or robots that work in proximity of humans. As enforcing safety during training might severely limit the agent's exploration, we propose here a new architecture that handles the trade-off between efficient progress and safety during exploration. As the exploration progresses, we update via Bayesian inference Dirichlet-Categorical models of the transition probabilities of the Markov decision process that describes the environment dynamics. This paper proposes a way to approximate moments of belief about the risk associated to the action selection policy. We construct those approximations, and prove the convergence results. We propose a novel method for leveraging the expectation approximations to derive an approximate bound on the confidence that the risk is below a certain level. This approach can be easily interleaved with RL and we present experimental results to showcase the performance of the overall architecture.
Inferring the Graph of Networked Dynamical Systems under Partial Observability and Spatially Colored Noise
Venue: arXiv cs.LG 2023
Year: 2023

In a Networked Dynamical System (NDS), each node is a system whose dynamics are coupled with the dynamics of neighboring nodes. The global dynamics naturally builds on this network of couplings and it is often excited by a noise input with nontrivial structure. The underlying network is unknown in many applications and should be inferred from observed data. We assume: i) Partial observability -- time series data is only available over a subset of the nodes; ii) Input noise -- it is correlated across distinct nodes while temporally independent, i.e., it is spatially colored. We present a feasibility condition on the noise correlation structure wherein there exists a consistent network inference estimator to recover the underlying fundamental dependencies among the observed nodes. Further, we describe a structure identification algorithm that exhibits competitive performance across distinct regimes of network connectivity, observability, and noise correlation.
Optimize and Reduce: A Top-Down Approach for Image Vectorization
Venue: arXiv cs.CV 2023
Year: 2023

Vector image representation is a popular choice when editability and flexibility in resolution are desired. However, most images are only available in raster form, making raster-to-vector image conversion (vectorization) an important task. Classical methods for vectorization are either domain-specific or yield an abundance of shapes which limits editability and interpretability. Learning-based methods, that use differentiable rendering, have revolutionized vectorization, at the cost of poor generalization to out-of-training distribution domains, and optimization-based counterparts are either slow or produce non-editable and redundant shapes. In this work, we propose Optimize & Reduce (O&R), a top-down approach to vectorization that is both fast and domain-agnostic. O&R aims to attain a compact representation of input images by iteratively optimizing Bézier curve parameters and significantly reducing the number of shapes, using a devised importance measure. We contribute a benchmark of five datasets comprising images from a broad spectrum of image complexities - from emojis to natural-like images. Through extensive experiments on hundreds of images, we demonstrate that our method is domain agnostic and outperforms existing works in both reconstruction and perceptual quality for a fixed number of shapes. Moreover, we show that our algorithm is $\times 10$ faster than the state-of-the-art optimization-based method.
Vesicoureteral Reflux Detection with Reliable Probabilistic Outputs
Venue: arXiv cs.LG 2023
Year: 2023

Vesicoureteral Reflux (VUR) is a pediatric disorder in which urine flows backwards from the bladder to the upper urinary tract. Its detection is of great importance as it increases the risk of a Urinary Tract Infection, which can then lead to a kidney infection since bacteria may have direct access to the kidneys. Unfortunately the detection of VUR requires a rather painful medical examination, called voiding cysteourethrogram (VCUG), that exposes the child to radiation. In an effort to avoid the exposure to radiation required by VCUG some recent studies examined the use of machine learning techniques for the detection of VUR based on data that can be obtained without exposing the child to radiation. This work takes one step further by proposing an approach that provides lower and upper bounds for the conditional probability of a given child having VUR. The important property of these bounds is that they are guaranteed (up to statistical fluctuations) to contain well-calibrated probabilities with the only requirement that observations are independent and identically distributed (i.i.d.). Therefore they are much more informative and reliable than the plain yes/no answers provided by other techniques.
SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing
Venue: arXiv cs.CV 2023
Year: 2023

Image diffusion models have been utilized in various tasks, such as text-to-image generation and controllable image synthesis. Recent research has introduced tuning methods that make subtle adjustments to the original models, yielding promising results in specific adaptations of foundational generative diffusion models. Rather than modifying the main backbone of the diffusion model, we delve into the role of skip connection in U-Net and reveal that hierarchical features aggregating long-distance information across encoder and decoder make a significant impact on the content and quality of image generation. Based on the observation, we propose an efficient generative tuning framework, dubbed SCEdit, which integrates and edits Skip Connection using a lightweight tuning module named SC-Tuner. Furthermore, the proposed framework allows for straightforward extension to controllable image synthesis by injecting different conditions with Controllable SC-Tuner, simplifying and unifying the network design for multi-condition inputs. Our SCEdit substantially reduces training parameters, memory usage, and computational expense due to its lightweight tuners, with backward propagation only passing to the decoder blocks. Extensive experiments conducted on text-to-image generation and controllable image synthesis tasks demonstrate the superiority of our method in terms of efficiency and performance. Project page: \url{https://scedit.github.io/}
Towards Cognitive Load Assessment Using Electrooculography Measures
Venue: arXiv cs.HC 2023
Year: 2023

Cognitive load assessment is crucial for understanding human performance in various domains. This study investigates the impact of different task conditions and time constraints on cognitive load using multiple measures, including subjective evaluations, performance metrics, and physiological eye-tracking data. Fifteen participants completed a series of primary and secondary tasks with different time limits. The NASA-TLX questionnaire, reaction time, inverse efficiency score, and eye-related features (blink, saccade, and fixation frequency) were utilized to assess cognitive load. The study results show significant differences in the level of cognitive load required for different tasks and when under time constraints. The study also found that there was a positive correlation (r = 0.331, p = 0.014) between how often participants blinked their eyes and the level of cognitive load required but a negative correlation (r = -0.290, p = 0.032) between how often participants made quick eye movements (saccades) and the level of cognitive load required. Additionally, the analysis revealed a significant negative correlation (r = -0.347, p = 0.009) and (r = -0.370, p = 0.005) between fixation and saccade frequencies under time constraints.
Synthetic Shifts to Initial Seed Vector Exposes the Brittle Nature of Latent-Based Diffusion Models
Venue: arXiv cs.CV 2023
Year: 2023

Recent advances in Conditional Diffusion Models have led to substantial capabilities in various domains. However, understanding the impact of variations in the initial seed vector remains an underexplored area of concern. Particularly, latent-based diffusion models display inconsistencies in image generation under standard conditions when initialized with suboptimal initial seed vectors. To understand the impact of the initial seed vector on generated samples, we propose a reliability evaluation framework that evaluates the generated samples of a diffusion model when the initial seed vector is subjected to various synthetic shifts. Our results indicate that slight manipulations to the initial seed vector of the state-of-the-art Stable Diffusion (Rombach et al., 2022) can lead to significant disturbances in the generated samples, consequently creating images without the effect of conditioning variables. In contrast, GLIDE (Nichol et al., 2022) stands out in generating reliable samples even when the initial seed vector is transformed. Thus, our study sheds light on the importance of the selection and the impact of the initial seed vector in the latent-based diffusion model.
A Red Teaming Framework for Securing AI in Maritime Autonomous Systems
Venue: arXiv cs.CR 2023
Year: 2023

Artificial intelligence (AI) is being ubiquitously adopted to automate processes in science and industry. However, due to its often intricate and opaque nature, AI has been shown to possess inherent vulnerabilities which can be maliciously exploited with adversarial AI, potentially putting AI users and developers at both cyber and physical risk. In addition, there is insufficient comprehension of the real-world effects of adversarial AI and an inadequacy of AI security examinations; therefore, the growing threat landscape is unknown for many AI solutions. To mitigate this issue, we propose one of the first red team frameworks for evaluating the AI security of maritime autonomous systems. The framework provides operators with a proactive (secure by design) and reactive (post-deployment evaluation) response to securing AI technology today and in the future. This framework is a multi-part checklist, which can be tailored to different systems and requirements. We demonstrate this framework to be highly effective for a red team to use to uncover numerous vulnerabilities within a real-world maritime autonomous systems AI, ranging from poisoning to adversarial patch attacks. The lessons learned from systematic AI red teaming can help prevent MAS-related catastrophic events in a world with increasing uptake and reliance on mission-critical AI.
Assessing SATNet's Ability to Solve the Symbol Grounding Problem
Venue: arXiv cs.AI 2023
Year: 2023

SATNet is an award-winning MAXSAT solver that can be used to infer logical rules and integrated as a differentiable layer in a deep neural network. It had been shown to solve Sudoku puzzles visually from examples of puzzle digit images, and was heralded as an impressive achievement towards the longstanding AI goal of combining pattern recognition with logical reasoning. In this paper, we clarify SATNet's capabilities by showing that in the absence of intermediate labels that identify individual Sudoku digit images with their logical representations, SATNet completely fails at visual Sudoku (0% test accuracy). More generally, the failure can be pinpointed to its inability to learn to assign symbols to perceptual phenomena, also known as the symbol grounding problem, which has long been thought to be a prerequisite for intelligent agents to perform real-world logical reasoning. We propose an MNIST based test as an easy instance of the symbol grounding problem that can serve as a sanity check for differentiable symbolic solvers in general. Naive applications of SATNet on this test lead to performance worse than that of models without logical reasoning capabilities. We report on the causes of SATNet's failure and how to prevent them.
ABiMed: An intelligent and visual clinical decision support system for medication reviews and polypharmacy management
Venue: arXiv cs.HC 2023
Year: 2023

Background: Polypharmacy, i.e. taking five drugs or more, is both a public health and an economic issue. Medication reviews are structured interviews of the patient by the community pharmacist, aiming at optimizing the drug treatment and deprescribing useless, redundant or dangerous drugs. However, they remain difficult to perform and time-consuming. Several clinical decision support systems were developed for helping clinicians to manage polypharmacy. However, most were limited to the implementation of clinical practice guidelines. In this work, our objective is to design an innovative clinical decision support system for medication reviews and polypharmacy management, named ABiMed. Methods: ABiMed associates several approaches: guidelines implementation, but the automatic extraction of patient data from the GP's electronic health record and its transfer to the pharmacist, and the visual presentation of contextualized drug knowledge using visual analytics. We performed an ergonomic assessment and qualitative evaluations involving pharmacists and GPs during focus groups and workshops. Results: We describe the proposed architecture, which allows a collaborative multi-user usage. We present the various screens of ABiMed for entering or verifying patient data, for accessing drug knowledge (posology, adverse effects, interactions), for viewing STOPP/START rules and for suggesting modification to the treatment. Qualitative evaluations showed that health professionals were highly interested by our approach, associating the automatic guidelines execution with the visual presentation of drug knowledge. Conclusions: The association of guidelines implementation with visual presentation of knowledge is a promising approach for managing polypharmacy. Future works will focus on the improvement and the evaluation of ABiMed.
Deciphering Compatibility Relationships with Textual Descriptions via Extraction and Explanation
Venue: arXiv cs.CL 2023
Year: 2023

Understanding and accurately explaining compatibility relationships between fashion items is a challenging problem in the burgeoning domain of AI-driven outfit recommendations. Present models, while making strides in this area, still occasionally fall short, offering explanations that can be elementary and repetitive. This work aims to address these shortcomings by introducing the Pair Fashion Explanation (PFE) dataset, a unique resource that has been curated to illuminate these compatibility relationships. Furthermore, we propose an innovative two-stage pipeline model that leverages this dataset. This fine-tuning allows the model to generate explanations that convey the compatibility relationships between items. Our experiments showcase the model's potential in crafting descriptions that are knowledgeable, aligned with ground-truth matching correlations, and that produce understandable and informative descriptions, as assessed by both automatic metrics and human evaluation. Our code and data are released at https://github.com/wangyu-ustc/PairFashionExplanation
CR-SFP: Learning Consistent Representation for Soft Filter Pruning
Venue: arXiv cs.CV 2023
Year: 2023

Soft filter pruning~(SFP) has emerged as an effective pruning technique for allowing pruned filters to update and the opportunity for them to regrow to the network. However, this pruning strategy applies training and pruning in an alternative manner, which inevitably causes inconsistent representations between the reconstructed network~(R-NN) at the training and the pruned network~(P-NN) at the inference, resulting in performance degradation. In this paper, we propose to mitigate this gap by learning consistent representation for soft filter pruning, dubbed as CR-SFP. Specifically, for each training step, CR-SFP optimizes the R-NN and P-NN simultaneously with different distorted versions of the same training data, while forcing them to be consistent by minimizing their posterior distribution via the bidirectional KL-divergence loss. Meanwhile, the R-NN and P-NN share backbone parameters thus only additional classifier parameters are introduced. After training, we can export the P-NN for inference. CR-SFP is a simple yet effective training framework to improve the accuracy of P-NN without introducing any additional inference cost. It can also be combined with a variety of pruning criteria and loss functions. Extensive experiments demonstrate our CR-SFP achieves consistent improvements across various CNN architectures. Notably, on ImageNet, our CR-SFP reduces more than 41.8\% FLOPs on ResNet18 with 69.2\% top-1 accuracy, improving SFP by 2.1\% under the same training settings. The code will be publicly available on GitHub.
Towards AI-driven Integrative Emissions Monitoring & Management for Nature-Based Climate Solutions
Venue: arXiv cs.LG 2023
Year: 2023

AI has been proposed as an important tool to support several efforts related to nature-based climate solutions such as the detection of wildfires that affect forests and vegetation-based offsets. While this and other use-cases provide important demonstrative value of the power of AI in climate change mitigation, such efforts have typically been undertaken in silos, without awareness of the integrative nature of real-world climate policy-making. In this paper, we propose a novel overarching framework for AI-aided integrated and comprehensive decision support for various aspects of nature-based climate decision-making. Focusing on vegetation-based solutions such as forests, we demonstrate how different AI-aided decision support models such as AI-aided wildfire detection, AI-aided vegetation carbon stock assessment, reversal risk mitigation, and disaster response planning can be integrated into a comprehensive framework. Rather than being disparate elements, we posit that the exchange of data and analytical results across elements of the framework, and careful mitigation of uncertainty propagation will provide tremendous value relative to the status-quo for real-world climate policy-making.
PR-NeuS: A Prior-based Residual Learning Paradigm for Fast Multi-view Neural Surface Reconstruction
Venue: arXiv cs.CV 2023
Year: 2023

Neural surfaces learning has shown impressive performance in multi-view surface reconstruction. However, most existing methods use large multilayer perceptrons (MLPs) to train their models from scratch, resulting in hours of training for a single scene. Recently, how to accelerate the neural surfaces learning has received a lot of attention and remains an open problem. In this work, we propose a prior-based residual learning paradigm for fast multi-view neural surface reconstruction. This paradigm consists of two optimization stages. In the first stage, we propose to leverage generalization models to generate a basis signed distance function (SDF) field. This initial field can be quickly obtained by fusing multiple local SDF fields produced by generalization models. This provides a coarse global geometry prior. Based on this prior, in the second stage, a fast residual learning strategy based on hash-encoding networks is proposed to encode an offset SDF field for the basis SDF field. Moreover, we introduce a prior-guided sampling scheme to help the residual learning stage converge better, and thus recover finer structures. With our designed paradigm, experimental results show that our method only takes about 3 minutes to reconstruct the surface of a single scene, while achieving competitive surface quality. Our code will be released upon publication.
Towards Establishing Dense Correspondence on Multiview Coronary Angiography: From Point-to-Point to Curve-to-Curve Query Matching
Venue: arXiv cs.CV 2023
Year: 2023

Coronary angiography is the gold standard imaging technique for studying and diagnosing coronary artery disease. However, the resulting 2D X-ray projections lose 3D information and exhibit visual ambiguities. In this work, we aim to establish dense correspondence in multi-view angiography, serving as a fundamental basis for various clinical applications and downstream tasks. To overcome the challenge of unavailable annotated data, we designed a data simulation pipeline using 3D Coronary Computed Tomography Angiography (CCTA). We formulated the problem of dense correspondence estimation as a query matching task over all points of interest in the given views. We established point-to-point query matching and advanced it to curve-to-curve correspondence, significantly reducing errors by minimizing ambiguity and improving topological awareness. The method was evaluated on a set of 1260 image pairs from different views across 8 clinically relevant angulation groups, demonstrating compelling results and indicating the feasibility of establishing dense correspondence in multi-view angiography.
PRP Rebooted: Advancing the State of the Art in FOND Planning
Venue: arXiv cs.AI 2023
Year: 2023

Fully Observable Non-Deterministic (FOND) planning is a variant of classical symbolic planning in which actions are nondeterministic, with an action's outcome known only upon execution. It is a popular planning paradigm with applications ranging from robot planning to dialogue-agent design and reactive synthesis. Over the last 20 years, a number of approaches to FOND planning have emerged. In this work, we establish a new state of the art, following in the footsteps of some of the most powerful FOND planners to date. Our planner, PR2, decisively outperforms the four leading FOND planners, at times by a large margin, in 17 of 18 domains that represent a comprehensive benchmark suite. Ablation studies demonstrate the impact of various techniques we introduce, with the largest improvement coming from our novel FOND-aware heuristic.
MineObserver 2.0: A Deep Learning & In-Game Framework for Assessing Natural Language Descriptions of Minecraft Imagery
Venue: arXiv cs.AI 2023
Year: 2023

MineObserver 2.0 is an AI framework that uses Computer Vision and Natural Language Processing for assessing the accuracy of learner-generated descriptions of Minecraft images that include some scientifically relevant content. The system automatically assesses the accuracy of participant observations, written in natural language, made during science learning activities that take place in Minecraft. We demonstrate our system working in real-time and describe a teacher support dashboard to showcase observations, both of which advance our previous work. We present the results of a study showing that MineObserver 2.0 improves over its predecessor both in perceived accuracy of the system's generated descriptions as well as in usefulness of the system's feedback. In future work we intend improve system-generated descriptions, give teachers more control and upgrade the system to perform continuous learning to more effectively and rapidly respond to novel observations made by learners.