Miracle-dz/UniFAR-FTUs huggingface dataset:
Project Aria: A New Tool for Egocentric Multi-Modal AI Research

Egocentric, multi-modal data as available on future augmented reality (AR) devices provides unique challenges and opportunities for machine perception. These future devices will need to be all-day wearable in a socially acceptable form-factor to support always available, context-aware and personalized AI applications. Our team at Meta Reality Labs Research built the Aria device, an egocentric, multi-modal data recording and streaming device with the goal to foster and accelerate research in this area. In this paper, we describe the Aria device hardware including its sensor configuration and the corresponding software tools that enable recording and processing of such data.
Semi-Supervised Semantic Depth Estimation using Symbiotic Transformer and NearFarMix Augmentation

In computer vision, depth estimation is crucial for domains like robotics, autonomous vehicles, augmented reality, and virtual reality. Integrating semantics with depth enhances scene understanding through reciprocal information sharing. However, the scarcity of semantic information in datasets poses challenges. Existing convolutional approaches with limited local receptive fields hinder the full utilization of the symbiotic potential between depth and semantics. This paper introduces a dataset-invariant semi-supervised strategy to address the scarcity of semantic information. It proposes the Depth Semantics Symbiosis module, leveraging the Symbiotic Transformer for achieving comprehensive mutual awareness by information exchange within both local and global contexts. Additionally, a novel augmentation, NearFarMix is introduced to combat overfitting and compensate both depth-semantic tasks by strategically merging regions from two images, generating diverse and structurally consistent samples with enhanced control. Extensive experiments on NYU-Depth-V2 and KITTI datasets demonstrate the superiority of our proposed techniques in indoor and outdoor environments.
Quantifying Temperature Effects on Fall Chinook Salmon

The motivation for this study was to recommend relationships for use in a model of San Joaquin fall Chinook salmon. This report reviews literature pertaining to relationships between water temperature and fall Chinook salmon. The report is organized into three sections that deal with temperature effects on development and timing of freshwater life stages, temperature effects on incubation survival for eggs and alevin, and temperature effects on juvenile survival. Recommendations are made for modeling temperature influences for all three life stages.
Out-of-distribution detection using normalizing flows on the data manifold

A common approach for out-of-distribution detection involves estimating an underlying data distribution, which assigns a lower likelihood value to out-of-distribution data. Normalizing flows are likelihood-based generative models providing a tractable density estimation via dimension-preserving invertible transformations. Conventional normalizing flows are prone to fail in out-of-distribution detection, because of the well-known curse of dimensionality problem of the likelihood-based models. According to the manifold hypothesis, real-world data often lie on a low-dimensional manifold. This study investigates the effect of manifold learning using normalizing flows on out-of-distribution detection. We proceed by estimating the density on a low-dimensional manifold, coupled with measuring the distance from the manifold, as criteria for out-of-distribution detection. However, individually, each of them is insufficient for this task. The extensive experimental results show that manifold learning improves the out-of-distribution detection ability of a class of likelihood-based models known as normalizing flows. This improvement is achieved without modifying the model structure or using auxiliary out-of-distribution data during training.
Uncertainty-aware Risk Assessment of Robotic Systems via Importance Sampling

In this paper, we introduce a probabilistic approach to risk assessment of robot systems by focusing on the impact of uncertainties. While various approaches to identifying systematic hazards (e.g., bugs, design flaws, etc.) can be found in current literature, little attention has been devoted to evaluating risks in robot systems in a probabilistic manner. Existing methods rely on discrete notions for dangerous events and assume that the consequences of these can be described by simple logical operations. In this work, we consider measurement uncertainties as one main contributor to the evolvement of risks. Specifically, we study the impact of temporal and spatial uncertainties on the occurrence probability of dangerous failures, thereby deriving an approach for an uncertainty-aware risk assessment. Secondly, we introduce a method to improve the statistical significance of our results: While the rare occurrence of hazardous events makes it challenging to draw conclusions with reliable accuracy, we show that importance sampling -- a technique that successively generates samples in regions with sparse probability densities -- allows for overcoming this issue. We demonstrate the validity of our novel uncertainty-aware risk assessment method in three simulation scenarios from the domain of human-robot collaboration. Finally, we show how the results can be used to evaluate arbitrary safety limits of robot systems.
Break-Point Date Estimation for Nonstationary Autoregressive and Predictive Regression Models

In this article, we study the statistical and asymptotic properties of break-point estimators in nonstationary autoregressive and predictive regression models for testing the presence of a single structural break at an unknown location in the full sample. Moreover, we investigate aspects such as how the persistence properties of covariates and the location of the break-point affects the limiting distribution of the proposed break-point estimators.
Towards Autonomous Multi-Modal Mobility Morphobot (M4) Robot: Traversability Estimation and 3D Path Planning

This thesis enhances the autonomy of the M4 (Multi-Modal Mobility Morphobot) robot, designed for Mars and rescue missions. The research enables the robot to autonomously select its locomotion mode and path in complex terrains. Focusing on walking and flying modes, a Gazebo simulation, and custom perception-navigations pipelines are developed. Leveraging deep learning, the robot determines optimal mode transitions based on a 2.5D map. Additionally, an energy efficient path planner based on 2.5D mapping is implemented and validated in simulations. The contributions demonstrate scalability for future mode integrations. The M4 robot showcases intelligent mode switching, efficient navigation, and reduced energy consumption, bringing us closer to fully autonomous multi-modal robots for exploration and rescue missions. This work paves the way for future advancements in autonomous robotics, with the ultimate vision of deploying the M4 robot for exploration and rescue tasks, making a significant impact in the quest for intelligent and versatile robotic systems.
A Milstein-type method for highly non-linear non-autonomous time-changed stochastic differential equations

A Milstein-type method is proposed for some highly non-linear non-autonomous time-changed stochastic differential equations (SDEs). The spatial variables in the coefficients of the time-changed SDEs satisfy the super-linear growth condition and the temporal variables obey some H\"older's continuity condition. The strong convergence in the finite time is studied and the convergence order is obtained.
Using Knowledge Representation and Task Planning for Robot-agnostic Skills on the Example of Contact-Rich Wiping Tasks

The transition to agile manufacturing, Industry 4.0, and high-mix-low-volume tasks require robot programming solutions that are flexible. However, most deployed robot solutions are still statically programmed and use stiff position control, which limit their usefulness. In this paper, we show how a single robot skill that utilizes knowledge representation, task planning, and automatic selection of skill implementations based on the input parameters can be executed in different contexts. We demonstrate how the skill-based control platform enables this with contact-rich wiping tasks on different robot systems. To achieve that in this case study, our approach needs to address different kinematics, gripper types, vendors, and fundamentally different control interfaces. We conducted the experiments with a mobile platform that has a Universal Robots UR5e 6 degree-of-freedom robot arm with position control and a 7 degree-of-freedom KUKA iiwa with torque control.
The Global R-linear Convergence of Nesterov's Accelerated Gradient Method with Unknown Strongly Convex Parameter

The Nesterov accelerated gradient (NAG) method is an important extrapolation-based numerical algorithm that accelerates the convergence of the gradient descent method in convex optimization. When dealing with an objective function that is $\mu$-strongly convex, selecting extrapolation coefficients dependent on $\mu$ enables global R-linear convergence. In cases where $\mu$ is unknown, a commonly adopted approach is to set the extrapolation coefficient using the original NAG method. This choice allows for achieving the optimal iteration complexity among first-order methods for general convex problems. However, it remains unknown whether the NAG method with an unknown strongly convex parameter exhibits global R-linear convergence for strongly convex problems. In this work, we answer this question positively by establishing the Q-linear convergence of certain constructed Lyapunov sequences. Furthermore, we extend our result to the global R-linear convergence of the accelerated proximal gradient method, which is employed for solving strongly convex composite optimization problems. Interestingly, these results contradict the findings of the continuous counterpart of the NAG method in [Su, Boyd, and Cand\'es, J. Mach. Learn. Res., 2016, 17(153), 1-43], where the convergence rate by the suggested ordinary differential equation cannot exceed the $O(1/{\tt poly}(k))$ for strongly convex functions.
U-SEANNet: A Simple, Efficient and Applied U-Shaped Network for Diagnosing Nasal Diseases from Nasal Endoscopic Images

Utilizing deep learning (DL) models to improve the early diagnosis of nasal diseases from nasal endoscopic images holds paramount importance. However, the lack of available datasets stymies advancements in this field. Furthermore, existing models fail to strike a good trade-off between model diagnosis performance, model complexity and parameter size, rendering them unsuitable for practical application. To bridge these gaps, we created the first large-scale nasal endoscopy dataset, named 7-NasEID, comprising 11,352 images that span six nasal diseases and normal samples. Building on this, we proposed U-SEANNet, an innovative architecture, under-pinned by depth-wise separable convolutions. Additionally, to augment its discernment capabilities for subtle variations in input images, we further proposed the Global-Local Channel Feature Fusion Module, enabling the U-SEANNet to focus salient channel features from both global and local contexts. Notably, U-SEANNet’s parameter size and GFLOPs are only 0.78M and 0.21, respectively. Employing the 7-NasalEID, we conducted the five-fold cross-validation on U-SEANNet, juxtaposing its performance against seventeen renowned architectures. The experimental results suggest U-SEANNet as the state-of-the-art (SOTA) model, achieves an accuracy of 93.58%, sensitivity of 90.17%, and specificity of 91.27%. These findings demonstrate U-SEANNet’s prodigious potential for diagnosing nasal diseases in practical use, providing the development of efficacy nasal diseases diagnosis tools with a new insight.
Polyhedral combinatorics of bisectors

For any polyhedral norm, the bisector of two points is a polyhedral complex. We study combinatorial aspects of this complex. We investigate the sensitivity of the presence of labelled maximal cells in the bisector relative to the position of the two points. We thereby extend work of Criado, Joswig and Santos (2022) who showed that for the tropical distance function the presence of maximal cells is encoded by a polyhedral fan, the bisection fan. We initiate the study of bisection cones and bisection fans with respect to arbitrary polyhedral norms. In particular, we show that the bisection fan always exists for polyhedral norms in two dimensions. Furthermore, we determine the bisection fan of the $\ell_{1}$-norm and the $\ell_{\infty}$-norm as well as the discrete Wasserstein distance in arbitrary dimensions. Intricate combinatorial structures, such as the resonance arrangement, make their appearance. We apply our results to obtain bounds on the combinatorial complexity of the bisectors.
Planning with Logical Graph-based Language Model for Instruction Generation

Despite the superior performance of large language models to generate natural language texts, it is hard to generate texts with correct logic according to a given task, due to the difficulties for neural models to capture implied rules from free-form texts. In this paper, we propose a novel graph-based language model, Logical-GLM, to infuse logic into language models for more valid text generation and interpretability. Specifically, we first capture information from natural language instructions and construct logical bayes graphs that generally describe domains. Next, we generate logical skeletons to guide language model training, infusing domain knowledge into language models. Finally, we alternately optimize the searching policy of graphs and language models until convergence. The experimental results show that Logical-GLM is both effective and efficient compared with traditional language models, despite using smaller-scale training data and fewer parameters. Our approach can generate instructional texts with more correct logic owing to the internalized domain knowledge. Moreover, the usage of logical graphs reflects the inner mechanism of the language models, which improves the interpretability of black-box models.
RESTORE: Graph Embedding Assessment Through Reconstruction

Following the success of Word2Vec embeddings, graph embeddings (GEs) have gained substantial traction. GEs are commonly generated and evaluated extrinsically on downstream applications, but intrinsic evaluations of the original graph properties in terms of topological structure and semantic information have been lacking. Understanding these will help identify the deficiency of the various families of GE methods when vectorizing graphs in terms of preserving the relevant knowledge or learning incorrect knowledge. To address this, we propose RESTORE, a framework for intrinsic GEs assessment through graph reconstruction. We show that reconstructing the original graph from the underlying GEs yields insights into the relative amount of information preserved in a given vector form. We first introduce the graph reconstruction task. We generate GEs from three GE families based on factorization methods, random walks, and deep learning (with representative algorithms from each family) on the CommonSense Knowledge Graph (CSKG). We analyze their effectiveness in preserving the (a) topological structure of node-level graph reconstruction with an increasing number of hops and (b) semantic information on various word semantic and analogy tests. Our evaluations show deep learning-based GE algorithm (SDNE) is overall better at preserving (a) with a mean average precision (mAP) of 0.54 and 0.35 for 2 and 3-hop reconstruction respectively, while the factorization-based algorithm (HOPE) is better at encapsulating (b) with an average Euclidean distance of 0.14, 0.17, and 0.11 for 1, 2, and 3-hop reconstruction respectively. The modest performance of these GEs leaves room for further research avenues on better graph representation learning.
Solving Fréchet Distance Problems by Algebraic Geometric Methods

We study several polygonal curve problems under the Fr\'{e}chet distance via algebraic geometric methods. Let $\mathbb{X}_m^d$ and $\mathbb{X}_k^d$ be the spaces of all polygonal curves of $m$ and $k$ vertices in $\mathbb{R}^d$, respectively. We assume that $k \leq m$. Let $\mathcal{R}^d_{k,m}$ be the set of ranges in $\mathbb{X}_m^d$ for all possible metric balls of polygonal curves in $\mathbb{X}_k^d$ under the Fr\'{e}chet distance. We prove a nearly optimal bound of $O(dk\log (km))$ on the VC dimension of the range space $(\mathbb{X}_m^d,\mathcal{R}_{k,m}^d)$, improving on the previous $O(d^2k^2\log(dkm))$ upper bound and approaching the current $\Omega(dk\log k)$ lower bound. Our upper bound also holds for the weak Fr\'{e}chet distance. We also obtain exact solutions that are hitherto unknown for curve simplification, range searching, nearest neighbor search, and distance oracle.
From Humean Laws to a Neo-Kantian Spacetime: A Dynamics-First View of Topology

Do the spacetime manifolds which feature in our best scientific theories reflect anything metaphysically weighty in the world (e.g., any fundamental substances or relations)? Should we extend our notions of space and time beyond the epistemological roles they play in helping us codify the dynamical behavior of matter? Kant famously answered ``No'' to both of these questions, contra Newton and Leibniz. This paper introduces novel technical and philosophical support for such a (Neo-)Kantian perspective on the metaphysics of space and time. To begin, I will make an explicit analogy between broadly Humean views of laws (e.g., Lewis, Demarest, etc.) and dynamics-first views of geometry (e.g., Brown). I will then continue this line of analogous views beyond the metaphysics of laws debate and the dynamical vs geometric spacetime debate by extending it into the context of spacetime topology. Namely, I will put forward a dynamics-first view of topology (in answer to Norton's problem of pre-geometry). This dynamics-first view of topology is supported by some powerful new techniques for topological redescription which I have recently developed in Grimmer (2023a) and Grimmer (2023b), namely the ISE Method. These new techniques allow us to remove and replace the topological underpinnings of our spacetime theories just as easily as we can switch between different coordinate systems. For instance, a theory set on a M\"obius strip might be redescribed as being set on the Euclidean plane and vice versa. Indeed, as Grimmer (2023b) has proved, the ISE Method gives us access to effectively every possible spacetime framing of a given theory's kinematical and dynamical content. Given this overabundance of candidate spacetime framings, it is then conceivable that one can pick out a theory's topological structure via something analogous to a Best Systems Analysis.
Fluctuation relation in continuous-time random walks driven by an external field

We study a fluctuation relation representing a non-equilibrium equality indicating that the ratio between the distribution of trajectories obtained by exchanging the initial and final positions is characterized by free energy differences for the duration of the trajectories. We examine the fluctuation relation for noninteracting charge carriers driven by an external electric field by using a continuous-time lattice random walk model with a general waiting-time distribution of transitions. The fluctuation relation is obtained regardless of the lattice structure factor or the form of the waiting-time distribution. However, the fluctuation relation is satisfied only after taking the continuum limit in the presence of a reflecting boundary. Moreover, in free space without boundary conditions, exchanging the initial and final positions is equivalent to exchanging the field (or drift) directions. However, we show that the exchanging field (or drift) directions is not relevant for studying the fluctuation relation under the reflecting boundary condition.
VesselShot: Few-shot learning for cerebral blood vessel segmentation

Angiography is widely used to detect, diagnose, and treat cerebrovascular diseases. While numerous techniques have been proposed to segment the vascular network from different imaging modalities, deep learning (DL) has emerged as a promising approach. However, existing DL methods often depend on proprietary datasets and extensive manual annotation. Moreover, the availability of pre-trained networks specifically for medical domains and 3D volumes is limited. To overcome these challenges, we propose a few-shot learning approach called VesselShot for cerebrovascular segmentation. VesselShot leverages knowledge from a few annotated support images and mitigates the scarcity of labeled data and the need for extensive annotation in cerebral blood vessel segmentation. We evaluated the performance of VesselShot using the publicly available TubeTK dataset for the segmentation task, achieving a mean Dice coefficient (DC) of 0.62(0.03).
Accretion onto Oscillating Cosmic String Loops

Cosmic string loops are non-linear density fluctuations which form in the early universe and could play an important role in explaining many phenomena which are in tension with the standard $\Lambda$CDM model. Hence, the details of the accretion process onto cosmic string loops should be studied in detail. Most previous works view loops as point masses and ignore the impact of a finite loop size. In this work, we utilize the Zel'dovich approximation to calculate the non-linear mass sourced by a static extended loop with a time-averaged density profile derived from the trajectory of the loop oscillation, and compare the result with what is obtained for a point-mass source. We find that the finite size of a loop mainly affects the evolution of turnaround shells during the early stages of accretion, converging to the point mass result after a critical redshift, $z^{(II)/(III)}_{c}$. For $z>z^{(II)/(III)}_{c}$, the total accreted mass surrounding a loop is suppressed relative to the point mass case and has a growth rate proportional to $(1+z)^{-3/2}$. As an immediate extension, we also qualitatively analyse the accretion onto moving point masses and onto moving extended loops. In addition to the reduction in the nonlinear mass, the loop finite size also changes the shape of the turnaround surface at early stages of accretion.
Building Trust in Conversational AI: A Comprehensive Review and Solution Architecture for Explainable, Privacy-Aware Systems using LLMs and Knowledge Graph

Conversational AI systems have emerged as key enablers of human-like interactions across diverse sectors. Nevertheless, the balance between linguistic nuance and factual accuracy has proven elusive. In this paper, we first introduce LLMXplorer, a comprehensive tool that provides an in-depth review of over 150 Large Language Models (LLMs), elucidating their myriad implications ranging from social and ethical to regulatory, as well as their applicability across industries. Building on this foundation, we propose a novel functional architecture that seamlessly integrates the structured dynamics of Knowledge Graphs with the linguistic capabilities of LLMs. Validated using real-world AI news data, our architecture adeptly blends linguistic sophistication with factual rigour and further strengthens data security through Role-Based Access Control. This research provides insights into the evolving landscape of conversational AI, emphasizing the imperative for systems that are efficient, transparent, and trustworthy.