Miracle-dz/UniFAR-FTUs huggingface dataset:
Graph Neural Networks in Vision-Language Image Understanding: A Survey

2D image understanding is a complex problem within Computer Vision, but it holds the key to providing human level scene comprehension. It goes further than identifying the objects in an image, and instead it attempts to understand the scene. Solutions to this problem form the underpinning of a range of tasks, including image captioning, Visual Question Answering (VQA), and image retrieval. Graphs provide a natural way to represent the relational arrangement between objects in an image, and thus in recent years Graph Neural Networks (GNNs) have become a standard component of many 2D image understanding pipelines, becoming a core architectural component especially in the VQA group of tasks. In this survey, we review this rapidly evolving field and we provide a taxonomy of graph types used in 2D image understanding approaches, a comprehensive list of the GNN models used in this domain, and a roadmap of future potential developments. To the best of our knowledge, this is the first comprehensive survey that covers image captioning, visual question answering, and image retrieval techniques that focus on using GNNs as the main part of their architecture.
When is Importance Weighting Correction Needed for Covariate Shift Adaptation?

This paper investigates when the importance weighting (IW) correction is needed to address covariate shift, a common situation in supervised learning where the input distributions of training and test data differ. Classic results show that the IW correction is needed when the model is parametric and misspecified. In contrast, recent results indicate that the IW correction may not be necessary when the model is nonparametric and well-specified. We examine the missing case in the literature where the model is nonparametric and misspecified, and show that the IW correction is needed for obtaining the best approximation of the true unknown function for the test distribution. We do this by analyzing IW-corrected kernel ridge regression, covering a variety of settings, including parametric and nonparametric models, well-specified and misspecified settings, and arbitrary weighting functions.
Long Distance GNSS-Denied Visual Inertial Navigation for Autonomous Fixed Wing Unmanned Air Vehicles: SO(3) Manifold Filter based on Virtual Vision Sensor

This article proposes a visual inertial navigation algorithm intended to diminish the horizontal position drift experienced by autonomous fixed-wing UAVs (unmanned air vehicles) in the absence of GNSS (Global Navigation Satellite System) signals. In addition to accelerometers, gyroscopes, and magnetometers, the proposed navigation filter relies on the accurate incremental displacement outputs generated by a VO (visual odometry) system, denoted here as a virtual vision sensor, or VVS, which relies on images of the Earth surface taken by an onboard camera and is itself assisted by filter inertial estimations. Although not a full replacement for a GNSS receiver since its position observations are relative instead of absolute, the proposed system enables major reductions in the GNSS-denied attitude and position estimation errors. The filter is implemented in the manifold of rigid body rotations or SO(3) in order to minimize the accumulation of errors in the absence of absolute observations. Stochastic high-fidelity simulations of two representative scenarios involving the loss of GNSS signals are employed to evaluate the results. The authors release the C++ implementation of both the visual inertial navigation filter and the high-fidelity simulation as open-source software.
An Introduction To Continuous Model Theory

We present an introduction to modern continuous model theory with an emphasis on its interactions with topics covered in this volume such as $C^*$-algebras and von Neumann algebras. The role of ultraproducts is highlighted and expositions of definable sets, imaginaries, quantifier elimination and separable categoricity are included.
Notable: On-the-fly Assistant for Data Storytelling in Computational Notebooks

Computational notebooks are widely used for data analysis. Their interleaved displays of code and execution results (e.g., visualizations) are welcomed since they enable iterative analysis and preserve the exploration process. However, the communication of data findings remains challenging in computational notebooks. Users have to carefully identify useful findings from useless ones, document them with texts and visual embellishments, and then organize them in different tools. Such workflow greatly increases their workload, according to our interviews with practitioners. To address the challenge, we designed Notable to offer on-the-fly assistance for data storytelling in computational notebooks. It provides intelligent support to minimize the work of documenting and organizing data findings and diminishes the cost of switching between data exploration and storytelling. To evaluate Notable, we conducted a user study with 12 data workers. The feedback from user study participants verifies its effectiveness and usability.
3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction

Rich data and powerful machine learning models allow us to design drugs for a specific protein target \textit{in silico}. Recently, the inclusion of 3D structures during targeted drug design shows superior performance to other target-free models as the atomic interaction in the 3D space is explicitly modeled. However, current 3D target-aware models either rely on the voxelized atom densities or the autoregressive sampling process, which are not equivariant to rotation or easily violate geometric constraints resulting in unrealistic structures. In this work, we develop a 3D equivariant diffusion model to solve the above challenges. To achieve target-aware molecule design, our method learns a joint generative process of both continuous atom coordinates and categorical atom types with a SE(3)-equivariant network. Moreover, we show that our model can serve as an unsupervised feature extractor to estimate the binding affinity under proper parameterization, which provides an effective way for drug screening. To evaluate our model, we propose a comprehensive framework to evaluate the quality of sampled molecules from different dimensions. Empirical studies show our model could generate molecules with more realistic 3D structures and better affinities towards the protein targets, and improve binding affinity ranking and prediction without retraining.
CAE-CNNLoc: An Edge-based WiFi Fingerprinting Indoor Localization Using Convolutional Neural Network and Convolutional Auto-Encoder

With the ongoing development of Indoor Location-Based Services, accurate location information of users in indoor environments has been a challenging issue in recent years. Due to the widespread use of WiFi networks, WiFi fingerprinting has become one of the most practical methods of locating mobile users. In addition to localization accuracy, some other critical factors such as cost, latency, and users' privacy should be considered in indoor localization systems. In this study, we propose a lightweight Convolutional Neural Network (CNN)-based method for edge devices (such as smartphones) to overcome the above issues by eliminating the need for a cloud/server in the localization system. To enable the use of the proposed model on resource-constraint edge devices, post-training optimization techniques including quantization, pruning and clustering are used to compress the network model. The proposed method is evaluated for three different open datasets, i.e., UJIIndoorLoc, Tampere and UTSIndoorLoc, as well as for our collected dataset named SBUK-D to verify its scalability. The results demonstrate the superiority of the proposed method compared to state-of-the-art studies. We also evaluate performance efficiency of our localization method on an android smartphone to demonstrate its applicability to edge devices. For UJIIndoorLoc dataset, our model with post-training optimizations obtains approximately 99% building accuracy, over 98% floor accuracy, and 4 m positioning mean error with the model size and inference time of 60 KB and 270 us, respectively, which demonstrate high accuracy as well as amenability to the resource-constrained edge devices.
Smart patterning for topological pumping of elastic surface waves

Topological pumping allows waves to navigate a sample undisturbed by disorders and defects. We demonstrate this phenomenon with elastic surface waves by strategically patterning an elastic surface to create a synthetic dimension. The surface is decorated with arrays of resonating pillars that are connected by spatially slow-varying coupling bridges and support eigenmodes located below the sound cone. We establish a connection between the collective dynamics of the pillars and that of electrons in a magnetic field by developing a tight-binding model and a WKB (Wentzel-Kramers-Brillouin) analysis. This enables us to predict the topological pumping pattern, which we validate through numerical and experimental steering of waves from one edge to the other. Furthermore, we observe the immune nature of the topologically pumped surface waves to disorder and defects. The combination of surface patterning and WKB analysis provides a versatile platform for controlling surface waves and exploring topological matter in higher dimensions.
Modeling of Cavitating Flow through Waterjet Propulsors

A computational-fluid-dynamics-based modeling effort to capture flow through an axial flow waterjet propulsor is presented. The effort covered the waterjet flow over a wide range of flow coefficients and into cavitation-driven breakdown. The computations are presented in cavitation at two values of flow coefficient through a series of decreasing operating inlet total pressure. The computational results are compared to experimental measurements. Suction-surface and tip-gap cavitation patterns are presented and compared to experimental photographs. Presented computational solutions are blade-passage steady and periodic. The computational results apply a powering iteration methodology to facilitate coupling of rotor, stator, and inflow and outflow ducting.
SGDA: Towards 3D Universal Pulmonary Nodule Detection via Slice Grouped Domain Attention

Lung cancer is the leading cause of cancer death worldwide. The best solution for lung cancer is to diagnose the pulmonary nodules in the early stage, which is usually accomplished with the aid of thoracic computed tomography (CT). As deep learning thrives, convolutional neural networks (CNNs) have been introduced into pulmonary nodule detection to help doctors in this labor-intensive task and demonstrated to be very effective. However, the current pulmonary nodule detection methods are usually domain-specific, and cannot satisfy the requirement of working in diverse real-world scenarios. To address this issue, we propose a slice grouped domain attention (SGDA) module to enhance the generalization capability of the pulmonary nodule detection networks. This attention module works in the axial, coronal, and sagittal directions. In each direction, we divide the input feature into groups, and for each group, we utilize a universal adapter bank to capture the feature subspaces of the domains spanned by all pulmonary nodule datasets. Then the bank outputs are combined from the perspective of domain to modulate the input group. Extensive experiments demonstrate that SGDA enables substantially better multi-domain pulmonary nodule detection performance compared with the state-of-the-art multi-domain learning methods.
Incorporating Human Path Preferences in Robot Navigation with Minimal Interventions

Robots that can effectively understand human intentions from actions are crucial for successful human-robot collaboration. In this work, we address the challenge of a robot navigating towards an unknown goal while also accounting for a human's preference for a particular path in the presence of obstacles. This problem is particularly challenging when both the goal and path preference are unknown a priori. To overcome this challenge, we propose a method for encoding and inferring path preference online using a partitioning of the space into polytopes. Our approach enables joint inference over the goal and path preference using a stochastic observation model for the human. We evaluate our method on an unknown-goal navigation problem with sparse human interventions, and find that it outperforms baseline approaches as the human's inputs become increasingly sparse. We find that the time required to update the robot's belief does not increase with the complexity of the environment, which makes our method suitable for online applications.
A switching Gaussian process latent force model for the identification of mechanical systems with a discontinuous nonlinearity

Abstract An approach for the identification of discontinuous and nonsmooth nonlinear forces, as those generated by frictional contacts, in mechanical systems that can be approximated by a single-degree-of-freedom model is presented. To handle the sharp variations and multiple motion regimes introduced by these nonlinearities in the dynamic response, the partially known physics-based model and noisy measurements of the system’s response to a known input force are combined within a switching Gaussian process latent force model (GPLFM). In this grey-box framework, multiple Gaussian processes are used to model the unknown nonlinear force across different motion regimes and a resetting model enables the generation of discontinuities. The states of the system, nonlinear force, and regime transitions are inferred by using filtering and smoothing techniques for switching linear dynamical systems. The proposed switching GPLFM is applied to a simulated dry friction oscillator and an experimental setup consisting of a single-storey frame with a brass-to-steel contact. Excellent results are obtained in terms of the identified nonlinear and discontinuous friction force for varying: (i) normal load amplitudes in the contact; (ii) measurement noise levels, and (iii) number of samples in the datasets. Moreover, the identified states, friction force, and sequence of motion regimes are used for evaluating: (1) uncertain system parameters; (2) the friction force–velocity relationship, and (3) the static friction force. The correct identification of the discontinuous nonlinear force and the quantification of any remaining uncertainty in its prediction enable the implementation of an accurate forward model able to predict the system’s response to different input forces.
PanIC: consistent information criteria for general model selection problems

Model selection is a ubiquitous problem that arises in the application of many statistical and machine learning methods. In the likelihood and related settings, it is typical to use the method of information criteria (IC) to choose the most parsimonious among competing models by penalizing the likelihood-based objective function. Theorems guaranteeing the consistency of IC can often be difficult to verify and are often specific and bespoke. We present a set of results that guarantee consistency for a class of IC, which we call PanIC (from the Greek root 'pan', meaning 'of everything'), with easily verifiable regularity conditions. The PanIC are applicable in any loss-based learning problem and are not exclusive to likelihood problems. We illustrate the verification of regularity conditions for model selection problems regarding finite mixture models, least absolute deviation and support vector regression, and principal component analysis, and we demonstrate the effectiveness of the PanIC for such problems via numerical simulations. Furthermore, we present new sufficient conditions for the consistency of BIC-like estimators and provide comparisons of the BIC to PanIC.
One-4-All: Neural Potential Fields for Embodied Navigation

A fundamental task in robotics is to navigate between two locations. In particular, real-world navigation can require long-horizon planning using high-dimensional RGB images, which poses a substantial challenge for end-to-end learning-based approaches. Current semi-parametric methods instead achieve long-horizon navigation by combining learned modules with a topological memory of the environment, often represented as a graph over previously collected images. However, using these graphs in practice requires tuning a number of pruning heuristics. These heuristics are necessary to avoid spurious edges, limit runtime memory usage and maintain reasonably fast graph queries in large environments. In this work, we present One-4-All (O4A), a method leveraging self-supervised and manifold learning to obtain a graph-free, end-to-end navigation pipeline in which the goal is specified as an image. Navigation is achieved by greedily minimizing a potential function defined continuously over image embeddings. Our system is trained offline on non-expert exploration sequences of RGB data and controls, and does not require any depth or pose measurements. We show that 04A can reach long-range goals in 8 simulated Gibson indoor environments and that resulting embeddings are topologically similar to ground truth maps, even if no pose is observed. We further demonstrate successful real-world navigation using a Jackal UGV platform.aaProject page https://montrealrobotics.ca/o4a/.
On backward smoothing algorithms

In the context of state-space models, skeleton-based smoothing algorithms rely on a backward sampling step which by default has a $\mathcal O(N^2)$ complexity (where $N$ is the number of particles). Existing improvements in the literature are unsatisfactory: a popular rejection sampling -- based approach, as we shall show, might lead to badly behaved execution time; another rejection sampler with stopping lacks complexity analysis; yet another MCMC-inspired algorithm comes with no stability guarantee. We provide several results that close these gaps. In particular, we prove a novel non-asymptotic stability theorem, thus enabling smoothing with truly linear complexity and adequate theoretical justification. We propose a general framework which unites most skeleton-based smoothing algorithms in the literature and allows to simultaneously prove their convergence and stability, both in online and offline contexts. Furthermore, we derive, as a special case of that framework, a new coupling-based smoothing algorithm applicable to models with intractable transition densities. We elaborate practical recommendations and confirm those with numerical experiments.
Modelling the control of bovine brucellosis in India

Brucellosis imposes substantial impacts on livestock production and public health worldwide. A stochastic, age-structured model incorporating herd demographics was developed describing within- and between-herd transmission of Brucella abortus in dairy cattle herds. The model was fitted to data from a cross-sectional study conducted in Punjab State of India and used to evaluate the effectiveness of control strategies under consideration. Based on model results, stakeholder acceptance and constraints regarding vaccine supply, vaccination of replacement calves in large farms should be prioritized. Test and removal applied at early stages of the control programme where seroprevalence is high would not constitute an effective or acceptable use of resources because significant numbers of animals would be ‘removed’ (culled or not used for breeding) based on false positive results. To achieve sustained reductions in brucellosis, policymakers must commit to maintaining vaccination in the long term, which may eventually reduce frequency of infection in the livestock reservoir to a low enough level for elimination to be a realistic objective. This work provides key strategic insights into the control of brucellosis in India, which has the largest cattle population globally, and a general modelling framework for evaluating control strategies in endemic settings.
It's a small world for parasites: evidence supporting the North American invasion of European Echinococcus multilocularis

Echinococcus multilocularis (Em), the causative agent of human alveolar echinococcosis (AE), is present in the Holarctic region, and several genetic variants deem to have differential infectivity and pathogenicity. An unprecedented outbreak of human AE cases in Western Canada infected with a European-like strain circulating in wild hosts warranted assessment of whether this strain was derived from a recent invasion or was endemic but undetected. Using nuclear and mitochondrial markers, we investigated the genetic diversity of Em in wild coyotes and red foxes from Western Canada, compared the genetic variants identified to global isolates and assessed their spatial distribution to infer possible invasion dynamics. Genetic variants from Western Canada were closely related to the original European clade, with lesser genetic diversity than that expected for a long-established strain and spatial genetic discontinuities within the study area, supporting the hypothesis of a relatively recent invasion with various founder events.
Reconstructing locomotor ecology of extinct avialans: a case study of Ichthyornis comparing sternum morphology and skeletal proportions

Avian skeletal morphology is associated with locomotor function, including flight style, swimming and terrestrial locomotion, and permits informed inferences on locomotion in extinct taxa. The fossil taxon Ichthyornis (Avialae: Ornithurae) has long been regarded as highly aerial, with flight similar to terns or gulls (Laridae), and skeletal features resembling foot-propelled diving adaptations. However, rigorous testing of locomotor hypotheses has yet to be performed on Ichthyornis, despite its notable phylogenetic position as one of the most crownward stem birds. We analysed separate datasets of three-dimensional sternal shape (geometric morphometrics) and skeletal proportions (linear measurements across the skeleton), to examine how well these data types predict locomotor traits in Neornithes. We then used this information to infer locomotor capabilities of Ichthyornis. We find strong support for both soaring and foot-propelled swimming capabilities in Ichthyornis. Further, sternal shape and skeletal proportions provide complementary information on avian locomotion: skeletal proportions allow better predictions of the capacity for flight, whereas sternal shape predicts variation in more specific locomotor abilities such as soaring, foot-propelled swimming and escape burst flight. These results have important implications for future studies of extinct avialan ecology and underscore the importance of closely considering sternum morphology in investigations of fossil bird locomotion.
False Discovery Control for Random Fields

This article extends false discovery rates to random fields, for which there are uncountably many hypothesis tests. We develop a method for finding regions in the field's domain where there is a significant signal while controlling either the proportion of area or the proportion of clusters in which false rejections occur. The method produces confidence envelopes for the proportion of false discoveries as a function of the rejection threshold. From the confidence envelopes, we derive threshold procedures to control either the mean or the specified tail probabilities of the false discovery proportion. An essential ingredient of this construnction is a new algorithm to compute a confidence superset for the set of all true-null locations. We demonstrate our method with applications to scan statistics and functional neuroimaging.