Miracle-dz/UniFAR-FTUs huggingface dataset:
Simple online learning with consistent oracle

We consider online learning in the model where a learning algorithm can access the class only via the \emph{consistent oracle} -- an oracle, that, at any moment, can give a function from the class that agrees with all examples seen so far. This model was recently considered by Assos et al.~(COLT'23). It is motivated by the fact that standard methods of online learning rely on computing the Littlestone dimension of subclasses, a computationally intractable problem. Assos et al.~gave an online learning algorithm in this model that makes at most $C^d$ mistakes on classes of Littlestone dimension $d$, for some absolute unspecified constant $C>0$. We give a novel algorithm that makes at most $O(256^d)$ mistakes. Our proof is significantly simpler and uses only very basic properties of the Littlestone dimension. We also show that there exists no algorithm in this model that makes less than $3^d$ mistakes.
Optimal spread for spanning subgraphs of Dirac hypergraphs

Let $G$ and $H$ be hypergraphs on $n$ vertices, and suppose $H$ has large enough minimum degree to necessarily contain a copy of $G$ as a subgraph. We give a general method to randomly embed $G$ into $H$ with good"spread". More precisely, for a wide class of $G$, we find a randomised embedding $f\colon G\hookrightarrow H$ with the following property: for every $s$, for any partial embedding $f'$ of $s$ vertices of $G$ into $H$, the probability that $f$ extends $f'$ is at most $O(1/n)^s$. This is a common generalisation of several streams of research surrounding the classical Dirac-type problem. For example, setting $s=n$, we obtain an asymptotically tight lower bound on the number of embeddings of $G$ into $H$. This recovers and extends recent results of Glock, Gould, Joos, K\"uhn, and Osthus and of Montgomery and Pavez-Sign\'e regarding enumerating Hamilton cycles in Dirac hypergraphs. Moreover, using the recent developments surrounding the Kahn--Kalai conjecture, this result implies that many Dirac-type results hold robustly, meaning $G$ still embeds into $H$ after a random sparsification of its edge set. This allows us to recover a recent result of Kang, Kelly, K\"uhn, Osthus, and Pfenninger and of Pham, Sah, Sawhney, and Simkin for perfect matchings, and obtain novel results for Hamilton cycles and factors in Dirac hypergraphs. Notably, our randomised embedding algorithm is self-contained and does not require Szemer\'edi's regularity lemma or iterative absorption.
OnUVS: Online Feature Decoupling Framework for High-Fidelity Ultrasound Video Synthesis

Ultrasound (US) imaging is indispensable in clinical practice. To diagnose certain diseases, sonographers must observe corresponding dynamic anatomic structures to gather comprehensive information. However, the limited availability of specific US video cases causes teaching difficulties in identifying corresponding diseases, which potentially impacts the detection rate of such cases. The synthesis of US videos may represent a promising solution to this issue. Nevertheless, it is challenging to accurately animate the intricate motion of dynamic anatomic structures while preserving image fidelity. To address this, we present a novel online feature-decoupling framework called OnUVS for high-fidelity US video synthesis. Our highlights can be summarized by four aspects. First, we introduced anatomic information into keypoint learning through a weakly-supervised training strategy, resulting in improved preservation of anatomical integrity and motion while minimizing the labeling burden. Second, to better preserve the integrity and textural information of US images, we implemented a dual-decoder that decouples the content and textural features in the generator. Third, we adopted a multiple-feature discriminator to extract a comprehensive range of visual cues, thereby enhancing the sharpness and fine details of the generated videos. Fourth, we constrained the motion trajectories of keypoints during online learning to enhance the fluidity of generated videos. Our validation and user studies on in-house echocardiographic and pelvic floor US videos showed that OnUVS synthesizes US videos with high fidelity.
Neural-Network-Driven Method for Optimal Path Planning via High-Accuracy Region Prediction

Sampling-based path planning algorithms suffer from heavy reliance on uniform sampling, which accounts for unreliable and time-consuming performance, especially in complex environments. Recently, neural-network-driven methods predict regions as sampling domains to realize a non-uniform sampling and reduce calculation time. However, the accuracy of region prediction hinders further improvement. We propose a sampling-based algorithm, abbreviated to Region Prediction Neural Network RRT* (RPNN-RRT*), to rapidly obtain the optimal path based on a high-accuracy region prediction. First, we implement a region prediction neural network (RPNN), to predict accurate regions for the RPNN-RRT*. A full-layer channel-wise attention module is employed to enhance the feature fusion in the concatenation between the encoder and decoder. Moreover, a three-level hierarchy loss is designed to learn the pixel-wise, map-wise, and patch-wise features. A dataset, named Complex Environment Motion Planning, is established to test the performance in complex environments. Ablation studies and test results show that a high accuracy of 89.13% is achieved by the RPNN for region prediction, compared with other region prediction models. In addition, the RPNN-RRT* performs in different complex scenarios, demonstrating significant and reliable superiority in terms of the calculation time, sampling efficiency, and success rate for optimal path planning.
Validity and Reliability of Inertial Measurement Units in Active Range of Motion Assessment in the Hip Joint

Measuring joint range of motion has traditionally occurred with a universal goniometer, incli-nometer, or expensive laboratory systems. The popularity of the inertial measurement unit (IMU) is growing, enabling the small even single portable device to measure the range of mo-tion. Until now, they were not used to assess hip joint range of motion. Our study aimed to check the validity of IMU in assessing a hip range of motion and compare them to other meas-urement devices - universal goniometer and inclinometer. Twenty participants completed three hip movements (flexion in standing and prone internal and external rotation) on both hips. Two testers independently assessed each movement with a goniometer, digital inclinometer, and IMU at different time points. To assess the agreement of active hip ROM between devices, Intra-class Correlation Coefficient (ICC) and Bland-Altman analysis were used. Furthermore, in-ter-rater and intra-rater reliability were also assessed by using ICC and Bland-Altman analysis. Limits of agreement (LOA) were calculated using Bland-Altman plots. IMU showed good to ex-cellent validity compared with the goniometer and digital inclinometer for all tested move-ments with ICCs ranging from 0.87-0.99 and LOAs <9{degrees}. Intra-rater reliability was good-to-excellent for all tested movements for all devices, with ICCs ranging from 0.87 to 0.99 and LOAs < 7{degrees}. Inter-rater reliability was lower and comparable for all measurement methods: moderate flexion (ICC: 0.58-0.59), LOAs <22,4 and poor for both rotations (ICC:-0.33-0.04), and LOAs < 7.8{degrees}). The present study shows that a single inertial measurement unit (RSQ Motion) could be successfully used to assess the active hip range of motion in healthy subjects with com-parable to other methods accuracy.
DDF-HO: Hand-Held Object Reconstruction via Conditional Directed Distance Field

Reconstructing hand-held objects from a single RGB image is an important and challenging problem. Existing works utilizing Signed Distance Fields (SDF) reveal limitations in comprehensively capturing the complex hand-object interactions, since SDF is only reliable within the proximity of the target, and hence, infeasible to simultaneously encode local hand and object cues. To address this issue, we propose DDF-HO, a novel approach leveraging Directed Distance Field (DDF) as the shape representation. Unlike SDF, DDF maps a ray in 3D space, consisting of an origin and a direction, to corresponding DDF values, including a binary visibility signal determining whether the ray intersects the objects and a distance value measuring the distance from origin to target in the given direction. We randomly sample multiple rays and collect local to global geometric features for them by introducing a novel 2D ray-based feature aggregation scheme and a 3D intersection-aware hand pose embedding, combining 2D-3D features to model hand-object interactions. Extensive experiments on synthetic and real-world datasets demonstrate that DDF-HO consistently outperforms all baseline methods by a large margin, especially under Chamfer Distance, with about 80% leap forward. Codes are available at https://github.com/ZhangCYG/DDFHO.
Nonlinear Deterministic Observer for Inertial Navigation Using Ultra-Wideband and IMU Sensor Fusion

Navigation in Global Positioning Systems (GPS)-denied environments requires robust estimators reliant on fusion of inertial sensors able to estimate rigid-body's orientation, position, and linear velocity. Ultra-wideband (UWB) and Inertial Measurement Unit (IMU) represent low-cost measurement technology that can be utilized for successful Inertial Navigation. This paper presents a nonlinear deterministic navigation observer in a continuous form that directly employs UWB and IMU measurements. The estimator is developed on the extended Special Euclidean Group $\mathbb{SE}_{2}$ (3) and ensures exponential convergence of the closed loop error signals starting from almost any initial condition. The discrete version of the proposed observer is tested using a publicly available real-world dataset of a drone flight.
The roles of parenting, resilience, and interpersonal relationships on adolescents' mental health and stress-related growth during COVID-19.

BACKGROUND The present longitudinal study investigated parenting style as a precursor for Chinese adolescents' stress-related growth and mental health difficulties during the COVID-19 pandemic, as well as the mediating roles of intrapersonal resilience and interpersonal relationships (i.e., peer and parent-adolescent). METHODS Chinese adolescents in a middle school (7th grade) and their parents in Beijing, China, were invited to complete a survey at two time points (T1: September 2020, T2: June 2021). A total of 206 adolescents (52.9% boys; Mage = 12.90 years, SDage = 0.33) and parents (17.5% fathers, 82.4% mothers; Mage = 43.50 years, SDage = 4.76 years) were included in this study. RESULTS Results showed that Chinese parents' authoritarian, not authoritative parenting, predicted adolescents' mental health difficulties nine months later. In addition, parent-adolescent relationships, but not peer relationships nor resilience, mediated the relations between parenting style and stress-related growth. Adolescents' resilience predicted fewer mental health difficulties. CONCLUSION It is important to target multiple ecologies (e.g., family) of adolescents for promoting positive adjustment.
Finiteness properties and Relatively Hyperbolic Groups

We show that properties $F_n$ and $FP_n$ hold for a relatively hyperbolic group if and only if they hold for all the peripheral subgroups. As an application we show that there are at least countably many distinct quasi-isometry classes of one-ended groups that are type $F_n$ but not $F_{n+1}$ and similarly of type $FP_n$ and not $FP_{n+1}$ for all positive integers $n$.
Expressivity of Graph Neural Networks Through the Lens of Adversarial Robustness

We perform the first adversarial robustness study into Graph Neural Networks (GNNs) that are provably more powerful than traditional Message Passing Neural Networks (MPNNs). In particular, we use adversarial robustness as a tool to uncover a significant gap between their theoretically possible and empirically achieved expressive power. To do so, we focus on the ability of GNNs to count specific subgraph patterns, which is an established measure of expressivity, and extend the concept of adversarial robustness to this task. Based on this, we develop efficient adversarial attacks for subgraph counting and show that more powerful GNNs fail to generalize even to small perturbations to the graph's structure. Expanding on this, we show that such architectures also fail to count substructures on out-of-distribution graphs.
Fast Training of NMT Model with Data Sorting

The Transformer model has revolutionized Natural Language Processing tasks such as Neural Machine Translation, and many efforts have been made to study the Transformer architecture, which increased its efficiency and accuracy. One potential area for improvement is to address the computation of empty tokens that the Transformer computes only to discard them later, leading to an unnecessary computational burden. To tackle this, we propose an algorithm that sorts translation sentence pairs based on their length before batching, minimizing the waste of computing power. Since the amount of sorting could violate the independent and identically distributed (i.i.d) data assumption, we sort the data partially. In experiments, we apply the proposed method to English-Korean and English-Luganda language pairs for machine translation and show that there are gains in computational time while maintaining the performance. Our method is independent of architectures, so that it can be easily integrated into any training process with flexible data lengths.
How to Mask in Error Correction Code Transformer: Systematic and Double Masking

In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art performance, outperforming other methods by large margins. To further enhance the performance of ECCT, we propose two novel methods. First, leveraging the systematic encoding technique of ECCs, we introduce a new masking matrix for ECCT, aiming to improve the performance and reduce the computational complexity. Second, we propose a novel transformer architecture of ECCT called a double-masked ECCT. This architecture employs two different mask matrices in a parallel manner to learn more diverse features of the relationship between codeword bits in the masked self-attention blocks. Extensive simulation results show that the proposed double-masked ECCT outperforms the conventional ECCT, achieving the state-of-the-art decoding performance with significant margins.
Inherent Redundancy in Spiking Neural Networks

Spiking Neural Networks (SNNs) are well known as a promising energy-efficient alternative to conventional artificial neural networks. Subject to the preconceived impression that SNNs are sparse firing, the analysis and optimization of inherent redundancy in SNNs have been largely overlooked, thus the potential advantages of spike-based neuromorphic computing in accuracy and energy efficiency are interfered. In this work, we pose and focus on three key questions regarding the inherent redundancy in SNNs. We argue that the redundancy is induced by the spatio-temporal invariance of SNNs, which enhances the efficiency of parameter utilization but also invites lots of noise spikes. Further, we analyze the effect of spatio-temporal invariance on the spatio-temporal dynamics and spike firing of SNNs. Then, motivated by these analyses, we propose an Advance Spatial Attention (ASA) module to harness SNNs’ redundancy, which can adaptively optimize their membrane potential distribution by a pair of individual spatial attention sub-modules. In this way, noise spike features are accurately regulated. Experimental results demonstrate that the proposed method can significantly drop the spike firing with better performance than state-of-the-art SNN baselines. Our code is available in https://github.com/BICLab/ASA-SNN.
Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations

Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.
Biaxial strain modulated electronic structures of layered two-dimensional MoSiGeN4 Rashba systems.

The two-dimensional (2D) MA2Z4 family has received extensive attention in manipulating its electronic structure and achieving intriguing physical properties. However, engineering the electronic properties remains a challenge. Herein, based on first-principles calculations, we systematically investigate the effect of biaxial strains on the electronic structure of 2D Rashba MoSiGeN4 (MSGN), and further explore how the interlayer interactions affect the Rashba spin splitting (RSS) in such strained layered MSGN systems. After applying biaxial strains, the band gap decreases monotonically with increasing tensile strains but increases when the compressive strains are applied. An indirect-direct-indirect band gap transition is induced by applying a moderate compressive strain (<5%) in the MSGN systems. Due to the symmetry breaking and moderate spin-orbit coupling (SOC), the monolayer MSGN possesses an isolated RSS near the Fermi level, which could be effectively regulated to the Lifshitz-type spin splitting (LSS) by biaxial strain. For instance, the LSS ← RSS → LSS transformation of the Fermi surface is presented in the monolayer and a more complex and changeable LSS ← RSS → LSS → RSS evolution is observed in bilayer and trilayer MSGN systems as the biaxial strain varies from -8% to 12%, which actually depends on the appearance, variation, and vanish of the Mexican hat band in the absence of SOC under different strains. The contribution of the Mo-dz2 orbital hybridized with the N-pz orbital in the highest valence band plays a dominant role in band evolution under biaxial strains, where the RSS → LSS evolution corresponds to the decreased Mo-dz2 orbital contribution. Our study highlights the biaxial strain controllable RSS, in particular the introduction and even the evolution of LSS near the Fermi surface, which makes the strained MSGN systems promising candidates for future applications in spintronic devices.
DeepContrast: Deep Tissue Contrast Enhancement using Synthetic Data Degradations and OOD Model Predictions

Microscopy images are crucial for life science research, allowing detailed inspection and characterization of cellular and tissue-level structures and functions. However, microscopy data are unavoidably affected by image degradations, such as noise, blur, or others. Many such degradations also contribute to a loss of image contrast, which becomes especially pronounced in deeper regions of thick samples. Today, best performing methods to increase the quality of images are based on Deep Learning approaches, which typically require ground truth (GT) data during training. Our inability to counteract blurring and contrast loss when imaging deep into samples prevents the acquisition of such clean GT data. The fact that the forward process of blurring and contrast loss deep into tissue can be mod-eled, allowed us to propose a new method that can circumvent the problem of unobtainable GT data. To this end, we first synthetically degraded the quality of microscopy images even further by using an approximate forward model for deep tissue image degradations. Then we trained a neural network that learned the inverse of this degradation function from our generated pairs of raw and degraded images. We demonstrated that networks trained in this way can be used out-of-distribution (OOD) to improve the quality of less severely degraded images, e.g. the raw data imaged in a microscope. Since the absolute level of degradation in such microscopy images can be stronger than the additional degradation introduced by our forward model, we also explored the effect of iterative predictions. Here, we observed that in each iteration the measured image contrast kept improving while detailed structures in the images got increasingly removed. Therefore, dependent on the desired downstream analysis, a balance between contrast improvement and retention of image details has to be found.
Max-affine regression via first-order methods

We consider regression of a max-affine model that produces a piecewise linear model by combining affine models via the max function. The max-affine model ubiquitously arises in applications in signal processing and statistics including multiclass classification, auction problems, and convex regression. It also generalizes phase retrieval and learning rectifier linear unit activation functions. We present a non-asymptotic convergence analysis of gradient descent (GD) and mini-batch stochastic gradient descent (SGD) for max-affine regression when the model is observed at random locations following the sub-Gaussianity and an anti-concentration with additive sub-Gaussian noise. Under these assumptions, a suitably initialized GD and SGD converge linearly to a neighborhood of the ground truth specified by the corresponding error bound. We provide numerical results that corroborate the theoretical finding. Importantly, SGD not only converges faster in run time with fewer observations than alternating minimization and GD in the noiseless scenario but also outperforms them in low-sample scenarios with noise.
Sarcasm Detection in a Disaster Context

During natural disasters, people often use social media platforms such as Twitter to ask for help, to provide information about the disaster situation, or to express contempt about the unfolding event or public policies and guidelines. This contempt is in some cases expressed as sarcasm or irony. Understanding this form of speech in a disaster-centric context is essential to improving natural language understanding of disaster-related tweets. In this paper, we introduce HurricaneSARC, a dataset of 15,000 tweets annotated for intended sarcasm, and provide a comprehensive investigation of sarcasm detection using pre-trained language models. Our best model is able to obtain as much as 0.70 F1 on our dataset. We also demonstrate that the performance on HurricaneSARC can be improved by leveraging intermediate task transfer learning. We release our data and code at https://github.com/tsosea2/HurricaneSarc.
Safety Filter Design for Neural Network Systems via Convex Optimization

With the increase in data availability, it has been widely demonstrated that neural networks (NN) can capture complex system dynamics precisely in a data-driven manner. However, the architectural complexity and nonlinearity of the NNs make it challenging to synthesize a provably safe controller. In this work, we propose a novel safety filter that relies on convex optimization to ensure safety for a NN system, subject to additive disturbances that are capable of capturing modeling errors. Our approach leverages tools from NN verification to over-approximate NN dynamics with a set of linear bounds, followed by an application of robust linear MPC to search for controllers that can guarantee robust constraint satisfaction. We demonstrate the efficacy of the proposed framework numerically on a nonlinear pendulum system.