Miracle-dz/UniFAR-FTUs huggingface dataset:
CAFS: Class Adaptive Framework for Semi-Supervised Semantic Segmentation

Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by selftraining with pseudo-labeling pixels having high confidences for unlabeled images. However, using only highconfidence pixels for self-training may result in losing much of the information in the unlabeled datasets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-adaptive semisupervision framework for semi-supervised semantic segmentation (CAFS) to cope with the loss of most information that occurs in existing high-confidence-based pseudolabeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CAFS constructs a validation set on a labeled dataset, to leverage the calibration performance for each class. On this basis, we propose a calibration aware class-wise adaptive thresholding and classwise adaptive oversampling using the analysis results from the validation set. Our proposed CAFS achieves state-ofthe-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0% and 80.4%, respectively. The code is available at https://github.com/cjf8899/CAFS.
Hydrodynamics of an odd active surfer in a chiral fluid

We theoretically and computationally study the low-Reynolds-number hydrodynamics of a linear active microswimmer surfing on a compressible thin fluid layer characterized by an odd viscosity. Since the underlying three-dimensional fluid is assumed to be very thin compared to any lateral size of the fluid layer, the model is effectively two-dimensional. In the limit of small odd viscosity compared to the even viscosities of the fluid layer, we obtain analytical expressions for the self-induced flow field, which includes non-reciprocal components due to the odd viscosity. On this basis, we fully analyze the behavior of a single linear swimmer, finding that it follows a circular path, the radius of which is, to leading order, inversely proportional to the magnitude of the odd viscosity. In addition, we show that a pair of swimmers exhibits a wealth of two-body dynamics that depends on the initial relative orientation angles as well as on the propulsion mechanism adopted by each swimmer. In particular, the pusher–pusher and pusher–puller-type swimmer pairs exhibit a generic spiral motion, while the puller–puller pair is found to either co-rotate in the steady state along a circular trajectory or exhibit a more complex chaotic behavior resulting from the interplay between hydrodynamic and steric interactions. Our theoretical predictions may pave the way toward a better understanding of active transport in active chiral fluids with odd viscosity, and may find potential applications in the quantitative microrheological characterization of odd-viscous fluids.
Hierarchical Knowledge Guided Learning for Real-World Retinal Disease Recognition

In the real world, medical datasets often exhibit a long-tailed data distribution (i.e., a few classes occupy the majority of the data, while most classes have only a limited number of samples), which results in a challenging long-tailed learning scenario. Some recently published datasets in ophthalmology AI consist of more than 40 kinds of retinal diseases with complex abnormalities and variable morbidity. Nevertheless, more than 30 conditions are rarely seen in global patient cohorts. From a modeling perspective, most deep learning models trained on these datasets may lack the ability to generalize to rare diseases where only a few available samples are presented for training. In addition, there may be more than one disease for the presence of the retina, resulting in a challenging label co-occurrence scenario, also known as multi-label, which can cause problems when some re-sampling strategies are applied during training. To address the above two major challenges, this paper presents a novel method that enables the deep neural network to learn from a long-tailed fundus database for various retinal disease recognition. Firstly, we exploit the prior knowledge in ophthalmology to improve the feature representation using a hierarchy-aware pre-training. Secondly, we adopt an instance-wise class-balanced sampling strategy to address the label co-occurrence issue under the long-tailed medical dataset scenario. Thirdly, we introduce a novel hybrid knowledge distillation to train a less biased representation and classifier. We conducted extensive experiments on four databases, including two public datasets and two in-house databases with more than one million fundus images. The experimental results demonstrate the superiority of our proposed methods with recognition accuracy outperforming the state-of-the-art competitors, especially for these rare diseases.
SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness

Energy-efficient deep neural network (DNN) accelerators are prone to non-idealities that degrade DNN performance at inference time. To mitigate such degradation, existing methods typically add perturbations to the DNN weights during training to simulate inference on noisy hardware. However, this often requires knowledge about the target hardware and leads to a trade-off between DNN performance and robustness, decreasing the former to increase the latter. In this work, we show that applying sharpness-aware training, by optimizing for both the loss value and loss sharpness, significantly improves robustness to noisy hardware at inference time without relying on any assumptions about the target hardware. In particular, we propose a new adaptive sharpness-aware method that conditions the worst-case perturbation of a given weight not only on its magnitude but also on the range of the weight distribution. This is achieved by performing sharpness-aware minimization scaled by outlier minimization (SAMSON). Our approach outperforms existing sharpness-aware training methods both in terms of model generalization performance in noiseless regimes and robustness in noisy settings, as measured on several architectures and datasets.
The Interplay between Radio AGN Activity and Their Host Galaxies

Radio activity in AGN (Active Galactic Nuclei) produce feedback on the host galaxy via the impact of the relativistic jets on the circumnuclear gas. Although radio jets can reach up to several times the optical radius of the host galaxy, in this review we focus on the observation of the feedback deposited locally in the central region of the host galaxies, in the form of outflows due to the jet-gas interaction. We begin by discussing how galaxy mergers and interactions are the most favored scenario for triggering radio AGN after gas accretion to the nuclear supermassive black hole and star formation enhancement in the nuclear region, observed in particular in the most luminous sources. We then discuss observational signatures of the process of jet-gas coupling, in particular the resulting outflows and their effects on the host galaxy. These include the presence of shock signatures and the detection of outflows not only along the radio jet but perpendicular to it in many sources. Although most of the studies are done via the observation of ionized gas, molecular gas is also being increasingly observed in outflow, contributing to the bulk of the mass outflow rate. Even though most radio sources present outflow kinetic powers that do not reach 1%Lbol, and thus do not seem to provide an immediate impact on the host galaxy, they act to heat the ISM gas, preventing star formation, slowing the galaxy mass build-up process and limiting the stellar mass growth, in a “maintenance mode” feedback.
NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects

We propose NEMTO, the first end-to-end neural rendering pipeline to model 3D transparent objects with complex geometry and unknown indices of refraction. Commonly used appearance modeling such as the Disney BSDF model cannot accurately address this challenging problem due to the complex light paths bending through refractions and the strong dependency of surface appearance on illumination. With 2D images of the transparent object as input, our method is capable of high-quality novel view and relighting synthesis. We leverage implicit Signed Distance Functions (SDF) to model the object geometry and propose a refraction-aware ray bending network to model the effects of light refraction within the object. Our ray bending network is more tolerant to geometric inaccuracies than traditional physically-based methods for rendering transparent objects. We provide extensive evaluations on both synthetic and real-world datasets to demonstrate our high-quality synthesis and the applicability of our method.
ADCNet: Learning from Raw Radar Data via Distillation

As autonomous vehicles and advanced driving assistance systems have entered wider deployment, there is an increased interest in building robust perception systems using radars. Radar-based systems are lower cost and more robust to adverse weather conditions than their LiDAR-based counterparts; however the point clouds produced are typically noisy and sparse by comparison. In order to combat these challenges, recent research has focused on consuming the raw radar data, instead of the final radar point cloud. We build on this line of work and demonstrate that by bringing elements of the signal processing pipeline into our network and then pre-training on the signal processing task, we are able to achieve state of the art detection performance on the RADIal dataset. Our method uses expensive offline signal processing algorithms to pseudo-label data and trains a network to distill this information into a fast convolutional backbone, which can then be finetuned for perception tasks. Extensive experiment results corroborate the effectiveness of the proposed techniques.
Learning Optical Flow and Scene Flow With Bidirectional Camera-LiDAR Fusion

In this paper, we study the problem of jointly estimating the optical flow and scene flow from synchronized 2D and 3D data. Previous methods either employ a complex pipeline that splits the joint task into independent stages, or fuse 2D and 3D information in an “early-fusion” or “late-fusion” manner. Such one-size-fits-all approaches suffer from a dilemma of failing to fully utilize the characteristic of each modality or to maximize the inter-modality complementarity. To address the problem, we propose a novel end-to-end framework, which consists of 2D and 3D branches with multiple bidirectional fusion connections between them in specific layers. Different from previous work, we apply a point-based 3D branch to extract the LiDAR features, as it preserves the geometric structure of point clouds. To fuse dense image features and sparse point features, we propose a learnable operator named bidirectional camera-LiDAR fusion module (Bi-CLFM). We instantiate two types of the bidirectional fusion pipeline, one based on the pyramidal coarse-to-fine architecture (dubbed CamLiPWC), and the other one based on the recurrent all-pairs field transforms (dubbed CamLiRAFT). On FlyingThings3D, both CamLiPWC and CamLiRAFT surpass all existing methods and achieve up to a 47.9% reduction in 3D end-point-error from the best published result. Our best-performing model, CamLiRAFT, achieves an error of 4.26% on the KITTI Scene Flow benchmark, ranking 1st among all submissions with much fewer parameters. Besides, our methods have strong generalization performance and the ability to handle non-rigid motion.
Saddle node scaling on approach to dislocation nucleation

Submitted for the MAR11 Meeting of The American Physical Society Saddle node scaling on approach to dislocation nucleation AKANKSHA GARG, ASAD HASAN, CRAIG MALONEY, Carnegie Mellon University — We study the process of dislocation nucleation in a perfect 2D hexagonal crystal under nano-indentation loading in a numerical model using energy minimization techniques and analysis of the energy eigenmodes. The nucleation event takes the form of a saddle-node catastrophe and is governed by associated scaling laws. In particular, on approach to nucleation, a single energy eigenmode descends through the spectrum and its eigenvalue vanishes as the square root of the distance to the nucleation point. The velocity of the system shows the same scaling behavior, and its normal mode decomposition demonstrates that it is dominated by the critical mode responsible for nucleation. Asad Hasan Carnegie Mellon University Date submitted: 23 Nov 2010 Electronic form version 1.4
Analysis of Stabilized Finite Volume Method for Poisson Equation

Abstract In this work, we systematically analyze a stabilized finite volume method for the Poisson equation. On stating the convergence of this method, optimal error estimates in different norms are obtained by establishing the adequate connections between the finite element and finite volume methods. Furthermore, some super-convergence results are established by using L 2 -projection method on a coarse mesh based on some regularity assumptions for Poisson equation. Finally, some numerical experiments are presented to confirm the theoretical findings.
Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images

We introduce the first active learning (AL) framework for high-accuracy instance segmentation of moveable parts from RGB images of real indoor scenes. As with most human-in-the-loop approaches, the key criterion for success in AL is to minimize human effort while still attaining high performance. To this end, we employ a transformer that utilizes a masked-attention mechanism to supervise the active segmentation. To enhance the network tailored to moveable parts, we introduce a coarse-to-fine AL approach which first uses an object-aware masked attention and then a pose-aware one, leveraging the hierarchical nature of the problem and a correlation between moveable parts and object poses and interaction directions. Our method achieves close to fully accurate (96% and higher) segmentation results, with semantic labels, on real images, with 82% time saving over manual effort, where the training data consists of only 11.45% annotated real photographs. At last, we contribute a dataset of 2,550 real photographs with annotated moveable parts, demonstrating its superior quality and diversity over the current best alternatives.
Deephys: Deep Electrophysiology, Debugging Neural Networks under Distribution Shifts

Deep Neural Networks (DNNs) often fail in out-of-distribution scenarios. In this paper, we introduce a tool to visualize and understand such failures. We draw inspiration from concepts from neural electrophysiology, which are based on inspecting the internal functioning of a neural networks by analyzing the feature tuning and invariances of individual units. Deep Electrophysiology, in short Deephys, provides insights of the DNN's failures in out-of-distribution scenarios by comparative visualization of the neural activity in in-distribution and out-of-distribution datasets. Deephys provides seamless analyses of individual neurons, individual images, and a set of set of images from a category, and it is capable of revealing failures due to the presence of spurious features and novel features. We substantiate the validity of the qualitative visualizations of Deephys thorough quantitative analyses using convolutional and transformers architectures, in several datasets and distribution shifts (namely, colored MNIST, CIFAR-10 and ImageNet).
Vortex lock-in phenomenon in the wake of a plunging airfoil

The flow over a NACA0012 airfoil, oscillated sinusoidally in plunge, is simulated numerically using a two-dimensional Navier-Stokes solver at a Reynolds number of 20,000. The wake of the airfoil is visualized using a numerical particle tracing method for high reduced frequencies (1.0 <k< <10.0) and small nondimensional amplitudes (h < 0.1). Anomalous vortex shedding modes (involving multiple vortices shed per half-cycle of airfoil motion) observed experimentally in the literature are reproduced numerically and are shown to be the result of interaction between the plunging frequency and a natural bluff-body shedding frequency. This results in a vortex lock-in phenomenon analogous to that seen for oscillating cylinders. However, the lock-in boundary is not symmetric about the natural shedding frequency, due to the sharp trailing edge forcing the flow to separate at the trailing edge on the windward side of the airfoil for the majority of the plunge cycle at higher frequencies and amplitudes.
Semiparametric inference for relative heterogeneous vaccine efficacy between strains in observational case-only studies (preprint)

The aim of this manuscript is to explore semiparametric methods for inferring subgroup-specific relative vaccine efficacy in a partially vaccinated population against multiple strains of a virus. We consider methods for observational case-only studies with informative missingness in viral strain type due to vaccination status, pre-vaccination variables, and also post-vaccination factors such as viral load. We establish general causal conditions under which the relative conditional vaccine efficacy between strains can be identified nonparametrically from the observed data-generating distribution. Assuming that the relative strain-specific conditional vaccine efficacy has a known parametric form, we propose semiparametric asymptotically linear estimators of the parameters based on targeted (debiased) machine learning estimators for partially linear logistic regression models. Finally, we apply our methods to estimate the relative strain-specific conditional vaccine efficacy in the ENSEMBLE COVID-19 vaccine trial.
Greedy Pruning with Group Lasso Provably Generalizes for Matrix Sensing and Neural Networks with Quadratic Activations

Pruning schemes have been widely used in practice to reduce the complexity of trained models with a massive number of parameters. In fact, several practical studies have shown that if a pruned model is fine-tuned with some gradient-based updates it generalizes well to new samples. Although the above pipeline, which we refer to as pruning + fine-tuning, has been extremely successful in lowering the complexity of trained models, there is very little known about the theory behind this success. In this paper, we address this issue by investigating the pruning + fine-tuning framework on the overparameterized matrix sensing problem with the ground truth $U_\star \in \mathbb{R}^{d \times r}$ and the overparameterized model $U \in \mathbb{R}^{d \times k}$ with $k \gg r$. We study the approximate local minima of the mean square error, augmented with a smooth version of a group Lasso regularizer, $\sum_{i=1}^k \| U e_i \|_2$. In particular, we provably show that pruning all the columns below a certain explicit $\ell_2$-norm threshold results in a solution $U_{\text{prune}}$ which has the minimum number of columns $r$, yet close to the ground truth in training loss. Moreover, in the subsequent fine-tuning phase, gradient descent initialized at $U_{\text{prune}}$ converges at a linear rate to its limit. While our analysis provides insights into the role of regularization in pruning, we also show that running gradient descent in the absence of regularization results in models which {are not suitable for greedy pruning}, i.e., many columns could have their $\ell_2$ norm comparable to that of the maximum. To the best of our knowledge, our results provide the first rigorous insights on why greedy pruning + fine-tuning leads to smaller models which also generalize well.
Distributed Differentiable Dynamic Game for Multi-robot Coordination

This paper develops a Distributed Differentiable Dynamic Game (D3G) framework, which can efficiently solve the forward and inverse problems in multi-robot coordination. We formulate multi-robot coordination as a dynamic game, where the behavior of a robot is dictated by its own dynamics and objective that also depends on others' behavior. In the forward problem, D3G enables all robots collaboratively to seek the Nash equilibrium of the game in a distributed manner, by developing a distributed shooting-based Nash solver. In the inverse problem, where each robot aims to find (learn) its objective (and dynamics) parameters to mimic given coordination demonstrations, D3G proposes a differentiation solver based on Differential Pontryagin's Maximum Principle, which allows each robot to update its parameters in a distributed and coordinated manner. We test the D3G in simulation with two types of robots given different task configurations. The results demonstrate the effectiveness of D3G for solving both forward and inverse problems in comparison with existing methods.
An update on the management of non-variceal upper gastrointestinal bleeding

Abstract Upper gastrointestinal bleeding (UGIB) continues to be a common gastrointestinal emergency that carries significant morbidity and mortality. The epidemiology of UGIB has been changing over the last few decades with an overall decrease in peptic ulcer disease and increase in the prevalence of other etiologies including vascular lesions and malignancy. Appropriate risk assessment and patient stratification are crucial to ensuring that optimal care is delivered to patients and some risk assessment tools have shown excellent ability to define a low-risk group who can be managed as outpatients safely. Regardless of the etiology of UGIB, resuscitative interventions by primary care providers remain the most important initial measures to improve the outcome for patients including hemodynamic stabilization, an appropriate blood transfusion strategy, with or without acid-lowering agents, while also providing subsequent urgent endoscopic assessment and intervention. In addition, with increasing use of antithrombotic agents in clinical practice and its associated risk of bleeding, the management of such agents in the acute setting has become a real challenge to all physicians. In this article, we provide an up-to-date, evidence-based, practical review of recent changes and advances in UGIB with a focus on non-variceal etiologies.