Miracle-dz/UniFAR-FTUs huggingface dataset:
Volumetric Disentanglement for 3D Scene Manipulation

Recently, advances in differential volumetric rendering enabled significant breakthroughs in the photo-realistic and fine-detailed reconstruction of complex 3D scenes, which is key for many virtual reality applications. However, in the context of augmented reality, one may also wish to effect semantic manipulations or augmentations of objects within a scene. To this end, we propose a volumetric framework for (i) disentangling or separating, the volumetric representation of a given foreground object from the background, and (ii) semantically manipulating the foreground object, as well as the background. Our framework takes as input a set of 2D masks specifying the desired foreground object for training views, together with the associated 2D views and poses, and produces a foreground-background disentanglement that respects the surrounding illumination, reflections, and partial occlusions, which can be applied to both training and novel views. Our method enables the separate control of pixel color and depth as well as 3D similarity transformations of both the foreground and background objects. We subsequently demonstrate the applicability of our framework on a number of downstream manipulation tasks including object camouflage, non-negative 3D object inpainting, 3D object translation, 3D object inpainting, and 3D text-based object manipulation. Full results are given in our project webpage at https://sagiebenaim.github. io/volumetric-disentanglement/
Adaptive Tree Backup Algorithms for Temporal-Difference Reinforcement Learning

Q($\sigma$) is a recently proposed temporal-difference learning method that interpolates between learning from expected backups and sampled backups. It has been shown that intermediate values for the interpolation parameter $\sigma \in [0,1]$ perform better in practice, and therefore it is commonly believed that $\sigma$ functions as a bias-variance trade-off parameter to achieve these improvements. In our work, we disprove this notion, showing that the choice of $\sigma=0$ minimizes variance without increasing bias. This indicates that $\sigma$ must have some other effect on learning that is not fully understood. As an alternative, we hypothesize the existence of a new trade-off: larger $\sigma$-values help overcome poor initializations of the value function, at the expense of higher statistical variance. To automatically balance these considerations, we propose Adaptive Tree Backup (ATB) methods, whose weighted backups evolve as the agent gains experience. Our experiments demonstrate that adaptive strategies can be more effective than relying on fixed or time-annealed $\sigma$-values.
Accurate Scoliosis Vertebral Landmark Localization on X-ray Images via Shape-constrained Multi-stage Cascaded CNNs

Vertebral landmark localization is a crucial step for variant spine-related clinical applications, which requires detecting the corner points of 17 vertebrae. However, the neighbor landmarks often disturb each other for the homogeneous appearance of vertebrae, which makes vertebral landmark localization extremely difficult. In this paper, we propose multi-stage cascaded convolutional neural networks (CNNs) to split the single task into two sequential steps, i.e., center point localization to roughly locate 17 center points of vertebrae, and corner point localization to find 4 corner points for each vertebra without distracted by others. Landmarks in each step are located gradually from a set of initialized points by regressing offsets via cascaded CNNs. Principal Component Analysis (PCA) is employed to preserve a shape constraint in offset regression to resist the mutual attraction of vertebrae. We evaluate our method on the AASCE dataset that consists of 609 tight spinal anterior-posterior X-ray images and each image contains 17 vertebrae composed of the thoracic and lumbar spine for spinal shape characterization. Experimental results demonstrate our superior performance of vertebral landmark localization over other state-of-the-arts with the relative error decreasing from 3.2e-3 to 7.2e-4.
Robust Calibration with Multi-domain Temperature Scaling

Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challenging when training distribution and test distribution are different, even the distribution shifts are mild. Despite the ubiquity of distribution shifts in real-world applications, existing uncertainty quantification approaches mainly study the in-distribution setting where the train and test distributions are the same. In this paper, we develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains. Our proposed method -- multi-domain temperature scaling -- uses the heterogeneity in the domains to improve calibration robustness under distribution shift. Through experiments on three benchmark data sets, we find our proposed method outperforms existing methods as measured on both in-distribution and out-of-distribution test sets.
Estimation of Over-parameterized Models via Fitting to Future Observations

From a model-building perspective, in this paper we propose a paradigm shift for ﬁtting over-parameterized models. Philosophically, the mindset is to ﬁt models to future observations rather than to the observed sample. Technically, choosing an imputation model for generating future observations, we ﬁt over-parameterized models to future observations via optimizing an approximation to the desired expected loss-function based on its sample counterpart and an adaptive simplicity-preference function. This technique is discussed in detail to both creating bootstrap imputation and ﬁnal estimation with bootstrap imputation. The method is illustrated with the many-normal-means problem, n < p linear regression, and deep convolutional neural networks for image classiﬁcation of MNIST digits. The numerical results demonstrate superior performance across these three different types of applications. For example, for the many-normal-means problem, our method uniformly dominates James-Stein and Efron’s g − modeling, and for the MNIST image classiﬁcation, it performs better than all existing methods and reaches arguably the best possible result. While this paper is largely expository because of the ambitious task of taking a look at over-parameterized models from the new perspective, fundamental theoretical properties are also investigated. We conclude the paper with a few remarks. show that when the model is obtained at iterations with the smallest
Invariant Percolation Properties in Random Isotropic Systems of Conductive Discorectangles on a Plane: From Disks to Sticks

Nanorod‐ and nanowire‐based films are promising 2D materials for the transparent conductive electrodes, including flexible and stretchable ones. To mimic the shape of elongated conductive nanoparticles and, at the same time, simplify the simulations, different simple geometrical figures are used. During simulation, the shape and the aspect ratio of the particles affect the electrical and optical properties of films. However, some features of the conductive films may be insensitive both to the shape of particles and to their aspect ratio. 2D random systems of isotropically placed, overlapping, identical discorectangles (stadia) with aspect ratios ranging from 1 (disks) to ∞ (zero‐width sticks) are studied. The particular case when only the junction resistance between conductive particles is taken into account is considered. The effect of the aspect ratio and the number density of conductive discorectangles on the behavior of the electrical conductivity, the local conductivity exponent, and the current‐carrying backbone is analyzed. The computer simulations demonstrate that some of the properties of random, isotropic 2D systems of conductive discorectangles are insensitive to the aspect ratios of the particles.
Linear MSRD Codes with Diﬀerent Matrix Sizes

A sum-rank-metric code attaining the Singleton bound is called maximum sum-rank distance (MSRD). MSRD codes have applications in space-time coding and construction of partial-MDS codes for repair in distributed storage. MSRD codes have been constructed in some parameter cases. In this paper we construct a F q -linear MSRD code over some ﬁeld F q with diﬀerent matrix sizes n 1 > n 2 > · · · > n t satisfying n i ≥ n 2 i +1 + · · · + n 2 t for i = 1 , 2 , . , t − 1 for any given minimum sum-rank distance. Many good linear sum-rank-metric codes over small ﬁelds with such diﬀerent matrix sizes are given. A lower bound on the dimensions of constructed F q 2 -linear sum-rank-metric codes over F q 2 with such diﬀerent matrix sizes and given minimum sum-rank distances is also presented.
Prepatterning of Papilio xuthus caterpillar camouflage is controlled by three homeobox genes: clawless, abdominal-A, and Abdominal-B

Three homeobox genes regulate prepatterning of caterpillar camouflage design in the swallowtail butterfly, Papilio xuthus. Color patterns often function as camouflage to protect insects from predators. In most swallowtail butterflies, younger larvae mimic bird droppings but change their pattern to mimic their host plants during their final molt. This pattern change is determined during the early fourth instar by juvenile hormone (JH-sensitive period), but it remains unclear how the prepatterning process is controlled. Using Papilio xuthus larvae, we performed transcriptome comparisons to identify three camouflage pattern–associated homeobox genes [clawless, abdominal-A, and Abdominal-B (Abd-B)] that are up-regulated during the JH-sensitive period in a region-specific manner. Electroporation-mediated knockdown of each gene at the third instar caused loss or change of original fifth instar patterns, but not the fourth instar mimetic pattern, and knockdown of Abd-B after the JH-sensitive period had no effect on fifth instar patterns. These results indicate the role of these genes during the JH-sensitive period and in the control of the prepatterning gene network.
Functional equations for transfer-matrix operators in open Hecke chain models

We consider integrable open chain models formulated in terms of the generators of affine Hecke algebras. We use the fusion procedure to construct the hierarchy of commutative elements, which are analogues of the commutative transfer matrices. These elements satisfy a set of functional relations generalizing functional relations for a family of transfer matrices in solvable spin chain models of the Uq(gl(n|m)) type.
Additive MIL: Intrinsic Interpretability for Pathology

Multiple Instance Learning ( MIL ) has been widely applied in pathology towards solving critical problems such as automating cancer diagnosis and grading, predicting patient prognosis, and therapy response. Deploying these models in a clinical setting requires careful inspection of these black boxes during development and deployment to identify failures and maintain physician trust. In this work, we propose a simple formulation of MIL models, which enables interpretability while maintaining similar predictive performance. Our Additive MIL models enable spatial credit assignment such that the contribution of each region in the image can be exactly computed and visualized. We show that our spatial credit assignment coincides with regions used by pathologists during diagnosis and improves upon classical attention heatmaps from attention MIL models. We show that any existing MIL model can be made additive with a simple change in function composition. We also show how these models can debug model failures, identify spurious features, and highlight class-wise regions of interest, enabling their use in high-stakes environments such as clinical decision-making.
Finite-Sample Maximum Likelihood Estimation of Location

We consider 1-dimensional location estimation, where we estimate a parameter $\lambda$ from $n$ samples $\lambda + \eta_i$, with each $\eta_i$ drawn i.i. d. from a known distribution $f$. For fixed $f$ the maximum-likelihood estimate (MLE) is well-known to be optimal in the limit as $n \to \infty$: it is asymptotically normal with variance matching the Cram\'er-Rao lower bound of $\frac{1}{n\mathcal{I}}$, where $\mathcal{I}$ is the Fisher information of $f$. However, this bound does not hold for finite $n$, or when $f$ varies with $n$. We show for arbitrary $f$ and $n$ that one can recover a similar theory based on the Fisher information of a smoothed version of $f$, where the smoothing radius decays with $n$.
The stationary Weyl equation and Cosserat elasticity

The paper deals with the Weyl equation which is the massless Dirac equation. We study the Weyl equation in the stationary setting, i.e. when the spinor field oscillates harmonically in time. We suggest a new geometric interpretation of the stationary Weyl equation. We think of our three-dimensional space as an elastic continuum and assume that material points of this continuum can experience no displacements, only rotations. This framework is a special case of the Cosserat theory of elasticity. The rotations of material points of the space continuum are described mathematically by attaching to each geometric point an orthonormal basis which gives a field of orthonormal bases called the coframe. As the dynamical variables (unknowns) of our theory, we choose the coframe and a density. We choose a particular potential energy which is conformally invariant and then incorporate time into our action in the standard Newtonian way, by subtracting kinetic energy. The main result of our paper is the theorem stating that in the stationary setting our model is equivalent to a pair of Weyl equations.
Stacked unsupervised learning with a network architecture found by supervised meta-learning

Stacked unsupervised learning (SUL) seems more biologically plausible than backpropagation, because learning is local to each layer. But SUL has fallen far short of backpropagation in practical applications, undermining the idea that SUL can explain how brains learn. Here we show an SUL algorithm that can perform completely unsupervised clustering of MNIST digits with comparable accuracy relative to unsupervised algorithms based on backpropagation. Our algorithm is exceeded only by self-supervised methods requiring training data augmentation by geometric distortions. The only prior knowledge in our unsupervised algorithm is implicit in the network architecture. Multiple convolutional"energy layers"contain a sum-of-squares nonlinearity, inspired by"energy models"of primary visual cortex. Convolutional kernels are learned with a fast minibatch implementation of the K-Subspaces algorithm. High accuracy requires preprocessing with an initial whitening layer, representations that are less sparse during inference than learning, and rescaling for gain control. The hyperparameters of the network architecture are found by supervised meta-learning, which optimizes unsupervised clustering accuracy. We regard such dependence of unsupervised learning on prior knowledge implicit in network architecture as biologically plausible, and analogous to the dependence of brain architecture on evolutionary history.
THE ROKHLIN PROPERTY FOR AUTOMORPHISMS ON SIMPLE C*-ALGEBRAS

Let A be the class of unital separable simple amenable C*- algebras A which satisfy the Universal Coefficient Theorem for which AMP has tracial rank zero for some supernatural number p of infinite type. Let A 2 A and letbe an automorphism of A. Suppose that � has the tracial Rokhlin property. Suppose also that there is an integer J � 1 such that (� J ) = (idA) in KL(A,A), we show that A oZ 2 A.
Contrastive learning unifies t-SNE and UMAP

Neighbor embedding methods t -SNE and UMAP are the de facto standard for visualizing high-dimensional datasets. They appear to use very different loss functions with different motivations, and the exact relationship between them has been unclear. Here we show that UMAP is effectively negative sampling applied to the t -SNE loss function. We explain the difference between negative sampling and noise-contrastive estimation (NCE), which has been used to optimize t -SNE under the name NCVis. We prove that, unlike NCE, negative sampling learns a scaled data distribution. When applied in the neighbor embedding setting, it yields more compact embeddings with increased attraction, explaining differences in appearance between UMAP and t -SNE. Further, we generalize the notion of negative sampling and obtain a spectrum of embeddings, encompassing visualizations similar to t -SNE, NCVis, and UMAP. Finally, we explore the connection between representation learning in the SimCLR setting and neighbor embeddings, and show that (i) t -SNE can be optimized using the InfoNCE loss and in a parametric setting; (ii) various contrastive losses with only few noise samples can yield competitive performance in the SimCLR setup.
Sparse Bayesian learning for complex‐valued rational approximations

Surrogate models are used to alleviate the computational burden in engineering tasks, which require the repeated evaluation of computationally demanding models of physical systems, such as the efficient propagation of uncertainties. For models that show a strongly non‐linear dependence on their input parameters, standard surrogate techniques, such as polynomial chaos expansion, are not sufficient to obtain an accurate representation of the original model response. It has been shown that for models with discontinuities or rational dependencies, for example, frequency response functions of dynamic systems, the use of a rational (Padé) approximation can significantly improve the approximation accuracy. In order to avoid overfitting issues in previously proposed standard least squares approaches, we introduce a sparse Bayesian learning approach to estimate the coefficients of the rational approximation. Therein the linearity in the numerator polynomial coefficients is exploited and the denominator polynomial coefficients as well as the problem hyperparameters are determined through type‐II‐maximum likelihood estimation. We apply a quasi‐Newton gradient‐descent algorithm to find the optimal denominator coefficients and derive the required gradients through application of ℂℝ$$ \mathit{\mathbb{CR}} $$ ‐calculus. The method is applied to the frequency response functions of an algebraic frame structure model as well as that of an orthotropic plate finite element model.
Learning with Capsules: A Survey

Capsule networks were proposed as an alternative approach to Convolutional Neural Networks (CNNs) for learning object-centric representations, which can be leveraged for improved generalization and sample complexity. Unlike CNNs, capsule networks are designed to explicitly model part-whole hierarchical relationships by using groups of neurons to encode visual entities, and learn the relationships between those entities. Promising early results achieved by capsule networks have motivated the deep learning community to continue trying to improve their performance and scalability across several application areas. However, a major hurdle for capsule network research has been the lack of a reliable point of reference for understanding their foundational ideas and motivations. The aim of this survey is to provide a comprehensive overview of the capsule network research landscape, which will serve as a valuable resource for the community going forward. To that end, we start with an introduction to the fundamental concepts and motivations behind capsule networks, such as equivariant inference in computer vision. We then cover the technical advances in the capsule routing mechanisms and the various formulations of capsule networks, e.g. generative and geometric. Additionally, we provide a detailed explanation of how capsule networks relate to the popular attention mechanism in Transformers, and highlight non-trivial conceptual similarities between them in the context of representation learning. Afterwards, we explore the extensive applications of capsule networks in computer vision, video and motion, graph representation learning, natural language processing, medical imaging and many others. To conclude, we provide an in-depth discussion regarding the main hurdles in capsule network research, and highlight promising research directions for future work.
FuSS: Fusing Superpixels for Improved Segmentation Consistency

In this work, we propose two different approaches to improve the semantic consistency of Open Set Semantic Segmentation. First, we propose a method called OpenGMM that extends the OpenPCS framework using a Gaussian Mixture of Models to model the distribution of pixels for each class in a multimodal manner. The second approach is a post-processing which uses superpixels to enforce highly homogeneous regions to behave equally, rectifying erroneous classified pixels within these regions, we also proposed a novel superpixel method called FuSS. All tests were performed on ISPRS Vaihingen and Potsdam datasets, and both methods were capable to improve quantitative and qualitative results for both datasets. Besides that, the post-process with FuSS achieved state-of-the-art results for both datasets. The official implementation is available at: \url{https://github.com/iannunes/FuSS}.
Physically Inspired Constraint for Unsupervised Regularized Ultrasound Elastography

Displacement estimation is a critical step of virtually all Ultrasound Elastography (USE) techniques. Two main features make this task unique compared to the general optical ﬂow problem: the high-frequency nature of ultrasound radio-frequency (RF) data and the governing laws of physics on the displacement ﬁeld. Recently, the architecture of the optical ﬂow networks has been modiﬁed to be able to use RF data. Also, semi-supervised and unsupervised techniques have been employed for USE by considering prior knowledge of displacement continuity in the form of the ﬁrst- and second-derivative regularizers. Despite these attempts, no work has considered the tissue compression pattern, and displacements in axial and lateral directions have been assumed to be independent. However, tissue motion pattern is governed by laws of physics in USE, rendering the axial and the lateral displacements highly correlated. In this paper, we propose Physically Inspired ConsTraint for Unsupervised Regularized Elastography (PICTURE), where we impose constraints on the Poisson’s ratio to improve lateral displacement estimates. Experiments on phantom and in vivo data show that PICTURE substantially improves the quality of the lateral displacement estimation.