Miracle-dz/UniFAR-FTUs huggingface dataset:
Misspecified Phase Retrieval with Generative Priors

In this paper, we study phase retrieval under model misspecification and generative priors. In particular, we aim to estimate an $n$-dimensional signal $\mathbf{x}$ from $m$ i.i. d. ~realizations of the single index model $y = f(\mathbf{a}^T\mathbf{x})$, where $f$ is an unknown and possibly random nonlinear link function and $\mathbf{a} \in \mathbb{R}^n$ is a standard Gaussian vector. We make the assumption $\mathrm{Cov}[y,(\mathbf{a}^T\mathbf{x})^2] \ne 0$, which corresponds to the misspecified phase retrieval problem. In addition, the underlying signal $\mathbf{x}$ is assumed to lie in the range of an $L$-Lipschitz continuous generative model with bounded $k$-dimensional inputs. We propose a two-step approach, for which the first step plays the role of spectral initialization and the second step refines the estimated vector produced by the first step iteratively. We show that both steps enjoy a statistical rate of order $\sqrt{(k\log L)\cdot (\log m)/m}$ under suitable conditions. Experiments on image datasets are performed to demonstrate that our approach performs on par with or even significantly outperforms several competing methods.
OPERA: Omni-Supervised Representation Learning with Hierarchical Supervisions

The pretrain-finetune paradigm in modern computer vision facilitates the success of self-supervised learning, which tends to achieve better transferability than supervised learning. However, with the availability of massive labeled data, a natural question emerges: how to train a better model with both self and full supervision signals? In this paper, we propose Omni-suPErvised Representation leArning with hierarchical supervisions (OPERA) as a solution. We provide a unified perspective of supervisions from labeled and unlabeled data and propose a unified framework of fully supervised and self-supervised learning. We extract a set of hierarchical proxy representations for each image and impose self and full supervisions on the corresponding proxy representations. Extensive experiments on both convolutional neural networks and vision transformers demonstrate the superiority of OPERA in image classification, segmentation, and object detection.1
Prospects for Detecting Fast Radio Bursts in the Globular Clusters of Nearby Galaxies

The recent detection of a repeating fast radio burst (FRB) in an old globular cluster in M81 challenges traditional FRB formation mechanisms based on the magnetic activity of young neutron stars formed in core-collapse supernovae. Furthermore, the detection of this repeater in such a nearby galaxy implies a high local universe rate of similar events in globular clusters. Building off the properties inferred from the M81 FRB, we predict the number of FRB sources in nearby (d ≲ 20 Mpc) galaxies with large globular cluster systems known. Incorporating the uncertain burst energy distribution, we estimate the rate of bursts detectable in these galaxies by radio instruments such as FAST and MeerKat. Of all local galaxies, we find M87 is the best candidate for FRB detections. We predict that M87's globular cluster system contains (10) FRB sources at present and that a dedicated radio survey (by either FAST or MeerKat) of (10) hr has a 90% probability of detecting a globular cluster FRB in M87. The detection of even a handful of additional globular cluster FRBs would provide invaluable constraints on FRB mechanisms and population properties. Previous studies have demonstrated young neutron stars formed following the collapse of dynamically formed massive white dwarf binary mergers may provide the most natural mechanism for these bursts. We explore the white dwarf merger scenario using a suite of N-body cluster models, focusing in particular on such mergers in M87's clusters. We describe a number of outstanding features of this scenario that in principle may be testable with an ensemble of observed FRBs in nearby globular clusters.
Multi-User Reinforcement Learning with Low Rank Rewards

In this work, we consider the problem of collaborative multi-user reinforcement learning. In this setting there are multiple users with the same state-action space and transition probabilities but with different rewards. Under the assumption that the reward matrix of the $N$ users has a low-rank structure -- a standard and practically successful assumption in the offline collaborative filtering setting -- the question is can we design algorithms with significantly lower sample complexity compared to the ones that learn the MDP individually for each user. Our main contribution is an algorithm which explores rewards collaboratively with $N$ user-specific MDPs and can learn rewards efficiently in two key settings: tabular MDPs and linear MDPs. When $N$ is large and the rank is constant, the sample complexity per MDP depends logarithmically over the size of the state-space, which represents an exponential reduction (in the state-space size) when compared to the standard ``non-collaborative'' algorithms.
Searching for Conformity Across Cosmic Time with Local Group and Local Volume Star Formation Histories

Conformity denotes the correlation of properties between pairs of galaxies as a function of separation. Correlations between properties such as the star formation rate (SFR), stellar mass, and specific star formation rate (sSFR) have implications for the impact of environment upon galaxy formation and evolution. Conformity between primary galaxies and satellites within the same dark matter halo has been well documented in simulations and observations. However, the existence of conformity at greater distances—known as two-halo conformity—remains uncertain. We investigate whether galaxies in the Local Volume to a distance of 4 Mpc show conformity by examining the SFR, sSFR, stellar mass, and quenched fraction as a function of physical separation. Making use of the star formation histories of these galaxies, we then extend this analysis back in time to offer the first probe of conformity inside our past light cone. At the present day, we find that the stellar mass or sSFR of a galaxy correlates with the median SFR of neighboring galaxies at a separation of 2–3 Mpc. At a lookback time of 1 Gyr, we find a correlation with the quenched fraction of neighboring galaxies, again at a 2–3 Mpc separation. These signals of conformity likely arise from the differences between the recent star formation histories of Local Group dwarf galaxies and those outside the Local Group. As current and future missions including JWST, Rubin, and Roman expand the sample of Local Volume galaxies, tests of conformity using star formation histories will provide an important tool for exploring spatiotemporal correlations between galaxies.
STSC-SNN: Spatio-Temporal Synaptic Connection with temporal convolution and attention for spiking neural networks

Spiking neural networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a Spatio-Temporal Synaptic Connection SNN (STSC-SNN) model to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Specifically, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance via varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.
Content-Adaptive Pixel Discretization to Improve Model Robustness

Preprocessing defenses such as pixel discretization are appealing to remove adversarial attacks due to their simplicity. However, they have been shown to be ineﬀective except on simple datasets like MNIST. We hypothesize that existing discretization approaches failed because using a ﬁxed codebook for the entire dataset limits their ability to balance image representation and codeword separability. We ﬁrst formally prove that adaptive codebooks can provide stronger robustness guarantees than ﬁxed codebooks as a preprocessing defense on some datasets. Based on that insight, we propose a content-adaptive pixel discretization defense called Essential Features , which discretizes the image to a per-image adaptive codebook to reduce the color space. We then ﬁnd that Essential Features can be further optimized by applying adaptive blurring before the discretization to push perturbed pixel values back to their original value before determining the codebook. Against adaptive attacks, we show that content-adaptive pixel discretization extends the range of datasets that beneﬁt in terms of both L 2 and L ∞ robustness where previously ﬁxed codebooks were found to have failed. Our ﬁndings suggest that content-adaptive pixel discretization should be part of the repertoire for making models robust. clear perturbations but a similar SSIM score as a pretrained ATTA baseline attack. This shows that neither SSIM nor L ∞ necessarily capture the true increase of perceptibility. We leave more exploration to future work.
Uncertainty-Aware Unsupervised Image Deblurring with Deep Priors Guided by Domain Knowledge

Non-blind deblurring methods achieve decent performance under the accurate blur kernel assumption. Since the kernel error is inevitable in practice, ringing artifacts are often introduced by non-blind deblurring. Recently, semi-blind deblurring methods can handle kernel uncertainty by introducing the prior of the kernel (or induced) error. However, how to design a suitable prior of the kernel (or induced) error remains challenging. Hand-crafted prior, incorporating domain knowledge, generally performs well but may lead to poor performance when kernel (or induced) error is complex. Data-driven prior, which excessively depends on the diversity and abundance of training data, is vulnerable to out-of-distribution blurs and images. To address this challenge, we suggest a data-free deep prior for the kernel induced error (termed as residual) expressed by a customized untrained deep neural network, which allows us to ﬂexibly adapt to different blurs and images in real scenarios. By organically integrating the respective strengths of deep priors and hand-crafted priors, we pro-pose an unsupervised semi-blind deblurring model which recovers the latent image from the blurry image and inaccurate blur kernel. To tackle the formulated model, an efﬁ-cient alternating minimization algorithm is developed. Extensive experiments demonstrate the superiority of the proposed method to both data-driven prior and hand-crafted prior based methods in terms of the image quality and the robustness to the
X-NeRF: Explicit Neural Radiance Field for Multi-Scene 360° Insufficient RGB-D Views

Neural Radiance Fields (NeRFs), despite their outstanding performance on novel view synthesis, often need dense input views. Many papers train one model for each scene respectively and few of them explore incorporating multimodal data into this problem. In this paper, we focus on a rarely discussed but important setting: can we train one model that can represent multiple scenes, with 360° insufficient views and RGB-D images? We refer insufficient views to few extremely sparse and almost non-overlapping views. To deal with it, X-NeRF, a fully explicit approach which learns a general scene completion process instead of a coordinate-based mapping, is proposed. Given a few insufficient RGB-D input views, X-NeRF first transforms them to a sparse point cloud tensor and then applies a 3D sparse generative Convolutional Neural Network (CNN) to complete it to an explicit radiance field whose volumetric rendering can be conducted fast without running networks during inference. To avoid overfitting, besides common rendering loss, we apply perceptual loss as well as view augmentation through random rotation on point clouds. The proposed methodology significantly out-performs previous implicit methods in our setting, indicating the great potential of proposed problem and approach. Codes and data are available at https://github.com/HaoyiZhu/XNeRF.
Global Spectral Filter Memory Network for Video Object Segmentation

This paper studies semi-supervised video object segmentation through boosting intra-frame interaction. Recent memory network-based methods focus on exploiting inter-frame temporal reference while paying little attention to intra-frame spatial dependency. Specifically, these segmentation model tends to be susceptible to interference from unrelated nontarget objects in a certain frame. To this end, we propose Global Spectral Filter Memory network (GSFM), which improves intra-frame interaction through learning long-term spatial dependencies in the spectral domain. The key components of GSFM is 2D (inverse) discrete Fourier transform for spatial information mixing. Besides, we empirically find low frequency feature should be enhanced in encoder (backbone) while high frequency for decoder (segmentation head). We attribute this to semantic information extracting role for encoder and fine-grained details highlighting role for decoder. Thus, Low (High) Frequency Module is proposed to fit this circumstance. Extensive experiments on the popular DAVIS and YouTube-VOS benchmarks demonstrate that GSFM noticeably outperforms the baseline method and achieves state-of-the-art performance. Besides, extensive analysis shows that the proposed modules are reasonable and of great generalization ability. Our source code is available at https://github.com/workforai/GSFM.
CASAPose: Class-Adaptive and Semantic-Aware Multi-Object Pose Estimation

Applications in the field of augmented reality or robotics often require joint localisation and 6D pose estimation of multiple objects. However, most algorithms need one network per object class to be trained in order to provide the best results. Analysing all visible objects demands multiple inferences, which is memory and time-consuming. We present a new single-stage architecture called CASAPose that determines 2D-3D correspondences for pose estimation of multiple different objects in RGB images in one pass. It is fast and memory efficient, and achieves high accuracy for multiple objects by exploiting the output of a semantic segmentation decoder as control input to a keypoint recognition decoder via local class-adaptive normalisation. Our new differentiable regression of keypoint locations significantly contributes to a faster closing of the domain gap between real test and synthetic training data. We apply segmentation-aware convolutions and upsampling operations to increase the focus inside the object mask and to reduce mutual interference of occluding objects. For each inserted object, the network grows by only one output segmentation map and a negligible number of parameters. We outperform state-of-the-art approaches in challenging multi-object scenes with inter-object occlusion and synthetic training.
A Kernel-Based View of Language Model Fine-Tuning

It has become standard to solve NLP tasks by fine-tuning pre-trained language models (LMs), especially in low-data settings. There is minimal theoretical understanding of empirical success, e.g., why fine-tuning a model with $10^8$ or more parameters on a couple dozen training points does not result in overfitting. We investigate whether the Neural Tangent Kernel (NTK) - which originated as a model to study the gradient descent dynamics of infinitely wide networks with suitable random initialization - describes fine-tuning of pre-trained LMs. This study was inspired by the decent performance of NTK for computer vision tasks (Wei et al., 2022). We extend the NTK formalism to Adam and use Tensor Programs (Yang, 2020) to characterize conditions under which the NTK lens may describe fine-tuning updates to pre-trained language models. Extensive experiments on 14 NLP tasks validate our theory and show that formulating the downstream task as a masked word prediction problem through prompting often induces kernel-based dynamics during fine-tuning. Finally, we use this kernel view to propose an explanation for the success of parameter-efficient subspace-based fine-tuning methods.
Measurement-based quantum computation in finite one-dimensional systems: string order implies computational power

We present a new framework for assessing the power of measurement-based quantum computation (MBQC) on short-range entangled symmetric resource states, in spatial dimension one. It requires fewer assumptions than previously known. The formalism can handle finitely extended systems (as opposed to the thermodynamic limit), and does not require translation-invariance. Further, we strengthen the connection between MBQC computational power and string order. Namely, we establish that whenever a suitable set of string order parameters is non-zero, a corresponding set of unitary gates can be realized with fidelity arbitrarily close to unity.
Metric Program Synthesis

We present a new domain-agnostic synthesis technique for generating programs from input-output examples. Our method, called metric program synthesis , relaxes the well-known observational equivalence idea (used widely in bottom-up enumerative synthesis) into a weaker notion of observational similarity , with the goal of reducing the search space that the synthesizer needs to explore. Our method clusters programs into equivalence classes based on a distance metric and constructs a version space that compactly represents “approximately correct” programs. Then, given a “close enough” program sampled from this version space, our approach uses a distance-guided repair algorithm to find a program that exactly matches the given input-output examples. We have implemented our proposed metric program synthesis technique in a tool called SyMetric and evaluate it in three different domains considered in prior work. Our evaluation shows that SyMetric outperforms other domain-agnostic synthesizers that use observational equivalence and that it achieves results competitive with domain-specific synthesizers that are either designed for or trained on those domains. our our performance
Once is Enough: A Light-Weight Cross-Attention for Fast Sentence Pair Modeling

Transformer-based models have achieved great success on sentence pair modeling tasks, such as answer selection and natural language inference (NLI). These models generally perform cross-attention over input pairs, leading to prohibitive computational costs. Recent studies propose dual-encoder and late interaction architectures for faster computation. However, the balance between the expressive of cross-attention and computation speedup still needs better coordinated. To this end, this paper introduces a novel paradigm MixEncoder for efficient sentence pair modeling. MixEncoder involves a light-weight cross-attention mechanism. It conducts query encoding only once while modeling the query-candidate interaction in parallel. Extensive experiments conducted on four tasks demonstrate that our MixEncoder can speed up sentence pairing by over 113x while achieving comparable performance as the more expensive cross-attention models.
Retrieval Augmentation for T5 Re-ranker using External Sources

Retrieval augmentation has shown promising improvements in different tasks. However, whether such augmentation can assist a large language model based re-ranker remains unclear. We investigate how to augment T5-based re-rankers using high-quality information retrieved from two external corpora -- a commercial web search engine and Wikipedia. We empirically demonstrate how retrieval augmentation can substantially improve the effectiveness of T5-based re-rankers for both in-domain and zero-shot out-of-domain re-ranking tasks.
Solving the Yamabe-Type Equations on Closed Manifolds by Iteration Schemes

We introduce a double iterative scheme and local variational method to solve the Yamabe-type equation − 4( n − 1) n − 2 ∆ g u + ( S g + β ) u = λu n +2 n − 2 for some constant β 6 0, locally on Riemannian domain (Ω , g ) with trivial Dirichlet condition and globally on closed manifolds ( M, g ); the dimensions of Ω and M are at least 3. In contrast to the traditional global variational method, these Yamabe-type equations on closed manifolds are analyzed by local analysis and monotone iteration scheme. In particular, we do not need to use the Weyl tensor. In particular, the sign of the ﬁrst eigenvalue η 1 of conformal Laplacian (cid:3) g u := − 4( n − 1) n − 2 ∆ g u + S g u plays the central role. When letting β → 0 from the left, we reproof the classical Yamabe problem as a natural consequence of the Yamabe-type equations. The results are classiﬁed by the sign of η 1 .
Quantification of moving target cyber defenses

Current network and information systems are static, making it simple for attackers to maintain an advantage. Adaptive defenses, such as Moving Target Defenses (MTD) have been developed as potential “game-changers” in an effort to increase the attacker’s workload. With many new methods being developed, it is difficult to accurately quantify and compare their overall costs and effectiveness. This paper compares the tradeoffs between current approaches to the quantification of MTDs. We present results from an expert opinion survey on quantifying the overall effectiveness, upfront and operating costs of a select set of MTD techniques. We find that gathering informed scientific opinions can be advantageous for evaluating such new technologies as it offers a more comprehensive assessment. We end by presenting a coarse ordering of a set of MTD techniques from most to least dominant. We found that seven out of 23 methods rank as the more dominant techniques. Five of which are techniques of either address space layout randomization or instruction set randomization. The remaining two techniques are applicable to software and computer platforms. Among the techniques that performed the worst are those primarily aimed at network randomization.