Miracle-dz/UniFAR-FTUs huggingface dataset:
Counterfactual Generative Models for Time-Varying Treatments

Estimating the counterfactual outcome of treatment is essential for decision-making in public health and clinical science, among others. Often, treatments are administered in a sequential, time-varying manner, leading to an exponentially increased number of possible counterfactual outcomes. Furthermore, in modern applications, the outcomes are high-dimensional and conventional average treatment effect estimation fails to capture disparities in individuals. To tackle these challenges, we propose a novel conditional generative framework capable of producing counterfactual samples under time-varying treatment, without the need for explicit density estimation. Our method carefully addresses the distribution mismatch between the observed and counterfactual distributions via a loss function based on inverse probability re-weighting, and supports integration with state-of-the-art conditional generative models such as the guided diffusion and conditional variational autoencoder. We present a thorough evaluation of our method using both synthetic and real-world data. Our results demonstrate that our method is capable of generating high-quality counterfactual samples and outperforms the state-of-the-art baselines.
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning

Real-life multilingual systems should be able to efficiently incorporate new languages as data distributions fed to the system evolve and shift over time. To do this, systems need to handle the issue of catastrophic forgetting, where the model performance drops for languages or tasks seen further in its past. In this paper, we study catastrophic forgetting, as well as methods to minimize this, in a massively multilingual continual learning framework involving up to 51 languages and covering both classification and sequence labeling tasks. We present LR ADJUST, a learning rate scheduling method that is simple, yet effective in preserving new information without strongly overwriting past knowledge. Furthermore, we show that this method is effective across multiple continual learning approaches. Finally, we provide further insights into the dynamics of catastrophic forgetting in this massively multilingual setup.
High-Similarity-Pass Attention for Single Image Super-Resolution

Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation.
Robust Category-Level 3D Pose Estimation from Synthetic Data

Obtaining accurate 3D object poses is vital for numerous computer vision applications, such as 3D reconstruction and scene understanding. However, annotating real-world objects is time-consuming and challenging. While synthetically generated training data is a viable alternative, the domain shift between real and synthetic data is a significant challenge. In this work, we aim to narrow the performance gap between models trained on synthetic data and few real images and fully supervised models trained on large-scale data. We achieve this by approaching the problem from two perspectives: 1) We introduce SyntheticP3D, a new synthetic dataset for object pose estimation generated from CAD models and enhanced with a novel algorithm. 2) We propose a novel approach (CC3D) for training neural mesh models that perform pose estimation via inverse rendering. In particular, we exploit the spatial relationships between features on the mesh surface and a contrastive learning scheme to guide the domain adaptation process. Combined, these two approaches enable our models to perform competitively with state-of-the-art models using only 10% of the respective real training images, while outperforming the SOTA model by 10.4% with a threshold of pi/18 using only 50% of the real training data. Our trained model further demonstrates robust generalization to out-of-distribution scenarios despite being trained with minimal real data.
Optical Forbidden Emission Line Spectro-astrometry of T CrA: Evidence for a Multiple System and Multiple Jets

Spectro-astrometry is applied to echelle spectra of the young intermediate-mass star T CrA. The aim is to better understand the origin of the [O i] and [S ii] emission from T CrA and further explore the usefulness of spectro-astrometry in the search for a reliable tracer of MHD disk winds. The analysis reveals a small-scale curved jet in an east–west direction and inclined parallel to the plane of the sky. It is the inclination of this jet that led to the classification of the forbidden emission lines as a low-velocity component. Thus, spectro-astrometry highlights here that for close to edge-on disks spatial information is necessary. The position angle of the jet is not perpendicular to the position angle of the accretion disk nor does it agree with older observations of outflows likely driven by T CrA. The mass outflow rate of 5–10 × 10−8 M ⊙ yr−1 is within the range for intermediate-mass stars. We conclude that more than one outflow is driven by the T CrA system and that the curvature seen in the first detection of an outflow from T CrA and in the data presented here is likely due to the multiplicity of the system.
Multicast and Unicast Superposition Transmission in MIMO OFDMA Systems With Statistical CSIT

We consider a downlink multicast and unicast superposition transmission in multi-layer Multiple-Input Multiple-Output (MIMO) Orthogonal Frequency Division Multiple Access (OFDMA) systems when only the statistical channel state information is available at the transmitter (CSIT). Multiple users can be scheduled by using the time/frequency resources in OFDMA, while for each scheduled user MIMO spatial multiplexing is used to transmit multiple information layers, i.e., single user (SU)-MIMO. The users only need to feedback to the base-station the rank-indicator and the long-term average channel signal-to-noise ratio (SNR), i.e., statistical CSIT, to indicate a suitable number of transmission layers, a suitable modulation and coding scheme and allow the base-station to perform user scheduling. This approach is especially relevant for the delivery of common (e.g., popular live event) and independent (e.g., user personalized) content to a high number of users in deployments in the lower frequency bands operating in Frequency-Division-Duplex (FDD) mode, e.g., sub-1 GHz. We show that the optimal resource allocation that maximizes the ergodic sum-rate involves greedy user selection per OFDM subchannel and superposition transmission of one multicast signal across all subchannels and single unicast signal per subchannel. The optimal resource allocation dynamically chooses between sending unicast only, multicast only, or their superposition in each subchannel based on the disparity between the best and the worst user channel conditions and minimum multicast rate target. Degree-of-freedom (DoF) analysis shows that while the lack of instantaneous CSI limits DoF of unicast messages to the minimum number of transmit antennas and receiver antennas, the multicast message obtains full DoF that increases linearly with the number of users. We present resource allocation algorithms consisting of user selection and power allocation between multicast and unicast signals in each OFDM subchannel. System level simulations in 5G rural macro-cell scenarios show overall network throughput gains in realistic network environments by superposition transmission of multicast and unicast signals.
Constraints on black hole charges in M87* and Sgr A* with the EHT observations

In May 2022 ICRANet organized the Workshop dedicated to the 80th anniversary of Professor Ruffini. This paper is based on the talk delivered at the meeting. Professor Ruffini was well known for Soviet scientific community not only due to his publications in leading journals but also due Russian translations of his books where he was an author or a contributor in collection of articles. But only in 1988 I had an opportunity to watch and listen professor R. Ruffini at the Conference dedicated to the century since the birthday of Alexander Alexandrovich Friedmann. This conference was organized in Leningrad (Soviet Union) in June during a short magic period when there are white nights there. In June 2023 we celebrate the 135th anniversary of Friedmann's birth. Friedmann and his closed friend V. K. Frederics were the founders of Soviet school of general relativity and George Gamow was one of the brilliant representative of the school and he was the author of the hot Universe model which is the most popular now. In the USSR a development of general relativity and relativistic cosmology was not smooth and only in sixties of the last century these branches of science freed from the total control of representatives of the ideology of Marxism -- Leninism. I also discussed a Soviet contribution in a discovery of cosmic microwave background radiation done by T. Shmaonov in 1957 and reasons why his supervisors did not connect these results with the hot Universe models discussed by G. Gamow. Author's results about observational features of supemassive black holes (including the black hole in our Galactic Center) are also briefly discussed, it was considered an opportunity to evaluate a (tidal) charge of Reissner -- Nordstr\"om black hole from observational estimates of shadow size in the Galactic Center and M87* done by the EHT Collaboration based its observations in April 2017.
Radio Observations of Six Young Type Ia Supernovae

Type Ia supernovae (SNe Ia) are important cosmological tools, probes of binary star evolution, and contributors to cosmic metal enrichment; yet, a definitive understanding of the binary star systems that produce them remains elusive. Of particular interest is the identity of the mass-donor companion to the exploding carbon–oxygen white dwarf (CO WD). In this work, we present early-time (first observation within 10 days post-explosion) radio observations of six nearby (within 40 Mpc) SNe Ia taken by the Jansky Very Large Array, which are used to constrain the presence of synchrotron emission from the interaction between ejecta and circumstellar material (CSM). The two motivations for these early-time observations are: (1) to constrain the presence of low-density winds and (2) to provide an additional avenue of investigation for those SNe Ia observed to have early-time optical/UV excesses that may be due to CSM interaction. We detect no radio emission from any of our targets. Toward our first aim, these non-detections further increase the sample of SNe Ia that rule out winds from symbiotic binaries and strongly accreting white dwarfs. and discuss the dependence on underlying model assumptions and how our observations represent a large increase in the sample of SNe Ia with low-density wind constraints. For the second aim, we present a radiation hydrodynamics simulation to explore radio emission from an SN Ia interacting with a compact shell of CSM, and find that relativistic electrons cannot survive to produce radio emission despite the rapid expansion of the shocked shell after shock breakout. The effects of model assumptions are discussed for both the wind and compact shell conclusions.
On Representations of Grothendieck Toposes

Results of a representation-theoretic nature have played a major role in topos theory since the beginnings of the subject. For example, Deligne's theorem on coherent toposes, which says that every coherent topos has a continuous embedding into a topos of the form Set I for a discrete set I, is a typical result in the representation theory of toposes. (A continuous functor between toposes is the left adjoint of a geometric morphism. For Grothendieck toposes, it is exactly the same as a continuous functor between them, considered as sites with their canonical topologies. By a continuous functor between sites on left exact categories, we mean a left exact functor taking covers to covers.) A representation-like result for toposes typically asserts that a topos that satisfies some abstract conditions is related to a topos of some concrete kind; the relation between them is usually an embedding of the first topos in the second (concrete) one, for which the embedding satisfies some additional properties (fullness, etc.).
Detecting Adversarial Data by Probing Multiple Perturbations Using Expected Perturbation Score

Adversarial detection aims to determine whether a given sample is an adversarial one based on the discrepancy between natural and adversarial distributions. Unfortunately, estimating or comparing two data distributions is extremely difficult, especially in high-dimension spaces. Recently, the gradient of log probability density (a.k.a., score) w.r. t. the sample is used as an alternative statistic to compute. However, we find that the score is sensitive in identifying adversarial samples due to insufficient information with one sample only. In this paper, we propose a new statistic called expected perturbation score (EPS), which is essentially the expected score of a sample after various perturbations. Specifically, to obtain adequate information regarding one sample, we perturb it by adding various noises to capture its multi-view observations. We theoretically prove that EPS is a proper statistic to compute the discrepancy between two samples under mild conditions. In practice, we can use a pre-trained diffusion model to estimate EPS for each sample. Last, we propose an EPS-based adversarial detection (EPS-AD) method, in which we develop EPS-based maximum mean discrepancy (MMD) as a metric to measure the discrepancy between the test sample and natural samples. We also prove that the EPS-based MMD between natural and adversarial samples is larger than that among natural samples. Extensive experiments show the superior adversarial detection performance of our EPS-AD.
Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data

Neural transducer (RNNT)-based target-speaker speech recognition (TS-RNNT) directly transcribes a target speaker's voice from a multi-talker mixture. It is a promising approach for streaming applications because it does not incur the extra computation costs of a target speech extraction frontend, which is a critical barrier to quick response. TS-RNNT is trained end-to-end given the input speech (i.e., mixtures and enrollment speech) and reference transcriptions. The training mixtures are generally simulated by mixing single-talker signals, but conventional TS-RNNT training does not utilize single-speaker signals. This paper proposes using knowledge distillation (KD) to exploit the parallel mixture/single-talker speech data. Our proposed KD scheme uses an RNNT system pretrained with the target single-talker speech input to generate pseudo labels for the TS-RNNT training. Experimental results show that TS-RNNT systems trained with the proposed KD scheme outperform a baseline TS-RNNT.
Implicit bias of SGD in L2-regularized linear DNNs: One-way jumps from high to low rank

The $L_{2}$-regularized loss of Deep Linear Networks (DLNs) with more than one hidden layers has multiple local minima, corresponding to matrices with different ranks. In tasks such as matrix completion, the goal is to converge to the local minimum with the smallest rank that still fits the training data. While rank-underestimating minima can be avoided since they do not fit the data, GD might get stuck at rank-overestimating minima. We show that with SGD, there is always a probability to jump from a higher rank minimum to a lower rank one, but the probability of jumping back is zero. More precisely, we define a sequence of sets $B_{1}\subset B_{2}\subset\cdots\subset B_{R}$ so that $B_{r}$ contains all minima of rank $r$ or less (and not more) that are absorbing for small enough ridge parameters $\lambda$ and learning rates $\eta$: SGD has prob. 0 of leaving $B_{r}$, and from any starting point there is a non-zero prob. for SGD to go in $B_{r}$.
Differentially Private Latent Diffusion Models

Diffusion models (DMs) are widely used for generating high-quality high-dimensional images in a non-differentially private manner. To address this challenge, recent papers suggest pre-training DMs with public data, then fine-tuning them with private data using DP-SGD for a relatively short period. In this paper, we further improve the current state of DMs with DP by adopting the Latent Diffusion Models (LDMs). LDMs are equipped with powerful pre-trained autoencoders that map the high-dimensional pixels into lower-dimensional latent representations, in which DMs are trained, yielding a more efficient and fast training of DMs. In our algorithm, DP-LDMs, rather than fine-tuning the entire DMs, we fine-tune only the attention modules of LDMs at varying layers with privacy-sensitive data, reducing the number of trainable parameters by roughly 90% and achieving a better accuracy, compared to fine-tuning the entire DMs. The smaller parameter space to fine-tune with DP-SGD helps our algorithm to achieve new state-of-the-art results in several public-private benchmark data pairs.Our approach also allows us to generate more realistic, high-dimensional images (256x256) and those conditioned on text prompts with differential privacy, which have not been attempted before us, to the best of our knowledge. Our approach provides a promising direction for training more powerful, yet training-efficient differentially private DMs, producing high-quality high-dimensional DP images.
Consistent Optimal Transport with Empirical Conditional Measures

Given samples from two joint distributions, we consider the problem of Optimal Transportation (OT) between them when conditioned on a common variable. We focus on the general setting where the conditioned variable may be continuous, and the marginals of this variable in the two joint distributions may not be the same. In such settings, standard OT variants cannot be employed, and novel estimation techniques are necessary. Since the main challenge is that the conditional distributions are not explicitly available, the key idea in our OT formulation is to employ kernelized-least-squares terms computed over the joint samples, which implicitly match the transport plan's marginals with the empirical conditionals. Under mild conditions, we prove that our estimated transport plans, as a function of the conditioned variable, are asymptotically optimal. For finite samples, we show that the deviation in terms of our regularized objective is bounded by $O(1/m^{1/4})$, where $m$ is the number of samples. We also discuss how the conditional transport plan could be modelled using explicit probabilistic models as well as using implicit generative ones. We empirically verify the consistency of our estimator on synthetic datasets, where the optimal plan is analytically known. When employed in applications like prompt learning for few-shot classification and conditional-generation in the context of predicting cell responses to treatment, our methodology improves upon state-of-the-art methods.
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation

Large language models (large LMs) are susceptible to producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context. In this work, we present a comprehensive investigation into self-contradiction for various instruction-tuned LMs, covering evaluation, detection, and mitigation. Our primary evaluation task is open-domain text generation, but we also demonstrate the applicability of our approach to shorter question answering. Our analysis reveals the prevalence of self-contradictions, e.g., in 17.7% of all sentences produced by ChatGPT. We then propose a novel prompting-based framework designed to effectively detect and mitigate self-contradictions. Our detector achieves high accuracy, e.g., around 80% F1 score when prompting ChatGPT. The mitigation algorithm iteratively refines the generated text to remove contradictory information while preserving text fluency and informativeness. Importantly, our entire framework is applicable to black-box LMs and does not require retrieval of external knowledge. Rather, our method complements retrieval-based methods, as a large portion of self-contradictions (e.g., 35.2% for ChatGPT) cannot be verified using online text. Our approach is practically effective and has been released as a push-button tool to benefit the public at https://chatprotect.ai/.
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels

Controlling artificial agents from visual sensory data is an arduous task. Reinforcement learning (RL) algorithms can succeed but require large amounts of interactions between the agent and the environment. To alleviate the issue, unsupervised RL proposes to employ self-supervised interaction and learning, for adapting faster to future tasks. Yet, as shown in the Unsupervised RL Benchmark (URLB; Laskin et al. 2021), whether current unsupervised strategies can improve generalization capabilities is still unclear, especially in visual control settings. In this work, we study the URLB and propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent, and a task-aware fine-tuning strategy combined with a new proposed hybrid planner, Dyna-MPC, to adapt the agent for downstream tasks. On URLB, our method obtains 93.59% overall normalized performance, surpassing previous baselines by a staggering margin. The approach is empirically evaluated through a large-scale empirical study, which we use to validate our design choices and analyze our models. We also show robust performance on the Real-Word RL benchmark, hinting at resiliency to environment perturbations during adaptation. Project website: https://masteringurlb.github. io/