Miracle-dz/UniFAR-FTUs huggingface dataset:
PCFG-Based Natural Language Interface Improves Generalization for Controlled Text Generation

Existing work on controlled text generation (CTG) assumes a control interface of categorical attributes. In this work, we propose a natural language (NL) interface, where we craft a PCFG to embed the control attributes into natural language commands, and propose variants of existing CTG models that take commands as input. In our experiments, we design tailored setups to test the model’s generalization abilities. We find our PCFG-based command generation approach is effective for handling unseen commands compared to fix-set templates. Further, our proposed NL models can effectively generalize to unseen attributes (a new ability enabled by the NL interface), as well as unseen attribute combinations. Interestingly, in model comparisons, the simple conditional generation approach, enhanced with our proposed NL interface, is shown to be a strong baseline in those challenging settings.
Dead ends and rationality of complete growth series

The complete growth series of a finitely generated group is given by $\sum_{n\ge 0} A_ns^n$, where $A_n$ is the sum of elements of length $n$ in the group semiring. We study the $\mathbb NG$-rationality and $\mathbb NG$-algebraicity of such series. We show that having dead ends of arbitrarily large depths is an obstruction to $\mathbb NG$-rationality. In the case of the $3$-dimensional Heisenberg group $H_3(\mathbb Z)$, we prove that the complete series is not $\mathbb NG$-algebraic for any generating set. Dead ends are also used to show that complete growth series of higher Heisenberg groups are not $\mathbb NG$-rational for specific generating sets. Using a more general version of this obstruction, we prove that complete growth series of some lamplighter groups are not $\mathbb NG$-rational either.
Identification of quantum entanglement with Siamese convolutional neural networks and semi-supervised learning

Quantum entanglement is a fundamental property commonly used in various quantum information protocols and algorithms. Nonetheless, the problem of identifying entanglement has still not reached a general solution for systems larger than two qubits. In this study, we use deep convolutional neural networks, a type of supervised machine learning, to identify quantum entanglement for any bipartition in a 3-qubit system. We demonstrate that training the model on synthetically generated datasets of random density matrices excluding challenging positive-under-partial-transposition entangled states (PPTES), which cannot be identified (and correctly labeled) in general, leads to good model accuracy even for PPTES states, that were outside the training data. Our aim is to enhance the model's generalization on PPTES. By applying entanglement-preserving symmetry operations through a triple Siamese network trained in a semi-supervised manner, we improve the model's accuracy and ability to recognize PPTES. Moreover, by constructing an ensemble of Siamese models, even better generalization is observed, in analogy with the idea of finding separate types of entanglement witnesses for different classes of states. The neural models' code and training schemes, as well as data generation procedures, are available at github.com/Maticraft/quantum_correlations.
MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation

Commonsense reasoning tasks such as commonsense knowledge graph completion and commonsense question answering require powerful representation learning. In this paper, we propose to learn commonsense knowledge representation by MICO, a Multi-alternative contrastve learning framework on COmmonsense knowledge graphs (MICO). MICO generates the commonsense knowledge representation by contextual interaction between entity nodes and relations with multi-alternative contrastive learning. In MICO, the head and tail entities in an $(h,r,t)$ knowledge triple are converted to two relation-aware sequence pairs (a premise and an alternative) in the form of natural language. Semantic representations generated by MICO can benefit the following two tasks by simply comparing the distance score between the representations: 1) zero-shot commonsense question answering task; 2) inductive commonsense knowledge graph completion task. Extensive experiments show the effectiveness of our method.
Controllable Style Transfer via Test-time Training of Implicit Neural Representation

We propose a controllable style transfer framework based on Implicit Neural Representation that pixel-wisely controls the stylized output via test-time training. Unlike traditional image optimization methods that often suffer from unstable convergence and learning-based methods that require intensive training and have limited generalization ability, we present a model optimization framework that optimizes the neural networks during test-time with explicit loss functions for style transfer. After being test-time trained once, thanks to the flexibility of the INR-based model, our framework can precisely control the stylized images in a pixel-wise manner and freely adjust image resolution without further optimization or training. We demonstrate several applications.
The Hidden Uniform Cluster Prior in Self-Supervised Learning

A successful paradigm in representation learning is to perform self-supervised pretraining using tasks based on mini-batch statistics (e.g., SimCLR, VICReg, SwAV, MSN). We show that in the formulation of all these methods is an overlooked prior to learn features that enable uniform clustering of the data. While this prior has led to remarkably semantic representations when pretraining on class-balanced data, such as ImageNet, we demonstrate that it can hamper performance when pretraining on class-imbalanced data. By moving away from conventional uniformity priors and instead preferring power-law distributed feature clusters, we show that one can improve the quality of the learned representations on real-world class-imbalanced datasets. To demonstrate this, we develop an extension of the Masked Siamese Networks (MSN) method to support the use of arbitrary features priors.
Model-checking lock-sharing systems against regular constraints

We study the verification of distributed systems where processes are finite automata with access to a shared pool of locks. We consider objectives that are boolean combinations of local regular constraints. We show that the problem, PSPACE-complete in general, falls in NP with the right assumptions on the system. We use restrictions on the number of locks a process can access and the order in which locks can be released. We provide tight complexity bounds, as well as a subcase of interest that can be solved in PTIME.
Number density distribution of near-infrared sources on a sub-degree scale in the Galactic center: Comparison with the Fe xxv Kα line at 6.7 keV

The stellar distribution derived from an $H$ and $K_{\mathrm S}$-band survey of the central region of our Galaxy is compared with the Fe XXV K$\alpha$ (6.7 keV) line intensity observed with the Suzaku satellite. The survey is for the Galactic coordinates $|l| \lesssim 3^{\circ}. 0$ and $|b| \lesssim 1^{\circ}. 0$ (equivalent to 0.8 kpc $\times$ 0.3 kpc for $R_0 = 8$ kpc), and the number-density distribution $N(K_{\mathrm S,0}; l, b)$ of stars is derived using the extinction-corrected magnitude $K_{\mathrm S,0}=10.5$. This is deep enough to probe the old red giant population and in turn to estimate the ($l$, $b$) distribution of faint X-ray point sources such as coronally active binaries and cataclysmic variables. In the Galactic plane ($b=0^{\circ}$), $N(10.5; l, b)$ increases to the Galactic center as $|l|^{-0.30 \pm 0.03}$ in the range of $-0^{\circ}. 1 \geq l \geq -0^{\circ}. 7$, but this increase is significantly slower than the increase ($|l|^{-0.44 \pm 0.02}$ ) of the Fe XXV K$\alpha$ line intensity. If normalized with the ratios in the outer region $1^{\circ}. 5 \leq |l| \leq 2^{\circ}. 8$, where faint X-ray point sources are argued to dominate the diffuse Galactic X-ray ridge emission, the excess of the Fe XXV K$\alpha$ line intensity over the stellar number density is at least a factor of two at $|l| = 0^{\circ}. 1$. This indicates that a significant part of the Galactic center diffuse emission arises from a truly diffuse optically-thin thermal plasma, and not from an unresolved collection of faint X-ray point sources related to the old stellar population.
The Distribution of Unstable Fixed Points in Chaotic Neural Networks

We analytically determine the number and distribution of fixed points in a canonical model of a chaotic neural network. This distribution reveals that fixed points and dynamics are confined to separate shells in phase space. Furthermore, the distribution enables us to determine the eigenvalue spectra of the Jacobian at the fixed points. Despite the radial separation of fixed points and dynamics, we find that nearby fixed points act as partially attracting landmarks for the dynamics.
Object-Category Aware Reinforcement Learning

Object-oriented reinforcement learning (OORL) is a promising way to improve the sample efficiency and generalization ability over standard RL. Recent works that try to solve OORL tasks without additional feature engineering mainly focus on learning the object representations and then solving tasks via reasoning based on these object representations. However, none of these works tries to explicitly model the inherent similarity between different object instances of the same category. Objects of the same category should share similar functionalities; therefore, the category is the most critical property of an object. Following this insight, we propose a novel framework named Object-Category Aware Reinforcement Learning (OCARL), which utilizes the category information of objects to facilitate both perception and reasoning. OCARL consists of three parts: (1) Category-Aware Unsupervised Object Discovery (UOD), which discovers the objects as well as their corresponding categories; (2) Object-Category Aware Perception, which encodes the category information and is also robust to the incompleteness of (1) at the same time; (3) Object-Centric Modular Reasoning, which adopts multiple independent and object-category-specific networks when reasoning based on objects. Our experiments show that OCARL can improve both the sample efficiency and generalization in the OORL domain.
Disentanglement of Correlated Factors via Hausdorff Factorized Support

A grand goal in deep learning research is to learn representations capable of generalizing across distribution shifts. Disentanglement is one promising direction aimed at aligning a model's representation with the underlying factors generating the data (e.g. color or background). Existing disentanglement methods, however, rely on an often unrealistic assumption: that factors are statistically independent. In reality, factors (like object color and shape) are correlated. To address this limitation, we consider the use of a relaxed disentanglement criterion -- the Hausdorff Factorized Support (HFS) criterion -- that encourages only pairwise factorized \emph{support}, rather than a factorial distribution, by minimizing a Hausdorff distance. This allows for arbitrary distributions of the factors over their support, including correlations between them. We show that the use of HFS consistently facilitates disentanglement and recovery of ground-truth factors across a variety of correlation settings and benchmarks, even under severe training correlations and correlation shifts, with in parts over $+60\%$ in relative improvement over existing disentanglement methods. In addition, we find that leveraging HFS for representation learning can even facilitate transfer to downstream tasks such as classification under distribution shifts. We hope our original approach and positive empirical results inspire further progress on the open problem of robust generalization. Code available at https://github.com/facebookresearch/disentangling-correlated-factors.
USB: A Unified Semi-supervised Learning Benchmark

Semi-supervised learning (SSL) improves model generalization by leveraging massive unlabeled data to augment limited labeled samples. However, currently, popular SSL evaluation protocols are often constrained to computer vision (CV) tasks. In addition, previous work typically trains deep neural networks from scratch, which is time-consuming and environmentally unfriendly. To address the above issues, we construct a Unified SSL Benchmark (USB) for classification by selecting 15 diverse, challenging, and comprehensive tasks from CV, natural language processing (NLP), and audio processing (Audio), on which we systematically evaluate the dominant SSL methods, and also open-source a modular and extensible codebase for fair evaluation of these SSL methods. We further provide the pre-trained versions of the state-of-the-art neural models for CV tasks to make the cost affordable for further tuning. USB enables the evaluation of a single SSL algorithm on more tasks from multiple domains but with less cost. Specifically, on a single NVIDIA V100, only 39 GPU days are required to evaluate FixMatch on 15 tasks in USB while 335 GPU days (279 GPU days on 4 CV datasets except for ImageNet) are needed on 5 CV tasks with TorchSSL.
Bayesian inference to study a signal with two or more decaying particles in a non-resonant background

We study the application of a Bayesian method to extract relevant information from data for the case of a signal consisting of two or more decaying particles and its background. The method takes advantage of the dependence that exists in the distributions of the decaying products at the event-by-event level and processes the information for the whole sample to infer the mixture fraction and the relevant parameters for signal and background distributions. The algorithm usually needs a numerical computation of the posterior, which we work out explicitly in a benchmark scenario of a simplified $pp\to hh \to b\bar b \gamma \gamma$ search. We perform a posterior predictive check on the results and we show how the signal fraction is correctly extracted from the sample, as well as many parameters in the signal and background distributions. The presented framework could be used for other searches such as $pp\to ZZ,\ WW,\ ZW$ and pair of Leptoquarks, among many others.
Limit laws in the lattice problem. III. Return to the case of boxes

We study the error of the number of points of a lattice $L$ that belong to a rectangle, centred at $0$, whose axes are parallel to the coordinate axes, dilated by a factor $t$ and then translated by a vector $X \in \mathbb{R}^{2}$. When we consider the second order moment of the error relatively to $X \in \mathbb{R}^{2}/L$, one shows that, when $t$ is random and becomes large and when the error is normalized by a quantity which behaves, in the admissible case, as $\sqrt{\log(t)}$, it converges in distribution to an explicit positive constant. In the case of a typical lattice $L$, we show that this result still holds but the normalisation is more important, around $\log(t)$. We also show that when $L=\mathbb{Z}^{2}$, the error, when normalized by $t$, converges in distribution when $t$ is random and becomes large and we compute the moments of the limit distribution.
Machine Learning in Transaction Monitoring: The Prospect of xAI

Banks hold a societal responsibility and regulatory requirements to mitigate the risk of financial crimes. Risk mitigation primarily happens through monitoring customer activity through Transaction Monitoring (TM). Recently, Machine Learning (ML) has been proposed to identify suspicious customer behavior, which raises complex socio-technical implications around trust and explainability of ML models and their outputs. However, little research is available due to its sensitivity. We aim to fill this gap by presenting empirical research exploring how ML supported automation and augmentation affects the TM process and stakeholders' requirements for building eXplainable Artificial Intelligence (xAI). Our study finds that xAI requirements depend on the liable party in the TM process which changes depending on augmentation or automation of TM. Context-relatable explanations can provide much-needed support for auditing and may diminish bias in the investigator's judgement. These results suggest a use case-specific approach for xAI to adequately foster the adoption of ML in TM.
Fitting peculiar spectral profiles in He I 10830Å absorption features

The new generation of solar instruments provides better spectral, spatial, and temporal resolution for a better understanding of the physical processes that take place on the Sun. Multiple-component profiles are more commonly observed with these instruments. Particularly, the He I 10830 A triplet presents such peculiar spectral profiles, which give information on the velocity and magnetic fine structure of the upper chromosphere. The purpose of this investigation is to describe a technique to efficiently fit the two blended components of the He I 10830 A triplet, which are commonly observed when two atmospheric components are located within the same resolution element. The observations used in this study were taken on 2015 April 17 with the very fast spectroscopic mode of the GREGOR Infrared Spectrograph (GRIS) attached to the 1.5-meter GREGOR solar telescope, located at the Observatorio del Teide, Tenerife, Spain. We apply a double-Lorentzian fitting technique using Levenberg-Marquardt least-squares minimization. This technique is very simple and much faster than inversion codes. Line-of-sight Doppler velocities can be inferred for a whole map of pixels within just a few minutes. Our results show sub- and supersonic downflow velocities of up to 32 km/s for the fast component in the vicinity of footpoints of filamentary structures. The slow component presents velocities close to rest.
THE INTERSTELLAR BUBBLES OF G38.9−0.4 AND THE IMPACT OF STELLAR FEEDBACK ON STAR FORMATION

We present a study of the star formation (SF) region G38.9−0.4 using publicly available multiwavelength Galactic plane surveys from ground- and space-based observatories. This region is composed of four bright mid-IR bubbles and numerous infrared dark clouds. Two bubbles, N 74 and N 75, each host a star cluster anchored by a single O9.5V star. We identified 162 young stellar objects (YSOs) and classify 54 as stage I, 7 as stage II, 6 as stage III, and 32 as ambiguous. We do not detect the classical signposts of triggered SF, i.e., star-forming pillars or YSOs embedded within bubble rims. We conclude that feedback-triggered SF has not occurred in G38.9−0.4. The YSOs are preferentially coincident with infrared dark clouds. This leads to a strong correlation between areal YSO mass surface density and gas mass surface density with a power law slope near 1.3, which closely matches the Schmidt–Kennicutt Law. The correlation is similar inside and outside the bubbles and may mean that the SF efficiency is neither enhanced nor suppressed in regions potentially influenced by stellar feedback. This suggests that gas density, regardless of how it is collected, is a more important driver of SF than stellar feedback. Larger studies should be able to quantify the fraction of all SF that is feedback-triggered by determining the fraction SF, feedback-compressed gas surrounding H ii regions relative to that already present in molecular clouds.
MOVE: Unsupervised Movable Object Segmentation and Detection

We introduce MOVE, a novel method to segment objects without any form of supervision. MOVE exploits the fact that foreground objects can be shifted locally relative to their initial position and result in realistic (undistorted) new images. This property allows us to train a segmentation model on a dataset of images without annotation and to achieve state of the art (SotA) performance on several evaluation datasets for unsupervised salient object detection and segmentation. In unsupervised single object discovery, MOVE gives an average CorLoc improvement of 7.2% over the SotA, and in unsupervised class-agnostic object detection it gives a relative AP improvement of 53% on average. Our approach is built on top of self-supervised features (e.g. from DINO or MAE), an inpainting network (based on the Masked AutoEncoder) and adversarial training.
Marginalized particle Gibbs for multiple state-space models coupled through shared parameters

We consider Bayesian inference from multiple time series described by a common state-space model (SSM) structure, but where diﬀerent subsets of parameters are shared between diﬀerent submodels. An important example is disease-dynamics, where parameters can be either disease or location speciﬁc. Parameter inference in these models can be improved by systematically aggregating information from the diﬀerent time series, most notably for short series. Particle Gibbs (PG) samplers are an eﬃcient class of algorithms for inference in SSMs, in particular when conjugacy can be exploited to marginalize out model parameters from the state update. We present two diﬀerent PG samplers that marginalize static model parameters on-the-ﬂy: one that updates one model at a time conditioned on the datasets for the other models, and one that concurrently updates all models by stacking them into a high-dimensional SSM. The distinctive features of each sampler make them suitable for diﬀerent modeling contexts. We provide insights on when each sampler should be used and show that they can be combined to form an eﬃcient PG sampler for a model with strong dependencies between states and parameters. The performance is illustrated on two linear-Gaussian examples and on a real-world example on the spread of mosquito-borne diseases.
Constructing $K-$optimal designs for different Scheff\'{e} models

To avoid multicollinearity in regression analysis, Ye and Zhou(2013) proposed $K-$optimality criterion. By far the most popular models for modeling the response of a mixture experiment are the Scheff\'{e} models. However, there have been no reports about constructing $K-$optimal designs for mixture models. The paper constructs $K-$optimal designs for first-order and second-order Scheff\'{e} models. The analytical solutions for first-order and second-order Scheff\'{e} models are obtained. A series of numerical results and examples are given to illustrate the theory.
MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks

Heterogeneous information network (HIN) is essential to study complicated networks containing multiple edge types and node types. Meta-path, a sequence of node types and edge types, is the core technique to embed HINs. Since manually curating meta-paths is time-consuming, there is a pressing need to develop automated meta-path generation approaches. Existing meta-path generation approaches cannot fully exploit the rich textual information in HINs, such as node names and edge type names. To address this problem, we propose MetaFill, a text-infilling-based approach for meta-path generation. The key idea of MetaFill is to formulate meta-path identification problem as a word sequence infilling problem, which can be advanced by pretrained language models (PLMs). We observed the superior performance of MetaFill against existing meta-path generation methods and graph embedding methods that do not leverage meta-paths in both link prediction and node classification on two real-world HIN datasets. We further demonstrated how MetaFill can accurately classify edges in the zero-shot setting, where existing approaches cannot generate any meta-paths. MetaFill exploits PLMs to generate meta-paths for graph embedding, opening up new avenues for language model applications in graph analysis.