Miracle-dz/UniFAR-FTUs huggingface dataset:
Testing the Number of Components in Finite Mixture Normal Regression Model with Panel Data

This paper develops the likelihood ratio-based test of the null hypothesis of a M0-component model against an alternative of (M0 + 1)-component model in the normal mixture panel regression by extending the Expectation-Maximization (EM) test of Chen and Li (2009a) and Kasahara and Shimotsu (2015) to the case of panel data. We show that, unlike the cross-sectional normal mixture, the first-order derivative of the density function for the variance parameter in the panel normal mixture is linearly independent of its second-order derivatives for the mean parameter. On the other hand, like the cross-sectional normal mixture, the likelihood ratio test statistic of the panel normal mixture is unbounded. We consider the Penalized Maximum Likelihood Estimator to deal with the unboundedness, where we obtain the data-driven penalty function via computational experiments. We derive the asymptotic distribution of the Penalized Likelihood Ratio Test (PLRT) and EM test statistics by expanding the log-likelihood function up to five times for the reparameterized parameters. The simulation experiment indicates good finite sample performance of the proposed EM test. We apply our EM test to estimate the number of production technology types for the finite mixture Cobb-Douglas production function model studied by Kasahara et al. (2022) used the panel data of the Japanese and Chilean manufacturing firms. We find the evidence of heterogeneity in elasticities of output for intermediate goods, suggesting that production function is heterogeneous across firms beyond their Hicks-neutral productivity terms.
Benchmarking the Ising Universality Class in $3 \le d<4$ dimensions

The Ising critical exponents $\eta$, $\nu$ and $\omega$ are determined up to one-per-thousand relative error in the whole range of dimensions $3 \le d<4$, using numerical conformal-bootstrap techniques. A detailed comparison is made with results by the resummed epsilon-expansion in varying dimension, the analytic bootstrap, Monte Carlo and non-perturbative renormalization-group methods, finding very good overall agreement. Precise conformal field theory data of scaling dimensions and structure constants are obtained as functions of dimension, improving on earlier findings, and providing benchmarks in $3 \le d<4$.
Flow Matching for Generative Modeling

We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. Training CNFs using Flow Matching on ImageNet leads to consistently better performance than alternative diffusion-based methods in terms of both likelihood and sample quality, and allows fast and reliable sample generation using off-the-shelf numerical ODE solvers.
LungViT: Ensembling Cascade of Texture Sensitive Hierarchical Vision Transformers for Cross-Volume Chest CT Image-to-Image Translation.

Chest computed tomography (CT) at inspiration is often complemented by an expiratory CT to identify peripheral airways disease. Additionally, co-registered inspiratory-expiratory volumes can be used to derive various markers of lung function. Expiratory CT scans, however, may not be acquired due to dose or scan time considerations or may be inadequate due to motion or insufficient exhale; leading to a missed opportunity to evaluate underlying small airways disease. Here, we propose LungViT - a generative adversarial learning approach using hierarchical vision transformers for translating inspiratory CT intensities to corresponding expiratory CT intensities. LungViT addresses several limitations of the traditional generative models including slicewise discontinuities, limited size of generated volumes, and their inability to model texture transfer at volumetric level. We propose a shifted-window hierarchical vision transformer architecture with squeeze-and-excitation decoder blocks for modeling dependencies between features. We also propose a multiview texture similarity distance metric for texture and style transfer in 3D. To incorporate global information into the training process and refine the output of our model, we use ensemble cascading. LungViT is able to generate large 3D volumes of size 320 × 320 × 320. We train and validate our model using a diverse cohort of 1500 subjects with varying disease severity. To assess model generalizability beyond the development set biases, we evaluate our model on an out-of-distribution external validation set of 200 subjects. Clinical validation on internal and external testing sets shows that synthetic volumes could be reliably adopted for deriving clinical endpoints of chronic obstructive pulmonary disease.
Learning to Reason With Relational Abstractions

Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps. However, the solution sequences are not formally structured and the resulting model-generated sequences may not reflect the kind of systematic reasoning we might expect an expert human to produce. In this paper, we study how to build stronger reasoning capability in language models using the idea of relational abstractions. We introduce new types of sequences that more explicitly provide an abstract characterization of the transitions through intermediate solution steps to the goal state. We find that models that are supplied with such sequences as prompts can solve tasks with a significantly higher accuracy, and models that are trained to produce such sequences solve problems better than those that are trained with previously used human-generated sequences and other baselines. Our work thus takes several steps toward elucidating and improving how language models perform on tasks requiring multi-step mathematical reasoning.
Matching Text and Audio Embeddings: Exploring Transfer-Learning Strategies for Language-Based Audio Retrieval

We present an analysis of large-scale pretrained deep learning models used for cross-modal (text-to-audio) retrieval. We use embeddings extracted by these models in a metric learning framework to connect matching pairs of audio and text. Shallow neural networks map the embeddings to a common dimensionality. Our system, which is an extension of our submission to the Language-based Audio Retrieval Task of the DCASE Challenge 2022, employs the RoBERTa foundation model as the text embedding extractor. A pretrained PANNs model extracts the audio embeddings. To improve the generalisation of our model, we investigate how pretraining with audio and associated noisy text collected from the online platform Freesound improves the performance of our method. Furthermore, our ablation study reveals that the proper choice of the loss function and fine-tuning the pretrained models are essential in training a competitive retrieval system.
Exploring the Fundamental Mechanism in Driving Highest-Velocity Ionized Outflows in Radio AGNs

We investigate the ionized gas kinematics relationship with X-ray, radio and accreting properties using a sample of 348 nearby (z<0.4) SDSS-FIRST-X-ray detected AGNs. X-ray properties of our sample are obtained from XMM-Newton, Swift and Chandra observations. We unveil the ionized gas outflows in our sample manifested by the non-gravitational broad component in [O iii]λ5007Å emission line profiles. From the comparison of the correlation of non-parametric outflow velocities (i.e., the velocity width, the maximal velocity of outflow and line dispersion) with X-ray luminosity and radio luminosity, we find that outflow velocities have similarly positive correlations with both X-ray and radio luminosity. After correcting for the gravitational component, we find that the [O iii] velocity dispersion normalized by stellar mass also increases with both X-ray luminosity and radio luminosity. We also find that, for a given X-ray (radio) luminosity, radio (X-ray) luminous AGNs have higher outflow velocities than non-radio (non-X-ray) luminous AGNs. Therefore, we find no clear preference between X-ray luminosity and radio luminosity in driving high-velocity ionized outflows and conclude that both AGN activity and small-scale jets contribute comparably. Moreover, there is no evidence that our obscured AGNs are preferentially associated with higher velocity outflows. Finally, we find a turning point around log(λEdd)≃−1.3 when we explore the dependency of outflow velocity on Eddington ratio. It can be interpreted considering the role of high radiation pressure (log(λEdd)≳−1.3) in causing drastic reduction in the covering factor of the circumnuclear materials.
Conditions for up-down asymmetry in the core of tokamak equilibria

A local magnetic equilibrium solution is sought around the magnetic axis in order to identify the key parameters defining the magnetic-surface's up-down asymmetry in the core of tokamak plasmas. The asymmetry is found to be determined essentially by the ratio of the toroidal current density flowing on axis to the fraction of the external field's odd perturbation that manages to propagate from the plasma boundary into the core. The predictions are tested and illustrated with experimentally relevant numerical equilibria. Hollow current-density distributions, and hence reverse magnetic shear, are seen to be crucial to bring into the core asymmetry values that are usually found only near the plasma edge.
Minimal-order Appointed-time Unknown Input Observers: Design and Applications

This paper presents a framework on minimal-order appointed-time unknown input observers for linear systems based on the pairwise observer structure. A minimal-order appointed-time observer is first proposed for the linear system without the unknown input, which can estimate the state exactly at the preset time by seeking for the unique solution of a system of linear equations. To further release the computational burden, another form of the appointed-time observer is designed. For the general linear system with the unknown input acting on both the system dynamics and the measured output, the model reconfiguration is made to decouple the effect of the unknown input, and the gap between the existing reduced-order appointed-time unknown input observer and the possible minimal-order appointed-time observer is revealed. Based on the reconstructed model, the minimal-order appointed-time unknown input observer is presented to realize state estimation of linear system with the unknown input at the arbitrarily small preset time. The minimal-order appointed-time unknown input observer is then applied to the design of fully distributed adaptive output-feedback attack-free consensus protocols for linear multi-agent systems.
To Softmax, or not to Softmax: that is the question when applying Active Learning for Transformer Models

Despite achieving state-of-the-art results in nearly all Natural Language Processing applications, fine-tuning Transformer-based language models still requires a significant amount of labeled data to work. A well known technique to reduce the amount of human effort in acquiring a labeled dataset is \textit{Active Learning} (AL): an iterative process in which only the minimal amount of samples is labeled. AL strategies require access to a quantified confidence measure of the model predictions. A common choice is the softmax activation function for the final layer. As the softmax function provides misleading probabilities, this paper compares eight alternatives on seven datasets. Our almost paradoxical finding is that most of the methods are too good at identifying the true most uncertain samples (outliers), and that labeling therefore exclusively outliers results in worse performance. As a heuristic we propose to systematically ignore samples, which results in improvements of various methods compared to the softmax function.
Effective Self-supervised Pre-training on Low-compute networks without Distillation

Despite the impressive progress of self-supervised learning (SSL), its applicability to low-compute networks has received limited attention. Reported performance has trailed behind standard supervised pre-training by a large margin, barring self-supervised learning from making an impact on models that are deployed on device. Most prior works attribute this poor performance to the capacity bottleneck of the low-compute networks and opt to bypass the problem through the use of knowledge distillation (KD). In this work, we revisit SSL for efficient neural networks, taking a closer at what are the detrimental factors causing the practical limitations, and whether they are intrinsic to the self-supervised low-compute setting. We find that, contrary to accepted knowledge, there is no intrinsic architectural bottleneck, we diagnose that the performance bottleneck is related to the model complexity vs regularization strength trade-off. In particular, we start by empirically observing that the use of local views can have a dramatic impact on the effectiveness of the SSL methods. This hints at view sampling being one of the performance bottlenecks for SSL on low-capacity networks. We hypothesize that the view sampling strategy for large neural networks, which requires matching views in very diverse spatial scales and contexts, is too demanding for low-capacity architectures. We systematize the design of the view sampling mechanism, leading to a new training methodology that consistently improves the performance across different SSL methods (e.g. MoCo-v2, SwAV, DINO), different low-size networks (e.g. MobileNetV2, ResNet18, ResNet34, ViT-Ti), and different tasks (linear probe, object detection, instance segmentation and semi-supervised learning). Our best models establish a new state-of-the-art for SSL methods on low-compute networks despite not using a KD loss term.
Time Will Change Things: An Empirical Study on Dynamic Language Understanding in Social Media Classification

Language features are ever-evolving in the real-world social media environment. Many trained models in natural language understanding (NLU), ineffective in semantic inference for unseen features, might consequently struggle with the deteriorating performance in dynamicity. To address this challenge, we empirically study social media NLU in a dynamic setup, where models are trained on the past data and test on the future. It better reflects the realistic practice compared to the commonly-adopted static setup of random data split. To further analyze model adaption to the dynamicity, we explore the usefulness of leveraging some unlabeled data created after a model is trained. The performance of unsupervised domain adaption baselines based on auto-encoding and pseudo-labeling and a joint framework coupling them both are examined in the experiments. Substantial results on four social media tasks imply the universally negative effects of evolving environments over classification accuracy, while auto-encoding and pseudo-labeling collaboratively show the best robustness in dynamicity.
CONSTRAINING THE MINIMUM MASS OF HIGH-REDSHIFT GALAXIES AND THEIR CONTRIBUTION TO THE IONIZATION STATE OF THE INTERGALACTIC MEDIUM

We model the latest HST WFPC3/IR observations of ≳100 galaxies at redshifts z = 7–8 in terms of a hierarchical galaxy formation model with starburst activity. Our model provides a distribution of UV luminosities per dark matter halo of a given mass and a natural explanation for the fraction of halos hosting galaxies. The observed luminosity function is best fit with a minimum halo mass per galaxy of , corresponding to a virial temperature of . Extrapolating to faint, undetected galaxies, the total production rate of ionizing radiation depends critically on this minimum mass. Future measurements with JWST should determine whether the entire galaxy population can comfortably account for the UV background required to keep the intergalactic medium ionized.
Passing through the Firewall

We propose that black hole information is encoded in non-local correlations between microscopic interior and exterior degrees of freedom. We give a simple qubit representation of this proposal, and show herein that for every black hole state, the apparent firewall can be removed via a universal, state independent unitary transformation. A central element in our discussion is the distinction between virtual qubits, which are in a specified vacuum state, and real qubits, that carry the free quantum information of the black hole. We outline how our proposal may be realized in AdS/CFT
RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank

Joint-Embedding Self Supervised Learning (JE-SSL) has seen a rapid development, with the emergence of many method variations but only few principled guidelines that would help practitioners to successfully deploy them. The main reason for that pitfall comes from JE-SSL's core principle of not employing any input reconstruction therefore lacking visual cues of unsuccessful training. Adding non informative loss values to that, it becomes difficult to deploy SSL on a new dataset for which no labels can help to judge the quality of the learned representation. In this study, we develop a simple unsupervised criterion that is indicative of the quality of the learned JE-SSL representations: their effective rank. Albeit simple and computationally friendly, this method -- coined RankMe -- allows one to assess the performance of JE-SSL representations, even on different downstream datasets, without requiring any labels. A further benefit of RankMe is that it does not have any training or hyper-parameters to tune. Through thorough empirical experiments involving hundreds of training episodes, we demonstrate how RankMe can be used for hyperparameter selection with nearly no reduction in final performance compared to the current selection method that involve a dataset's labels. We hope that RankMe will facilitate the deployment of JE-SSL towards domains that do not have the opportunity to rely on labels for representations' quality assessment.
What Do I Need to Design for Co-Design? Supporting Co-design as a Designerly Practice

Co-design practices have been used for decades to support participatory engagement in design work. However, despite a wide range of materials that describe the design and commitments of numerous co-design experiences, few descriptions of the knowledge that guides designers when creating these experiences exist. Thus, we ask: What kind of knowledge do designers need to design co-design experiences? What form(s) could intermediate-level knowledge for co-design take? To answer these questions, we adopted a co/auto-ethnographic and Research-through-Design approach to reflexively engage with our design decisions, outcomes, and challenges related to two virtual co-design workshops. We constructed a set of four multi-dimensional facets(Rhythms of Engagement, Material Engagement, Ludic Engagement, and Conceptual Achievement) and three roles (designer, researcher, facilitator) to consider when creating co-design experiences. We illustrate these facets and roles through examples, building new \textit{intermediate-level knowledge} to support future co-design research and design, framing co-design as a designerly practice.
Spinor Structure and Modulo 8 Periodicity

Spinor structure is understood as a totality of tensor products of biquaternion algebras, and the each tensor product is associated with an irreducible representation of the Lorentz group. A so-defined algebraic structure allows one to apply modulo 8 periodicity of Clifford algebras on the system of real and quaternionic representations of the Lorentz group. It is shown that modulo 8 periodic action of the Brauer-Wall group generates modulo 2 periodic relations on the system of representations, and all the totality of representations under this action forms a self-similar fractal structure. Some relations between spinors, twistors and qubits are discussed in the context of quantum information and decoherence theory.
Robust thresholds: Counting triangle factors and A shorter proof of the robust Corrádi-Hajnal Theorem

A BSTRACT . We show that the distribution of perfect matchings in an r -uniform hypergraph on n vertices induced by the edges of an r -partite super-regular graph is O (1 /n r − 1 ) -spread. This yields a short proof of the robust Corr´adi–Hajnal Theorem recently obtained in [1], that there is C > 0 such that, for a graph G with minimum degree 2 n/ 3 , a random subgraph of G where each edge is retained with probability C (log n ) 1 / 3 n − 2 / 3 contains a triangle factor with high probability. We also show that the number of triangle factors in a graph with minimum degree 2 n/ 3 is at least ( cn ) 2 n/ 3 for a constant c > 0 , addressing a question in [1].
A large 12C/13C isotopic ratio in M82 and NGC253

To derive carbon isotopic ratios from optically thin tracers in the central regions of the starburst galaxies M82 and NGC253. We present high sensitivity observations of CCH and two of its 13C isotopologues, C13CH and 13CCH, as well as the optically thin emission from C18O and 13C18O. We assume the column density ratio between isotopologues is representative of the 12C13C isotopic ratio. From CCH, lower limits to the 12C/13C isotopic ratio of 138 in M82, and 81 in NGC253, are derived. Lower limits to the 12C/13C ratios from CO isotopologues support these. 13C18O is tentatively detected in NGC253, which is the first reported detection in the extragalactic ISM. Based on these limits, we infer ratios of 16O/18O>350 and >300 in M82 and NGC253, respectively, and 32S/34S>16 in NGC253. and the H2 column density determination through the optically thin tracers 13CO and C18O. The derived CCH fractional abundances toward these galaxies of <~1.1\times10^-8 are in good agreement with those of molecular clouds in the Galactic disk. Our lower limits to the 12C/13C ratio from CCH are a factor of 2-3 larger than previous limits. The results are discussed in the context of molecular and nucleo-chemical evolution. The large 12C/13C isotopic ratio of the molecular ISM in these starburst galaxies suggest that the gas has been recently accreted toward their nuclear regions.