Miracle-dz/UniFAR-FTUs huggingface dataset:
BERTVision - A Parameter-Efficient Approach for Question Answering

We present a highly parameter efficient approach for Question Answering that significantly reduces the need for extended BERT fine-tuning. Our method uses information from the hidden state activations of each BERT transformer layer, which is discarded during typical BERT inference. Our best model achieves maximal BERT performance at a fraction of the training time and GPU or TPU expense. Performance is further improved by ensembling our model with BERTs predictions. Furthermore, we find that near optimal performance can be achieved for QA span annotation using less training data. Our experiments show that this approach works well not only for span annotation, but also for classification, suggesting that it may be extensible to a wider range of tasks.
Density of rational points on manifolds and Diophantine approximation on hypersurfaces

In this article, we establish an analogue of the dimension growth conjecture, which is regarding the density of rational points on projective varieties, for compact submanifolds of R with non-vanishing curvature. We also establish the convergence theory for the set of simultaneously ψapproximable points lying on a generic hypersurface, thereby settling the generalized Baker-Schmidt problem in the simultaneous setting for generic hypersurfaces. These results are obtained as consequences of an optimal upper bound for the density of rational points near manifolds of the form {(x, f(x)) ∈ R : x ∈ Bν(x0)} with non-zero Hessian matrix of f at x0 and ν > 0 sufficiently small.
Tactile Materials in Practice: Understanding the Experiences of Teachers of the Visually Impaired

Teachers of the visually impaired (TVIs) regularly present tactile materials (tactile graphics, 3D models, and real objects) to students with vision impairments. Researchers have been increasingly interested in designing tools to support the use of tactile materials, but we still lack an in-depth understanding of how tactile materials are created and used in practice today. To address this gap, we conducted interviews with 21 TVIs and a 3-week diary study with eight of them. We found that tactile materials were regularly used for academic as well as non-academic concepts like tactile literacy, motor ability, and spatial awareness. Real objects and 3D models served as “stepping stones” to tactile graphics and our participants preferred to teach with 3D models, despite finding them difficult to create, obtain, and modify. Use of certain materials also carried social implications; participants selected materials that fostered student independence and allow classroom inclusion. We contribute design considerations, encouraging future work on tactile materials to enable student and TVI co-creation, facilitate rapid prototyping, and promote movement and spatial awareness. To support future research in this area, our paper provides a fundamental understanding of current practices. We bridge these practices to established pedagogical approaches and highlight opportunities for growth regarding this important genre of educational materials.
ALP induced polarization effects on photons from galaxy clusters

Many extensions of the Standard Model of particle physics and in particular superstring and superbrane theories predict the existence of axion-like particles (ALPs). ALPs are very elusive, extremely light and interact primarily with photons. In the presence of an external magnetic field two effects show up: (i) photon-ALP oscillations and (ii) a change of the photon polarization state. The astrophysical context represents the best opportunity to get indirect evidence for the ALP existence thanks to various effects that the photon-ALP interaction produces in the sky. Great attention has been paid so far to photon-ALP oscillations, since they modify the transparency of the crossed media at very high energies and so the final spectra of faraway sources exhibit a flux excess and a characteristic oscillatory behavior. Two hints at the ALP existence have hitherto been discovered. But less interest has been attracted by the modification of the photon polarization. In this paper we address it in the X-ray and in the high energy (HE) bands. Specifically, we analyze the photon degree of linear polarization and the polarization angle induced by the photon-ALP interaction for photons generated in the central region of two galaxy clusters: Perseus and Coma. We find a substantial departure from conventional physics in both considered bands. We conclude that the ALP-induced polarization effects are more likely detectable with the proposed missions like COSI (approved to launch), e-ASTROGAM and AMEGO in the HE range. Still, possible ALP-induced effects on photon polarization could also be detected by IXPE (already operative) and by the proposed eXTP, XL-Calibur, NGXP and XPP in the X-ray band.
Learning Multi-Object Dynamics with Compositional Neural Radiance Fields

We present a method to learn compositional multi-object dynamics models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks. NeRFs have become a popular choice for representing scenes due to their strong 3D prior. However, most NeRF approaches are trained on a single scene, representing the whole scene with a global model, making generalization to novel scenes, containing different numbers of objects, challenging. Instead, we present a compositional, object-centric auto-encoder framework that maps multiple views of the scene to a set of latent vectors representing each object separately. The latent vectors parameterize individual NeRFs from which the scene can be reconstructed. Based on those latent vectors, we train a graph neural network dynamics model in the latent space to achieve compositionality for dynamics prediction. A key feature of our approach is that the latent vectors are forced to encode 3D information through the NeRF decoder, which enables us to incorporate structural priors in learning the dynamics models, making long-term predictions more stable compared to several baselines. Simulated and real world experiments show that our method can model and learn the dynamics of compositional scenes including rigid and deformable objects. Video: https://dannydriess.github. io/compnerfdyn/
Predicting the impact of treatments over time with uncertainty aware neural differential equations

Predicting the impact of treatments from observational data only still represents a majorchallenge despite recent significant advances in time series modeling. Treatment assignments are usually correlated with the predictors of the response, resulting in a lack of data support for counterfactual predictions and therefore in poor quality estimates. Developments in causal inference have lead to methods addressing this confounding by requiring a minimum level of overlap. However,overlap is difficult to assess and usually notsatisfied in practice. In this work, we propose Counterfactual ODE (CF-ODE), a novel method to predict the impact of treatments continuously over time using Neural Ordinary Differential Equations equipped with uncertainty estimates. This allows to specifically assess which treatment outcomes can be reliably predicted. We demonstrate over several longitudinal data sets that CF-ODE provides more accurate predictions and more reliable uncertainty estimates than previously available methods.
Formation and Morphology of the First Galaxies in the Cosmic Morning

We investigate the formation and morphological evolution of the first galaxies in the cosmic morning (10 ≳ z ≳ 4) using the Horizon Run 5 (HR5) simulation. For galaxies above the stellar mass M⋆,min=2×109M⊙ , we classify them into disk, spheroid, and irregular types according to their asymmetry and stellar-mass morphology. We find that about two-thirds of the galaxies have a Sérsic index <1.5, reflecting the dominance of disk-type morphology in the cosmic morning. The rest are evenly distributed as incidental and transient irregulars or spheroids. These fractions are roughly independent of redshift and stellar mass up to ∼1010 M ⊙. Almost all the first galaxies with M⋆>M⋆,min at z > 4 form at initial peaks of the matter-density field. Large-scale structures in the universe emerge and grow like cosmic rhizomes as the underlying matter-density fluctuations grow and form associations of galaxies in rare overdense regions and the realm of the galactic world is stretched into relatively lower-density regions along evolving filaments. The cosmic web of galaxies forms at lower redshifts when most rhizomes globally percolate. The primordial angular momentum produced by the induced tidal torques on protogalactic regions is correlated with the internal kinematics of galaxies and tightly aligned with the angular momentum of the total galaxy mass. The large-scale tidal field imprinted in the initial conditions seems responsible for the dominance of disk morphology and for the tendency of galaxies to reacquire a disk postdistortion.
Attainability and Optimality: The Equalized Odds Fairness Revisited

Fairness of machine learning algorithms has been of increasing interest. In order to suppress or eliminate discrimination in prediction, various notions as well as approaches have been proposed to impose fairness. Given a notion of fairness, an essential problem is then whether or not it can always be attained, even if with an unlimited amount of data. This issue is, however, not well addressed yet. In this paper, focusing on the Equalized Odds notion of fairness, we consider the attainability of this criterion and, furthermore, if it is attainable, the optimality of the prediction performance under various settings. In particular, for prediction performed by a deterministic function of input features, we give conditions under which Equalized Odds can hold true; if the stochastic prediction is acceptable, we show that under mild assumptions, fair predictors can always be derived. For classification, we further prove that compared to enforcing fairness by post-processing, one can always benefit from exploiting all available features during training and get potentially better prediction performance while remaining fair. Moreover, while stochastic prediction can attain Equalized Odds with theoretical guarantees, we also discuss its limitation and potential negative social impacts.
FreeSOLO: Learning to Segment Objects without Annotations

Instance segmentation is a fundamental vision task that aims to recognize and segment each object in an image. However, it requires costly annotations such as bounding boxes and segmentation masks for learning. In this work, we propose a fully unsupervised learning method that learns class-agnostic instance segmentation without any annotations. We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO. Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner. FreeSOLO achieves 9.8% $AP_{50}$ on the challenging COCO dataset, which even outperforms several segmentation proposal methods that use manual annotations. For the first time, we demonstrate unsupervised class-agnostic instance segmen-tation successfully. FreeSOLO's box localization significantly outperforms state-of-the-art unsupervised object de-tection/discovery methods, with about 100% relative improvements in COCO AP. FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by $+9.8\%$ AP when fine-tuning instance segmentation with only 5% COCO masks. Code is available at: github.com/NVlabs/FreeSOLO
Semiseparable functors

In this paper we introduce and investigate the notion of semiseparable functor. One of its first features is that it allows a novel description of separable and naturally full functors in terms of faithful and full functors, respectively. To any semiseparable functor we attach an invariant, given by an idempotent natural transformation, which controls when the functor is separable and yields a characterization of separable functors in terms of (dual) Maschke and conservative functors. We prove that any semiseparable functor admits a canonical factorization as a naturally full functor followed by a separable functor. Here the main tool is the construction of the coidentifier category attached to the associated idempotent natural transformation. Then we move our attention to the semiseparability of functors that have an adjoint. First we obtain a Rafael-type Theorem. Next we characterize the semiseparability of adjoint functors in terms of the (co)separability of the associated (co)monads and the natural fullness of the corresponding (co)comparison functor. We also focus on functors that are part of an adjoint triple. In particular, we describe bireflections as semiseparable (co)reflections, or equivalently, as either Frobenius or naturally full (co)reflections. As an application of our results, we study the semiseparability of functors traditionally attached to ring homomorphisms, coalgebra maps, corings and bimodules, introducing the notions of semicosplit coring and semiseparability relative to a bimodule which extend those of cosplit coring and Sugano’s separability relative to a bimodule, respectively.
Self-Training: A Survey

Semi-supervised algorithms aim to learn prediction functions from a small set of labeled observations and a large set of unlabeled observations. Because this framework is relevant in many applications, they have received a lot of interest in both academia and industry. Among the existing techniques, self-training methods have undoubtedly attracted greater attention in recent years. These models are designed to find the decision boundary on low density regions without making additional assumptions about the data distribution, and use the unsigned output score of a learned classifier, or its margin, as an indicator of confidence. The working principle of self-training algorithms is to learn a classifier iteratively by assigning pseudo-labels to the set of unlabeled training samples with a margin greater than a certain threshold. The pseudo-labeled examples are then used to enrich the labeled training data and to train a new classifier in conjunction with the labeled training set. In this paper, we present self-training methods for binary and multi-class classification; as well as their variants and two related approaches, namely consistency-based approaches and transductive learning. We examine the impact of significant self-training features on various methods, using different general and image classification benchmarks, and we discuss our ideas for future research in self-training. To the best of our knowledge, this is the first thorough and complete survey on this subject.
Self-Distilled StyleGAN: Towards Generation from Internet Photos

StyleGAN is known to produce high-fidelity images, while also offering unprecedented semantic editing. However, these fascinating abilities have been demonstrated only on a limited set of datasets, which are usually structurally aligned and well curated. In this paper, we show how StyleGAN can be adapted to work on raw uncurated images collected from the Internet. Such image collections impose two main challenges to StyleGAN: they contain many outlier images, and are characterized by a multi-modal distribution. Training StyleGAN on such raw image collections results in degraded image synthesis quality. To meet these challenges, we proposed a StyleGAN-based self-distillation approach, which consists of two main components: (i) A generative-based self-filtering of the dataset to eliminate outlier images, in order to generate an adequate training set, and (ii) Perceptual clustering of the generated images to detect the inherent data modalities, which are then employed to improve StyleGAN’s “truncation trick” in the image synthesis process. The presented technique enables the generation of high-quality images, while minimizing the loss in diversity of the data. Through qualitative and quantitative evaluation, we demonstrate the power of our approach to new challenging and diverse domains collected from the Internet. New datasets and pre-trained models are provided in our project website https://self-distilled-stylegan.github. io/.
Flat latent manifolds for music improvisation between human and machine

The use of machine learning in artistic music generation leads to controversial discussions of the quality of art, for which objective quantiﬁcation is nonsen-sical. We therefore consider a music-generating algorithm as a counterpart to a human musician, in a setting where reciprocal improvisation is to lead to new experiences, both for the musician and the audience. To obtain this behaviour, we resort to the framework of recurrent Variational Auto-Encoders (VAE) and learn to generate music, seeded by a human musician. In the learned model, we generate novel musical sequences by interpolation in latent space. Standard VAEs however do not guarantee any form of smoothness in their latent representation. This translates into abrupt changes in the generated music sequences. To overcome these limitations, we regularise the decoder and endow the latent space with a ﬂat Riemannian manifold, i.e., a manifold that is isometric to the Euclidean space. As a result, linearly interpolating in the latent space yields realistic and smooth musical changes that ﬁt the type of machine–musician interactions we aim for. We provide empirical evidence for our method via a set of experiments on music datasets and we deploy our model for an interactive jam session with a professional drummer. The live performance provides qualitative evidence that the latent representation can be intuitively interpreted and exploited by the drummer to drive the interplay. Beyond the musical application, our approach showcases an instance of human-centred design of machine-learning models, driven by interpretability and the interaction with the end user.
Probing Predictions on OOD Images via Nearest Categories

We study out-of-distribution (OOD) prediction behavior of neural networks when they classify images from unseen classes or corrupted images. To probe the OOD behavior, we introduce a new measure, nearest category generalization (NCG), where we compute the fraction of OOD inputs that are classified with the same label as their nearest neighbor in the training set. Our motivation stems from understanding the prediction patterns of adversarially robust networks, since previous work has identified unexpected consequences of training to be robust to norm-bounded perturbations. We find that robust networks have consistently higher NCG accuracy than natural training, even when the OOD data is much farther away than the robustness radius. This implies that the local regularization of robust training has a significant impact on the network's decision regions. We replicate our findings using many datasets, comparing new and existing training methods. Overall, adversarially robust networks resemble a nearest neighbor classifier when it comes to OOD data. Code available at https://github.com/yangarbiter/nearest-category-generalization.
XAutoML: A Visual Analytics Tool for Establishing Trust in Automated Machine Learning

In the last ten years, various automated machine learning (AutoML) systems have been proposed to build end-to-end machine learning (ML) pipelines with minimal human interaction. Even though such automatically synthesized ML pipelines are able to achieve a competitive performance, recent studies have shown that users do not trust models constructed by AutoML due to missing transparency of AutoML systems and missing explanations for the constructed ML pipelines. In a requirements analysis study with 26 domain experts, data scientists, and AutoML researchers from different professions with vastly different expertise in ML, we collect detailed informational needs to establish trust in AutoML. We propose XAutoML, an interactive visual analytics tool for explaining arbitrary AutoML optimization procedures and ML pipelines constructed by AutoML. XAutoML combines interactive visualizations with established techniques from explainable artificial intelligence (XAI) to make the complete AutoML procedure transparent and explainable. By integrating XAutoML with JupyterLab, experienced users can extend the visual analytics with ad-hoc visualizations based on information extracted from XAutoML. We validate our approach in a user study with the same diverse user group from the requirements analysis. All participants were able to extract useful information from XAutoML, leading to a significantly increased trust in ML pipelines produced by AutoML and the AutoML optimization itself.
Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction

In this paper, we propose an effective yet efficient model PAIE for both sentence-level and document-level Event Argument Extraction (EAE), which also generalizes well when there is a lack of training data. On the one hand, PAIE utilizes prompt tuning for extractive objectives to take the best advantages of Pre-trained Language Models (PLMs). It introduces two span selectors based on the prompt to select start/end tokens among input texts for each role. On the other hand, it captures argument interactions via multi-role prompts and conducts joint optimization with optimal span assignments via a bipartite matching loss. Also, with a flexible prompt design, PAIE can extract multiple arguments with the same role instead of conventional heuristic threshold tuning. We have conducted extensive experiments on three benchmarks, including both sentence- and document-level EAE. The results present promising improvements from PAIE (3.5% and 2.3% F1 gains in average on three benchmarks, for PAIE-base and PAIE-large respectively). Further analysis demonstrates the efficiency, generalization to few-shot settings, and effectiveness of different extractive prompt tuning strategies. Our code is available at https://github.com/mayubo2333/PAIE.
Influences of Internal Forcing on Atmospheric Circulations of Irradiated Giant Planets

Close-in giant planets with strong stellar irradiation show atmospheric circulation patterns with strong equatorial jets and global-scale stationary waves. So far, almost all modeling works on atmospheric circulations of such giant planets have mainly considered external radiation alone, without taking into account the role of internal heat fluxes or just treating it in very simplified ways. Here, we study atmospheric circulations of strongly irradiated giant planets by considering the effect of internal forcing, which is characterized by small-scale stochastic interior thermal perturbations, using a three-dimensional atmospheric general circulation model. We show that the perturbation-excited waves can largely modify atmospheric circulation patterns in the presence of relatively strong internal forcing. Specifically, our simulations demonstrate three circulation regimes: a superrotation regime, a midlatitude-jet regime, and a quasi-periodic oscillation regime, depending on the relative importance of external and internal forcings. It is also found that strong internal forcing can cause noticeable modifications of the thermal phase curves.