Miracle-dz/UniFAR-FTUs huggingface dataset:
Predicting regrowth of low-grade gliomas after radiotherapy

Diffuse low grade gliomas are invasive and incurable brain tumors that inevitably transform into higher grade ones. A classical treatment to delay this transition is radiotherapy (RT). Following RT, the tumor gradually shrinks during a period of typically 6 months to 4 years before regrowing. To improve the patient’s health-related quality of life and help clinicians build personalized follow-ups, one would benefit from predictions of the time during which the tumor is expected to decrease. The challenge is to provide a reliable estimate of this regrowth time shortly after RT (i.e. with few data), although patients react differently to the treatment. To this end, we analyze the tumor size dynamics from a batch of 20 high-quality longitudinal data, and propose a simple and robust analytical model, with just 4 parameters. From the study of their correlations, we build a statistical constraint that helps determine the regrowth time even for patients for which we have only a few measurements of the tumor size. We validate the procedure on the data and predict the regrowth time at the moment of the first MRI after RT, with precision of, typically, 6 months. Using virtual patients, we study whether some forecast is still possible just three months after RT. We obtain some reliable estimates of the regrowth time in 75% of the cases, in particular for all “fast-responders”. The remaining 25% represent cases where the actual regrowth time is large and can be safely estimated with another measurement a year later. These results show the feasibility of making personalized predictions of the tumor regrowth time shortly after RT.
ELUDE: Generating interpretable explanations via a decomposition into labelled and unlabelled features

Deep learning models have achieved remarkable success in diﬀerent areas of machine learning over the past decade; however, the size and complexity of these models make them diﬃcult to understand. In an eﬀort to make them more interpretable, several recent works focus on explaining parts of a deep neural network through human-interpretable, semantic attributes. However, it may be impossible to completely explain complex models using only semantic attributes. In this work, we propose to augment these attributes with a small set of uninterpretable features. Speciﬁcally, we develop a novel explanation framework ELUDE (Expla-nation via Labelled and Unlabelled DEcomposition) that decomposes a model’s prediction into two parts: one that is explainable through a linear combination of the semantic attributes, and another that is dependent on the set of uninterpretable features. By identifying the latter, we are able to analyze the “unexplained” portion of the model, obtaining insights into the information used by the model. We show that the set of unlabelled features can generalize to multiple models trained with the same feature space and compare our work to two popular attribute-oriented methods, Interpretable Basis Decomposition and Concept Bottleneck, and discuss the additional insights ELUDE provides. because the unlabelled feature subspace Uf ( x ) is low-dimensional). We also show that current interpretable-by-degin models are still far from being fully interpretable: when evaluating an interpretable-by-design Concept Bottleneck model, the labelled attributes explain less than 30% of the entire model.
Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexity

We propose a general framework to design posterior sampling methods for model-based RL. We show that the proposed algorithms can be analyzed by reducing regret to Hellinger distance in conditional probability estimation. We further show that optimistic posterior sampling can control this Hellinger distance, when we measure model error via data likelihood. This technique allows us to design and analyze unified posterior sampling algorithms with state-of-the-art sample complexity guarantees for many model-based RL settings. We illustrate our general result in many special cases, demonstrating the versatility of our framework.
Explainable expected goal models for performance analysis in football analytics

The expected goal provides a more representative measure of the team and player performance which also suit the low-scoring nature of football instead of score in modern football. The score of a match involves randomness and often may not represent the performance of the teams and players, therefore it has been popular to use the alternative statistics in recent years such as shots on target, ball possessions, and drills. To measure the probability of a shot being a goal by the expected goal, several features are used to train an expected goal model which is based on the event and tracking football data. The selection of these features, the size and date of the data, and the model which are used as the parameters that may affect the performance of the model. Using black-box machine learning models for increasing the predictive performance of the model decreases its interpretability that causes the loss of information that can be gathered from the model. This paper proposes an accurate expected goal model trained consisting of 315,430 shots from seven seasons between 2014-15 and 2020-21 of the top-five European football leagues. Moreover, this model is explained by using explainable artificial intelligence tool to obtain an explainable expected goal model for evaluating a team or player performance. To the best of our knowledge, this is the first paper that demonstrates a practical application of an explainable artificial intelligence tool aggregated profiles to explain a group of observations on an accurate expected goal model for monitoring the team and player performance. Moreover, these methods can be generalized to other sports branches.
Detection of long-range entanglement in gapped quantum spin liquids by local measurements

Topological order, reﬂected in long range patterns of entanglement, is quantiﬁed by the topological entanglement entropy (TEE) γ . We show that for gapped quantum spin liquids (QSL) it is possible to extract γ using two-spin local correlators. We demonstrate our method for the gapped Z 2 Kitaev spin liquid on a honeycomb lattice with anisotropic interactions. We show that the γ = log 2 for Z 2 topological order can be simply extracted from local two-spin correlators across two diﬀerent bonds, with an accuracy comparable or higher than the Kitaev-Preskill construction. This implies that the diﬀerent superselection sectors of Z 2 gauge theory determined by global Wilson loop operators can be fully reﬂected locally in the matter majorana sector.
AVATAR: Unconstrained Audiovisual Speech Recognition

Audio-visual automatic speech recognition (AV-ASR) is an extension of ASR that incorporates visual cues, often from the movements of a speaker's mouth. Unlike works that simply focus on the lip motion, we investigate the contribution of entire visual frames (visual actions, objects, background etc.). This is particularly useful for unconstrained videos, where the speaker is not necessarily visible. To solve this task, we propose a new sequence-to-sequence AudioVisual ASR TrAnsformeR (AVATAR) which is trained end-to-end from spectrograms and full-frame RGB. To prevent the audio stream from dominating training, we propose different word-masking strategies, thereby encouraging our model to pay attention to the visual stream. We demonstrate the contribution of the visual modality on the How2 AV-ASR benchmark, especially in the presence of simulated noise, and show that our model outperforms all other prior work by a large margin. Finally, we also create a new, real-world test bed for AV-ASR called VisSpeech, which demonstrates the contribution of the visual modality under challenging audio conditions.
Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition

Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems designed for normal speech. Their practical application to atypical task domains such as elderly and disordered speech across languages is often limited by the difficulty in collecting such specialist data from target speakers. This paper presents a cross-domain and cross-lingual A2A inversion approach that utilizes the parallel audio and ultrasound tongue imaging (UTI) data of the 24-hour TaL corpus in A2A model pre-training before being cross-domain and cross-lingual adapted to three datasets across two languages: the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech corpora; and the English TORGO dysarthric speech data, to produce UTI based articulatory features. Experiments conducted on three tasks suggested incorporating the generated articulatory features consistently outperformed the baseline TDNN and Conformer ASR systems constructed using acoustic features only by statistically significant word or character error rate reductions up to 4.75%, 2.59% and 2.07% absolute (14.69%, 10.64% and 22.72% relative) after data augmentation, speaker adaptation and cross system multi-pass decoding were applied.
Federated Learning with Uncertainty via Distilled Predictive Distributions

Most existing federated learning methods are unable to estimate model/predictive uncertainty since the client models are trained using the standard loss function minimization approach which ignores such uncertainties. In many situations, however, especially in limited data settings, it is beneficial to take into account the uncertainty in the model parameters at each client as it leads to more accurate predictions and also because reliable estimates of uncertainty can be used for tasks, such as out-of-distribution (OOD) detection, and sequential decision-making tasks, such as active learning. We present a framework for federated learning with uncertainty where, in each round, each client infers the posterior distribution over its parameters as well as the posterior predictive distribution (PPD), distills the PPD into a single deep neural network, and sends this network to the server. Unlike some of the recent Bayesian approaches to federated learning, our approach does not require sending the whole posterior distribution of the parameters from each client to the server but only the PPD in the distilled form as a deep neural network. In addition, when making predictions at test time, it does not require computationally expensive Monte-Carlo averaging over the posterior distribution because our approach always maintains the PPD in the form of a single deep neural network. Moreover, our approach does not make any restrictive assumptions, such as the form of the clients' posterior distributions, or of their PPDs. We evaluate our approach on classification in federated setting, as well as active learning and OOD detection in federated settings, on which our approach outperforms various existing federated learning baselines.
Specific Heat of Thin Phonon Cavities at Low Temperature: Very High Values Revealed by ZeptoJoule Calorimetry

Specific heat of phonon cavities is investigated in order to analyse the effect of phonon confinement on thermodynamic properties. The specific heat of free standing very thin SiN membranes in the low dimensional limit is measured down to very low temperatures (from 6~K to 50~mK). In the whole temperature range, we measured an excess of specific heat orders of magnitude bigger than the typical value observed in amorphous solids. Below 1~K, a cross-over in $c_p$ to a lower power law is seen, and the value of specific heat of thinner membranes becomes larger than that of thicker ones demonstrating a significant contribution coming from the surface. We show that this high value of the specific heat cannot be explained by the sole contribution of 2D phonon modes (Lamb waves). The excess specific heat, being thickness dependent, could come from tunneling two level systems (TLS) that form in low density regions of amorphous solids located on the surfaces. We also show that the specific heat is strongly tuned by the internal stress of the membrane by orders of magnitude, giving unprecedentedly high values, making low stress SiN very efficient for energy storage at very low temperature.
If it Bleeds, it Leads: A Computational Approach to Covering Crime in Los Angeles

NOTE (2022-06-14): This work was done for a class project in Fall 2019 and is shown, un-less noted, without changes. Developing and improving computational approaches to covering news can increase journalistic output and improve the way stories are covered. In this work we approach the problem of covering crime stories in Los Angeles. We present a machine-in-the-loop system that covers individual crimes by (1) learning the prototypical coverage archetypes from classical news articles on crime to learn their structure and (2) using output from the Los Angeles Police department to generate “lede para-graphs”, ﬁrst structural unit of crime-articles. We introduce a probabilistic graphical model for learning article structure and a rule-based system for generating ledes. We hope our work can lead to systems that use these com-ponents together to form the skeletons of news articles covering crime. 1
CARD: Classification and Regression Diffusion Models

Learning the distribution of a continuous or categorical response variable $\boldsymbol y$ given its covariates $\boldsymbol x$ is a fundamental problem in statistics and machine learning. Deep neural network-based supervised learning algorithms have made great progress in predicting the mean of $\boldsymbol y$ given $\boldsymbol x$, but they are often criticized for their ability to accurately capture the uncertainty of their predictions. In this paper, we introduce classification and regression diffusion (CARD) models, which combine a denoising diffusion-based conditional generative model and a pre-trained conditional mean estimator, to accurately predict the distribution of $\boldsymbol y$ given $\boldsymbol x$. We demonstrate the outstanding ability of CARD in conditional distribution prediction with both toy examples and real-world datasets, the experimental results on which show that CARD in general outperforms state-of-the-art methods, including Bayesian neural network-based ones that are designed for uncertainty estimation, especially when the conditional distribution of $\boldsymbol y$ given $\boldsymbol x$ is multi-modal. In addition, we utilize the stochastic nature of the generative model outputs to obtain a finer granularity in model confidence assessment at the instance level for classification tasks.
Sparse Structure Search for Parameter-Efficient Tuning

Adapting large pre-trained models (PTMs) through fine-tuning imposes prohibitive computational and storage burdens. Recent studies of parameter-efficient tuning (PET) find that only optimizing a small portion of parameters conditioned on PTMs could yield on-par performance compared to conventional fine-tuning. Generally, PET methods exquisitely design parameter-efficient modules (PET modules) which could be applied to arbitrary fine-grained positions inside PTMs. However, the effectiveness of these fine-grained positions largely relies on sophisticated manual designation, thereby usually producing sub-optimal results. In contrast to the manual designation, we explore constructing PET modules in an automatic manner. We automatically \textbf{S}earch for the \textbf{S}parse \textbf{S}tructure of \textbf{P}arameter-\textbf{E}fficient \textbf{T}uning (S$^3$PET). Based on a unified framework of various PET methods, S$^3$PET conducts the differentiable PET structure search through bi-level optimization and proposes shifted global sigmoid method to explicitly control the number of trainable parameters. Extensive experiments show that S$^3$PET surpasses manual and random structures with less trainable parameters. The searched structures preserve more than 99\% fine-tuning performance with 0.01\% trainable parameters. Moreover, the advantage of S$^3$PET is amplified with extremely low trainable parameters budgets (0.0009\%$\sim$0.01\%). The searched structures are transferable and explainable, providing suggestions and guidance for the future design of PET methods.
Probabilistic spatial transformer networks

Spatial Transformer Networks (STNs) estimate image transformations that can improve downstream tasks by `zooming in' on relevant regions in an image. However, STNs are hard to train and sensitive to mis-predictions of transformations. To circumvent these limitations, we propose a probabilistic extension that estimates a stochastic transformation rather than a deterministic one. Marginalizing transformations allows us to consider each image at multiple poses, which makes the localization task easier and the training more robust. As an additional benefit, the stochastic transformations act as a localized, learned data augmentation that improves the downstream tasks. We show across standard imaging benchmarks and on a challenging real-world dataset that these two properties lead to improved classification performance, robustness and model calibration. We further demonstrate that the approach generalizes to non-visual domains by improving model performance on time-series data.
Learning the Structure of Large Networked Systems Obeying Conservation Laws

Many networked systems such as electric networks, the brain, and social networks of opinion dynamics are known to obey conservation laws. Examples of this phenomenon include the Kirchoff laws in electric networks and opinion consensus in social networks. Conservation laws in networked systems may be modeled as balance equations of the form $X = B^{*} Y$, where the sparsity pattern of $B^{*}$ captures the connectivity of the network, and $Y, X \in \mathbb{R}^p$ are vectors of"potentials"and"injected flows"at the nodes respectively. The node potentials $Y$ cause flows across edges and the flows $X$ injected at the nodes are extraneous to the network dynamics. In several practical systems, the network structure is often unknown and needs to be estimated from data. Towards this, one has access to samples of the node potentials $Y$, but only the statistics of the node injections $X$. Motivated by this important problem, we study the estimation of the sparsity structure of the matrix $B^{*}$ from $n$ samples of $Y$ under the assumption that the node injections $X$ follow a Gaussian distribution with a known covariance $\Sigma_X$. We propose a new $\ell_{1}$-regularized maximum likelihood estimator for this problem in the high-dimensional regime where the size of the network $p$ is larger than sample size $n$. We show that this optimization problem is convex in the objective and admits a unique solution. Under a new mutual incoherence condition, we establish sufficient conditions on the triple $(n,p,d)$ for which exact sparsity recovery of $B^{*}$ is possible with high probability; $d$ is the degree of the graph. We also establish guarantees for the recovery of $B^{*}$ in the element-wise maximum, Frobenius, and operator norms. Finally, we complement these theoretical results with experimental validation of the performance of the proposed estimator on synthetic and real-world data.
X-Ray Observations of the Isolated Black Hole Candidate OGLE-2011-BLG-0462 and Other Collapsed Objects Discovered through Gravitational Microlensing

Isolated black holes and neutron stars can be revealed through the observation of long-duration gravitational microlensing events. A few candidates have been found in surveys of stars in the direction of the Galactic bulge. Recently, thanks to the addition of astrometric information at milliarcsecond level, it has been possible to reduce the uncertainties in the masses and distances for some of these “dark” gravitational lenses and select the most promising candidates. These isolated compact objects might emit X-rays powered by accretion from the interstellar medium. Using data of the Chandra, XMM-Newton, and INTEGRAL satellites, we searched for X-ray emission in the isolated black hole candidate OGLE-2011-BLG-0462, and in several other putative collapsed objects found with gravitational microlensing. OGLE-2011-BLG-0462 has been recently interpreted as a 7.1 M ⊙ black hole at a distance of 1.6 kpc, although a different group obtained a mass range (1.6–4.4 M ⊙) that cannot exclude a massive neutron star. We have derived upper limits on the flux from OGLE-2011-BLG-0462 of 9 × 10−15 erg cm−2 s−1 in the 0.5–7 keV range and ∼2 × 10−12 erg cm−2 s−1 in the 17–60 keV range. The implied X-ray luminosity is consistent with the small radiative efficiency expected for a black hole and disfavors a neutron star interpretation. Limits down to a factor of about five lower are obtained for the soft X-ray flux of other candidates, but their interpretation is affected by larger uncertainties in the masses, distances, and spatial velocities.
Masked Frequency Modeling for Self-Supervised Visual Pre-Training

We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models. Instead of randomly inserting mask tokens to the input embeddings in the spatial domain, in this paper, we shift the perspective to the frequency domain. Specifically, MFM first masks out a portion of frequency components of the input image and then predicts the missing frequencies on the frequency spectrum. Our key insight is that predicting masked components in the frequency domain is more ideal to reveal underlying image patterns rather than predicting masked patches in the spatial domain, due to the heavy spatial redundancy. Our findings suggest that with the right configuration of mask-and-predict strategy, both the structural information within high-frequency components and the low-level statistics among low-frequency counterparts are useful in learning good representations. For the first time, MFM demonstrates that, for both ViT and CNN, a simple non-Siamese framework can learn meaningful representations even using none of the following: (i) extra data, (ii) extra model, (iii) mask token. Experimental results on image classification and semantic segmentation, as well as several robustness benchmarks show the competitive performance and advanced robustness of MFM compared with recent masked image modeling approaches. Furthermore, we also comprehensively investigate the effectiveness of classical image restoration tasks for representation learning from a unified frequency perspective and reveal their intriguing relations with our MFM approach.