Miracle-dz/UniFAR-FTUs huggingface dataset:
Bruno Touschek in Germany after the War: 1945-46

Bruno Touschek was an Austrian born theoretical physicist, who proposed and built the first electron-positron collider in 1960 in the Frascati National Laboratories in Italy. In this note we reconstruct a crucial period of Bruno Touschek's life so far scarcely explored, which runs from Summer 1945 to the end of 1946. We shall describe his university studies in G\"ottingen, placing them in the context of the reconstruction of German science after 1945. The influence of Werner Heisenberg and other prominent German physicists will be highlighted. In parallel, we shall show how the decisions of the Allied powers towards restructuring science and technology in the UK after the war effort, determined Touschek's move to the University of Glasgow in 1947.
Intracranial Hemorrhage Segmentation Using Deep Convolutional Model

Traumatic brain injuries may cause intracranial hemorrhages (ICH). ICH could lead to disability or death if it is not accurately diagnosed and treated in a time-sensitive procedure. The current clinical protocol to diagnose ICH is examining Computerized Tomography (CT) scans by radiologists to detect ICH and localize its regions. However, this process relies heavily on the availability of an experienced radiologist. In this paper, we designed a study protocol to collect a dataset of 82 CT scans of subjects with a traumatic brain injury. Next, the ICH regions were manually delineated in each slice by a consensus decision of two radiologists. The dataset is publicly available online at the PhysioNet repository for future analysis and comparisons. In addition to publishing the dataset, which is the main purpose of this manuscript, we implemented a deep Fully Convolutional Networks (FCNs), known as U-Net, to segment the ICH regions from the CT scans in a fully-automated manner. The method as a proof of concept achieved a Dice coefficient of 0.31 for the ICH segmentation based on 5-fold cross-validation.
Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation

Neural conditional text generation systems have achieved significant progress in recent years, showing the ability to produce highly fluent text. However, the inherent lack of controllability in these systems allows them to hallucinate factually incorrect phrases that are unfaithful to the source, making them often unsuitable for many real world systems that require high degrees of precision. In this work, we propose a novel confidence oriented decoder that assigns a confidence score to each target position. This score is learned in training using a variational Bayes objective, and can be leveraged at inference time using a calibration technique to promote more faithful generation. Experiments on a structured data-to-text dataset -- WikiBio -- show that our approach is more faithful to the source than existing state-of-the-art approaches, according to both automatic metrics and human evaluation.
Exploration via Sample-Efficient Subgoal Design

The problem of exploration in unknown environments continues to pose a challenge for reinforcement learning algorithms, as interactions with the environment are usually expensive or limited. The technique of setting subgoals with an intrinsic shaped reward allows for the use of supplemental feedback to aid an agent in environment with sparse and delayed rewards. In fact, it can be an effective tool in directing the exploration behavior of the agent toward useful parts of the state space. In this paper, we consider problems where an agent faces an unknown task in the future and is given prior opportunities to "practice" on related tasks where the interactions are still expensive. We propose a one-step Bayes-optimal algorithm for selecting subgoal designs, along with the number of episodes and the episode length, to efficiently maximize the expected performance of an agent. We demonstrate its excellent performance on a variety of tasks and also prove an asymptotic optimality guarantee.
Communication Efficient Decentralized Training with Multiple Local Updates

Decentralized optimization has been demonstrated to be very useful in machine learning. This work studies the communication-efficiency issue in decentralized optimization. We analyze the Periodic Decentralized Stochastic Gradient Descent (PD-SGD) algorithm, a straightforward combination of federated averaging and decentralized SGD. For the setting of for non-convex objective and non-identically distributed data, we prove that PD-SGD converges to a critical point. In particular, the number of local SGDs trades off communication and local computation. From an algorithmic perspective, we analyze a novel version of PD-SGD, which alternates between multiple local updates and multiple decentralized SGDs. We also show that when we periodically shrink the length of local updates, this generalized PD-SGD can better balance the communication-convergence trade-off both theoretically and empirically.
Aggregated Gradient Langevin Dynamics

In this paper, we explore a general Aggregated Gradient Langevin Dynamics framework (AGLD) for the Markov Chain Monte Carlo (MCMC) sampling. We investigate the nonasymptotic convergence of AGLD with a unified analysis for different data accessing (e.g. random access, cyclic access and random reshuffle) and snapshot updating strategies, under convex and nonconvex settings respectively. It is the first time that bounds for I/O friendly strategies such as cyclic access and random reshuffle have been established in the MCMC literature. The theoretic results also indicate that methods in AGLD possess the merits of both the low per-iteration computational complexity and the short mixture time. Empirical studies demonstrate that our framework allows to derive novel schemes to generate high-quality samples for large-scale Bayesian posterior learning tasks.
Boosting Mapping Functionality of Neural Networks via Latent Feature Generation based on Reversible Learning

This paper addresses a boosting method for mapping functionality of neural networks in visual recognition such as image classification and face recognition. We present reversible learning for generating and learning latent features using the network itself. By generating latent features corresponding to hard samples and applying the generated features in a training stage, reversible learning can improve a mapping functionality without additional data augmentation or handling the bias of dataset. We demonstrate an efficiency of the proposed method on the MNIST,Cifar-10/100, and Extremely Biased and poorly categorized dataset (EBPC dataset). The experimental results show that the proposed method can outperform existing state-of-the-art methods in visual recognition. Extensive analysis shows that our method can efficiently improve the mapping capability of a network.
Domain-agnostic Question-Answering with Adversarial Training

Adapting models to new domain without finetuning is a challenging problem in deep learning. In this paper, we utilize an adversarial training framework for domain generalization in Question Answering (QA) task. Our model consists of a conventional QA model and a discriminator. The training is performed in the adversarial manner, where the two models constantly compete, so that QA model can learn domain-invariant features. We apply this approach in MRQA Shared Task 2019 and show better performance compared to the baseline model.
Ordering-Based Causal Structure Learning in the Presence of Latent Variables

We consider the task of learning a causal graph in the presence of latent confounders given i.i. d. ~samples from the model. While current algorithms for causal structure discovery in the presence of latent confounders are constraint-based, we here propose a score-based approach. We prove that under assumptions weaker than faithfulness, any sparsest independence map (IMAP) of the distribution belongs to the Markov equivalence class of the true model. This motivates the \emph{Sparsest Poset} formulation - that posets can be mapped to minimal IMAPs of the true model such that the sparsest of these IMAPs is Markov equivalent to the true model. Motivated by this result, we propose a greedy algorithm over the space of posets for causal structure discovery in the presence of latent confounders and compare its performance to the current state-of-the-art algorithms FCI and FCI+ on synthetic data.
Generative Hierarchical Models for Parts, Objects, and Scenes

Compositional structures between parts and objects are inherent in natural scenes. Modeling such compositional hierarchies via unsupervised learning can bring various benefits such as interpretability and transferability, which are important in many downstream tasks. In this paper, we propose the first deep latent variable model, called RICH, for learning Representation of Interpretable Compositional Hierarchies. At the core of RICH is a latent scene graph representation that organizes the entities of a scene into a tree structure according to their compositional relationships. During inference, taking top-down approach, RICH is able to use higher-level representation to guide lower-level decomposition. This avoids the difficult problem of routing between parts and objects that is faced by bottom-up approaches. In experiments on images containing multiple objects with different part compositions, we demonstrate that RICH is able to learn the latent compositional hierarchy and generate imaginary scenes.
Context-Driven Data Mining through Bias Removal and Data Incompleteness Mitigation

The results of data mining endeavors are majorly driven by data quality. Throughout these deployments, serious show-stopper problems are still unresolved, such as: data collection ambiguities, data imbalance, hidden biases in data, the lack of domain information, and data incompleteness. This paper is based on the premise that context can aid in mitigating these issues. In a traditional data science lifecycle, context is not considered. Context-driven Data Science Lifecycle (C-DSL); the main contribution of this paper, is developed to address these challenges. Two case studies (using data-sets from sports events) are developed to test C-DSL. Results from both case studies are evaluated using common data mining metrics such as: coefficient of determination (R2 value) and confusion matrices. The work presented in this paper aims to re-define the lifecycle and introduce tangible improvements to its outcomes.
A ν-support vector quantile regression model with automatic accuracy control

Quantile regression models have become popular among researchers these days. These models are being used frequently for obtaining the probabilistic forecast in different real-world applications. The Support Vector Quantile Regression (SVQR) model can obtain the conditional quantile estimate using kernel function in a non-parametric framework. The ϵ-SVQR model successfully incorporates the concept of the asymmetric ϵ-insensitive tube in the SVQR model and enables it to obtain a sparse and accurate solution. But, it requires a good choice of the user-defined parameter. A bad choice of ϵ value may result in poor predictions in the ϵ-SVQR model. In this paper, we propose a novel ‘ν-Support Vector Quantile Regression’ (ν-SVQR) model for quantile estimation. It can efficiently obtain a suitable asymmetric ϵ-insensitive zone according to the variance present in the data. The proposed ν-SVQR model uses the ν fraction of training data points for the estimation of the quantiles. In the ν-SVQR model, training points asymptotically appear above and below the asymmetric ϵ-insensitive tube in the ratio of 1 − τ and τ. Apart from these, there are other interesting properties of the proposed ν-SVQR model, which we have briefly described in this paper. These properties have been empirically verified using simulated and real-world data sets also.
Minimal automaton for multiplying and translating the Thue-Morse set

The Thue-Morse set $\mathcal{T}$ is the set of those non-negative integers whose binary expansions have an even number of $1$'s. The name of this set comes from the fact that its characteristic sequence is given by the famous Thue-Morse word $${\tt 0110100110010110\cdots},$$ which is the fixed point starting with ${\tt 0}$ of the word morphism ${\tt 0\mapsto 01}$, ${\tt 1\mapsto 10}$. The numbers in $\mathcal{T}$ are commonly called the evil numbers. We obtain an exact formula for the state complexity of the set $m\mathcal{T}+r$ (i.e. the number of states of its minimal automaton) with respect to any base $b$ which is a power of $2$. Our proof is constructive and we are able to explicitly provide the minimal automaton of the language of all $2^p$-expansions of the set of integers $m\mathcal{T}+r$ for any positive integers $p$ and $m$ and any remainder $r\in\{0,\ldots,m{-}1\}$. The proposed method is general for any $b$-recognizable set of integers.
Detecting muscle activation using ultrasound speed of sound inversion with deep learning

Functional muscle imaging is essential for diagnostics of a multitude of musculoskeletal afflictions such as degenerative muscle diseases, muscle injuries, muscle atrophy, and neurological related issues such as spasticity. However, there is currently no solution, imaging or otherwise, capable of providing a map of active muscles over a large field of view in dynamic scenarios.In this work, we look at the feasibility of applying longitudinal sound speed measurements to the task of dynamic muscle imaging of contraction or activation. We perform the assessment using a deep learning network applied to prebeamformed ultrasound channel data for sound speed inversion.Preliminary results show that dynamic muscle contraction can be detected in the calf and that this contraction can be positively assigned to the operating muscles. Potential frame rates in the hundreds to thousands of frames per second are necessary to accomplish this.
Vancouver Sun Run In Training clinics : an ordinal severity outcome measure and model of associated risk factors for running related pain

This study determined the feasibility of implementing a severity outcome measure for the classification of running related pain, in addition to constructing a multivariate model of associated risk factors for injury in novice runners. Throughout the Greater Vancouver Metropolitan Area, 421 participants responded in 29 separate In Training clinics. Subjects, on average, self-selected their running experience as limited with average physical fitness, average competitive motive and at least vestigial symptoms from a previous injury. Participants in this 13-week training program were surveyed during the 6 and 12 week of the In Training clinics' 13-week duration. Questionnaire items included a VAS scale for indication of degree of rehabilitation from prior injury, physical fitness, competitive motive, weekly distance, running frequency and running experience. The Training Function Score is designed to quantify, using an ordinal scale, function limitations that result from running injuries. Subjects also indicated whether they were "currently experiencing an injury as a result of injury". Overall, 36.1% of subjects experienced an injury during the training program. Univariate regression showed degree of rehabilitation, physical fitness, competitive motive and weekly distance as significantly associated with injury. Multivariate results, however, indicate only degree of rehabilitation while physical fitness accounts for 25.4% of the explained variation in the severity outcome measure. Pratt analysis further shows that 90.1% of that explained variation is accounted for by degree of rehabilitation only. Initial psychometric characteristics demonstrate outcome measured ability to discriminate injured from non-injured (uninjured = 82.91, injured = 64.56 and clinically injured = 46.29*; significant difference p<.001) and evaluate severity (significant (p<.011), regression association with severity outcome measure and increasing subjective definition of injury scale). Moderately sedentary individuals commencing a running program for the first time should be cautious of the affect prior history of injury has toward re-injury. This study suggests that an outcome measure for generalized running injuries is possible with respect to discriminating injured from non-injured runners, as well as evaluating the severity of those injuries. While such an outcome measure for general running injuries is feasible, proper validation procedures and methodology must be established before such a scale is used formally.
Investigation on young radio AGNs based on SDSS spectroscopy

The gigahertz peaked spectrum (GPS) sources, compact steep spectrum (CSS) radio sources, and high-frequency peaker (HFP) radio sources are thought to be young radio active galactic nuclei (AGNs) at the early stage of AGN evolution. We investigated the optical properties of the largest sample of 126 young radio AGNs based on the spectra in SDSS DR12. We find that the black hole masses MBH range from 107.32 to 109.84$\rm M_{\odot }$ and the Eddington ratios Redd vary from 10−4.93 to 100.37, suggesting that young radio AGNs have various accretion activities and not all are accreting at high accretion rate. Our young radio sources generally follow the evolutionary trend towards large-scale radio galaxies with increasing linear size and decreasing accretion rate in the radio power–linear size diagram. The radio properties of low-luminosity young radio AGNs with low Redd are discussed. The line width of [O iii] λ5007 core (σ[O iii]) is found to be a good surrogate of stellar velocity dispersion σ*. The radio luminosity $L_{\rm 5\, GHz}$ correlates strongly with [O iii] core luminosity L[O iii], suggesting that radio activity and accretion are closely related in young radio sources. We find one object that can be defined as a narrow-line Seyfert 1 galaxy, representing a population of young AGNs with both young jet and early accretion activity. The optical variabilities of 15 quasars with multi-epoch spectroscopy were investigated. Our results show that the optical variability in young AGN quasars presents low variations ($\leqslant \! 60{{\ \rm per\ cent}}$) similar to the normal radio-quiet quasars.
S4NN: temporal backpropagation for spiking neural networks with one spike per neuron

We propose a new supervised learning rule for multilayer spiking neural networks (SNNs) that use a form of temporal coding known as rank-order-coding. With this coding scheme, all neurons fire exactly one spike per stimulus, but the firing order carries information. In particular, in the readout layer, the first neuron to fire determines the class of the stimulus. We derive a new learning rule for this sort of network, named S4NN, akin to traditional error backpropagation, yet based on latencies. We show how approximated error gradients can be computed backward in a feedforward network with any number of layers. This approach reaches state-of-the-art performance with supervised multi-fully connected layer SNNs: test accuracy of 97.4% for the MNIST dataset, and 99.2% for the Caltech Face/Motorbike dataset. Yet, the neuron model that we use, nonleaky integrate-and-fire, is much simpler than the one used in all previous works. The source codes of the proposed S4NN are publicly available at https://github.com/SRKH/S4NN.
Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements

Physics-informed neural networks (PINNs) were recently proposed in [18] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution, while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a non-linear advection–diffusion–reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimise the hyper-parameters of sPINN by using the Bayesian optimisation method (meta-learning) and compare the results with the empirically selected hyper-parameters of sPINN. In particular, for the first part in solving the inverse deterministic ADR, we assume that we only have a few high-fidelity measurements, whereas the rest of the data is of lower fidelity. Hence, the PINN is trained using a composite multi-fidelity network, first introduced in [12], that learns the correlations between the multi-fidelity data and predicts the unknown values of diffusivity, transport velocity and two reaction constants as well as the concentration field. For the stochastic ADR, we employ a Karhunen–Loève (KL) expansion to represent the stochastic diffusivity, and arbitrary polynomial chaos (aPC) to represent the stochastic solution. Correspondingly, we design multiple NNs to represent the mean of the solution and learn each aPC mode separately, whereas we employ a separate NN to represent the mean of diffusivity and another NN to learn all modes of the KL expansion. For the inverse problem, in addition to stochastic diffusivity and concentration fields, we also aim to obtain the (unknown) deterministic values of transport velocity and reaction constants. The available data correspond to 7spatial points for the diffusivity and 20 space–time points for the solution, both sampled 2000 times. We obtain good accuracy for the deterministic parameters of the order of 1–2% and excellent accuracy for the mean and variance of the stochastic fields, better than three digits of accuracy. In the second part, we consider the previous stochastic inverse problem, and we use Bayesian optimisation to find five hyper-parameters of sPINN, namely the width, depth and learning rate of two NNs for learning the modes. We obtain much deeper and wider optimal NNs compared to the manual tuning, leading to even better accuracy, i.e., errors less than 1% for the deterministic values, and about an order of magnitude less for the stochastic fields.
BALQSO spectra explained by shock disruption of galactic clouds

Blue-shifted broad absorption lines (BALs) detected in quasar’s spectra are indicative of active galactic nuclei (AGN) outflows. We show, using 2D hydrodynamical simulations, that disruption of interstellar clouds by a fast AGN wind can lead to formation of cold, dense high-speed blobs that give rise to broad absorption features in the transmission spectrum of the AGN continuum source. For a wind velocity of 0.1c and sufficiently high cloud density (nc ≳ 104 cm−3; depending on size), absorption troughs with velocities up to about 3000 km s−1can be produced. For slower winds and/or lower cloud density, the anticipated velocity of the absorbing clouds should be smaller.