Miracle-dz/UniFAR-FTUs huggingface dataset:
A Framework for Reinforcement Learning and Planning

Sequential decision making, commonly formalized as Markov Decision Process optimization, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are planning and reinforcement learning. Both research fields largely have their own research communities. However, if both research fields solve the same problem, then we should be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying framework for reinforcement learning and planning (FRAP), which identifies the underlying dimensions on which any planning or learning algorithm has to decide. At the end of the paper, we compare - in a single table - a variety of well-known planning, model-free and model-based RL algorithms along the dimensions of our framework, illustrating the validity of the framework. Altogether, FRAP provides deeper insight into the algorithmic space of planning and reinforcement learning, and also suggests new approaches to integration of both fields.
Tensor Estimation With Structured Priors

We consider rank-one symmetric tensor estimation when the tensor is corrupted by gaussian noise and the spike forming the tensor is a structured signal coming from a generalized linear model. The latter is a mathematically tractable model of a non-trivial hidden lower-dimensional latent structure in a signal. We work in a large dimensional regime with fixed ratio of signal-to-latent space dimensions. Remarkably, in this asymptotic regime, the mutual information between the spike and the observations can be expressed as a finite-dimensional variational problem, and it is possible to deduce the minimum-mean-square-error from its solution. We discuss, on examples, properties of the phase transitions as a function of the signal-to-noise ratio. Typically, the critical signal-to-noise ratio decreases with increasing signal-to-latent space dimensions. We discuss the limit of vanishing ratio of signal-to-latent space dimensions and determine the limiting tensor estimation problem. We also point out similarities and differences with the case of matrices.
The Sci-hub Effect: Sci-hub downloads lead to more article citations

Citations are often used as a metric of the impact of scientific publications. Here, we examine how the number of downloads from Sci-hub as well as various characteristics of publications and their authors predicts future citations. Using data from 12 leading journals in economics, consumer research, neuroscience, and multidisciplinary research, we found that articles downloaded from Sci-hub were cited 1.72 times more than papers not downloaded from Sci-hub and that the number of downloads from Sci-hub was a robust predictor of future citations. Among other characteristics of publications, the number of figures in a manuscript consistently predicts its future citations. The results suggest that limited access to publications may limit some scientific research from achieving its full impact.
Calibrating the binary black hole population in nuclear star clusters through tidal disruption events

As the sensitivity of gravitational wave (GW) instruments improves and new networks start operating, hundreds of merging stellar-mass black holes (SBHs) and intermediate-mass black holes (IMBHs) are expected to be observed in the next few years. The origin and distribution of SBH and IMBH binaries in various dynamical environments is a fundamental scientific question in GW astronomy. In this paper we discuss the electromagnetic window provided by tidal disruption events (TDEs) of stars, into the assembly and merger of binary SBHs and IMBHs in nuclear star clusters (NSCs). We discuss how the host NSC mass and density and the slope of the black-hole mass function set the orbital properties and the masses of the binaries that undergo a TDE. For typical NSC properties, we predict a merger rate of $\sim 10^{-6}$--$10^{-7}\ {\rm yr}^{-1}$ per galaxy. The lightcurve of TDEs in NSCs could be interrupted and modulated by the companion black hole on the orbital period of the binary. The ejected mass associated with TDEs can produce optical transients of luminosity $\sim 10^{42}$--$10^{44}$ erg s$^{-1}$ over timescales of days to months. These should be readily detectable by optical transient surveys such as the Zwicky Transient Facility and LSST.
Streaming Transformer ASR with Blockwise Synchronous Inference

The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks in end-to-end (E2E) automatic speech recognition (ASR) systems. However, Transformer has a drawback in that the entire input sequence is required to compute self-attention. We have proposed a block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps encode not only local acoustic information but also global linguistic, channel, and speaker attributes. In this paper, we extend block processing towards an entire streaming E2E ASR system without additional training, by introducing a blockwise synchronous decoding process inspired by a neural transducer into the Transformer decoder. We further apply a knowledge distillation technique with which training of the streaming Transformer is guided by the ordinary batch Transformer model. Evaluations of the HKUST and AISHELL-1 Mandarin tasks and LibriSpeech English task show that our proposed streaming Transformer outperforms conventional online approaches including monotonic chunkwise attention (MoChA). We also confirm that the knowledge distillation technique improves the accuracy further. Our streaming ASR models achieve comparable/superior performance to the batch models and other streaming-based transformer methods in all the tasks.
Incremental Inference of Collective Graphical Models

We consider incremental inference problems from aggregate data for collective dynamics. In particular, we address the problem of estimating the aggregate marginals of a Markov chain from noisy aggregate observations in an incremental (online) fashion. We propose a sliding window Sinkhorn belief propagation (SW-SBP) algorithm that utilizes a sliding window filter of the most recent noisy aggregate observations along with encoded information from discarded observations. Our algorithm is built upon the recently proposed multi-marginal optimal transport based SBP algorithm that leverages standard belief propagation and Sinkhorn algorithm to solve inference problems from aggregate data. We demonstrate the performance of our algorithm on applications such as inferring population flow from aggregate observations.
Graph modelling approaches for motorway traffic flow prediction

Traffic flow prediction, particularly in areas that experience highly dynamic flows such as motorways, is a major issue faced in traffic management. Due to increasingly large volumes of data being generated every minute, deep learning methods have been used extensively in the latest years for both short and long term traffic flow prediction. However, such models, despite their efficiency, need large amounts of historical information to be provided, and they take a considerable amount of time and computing resources to train, validate and test. This paper presents two new spatial-temporal approaches for building accurate short-term predictions along a popular motorway in Sydney Australia, by making use of the graph structure of the motorway network (including exits and entries). Our proposed methods are proximity-based, and they use the most recent available traffic flow information of the upstream counting stations closest to a given target station. Where such information is not available they employ daily historical means instead. We show that for short-term predictions (less than 10 minutes into the future), our proposed graph-based approaches outperform state-of-the-art deep learning models, such as long-term short memory, convolutional neuronal networks or hybrid models.
Learning Optimal Distributionally Robust Individualized Treatment Rules

Abstract Recent development in the data-driven decision science has seen great advances in individualized decision making. Given data with individual covariates, treatment assignments and outcomes, policy makers best individualized treatment rule (ITR) that maximizes the expected outcome, known as the value function. Many existing methods assume that the training and testing distributions are the same. However, the estimated optimal ITR may have poor generalizability when the training and testing distributions are not identical. In this article, we consider the problem of finding an optimal ITR from a restricted ITR class where there are some unknown covariate changes between the training and testing distributions. We propose a novel distributionally robust ITR (DR-ITR) framework that maximizes the worst-case value function across the values under a set of underlying distributions that are “close” to the training distribution. The resulting DR-ITR can guarantee the performance among all such distributions reasonably well. We further propose a calibrating procedure that tunes the DR-ITR adaptively to a small amount of calibration data from a target population. In this way, the calibrated DR-ITR can be shown to enjoy better generalizability than the standard ITR based on our numerical studies. Supplementary materials for this article are available online.
Option pricing under a discrete-time Markov switching stochastic volatility with co-jump model

We consider option pricing using a discrete-time Markov switching stochastic volatility with co-jump model, which can capture asset price features such as leptokurtosis, skewness, volatility clustering, and varying mean-reversion speed of volatility. For pricing European options, we develop a computationally efficient method for obtaining the probability distribution of average integrated variance (AIV), which is key to option pricing under stochastic-volatility-type models. Building upon the efficiency of the European option pricing approach, we are able to price an American-style option, by converting its pricing into the pricing of a portfolio of European options. Our work also provides constructive guidance for analyzing derivatives based on variance, e.g., the variance swap. Numerical results indicate our methods can be implemented very efficiently and accurately.
Designing and Learning Trainable Priors with Non-Cooperative Games

We introduce a general framework for designing and learning neural networks whose forward passes can be interpreted as solving convex optimization problems, and whose architectures are derived from an optimization algorithm. We focus on non-cooperative convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end. The priors used in this presentation include variants of total variation, Laplacian regularization, sparse coding on learned dictionaries, and non-local self similarities. Our models are parameter efficient and fully interpretable, and our experiments demonstrate their effectiveness on a large diversity of tasks ranging from image denoising and compressed sensing for fMRI to dense stereo matching.
THEaiTRE: Artificial Intelligence to Write a Theatre Play

We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts. This paper reviews related work and drafts an approach we intend to follow. We plan to adopt generative neural language models and hierarchical generation approaches, supported by summarization and machine translation methods, and complemented with a human-in-the-loop approach.
Covariance-engaged Classification of Sets via Linear Programming

Set classification aims to classify a set of observations as a whole, as opposed to classifying individual observations separately. To formally understand the unfamiliar concept of binary set classification, we first investigate the optimal decision rule under the normal distribution, which utilizes the empirical covariance of the set to be classified. We show that the number of observations in the set plays a critical role in bounding the Bayes risk. Under this framework, we further propose new methods of set classification. For the case where only a few parameters of the model drive the difference between two classes, we propose a computationally-efficient approach to parameter estimation using linear programming, leading to the Covariance-engaged LInear Programming Set (CLIPS) classifier. Its theoretical properties are investigated for both independent case and various (short-range and long-range dependent) time series structures among observations within each set. The convergence rates of estimation errors and risk of the CLIPS classifier are established to show that having multiple observations in a set leads to faster convergence rates, compared to the standard classification situation in which there is only one observation in the set. The applicable domains in which the CLIPS performs better than competitors are highlighted in a comprehensive simulation study. Finally, we illustrate the usefulness of the proposed methods in classification of real image data in histopathology.
Effect of Echinacea purpurea on growth and survival of guppy (Poecilia reticulata) challenged with Aeromonas bestiarum

The effects of Echinacea purpurea supplementation on growth performance, nutrient utilization, body composition and disease resistance were studied in the ornamental fish Poecilia reticulata. Five diets were used, each differing in Echinacea content (0 g kg−1 diet – control, 5 g kg−1 diet – group 1, 10 g kg−1 diet – group 2, 20 g kg−1 diet – group 3, 30 g kg−1 diet – group 4 and 60 g kg−1 diet – group 5), and the fish were fed twice daily at a rate of 30 g kg−1 body weight per day for 67 days. The gain in the body weight and the condition factor were significantly increased in groups 2-5, whereas specific growth rate and feed conversion ratio were significantly increased in the supplemented groups (P < 0.05). Cumulative mortalities after challenge infection with the fish pathogen Aeromonas bestiarum were the lowest in the groups supplemented with Echinacea. Log-rank tests showed significant differences between the supplemented groups 1, 2, 3, 4 and 5 and the control group (P = 0.0074, P = 0.0075, P = 0.00507, P = 0.00001 and P = 0.00001, respectively). The results of this study indicate that Echinacea improves body weight gain and resistance against challenge infection in fish.
Receptor Ligand: In Vitro Autoradiography and High-Resolution Small Animal PET

Adenosine modulates brain activity through 4 G protein– coupled receptors, primarily adenosine A1 receptors (A1ARs). A1ARs are heterogeneously distributed throughout the brain and participate in many physiologic processes—for example, the induction of sleep and feedback inhibition of excitatory neurotransmission. There is also evidence that A1ARs are involved in brain pathologies, including cerebral ischemia, epilepsy, and neurodegeneration. Therefore, measuring A1ARs in the living brain has been a long-standing goal. This report describes the preclinical evaluation of 18 F-8-cyclopentyl-3-(3-fluoropropyl)-1propylxanthine (18F-CPFPX), a novel A1AR PET ligand. Methods: CPFPX, a xanthine-based A1AR antagonist, was labeled with either 18 Fo r3H, maintaining identical chemical structures, and evaluated in rats as a putative radioligand for in vivo or in vitro imaging of brain A1ARs by quantitative receptor autoradiography and the combination of high-resolution small animal PET and MRI. Results: 3 H-CPFPX bound with nanomolar affinity (Kd, 4.4 nmol/L) to A1ARs and showed a distribution typical of cerebral A1ARs. In extensive in vitro competition studies, 3 HCPFPX proved to be a highly selective and specific A1AR radioligand. Neither the nonxanthine-type adenosine A2A receptor antagonist ZM 241385 nor multiple cholinergic, serotoninergic, and glutamatergic receptor compounds competed for 3HCPFPX below the micromolar level. In vivo animal PET and ex vivo autoradiographic experiments measured radioactivity in discrete brain regions after intravenous injection of 18F-CPFPX. 18 F-CPFPX had excellent in vivo stability and penetrated the blood– brain barrier immediately after injection due to its high lipophilicity. Brain uptake was rapid and particularly high in gray matter regions. Retention of 18F-CPFPX was highest in the cerebellum, thalamus, and neocortex with evidence of saturable binding. Low binding potentials were found in the midbrain. In vivo displacement PET experiments with the A1AR antagonist 8-cyclopentyl-1,3-dipropylxanthine showed a 72% 8% displacement of 18F-CPFPX. Conclusion: 18F-CPFPX is a highly selective and specific ligand for A1ARs and a suitable radioligand for noninvasive PET imaging of A1ARs in the living brain. These studies also support the application of high-resolution animal PET as an effective in vivo imaging tool in the evaluation process of new radioligands.
A modified Armitage test for more than a linear trend on proportions

The Armitage test for linear trend in proportions can be modified using the multiple marginal model approach for three regression models with arithmetic, ordinal and logarithmic dose scores simultaneously, to be powerful against a wide range of possible dose response relationships. Moreover, it can be used for particular designs in the generalized linear (mixed) model for the three common effect sizes odds ratio, risk ratio and risk difference. The related R package tukeytrend allows simple generalizations, e.g. the analysis 2-by-k table data with a possible plateau shape or analysing overdispersed proportions. The evaluation of further real data examples are available in a vignette to that R package.
Growth of Ceratium hirundinella in a subtropical Australian reservoir: the role of vertical migration

A study into the photophysiology, growth and migration of Ceratium hirundinella in Chaffey Reservoir in subtropical northern New South Wales, Australia, revealed that a proportion of cells formed subsurface accumulations at depths that optimized light intensity (212-552 µmol photons m -2 s -1 ) for photosynthesis and cell growth. At high incident irradiance, Ceratium migrated downwards from the near-surface waters, avoiding high-light-induced, slow-recovering non-photochemical quenching of photosystem II. Overnight deepening of the surface mixed layer by convective cooling produced homogeneous distributions of Ceratium with a significant proportion of the population below the depth where light saturation of photosynthesis occurred. Ceratium migrated towards the surface from suboptimal light intensities, at a velocity of 1.6-2.7 � 10 -4 m s -1 . Subsurface accumu- lations occurred under a variety of turbulence intensities; however, accumulation was significantly reduced when the turbulent velocity scale in the mixed layer was >5 � 10-3 m s-1, beyond which turbulent diffusion dominated advection by swimming. The formation of subsurface accumulations with increased computed water column integral photosynthesis by 35% compared to a uniform cell distribution.
Cataloguing the radio-sky with unsupervised machine learning: a new approach for the SKA era

We develop a new analysis approach towards identifying related radio components and their corresponding infrared host galaxy based on unsupervised machine learning methods. By exploiting PINK, a self-organising map algorithm, we are able to associate radio and infrared sources without the a priori requirement of training labels. We present an example of this method using $894,415$ images from the FIRST and WISE surveys centred towards positions described by the FIRST catalogue. We produce a set of catalogues that complement FIRST and describe 802,646 objects, including their radio components and their corresponding AllWISE infrared host galaxy. Using these data products we (i) demonstrate the ability to identify objects with rare and unique radio morphologies (e.g. 'X'-shaped galaxies, hybrid FR-I/FR-II morphologies), (ii) can identify the potentially resolved radio components that are associated with a single infrared host and (iii) introduce a "curliness" statistic to search for bent and disturbed radio morphologies, and (iv) extract a set of 17 giant radio galaxies between 700-1100 kpc. As we require no training labels, our method can be applied to any radio-continuum survey, provided a sufficiently representative SOM can be trained.
Differential response by males and females to manipulation of partner contribution in the great tit (Parus major)

1. In birds with bi-parental care, handicapping is often assumed to decrease the amount of parental care of the handicapped partner. We discuss how handicapping could alter the shape of the handicapped bird's survival-effort curve (theoretical curve relating the survival of a parent to its effort) and show that the optimal response could yield a decrease, no response or even an increase in effort of the handicapped bird. 2. Male or female great tits Parus major (L.) were handicapped during the nestling period by clipping a number of feathers in order to study the effects on parental care and body condition. 3. Handicapped males significantly decreased their feeding rates, while handicapped females did not. Condition of handicapped females significantly deteriorated, while condition of handicapped males did not change during the experiment. Females with a handicapped partner fully compensated for their partner's decrease in work rate, while males with a handicapped partner did not show any compensation and even tended to decrease their feeding rates. 4. Using an inverse optimality approach, we reconstructed the theoretical curve relating the survival of a parent to its effort on the basis of the experimental effects. The handicapped male's survival-effort curve appeared to be slightly steeper than that of handicapped females. This suggests that handicapped males suffer more from an increase in effort than handicapped females.
On the convergence of the Metropolis algorithm with fixed-order updates for multivariate binary probability distributions

The Metropolis algorithm is arguably the most fundamental Markov chain Monte Carlo (MCMC) method. But the algorithm is not guaranteed to converge to the desired distribution in the case of multivariate binary distributions (e.g., Ising models or stochastic neural networks such as Boltzmann machines) if the variables (sites or neurons) are updated in a fixed order, a setting commonly used in practice. The reason is that the corresponding Markov chain may not be irreducible. We propose a modified Metropolis transition operator that behaves almost always identically to the standard Metropolis operator and prove that it ensures irreducibility and convergence to the limiting distribution in the multivariate binary case with fixed-order updates. The result provides an explanation for the behaviour of Metropolis MCMC in that setting and closes a long-standing theoretical gap. We experimentally studied the standard and modified Metropolis operator for models were they actually behave differently. If the standard algorithm also converges, the modified operator exhibits similar (if not better) performance in terms of convergence speed.