Miracle-dz/UniFAR-FTUs huggingface dataset:
Improved upper limits on the 21 cm signal power spectrum of neutral hydrogen at z ≈ 9.1 from LOFAR

A new upper limit on the 21 cm signal power spectrum at a redshift of z ≈ 9.1 is presented, based on 141 h of data obtained with the Low-Frequency Array (LOFAR). The analysis includes significant improvements in spectrally smooth gain-calibration, Gaussian Process Regression (GPR) foreground mitigation and optimally weighted power spectrum inference. Previously seen ‘excess power’ due to spectral structure in the gain solutions has markedly reduced but some excess power still remains with a spectral correlation distinct from thermal noise. This excess has a spectral coherence scale of 0.25–0.45 MHz and is partially correlated between nights, especially in the foreground wedge region. The correlation is stronger between nights covering similar local sidereal times. A best 2-σ upper limit of $\Delta ^2_{21} \lt (73)^2\, \mathrm{mK^2}$ at $k = 0.075\, \mathrm{h\, cMpc^{-1}}$ is found, an improvement by a factor ≈8 in power compared to the previously reported upper limit. The remaining excess power could be due to residual foreground emission from sources or diffuse emission far away from the phase centre, polarization leakage, chromatic calibration errors, ionosphere, or low-level radiofrequency interference. We discuss future improvements to the signal processing chain that can further reduce or even eliminate these causes of excess power.
Learning Parities with Neural Networks

In recent years we see a rapidly growing line of research which shows learnability of various models via common neural network algorithms. Yet, besides a very few outliers, these results show learnability of models that can be learned using linear methods. Namely, such results show that learning neural-networks with gradient-descent is competitive with learning a linear classifier on top of a data-independent representation of the examples. This leaves much to be desired, as neural networks are far more successful than linear methods. Furthermore, on the more conceptual level, linear models don't seem to capture the "deepness" of deep networks. In this paper we make a step towards showing leanability of models that are inherently non-linear. We show that under certain distributions, sparse parities are learnable via gradient decent on depth-two network. On the other hand, under the same distributions, these parities cannot be learned efficiently by linear methods.
Network Theoretic Analysis of Maximum a Posteriori Detectors for Sensor Analysis and Design

In this paper we characterize the performance of a class of maximum-a-posteriori (MAP) detectors for network systems driven by unknown stochastic inputs, as a function of the location of the sensors and the topology of the network. We consider two scenarios: one in which the changes occurs in the mean of the input, and the other where the changes are allowed to happen in the covariance (or power) of the input. In both the scenarios, to detect the changes, we associate suitable MAP detectors for a given set of sensors, and study its detection performance as function of the network topology, and the graphical distance between the input nodes and the sensors location. When the input and measurement noise follow a Gaussian distribution, we show that, as the number of measurements goes to infinity, the detectors' performance can be studied using the input to output gain of the transfer function of the network system. Using this characterization, we derive conditions under which the detection performance obtained when the sensors are located on a network cut is not worse (resp. not better) than the performance obtained by measuring all nodes of the subnetwork induced by the cut and not containing the input nodes. Our results provide structural insights into the sensor placement from a detection-theoretic viewpoint. Finally, we illustrate our findings via numerical examples.
A Lagrangian Approach to Information Propagation in Graph Neural Networks

In many real world applications, data are characterized by a complex structure, that can be naturally encoded as a graph. In the last years, the popularity of deep learning techniques has renewed the interest in neural models able to process complex patterns. In particular, inspired by the Graph Neural Network (GNN) model, different architectures have been proposed to extend the original GNN scheme. GNNs exploit a set of state variables, each assigned to a graph node, and a diffusion mechanism of the states among neighbor nodes, to implement an iterative procedure to compute the fixed point of the (learnable) state transition function. In this paper, we propose a novel approach to the state computation and the learning algorithm for GNNs, based on a constraint optimisation task solved in the Lagrangian framework. The state convergence procedure is implicitly expressed by the constraint satisfaction mechanism and does not require a separate iterative phase for each epoch of the learning procedure. In fact, the computational structure is based on the search for saddle points of the Lagrangian in the adjoint space composed of weights, neural outputs (node states), and Lagrange multipliers. The proposed approach is compared experimentally with other popular models for processing graphs.
Motion Deblurring using Spatiotemporal Phase Aperture Coding

Motion blur is a known issue in photography, as it limits the exposure time while capturing moving objects. Extensive research has been carried to compensate for it. In this work, a computational imaging approach for motion deblurring is proposed and demonstrated. Using dynamic phase-coding in the lens aperture during the image acquisition, the trajectory of the motion is encoded in an intermediate optical image. This encoding embeds both the motion direction and extent by coloring the spatial blur of each object. The color cues serve as prior information for a blind deblurring process, implemented using a convolutional neural network (CNN) trained to utilize such coding for image restoration. We demonstrate the advantage of the proposed approach over blind-deblurring with no coding and other solutions that use coded acquisition, both in simulation and real-world experiments.
Double/Debiased Machine Learning for Dynamic Treatment Effects

We consider the estimation of treatment effects in settings when multiple treatments are assigned over time and treatments can have a causal effect on future outcomes. We formulate the problem as a linear state space Markov process with a high dimensional state and propose an extension of the double/debiased machine learning framework to estimate the dynamic effects of treatments. Our method allows the use of arbitrary machine learning methods to control for the high dimensional state, subject to a mean square error guarantee, while still allowing parametric estimation and construction of confidence intervals for the dynamic treatment effect parameters of interest. Our method is based on a sequential regression peeling process, which we show can be equivalently interpreted as a Neyman orthogonal moment estimator. This allows us to show root-n asymptotic normality of the estimated causal effects.
V4D: 4D Convolutional Neural Networks for Video-level Representation Learning

Most existing 3D CNNs for video representation learning are clip-based methods, and thus do not consider video-level temporal evolution of spatio-temporal features. In this paper, we propose Video-level 4D Convolutional Neural Networks, referred as V4D, to model the evolution of long-range spatio-temporal representation with 4D convolutions, and at the same time, to preserve strong 3D spatio-temporal representation with residual connections. Specifically, we design a new 4D residual block able to capture inter-clip interactions, which could enhance the representation power of the original clip-level 3D CNNs. The 4D residual blocks can be easily integrated into the existing 3D CNNs to perform long-range modeling hierarchically. We further introduce the training and inference methods for the proposed V4D. Extensive experiments are conducted on three video recognition benchmarks, where V4D achieves excellent results, surpassing recent 3D CNNs by a large margin.
ELM-Based Frame Synchronization in Burst-Mode Communication Systems With Nonlinear Distortion

In burst-mode communication systems, the quality of frame synchronization (FS) at receivers significantly impacts the overall system performance. To guarantee FS, an extreme learning machine (ELM)-based synchronization method is proposed to overcome the nonlinear distortion caused by nonlinear devices or blocks. In the proposed method, a preprocessing is first performed to capture the coarse features of synchronization metric (SM) by using empirical knowledge. Then, an ELM-based FS network is employed to reduce system’s nonlinear distortion and improve SMs. Experimental results indicate that, compared with existing methods, our approach could significantly reduce the error probability of FS while improve the performance in terms of robustness and generalization.
Automated Cardiothoracic Ratio Calculation and Cardiomegaly Detection using Deep Learning Approach

We propose an algorithm for calculating the cardiothoracic ratio (CTR) from chest X-ray films. Our approach applies a deep learning model based on U-Net with VGG16 encoder to extract lung and heart masks from chest X-ray images and calculate CTR from the extents of obtained masks. Human radiologists evaluated our CTR measurements, and $76.5\%$ were accepted to be included in medical reports without any need for adjustment. This result translates to a large amount of time and labor saved for radiologists using our automated tools.
The ALMA Spectroscopic Survey in the HUDF: Deep 1.2 mm Continuum Number Counts

We present the results from the 1.2 mm continuum image obtained as part of the Atacama Large Millimeter/submillimeter Array Spectroscopic Survey in the Hubble Ultra Deep Field. The 1.2 mm continuum image has a size of 2.9 (4.2) arcmin2 within a primary beam response of 50% (10%) and an rms value of . We detect 35 sources at high significance (Fidelity ≥0.5); 32 have well-characterized near-infrared Hubble Space Telescope counterparts. We estimate the 1.2 mm number counts to flux levels of in two different ways: we first use the detected sources to constrain the number counts and find a significant flattening of the counts below Sν ∼ 0.1 mJy. In a second approach, we constrain the number counts using a probability of deflection statistics (P(D)) analysis. For this latter approach, we describe new methods to accurately measure the noise in interferometric imaging (employing jackknifing in the cube and in the visibility plane). This independent measurement confirms the flattening of the number counts. Our analysis of the differential number counts shows that we are detecting ∼93% (∼100% if we include the lower fidelity detections) of the total continuum dust emission associated with galaxies in the Hubble Ultra Deep Field. The ancillary data allow us to study the dependence of the 1.2 mm number counts on redshift (z = 0−4), galaxy dust mass ( ), stellar mass ( ), and star formation rate ( ). In an accompanying paper we show that the number counts are crucial to constrain galaxy evolution models and the understanding of star-forming galaxies at high redshift.
A Survey of Deep Learning Techniques for Neural Machine Translation

In recent years, natural language processing (NLP) has got great development with deep learning techniques. In the sub-field of machine translation, a new approach named Neural Machine Translation (NMT) has emerged and got massive attention from both academia and industry. However, with a significant number of researches proposed in the past several years, there is little work in investigating the development process of this new technology trend. This literature survey traces back the origin and principal development timeline of NMT, investigates the important branches, categorizes different research orientations, and discusses some future research trends in this field.
Conditional Self-Attention for Query-based Summarization

Self-attention mechanisms have achieved great success on a variety of NLP tasks due to its flexibility of capturing dependency between arbitrary positions in a sequence. For problems such as query-based summarization (Qsumm) and knowledge graph reasoning where each input sequence is associated with an extra query, explicitly modeling such conditional contextual dependencies can lead to a more accurate solution, which however cannot be captured by existing self-attention mechanisms. In this paper, we propose \textit{conditional self-attention} (CSA), a neural network module designed for conditional dependency modeling. CSA works by adjusting the pairwise attention between input tokens in a self-attention module with the matching score of the inputs to the given query. Thereby, the contextual dependencies modeled by CSA will be highly relevant to the query. We further studied variants of CSA defined by different types of attention. Experiments on Debatepedia and HotpotQA benchmark datasets show CSA consistently outperforms vanilla Transformer and previous models for the Qsumm problem.
A Resolution in Algorithmic Fairness: Calibrated Scores for Fair Classifications

Calibration and equal error rates are fundamental conditions for algorithmic fairness that have been shown to conflict with each other, suggesting that they cannot be satisfied simultaneously. This paper shows that the two are in fact compatible and presents a method for reconciling them. In particular, we derive necessary and sufficient conditions for the existence of calibrated scores that yield classifications achieving equal error rates. We then present an algorithm that searches for the most informative score subject to both calibration and minimal error rate disparity. Applied empirically to credit lending, our algorithm provides a solution that is more fair and profitable than a common alternative that omits sensitive features.
Pulsating campaigns of human prophylaxis driven by risk perception palliate oscillations of direct contact transmitted diseases

Human behavioral responses play an important role in the impact of disease outbreaks and yet they are often overlooked in epidemiological models. Understanding to what extent behavioral changes determine the outcome of spreading epidemics is essential to design effective intervention policies. Here we explore, analytically, the interplay between the personal decision to protect oneself from infection and the spreading of an epidemic. We do so by coupling a decision game based on the perceived risk of infection with a Susceptible-Infected-Susceptible model. Interestingly, we find that the simple decision on whether to protect oneself is enough to modify the course of the epidemics, by generating sustained steady oscillations in the prevalence. We deem these oscillations detrimental, and propose two intervention policies aimed at modifying behavioral patterns to help alleviate them. Surprisingly, we find that pulsating campaigns, compared to continuous ones, are more effective in diminishing such oscillations.
MuPix and ATLASPix -- Architectures and Results.

High Voltage Monolithic Active Pixel Sensors (HV-MAPS) are based on a commercial High Voltage CMOS process and collect charge by drift inside a reversely biased diode. HV-MAPS represent a promising technology for future pixel tracking detectors. Two recent developments are presented. The MuPix has a continuous readout and is being developed for the Mu3e experiment whereas the ATLASPix is being developed for LHC applications with a triggered readout. Both variants have a fully monolithic design including state machines, clock circuitries and serial drivers. Several prototypes and design variants were characterised in the lab and in testbeam campaigns to measure efficiencies, noise, time resolution and radiation tolerance. Results from recent MuPix and ATLASPix prototypes are presented and prospects for future improvements are discussed.
Black Hole Mass Function of Coalescing Neutron Star Black Hole Binary Systems: The Prospect of Reconstruction with the Gravitational Wave Observations

The discovery of gravitational waves from compact objects coalescence opens a brand-new window to observe the universe. With more events being detected in the future, statistical examinations would be essential to better understand the underlying astrophysical processes. In this work we investigate the prospect of measuring the mass function of black holes that are merging with the neutron stars. Applying Bayesian parameter estimation for hundreds of simulated neutron star–black hole (NSBH) mergers, we find that the parameters for most of the injected events can be well recovered. We also take a Bayesian hierarchical model to reconstruct the population properties of the masses of black holes, in the presence of a low mass gap, both the mass gap and power-law index (α) of black hole mass function can be well measured, thus we can reveal where the α is different for binary black hole (BBH) and NSBH systems. In the absence of a low mass gap, the gravitational wave data as well as the electromagnetic data can be used to pin down the nature of the merger event and then measure the mass of these very light black holes. However, as a result of the misclassification of BBH into NSBH, the measurement of α is more challenging and further dedicated efforts are needed.
WISE mid-infrared properties of compact active galactic nuclei selected from the high radio frequency AT20G survey

Past studies of compact active galactic nuclei (AGNs), the dominant population at high radio frequencies, selected them using flat spectral index criteria. This biases the sample due to the steepening of AGN spectra at high radio frequencies. We improve upon this by selecting 3610 compact AGNs using their angular size information ($\lesssim$0.15 arcsec scale) from the Australia Telescope 20 GHz (AT20G) high-angular resolution catalogue. We cross-match these against the Wide-field Infrared Survey Explorer AllWISE catalogue and present a catalogue with 3300 (91 per cent) matches, 91 (3 per cent) rejects, and 219 (6 per cent) non-detections that are excellent high-redshift candidates. Of the matched compact AGNs, 92 per cent exhibit QSO mid-infrared colours (W1−W2 > 0.5). Therefore, our sample of high frequency compact sources has a very high rate of identification with mid-infrared QSOs. We find counterparts for 88 per cent of 387 compact steep-spectrum (CSS) sources in the AT20G survey, 82 ± 5 per cent of which exhibit QSO mid-infrared colours and have moderate redshifts (zmedian = 0.82), while those dominated by host galaxy colours in mid-infrared have lower redshifts (zmedian = 0.13). The latter classified into late- and early-type galaxies using their mid-infrared colours shows a majority (68 ± 4 per cent) have colours characteristic of late-type galaxies. Thus, we find that a larger fraction of these CSS sources are embedded in hosts with higher gas densities than average early-type galaxies. We compare mid-infrared colours of our AGNs against those reported for AGNs primarily selected using non-radio techniques. This shows that mid-infrared SED of high frequency selected compact radio AGN is comparatively less red, possibly due to contributions from their hosts.
Combining SchNet and SHARC: The SchNarc Machine Learning Approach for Excited-State Dynamics

In recent years, deep learning has become a part of our everyday life and is revolutionizing quantum chemistry as well. In this work, we show how deep learning can be used to advance the research field of photochemistry by learning all important properties—multiple energies, forces, and different couplings—for photodynamics simulations. We simplify such simulations substantially by (i) a phase-free training skipping costly preprocessing of raw quantum chemistry data; (ii) rotationally covariant nonadiabatic couplings, which can either be trained or (iii) alternatively be approximated from only ML potentials, their gradients, and Hessians; and (iv) incorporating spin–orbit couplings. As the deep-learning method, we employ SchNet with its automatically determined representation of molecular structures and extend it for multiple electronic states. In combination with the molecular dynamics program SHARC, our approach termed SchNarc is tested on two polyatomic molecules and paves the way toward efficient photodynamics simulations of complex systems.
Learning by Semantic Similarity Makes Abstractive Summarization Better

One of the obstacles of abstractive summarization is the presence of various potentially correct predictions. Widely used objective functions for supervised learning, such as cross-entropy loss, cannot handle alternative answers effectively. Rather, they act as a training noise. In this paper, we propose Semantic Similarity strategy that can consider semantic meanings of generated summaries while training. Our training objective includes maximizing semantic similarity score which is calculated by an additional layer that estimates semantic similarity between generated summary and reference summary. By leveraging pre-trained language models, our model achieves a new state-of-the-art performance, ROUGE-L score of 41.5 on CNN/DM dataset. To support automatic evaluation, we also conducted human evaluation and received higher scores relative to both baseline and reference summaries.
Theoretical investigation of the Humphreys–Davidson limit at high and low metallicity

Context. Current massive star evolution grids are not able to simultaneously reproduce the empirical upper luminosity limit of red supergiants, the Humphrey–Davidson (HD) limit, nor the blue-to-red (B/R) supergiant ratio at high and low metallicity. Although previous studies have shown that the treatment of convection and semi-convection plays a role in the post-main-sequence (MS) evolution to blue or red supergiants (RSGs), a unified treatment for all metallicities has not been achieved so far. Aims. We focus on developing a better understanding of what drives massive star evolution to blue and red supergiant phases, with the ultimate aim of reproducing the HD limit at varied metallicities. We discuss the consequences of classifying B and R in the B/R ratio and clarify what is required to quantify a relatable theoretical B/R ratio for comparison with observations. Methods. For solar, Large Magellanic Cloud (50% solar), and Small Magellanic Cloud (20% solar) metallicities, we develop eight grids of MESA models for the mass range 20–60 M⊙ to probe the effect of semi-convection and overshooting on the core helium-burning phase. We compare rotating and non-rotating models with efficient (αsemi = 100) and inefficient semi-convection (αsemi = 0.1), with high and low amounts of core overshooting (αov of 0.1 or 0.5). The red and blue supergiant evolutionary phases are investigated by comparing the fraction of core He-burning lifetimes spent in each phase for a range of masses and metallicities. Results. We find that the extension of the convective core by overshooting αov = 0.5 has an effect on the post-MS evolution that can disable semi-convection, leading to more RSGs, but a lack of BSGs. We therefore implement αov = 0.1, which switches on semi-convective mixing, but for standard αsemi = 1 would result in an HD limit that is higher than observed at low Z (Large and Small Magellanic Clouds). Therefore, we need to implement very efficient semi-convection of αsemi = 100, which reproduces the HD limit at log L/L⊙ ∼ 5.5 for the Magellanic Clouds while simultaneously reproducing the Galactic HD limit of log L/L⊙ ∼ 5.8 naturally. The effect of semi-convection is not active at high metallicities because the envelope structure is depleted by strong mass loss such that semi-convective regions could not form. Conclusions. Metallicity-dependent mass loss plays an indirect, yet decisive role in setting the HD limit as a function of Z. For a combination of efficient semi-convection and low overshooting with standard Ṁ(Z), we find a natural HD limit at all metallicities.