Miracle-dz/UniFAR-FTUs huggingface dataset:
The Redshift Evolution of Ultraluminous X-Ray Sources out to z ∼ 0.5: Comparison with X-Ray Binary Populations and Contribution to the Cosmic X-Ray Background

Ultraluminous X-ray sources (ULXs) are thought to be powerful X-ray binaries (XRBs) and may contribute significantly to the redshift-dependent X-ray emission from star-forming galaxies. We have assembled a uniform sample of 259 ULXs over the redshift range z = 0.002–0.51 to constrain their physical nature and their contribution to the cosmic X-ray background. The sample is constructed by crossmatching galaxies from the Sloan Digital Sky Survey with the Chandra Source Catalog and selecting off-nuclear X-ray sources after applying astrometric corrections. The fraction of contaminants is ∼30% and shows no evolution with redshift. The host-galaxy star formation rates (SFRs) are systematically elevated relative to the parent sample when matched in host stellar mass. The specific SFRs suggest a slight preference for high-mass XRBs, and the X-ray luminosity scaling relations with host-galaxy stellar mass and SFR indicate that the highest-redshift sources represent relatively luminous XRB populations that dominate their host-galaxy X-ray emission. The fraction of galaxies hosting at least one ULX of a given luminosity increases with redshift over the full range of our sample, as expected if ULXs are preferentially found in galaxies with high SFRs and low metallicities. At z ∼ 0.5, the ULX X-ray flux is consistent with the X-ray emission from star-forming galaxies. Moreover, ULXs may account for up to ∼40% of the integrated flux from XRBs in the normal galaxy population out to z ∼ 0.5, suggesting they may contribute significantly to the overall ionizing radiation from galaxies.
Exploring Proxying QUIC and HTTP/3 for Satellite Communication

Low-Earth Orbit satellites have gained momentum to provide Internet connectivity, augmenting those in the long-established geostationary orbits. At the same time, QUIC has been developed as the new transport protocol for the web. While QUIC traffic is fully encrypted, intermediaries such as performance enhancing proxies (PEPs) - in the past essential for Internet over satellite performance - can no longer tamper with and optimize transport connections. In this paper, we present a satellite emulation testbed and use it to compare QUIC and TCP as well as HTTP/3 and HTTP/1.1 with and without minimal PEP functionality. Evaluating goodput over time, we find that the slow start threshold is reached up to 2s faster for QUIC PEP in comparison to QUIC Non-PEP. Moreover, we find that HTTP/3 and HTTP/3-PEP outperform HTTP/1.1 and HTTP/1.1-PEP in multiple web performance scenarios, where HTTP/3-PEP improves over HTTP/3 for Page Load Time by over 7s in edge cases. Hence, our findings hint that these performance gains may warrant exploring PEPs for QUIC.
Graviton partial waves and causality in higher dimensions

Do gravitational interactions respect the basic principles of relativity and quantum mechanics? We show that any graviton S-matrix that satisfies these assumptions cannot significantly differ from General Relativity at low energies. We provide sharp bounds on the size of potential corrections in terms of the mass M of new higher-spin states, in spacetime dimensions D>=5 where the S-matrix does not suffer from infrared ambiguities. The key novel ingredient is the full set of SO(D-1) partial waves for this process, which we show how to efficiently compute with Young tableau manipulations. We record new bounds on the central charges of holographic conformal theories.
Neural Combinatorial Optimization: a New Player in the Field

Neural Combinatorial Optimization attempts to learn good heuristics for solving a set of problems using Neural Network models and Reinforcement Learning. Recently, its good performance has encouraged many practitioners to develop neural architectures for a wide variety of combinatorial problems. However, the incorporation of such algorithms in the conventional optimization framework has raised many questions related to their performance and the experimental comparison with other methods such as exact algorithms, heuristics and metaheuristics. This paper presents a critical analysis on the incorporation of algorithms based on neural networks into the classical combinatorial optimization framework. Subsequently, a comprehensive study is carried out to analyse the fundamental aspects of such algorithms, including performance, transferability, computational cost and generalization to larger-sized instances. To that end, we select the Linear Ordering Problem as a case of study, an NP-hard problem, and develop a Neural Combinatorial Optimization model to optimize it. Finally, we discuss how the analysed aspects apply to a general learning framework, and suggest new directions for future work in the area of Neural Combinatorial Optimization algorithms.
OmniKnight: Multilingual Neural Machine Translation with Language-Specific Self-Distillation

Although all-in-one-model multilingual neural machine translation (MNMT) has achieved re-markable progress in the recent years, its selected best overall checkpoint fails to achieve the best performance simultaneously in all language pairs. It is because that the best checkpoints for each individual language pair ( i.e., language-speciﬁc best checkpoints ) scatter in different epochs. In this paper, we present a novel training strategy dubbed L anguage-S peciﬁc S elf- D istillation (LSSD) for bridging the gap between language-speciﬁc best checkpoints and the overall best checkpoint. In detail, we regard each language-speciﬁc best checkpoint as a teacher to distill the overall best checkpoint. Moreover, we system-atically explore three variants of our LSSD , which perform distillation statically, selec-tively, and adaptively. Experimental results on two widely-used benchmarks show that LSSD obtains consistent improvements towards all language pairs and achieves the state-of-the-art. 1
Training Mixed-Domain Translation Models via Federated Learning

Training mixed-domain translation models is a complex task that demands tailored architec- tures and costly data preparation techniques. In this work, we leverage federated learning (FL) in order to tackle the problem. Our investiga- tion demonstrates that with slight modifications in the training process, neural machine trans- lation (NMT) engines can be easily adapted when an FL-based aggregation is applied to fuse different domains. Experimental results also show that engines built via FL are able to perform on par with state-of-the-art baselines that rely on centralized training techniques.We evaluate our hypothesis in the presence of five datasets with different sizes, from different domains, to translate from German into English and discuss how FL and NMT can mutually benefit from each other. In addition to provid- ing benchmarking results on the union of FL and NMT, we also propose a novel technique to dynamically control the communication band- width by selecting impactful parameters during FL updates. This is a significant achievement considering the large size of NMT engines that need to be exchanged between FL parties.
Hidden behind the obvious: misleading keywords and implicitly abusive language on social media

While social media offers freedom of self-expression, abusive language carry significant negative social impact. Driven by the importance of the issue, research in the automated detection of abusive language has witnessed growth and improvement. However, these detection models display a reliance on strongly indicative keywords, such as slurs and profanity. This means that they can falsely (1a) miss abuse without such keywords or (1b) flag non-abuse with such keywords, and that (2) they perform poorly on unseen data. Despite the recognition of these problems, gaps and inconsistencies remain in the literature. In this study, we analyse the impact of keywords from dataset construction to model behaviour in detail, with a focus on how models make mistakes on (1a) and (1b), and how (1a) and (1b) interact with (2). Through the analysis, we provide suggestions for future research to address all three problems.
Heterogeneous Treatment Effects for Networks, Panels, and other Outcome Matrices

We are interested in the distribution of treatment effects for an experiment where units are randomized to a treatment but outcomes are measured for pairs of units. For example, we might measure risk sharing links between households enrolled in a microfinance program, employment relationships between workers and firms exposed to a trade shock, or bids from bidders to items assigned to an auction format. Such a double randomized experimental design may be appropriate when there are social interactions, market externalities, or other spillovers across units assigned to the same treatment. Or it may describe a natural or quasi experiment given to the researcher. In this paper, we propose a new empirical strategy that compares the eigenvalues of the outcome matrices associated with each treatment. Our proposal is based on a new matrix analog of the Fr\'echet-Hoeffding bounds that play a key role in the standard theory. We first use this result to bound the distribution of treatment effects. We then propose a new matrix analog of quantile treatment effects that is given by a difference in the eigenvalues. We call this analog spectral treatment effects.
Modeling and Correcting Bias in Sequential Evaluation

We consider the problem of sequential evaluation, in which an evaluator observes candidates in a sequence and assigns scores to these candidates in an online, irrevocable fashion. Sequential bias refers to dependencies between the evaluation outcome and the order in which the candidates appear, and extensive empirical studies have established its existence in many applications. Motivated by the psychology literature, we propose a natural model for the evaluator's rating process that captures the lack of calibration inherent to such a task. We conduct crowdsourcing experiments to demonstrate various facets of our model, propose a near-linear time, online algorithm for bias correction, and show that it is near-optimal in two canonical ranking metrics.
Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?

Several popular Transformer based language models have been found to be successful for text-driven brain encoding. However, existing literature leverages only pretrained text Transformer models and has not explored the efficacy of task-specific learned Transformer representations. In this work, we explore transfer learning from representations learned for ten popular natural language processing tasks (two syntactic and eight semantic) for predicting brain responses from two diverse datasets: Pereira (subjects reading sentences from paragraphs) and Narratives (subjects listening to the spoken stories). Encoding models based on task features are used to predict activity in different regions across the whole brain. Features from coreference resolution, NER, and shallow syntax parsing explain greater variance for the reading activity. On the other hand, for the listening activity, tasks such as paraphrase generation, summarization, and natural language inference show better encoding performance. Experiments across all 10 task representations provide the following cognitive insights: (i) language left hemisphere has higher predictive brain activity versus language right hemisphere, (ii) posterior medial cortex, temporo-parieto-occipital junction, dorsal frontal lobe have higher correlation versus early auditory and auditory association cortex, (iii) syntactic and semantic tasks display a good predictive performance across brain regions for reading and listening stimuli resp.
Inducing and Using Alignments for Transition-based AMR Parsing

Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments. These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints. Parsers also train on a point-estimate of the alignment pipeline, neglecting the uncertainty due to the inherent ambiguity of alignment. In this work we explore two avenues for overcoming these limitations. First, we propose a neural aligner for AMR that learns node-to-word alignments without relying on complex pipelines. We subsequently explore a tighter integration of aligner and parser training by considering a distribution over oracle action sequences arising from aligner uncertainty. Empirical results show this approach leads to more accurate alignments and generalization better from the AMR2.0 to AMR3.0 corpora. We attain a new state-of-the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0.
A Falsificationist Account of Artificial Neural Networks

Machine learning operates at the intersection of statistics and computer science. This raises the question as to its underlying methodology. While much emphasis has been put on the close link between the process of learning from data and induction, the falsiﬁcationist component of machine learning has received minor attention. In this paper, we argue that the idea of falsiﬁcation is central to the methodology of machine learning. It is commonly thought that machine learning algorithms infer general prediction rules from past observations. This is akin to a statistical procedure by which estimates are obtained from a sample of data. But machine learning algorithms can also be described as choosing one prediction rule from an entire class of functions. In particular, the algorithm that determines the weights of an artiﬁcial neural network operates by empirical risk minimization and rejects prediction rules that lack empirical adequacy. It also exhibits a behavior of implicit regularization that pushes hypothesis choice toward simpler prediction rules. We argue that taking both aspects together gives rise to a falsiﬁcationist account of artiﬁcial neural networks.
Residual Graph Convolutional Recurrent Networks For Multi-step Traffic Flow Forecasting

Traﬃc ﬂow forecasting is essential for traﬃc planning, control and management. The main challenge of traﬃc forecasting tasks is accurately capturing traﬃc networks’ spatial and temporal correlation. Although there are many traﬃc forecasting methods, most of them still have limitations in capturing spatial and temporal correlations. To improve traﬃc forecasting accuracy, we propose a new Spatial-temporal forecasting model, namely the Residual Graph Convolutional Recurrent Network (RGCRN). The model uses our proposed Residual Graph Convolutional Network (ResGCN) to capture the ﬁne-grained spatial correlation of the traﬃc road network and then uses a Bi-directional Gated Recurrent Unit (BiGRU) to model time series with spatial information and obtains the temporal correlation by analysing the change in information transfer between the forward and reverse neurons of the time series data. Our comparative experimental results on two real datasets show that RGCRN improves on average by 20 . 66% compared to the best baseline model. You can get our source code and data through https://github.com/zhangshqii/RGCRN .
Model-Free Opponent Shaping

In general-sum games, the interaction of self-interested learning agents commonly leads to collectively worst-case outcomes, such as defect-defect in the iterated prisoner's dilemma (IPD). To overcome this, some methods, such as Learning with Opponent-Learning Awareness (LOLA), shape their opponents' learning process. However, these methods are myopic since only a small number of steps can be anticipated, are asymmetric since they treat other agents as naive learners, and require the use of higher-order derivatives, which are calculated through white-box access to an opponent's differentiable learning algorithm. To address these issues, we propose Model-Free Opponent Shaping (M-FOS). M-FOS learns in a meta-game in which each meta-step is an episode of the underlying inner game. The meta-state consists of the inner policies, and the meta-policy produces a new inner policy to be used in the next episode. M-FOS then uses generic model-free optimisation methods to learn meta-policies that accomplish long-horizon opponent shaping. Empirically, M-FOS near-optimally exploits naive learners and other, more sophisticated algorithms from the literature. For example, to the best of our knowledge, it is the first method to learn the well-known Zero-Determinant (ZD) extortion strategy in the IPD. In the same settings, M-FOS leads to socially optimal outcomes under meta-self-play. Finally, we show that M-FOS can be scaled to high-dimensional settings.
Semantic Diversity in Dialogue with Natural Language Inference

Generating diverse, interesting responses to chitchat conversations is a problem for neural conversational agents. This paper makes two substantial contributions to improving diversity in dialogue generation. First, we propose a novel metric which uses Natural Language Inference (NLI) to measure the semantic diversity of a set of model responses for a conversation. We evaluate this metric using an established framework (Tevet and Berant, 2021) and find strong evidence indicating NLI Diversity is correlated with semantic diversity. Specifically, we show that the contradiction relation is more useful than the neutral relation for measuring this diversity and that incorporating the NLI model’s confidence achieves state-of-the-art results. Second, we demonstrate how to iteratively improve the semantic diversity of a sampled set of responses via a new generation procedure called Diversity Threshold Generation, which results in an average 137% increase in NLI Diversity compared to standard generation procedures.
Predicting Issue Types with seBERT

Pre-trained transformer models are the current state-of-the-art for natural language models processing. seBERT is such a model, that was developed based on the BERT architecture, but trained from scratch with software engineering data. We fine-tuned this model for the NLBSE challenge for the task of issue type prediction. Our model dominates the baseline fastText for all three issue types in both recall and precision to achieve an overall F1-score of 85.7%, which is an increase of 4.1% over the baseline.
Unique Characterisability and Learnability of Temporal Instance Queries

We aim to determine which temporal instance queries can be uniquely characterised by a (polynomial-size) set of positive and negative temporal data examples. We start by considering queries formulated in fragments of propositional linear temporal logic LTL that correspond to conjunctive queries (CQs) or extensions thereof induced by the until operator. Not all of these queries admit polynomial characterisations, but by imposing a further restriction to path-shaped queries we identify natural classes that do. We then investigate how far the obtained characterisations can be lifted to temporal knowledge graphs queried by 2D languages combining LTL with concepts in description logics EL or ELI (i.e., tree-shaped CQs). While temporal operators in the scope of description logic constructors can destroy polynomial characterisability, we obtain general transfer results for the case when description logic constructors are within the scope of temporal operators. Finally, we apply our characterisations to establish (polynomial) learnability of temporal instance queries using membership queries in the active learning framework.
An Efficient and Simplified Model for Forecasting using SRM

Learning form continuous financial systems play a vital role in enterprise operations. One of the most sophisticated non-parametric supervised learning classifiers, SVM (Support Vector Machines), provides robust and accurate results, however it may require intense computation and other resources. The heart of SLT (Statistical Learning Theory), SRM (Structural Risk Minimization) Principle can also be used for model selection. In this paper, we focus on comparing the performance of model estimation using SRM with SVR (Support Vector Regression) for forecasting the retail sales of consumer products. The potential benefits of an accurate sales forecasting technique in businesses are immense. Retail sales forecasting is an integral part of strategic business planning in areas such as sales planning, marketing research, pricing, production planning and scheduling. Performance comparison of support vector regression with model selection using SRM shows comparable results to SVR but in a computationally efficient manner. This research targeted the real life data to conclude the results after investigating the computer generated datasets for different types of model building.
Cost-Aware Comparison of LiDAR-based 3D Object Detectors

Considerable research efforts have been devoted to LiDAR-based 3D object detection and its empirical performance has been significantly improved. While the progress has been encouraging, we observe an overlooked issue: it is not yet common practice to compare different 3D detectors under the same cost, e.g. , inference latency. This makes it difficult to quantify the true performance gain brought by recently proposed architecture designs. The goal of this work is to conduct a fair comparison of LiDAR-based 3D object detectors. Specifically, we focus on SECOND, a simple grid-based one-stage detector, and analyze its performance under different costs by scaling its original architecture. Then we compare the family of scaled SECOND with recent 3D detection methods, such as Voxel R-CNN and PV-RCNN++. The results are surprising. We find that, if allowed to use the same latency, SECOND can match the performance of PV-RCNN++, the current state-of-the-art method on the Waymo Open Dataset. Scaled SECOND also easily outperforms many recent 3D detection methods published during the past year. We recommend future research control the inference cost in their empirical comparison and include the family of scaled SECOND as a strong baseline when presenting novel 3D detection methods.