Miracle-dz/UniFAR-FTUs huggingface dataset:
MLP-Mixer: An all-MLP Architecture for Vision

Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this paper we show that while convolutions and attention are both sufficient for good performance, neither of them are necessary. We present MLP-Mixer, an architecture based exclusively on multi-layer perceptrons (MLPs). MLP-Mixer contains two types of layers: one with MLPs applied independently to image patches (i.e."mixing"the per-location features), and one with MLPs applied across patches (i.e."mixing"spatial information). When trained on large datasets, or with modern regularization schemes, MLP-Mixer attains competitive scores on image classification benchmarks, with pre-training and inference cost comparable to state-of-the-art models. We hope that these results spark further research beyond the realms of well established CNNs and Transformers.
Implicit Regularization in Deep Tensor Factorization

Attempts of studying implicit regularization associated to gradient descent (GD) have identified matrix completion as a suitable test-bed. Late findings suggest that this phenomenon cannot be phrased as a minimization-norm problem, implying that a paradigm shift is required and that dynamics has to be taken into account. In the present work we address the more general setup of tensor completion by leveraging two popularized tensor factorization, namely Tucker and TensorTrain (TT). We track relevant quantities such as tensor nuclear norm, effective rank, generalized singular values and we introduce deep Tucker and TT unconstrained factorization to deal with the completion task. Experiments on both synthetic and real data show that gradient descent promotes solution with low-rank, and validate the conjecture saying that the phenomenon has to be addressed from a dynamical perspective.
Supervised multi-specialist topic model with applications on large-scale electronic health record data

Motivation: Electronic health record (EHR) data provides a new venue to elucidate disease comorbidities and latent phenotypes for precision medicine. To fully exploit its potential, a realistic data generative process of the EHR data needs to be modelled. Materials and Methods: We present MixEHR-S to jointly infer specialist-disease topics from the EHR data. As the key contribution, we model the specialist assignments and ICD-coded diagnoses as the latent topics based on patient's underlying disease topic mixture in a novel unified supervised hierarchical Bayesian topic model. For efficient inference, we developed a closed-form collapsed variational inference algorithm to learn the model distributions of MixEHR-S. Results: We applied MixEHR-S to two independent large-scale EHR databases in Quebec with three targeted applications: (1) Congenital Heart Disease (CHD) diagnostic prediction among 154,775 patients; (2) Chronic obstructive pulmonary disease (COPD) diagnostic prediction among 73,791 patients; (3) future insulin treatment prediction among 78,712 patients diagnosed with diabetes as a mean to assess the disease exacerbation. In all three applications, MixEHR-S conferred clinically meaningful latent topics among the most predictive latent topics and achieved superior target prediction accuracy compared to the existing methods, providing opportunities for prioritizing high-risk patients for healthcare services. Availability and implementation: MixEHR-S source code and scripts of the experiments are freely available at https://github.com/li-lab-mcgill/mixehrS
Real-Time Guarantees for Critical Traffic in IEEE 802.1Qbv TSN Networks with Unscheduled and Unsynchronized End-Systems

Time-Sensitive Networking (TSN) aims to extend the IEEE 802.1Q Ethernet standard with real-time and time-aware capabilities. Each device's transmission of time-critical frames is done according to a so-called Gate Control List (GCL) schedule via the timed-gate mechanism described in IEEE 802.1Qbv. Most schedule generation mechanisms for TSN have a constraining assumption that both switches and end-systems in the network must have at least the TSN capabilities related to scheduled gates and time synchronization. However, many TSN networks use off-the-shelf end-systems, e.g., for providing sensor data, which are not scheduled and/or synchronized. In this paper, we propose a more flexible scheduling strategy that considers a worst-case delay analysis within the scheduling synthesis step, leveraging the solution's optimality to support TSN networks with unscheduled and unsynchronized end-systems while still being able to guarantee bounded latency for critical messages. Our method enables real-world systems that feature off-the-shelf microcontrollers and sensor nodes without TSN capabilities connected to state-of-the-art TSN networks to communicate critical messages in a real-time fashion. We evaluate our approach using both synthetic and real-world test cases, comparing it with existing scheduling mechanisms. Furthermore, we use OMNET++ to validate the generated GCL schedules.
Strain-tunable metamagnetic critical endpoint in Mott insulating rare-earth titanates

Rare-earth titanates are Mott insulators whose magnetic ground state—antiferromagnetic (AFM) or ferromagnetic (FM)—can be tuned by the radius of the rare-earth element. Here, we combine phenomenology and ﬁrst-principles calculations to shed light on the generic magnetic phase diagram of a chemically substituted titanate on the rare-earth site that interpolates between an AFM and a FM state. Octahedral rotations present in these perovskites cause the AFM order to acquire a small FM component—and vice-versa—removing any multicritical point from the phase diagram. However, for a wide parameter range, a ﬁrst-order metamagnetic transition line terminating at a critical endpoint survives inside the magnetically ordered phase. Like the liquid-gas transition, a Widom line emerges from the endpoint, characterized by enhanced ﬂuctuations. In contrast to metallic FMs, this metamagnetic transition involves two symmetry-equivalent and insulating canted spin states. Moreover, instead of a magnetic ﬁeld, we show that uniaxial strain can be used to tune this transition to zero temperature, inducing a quantum critical endpoint.
The Distributed Dual Ascent Algorithm is Robust to Asynchrony

The distributed dual ascent is an established algorithm to solve strongly convex multi-agent optimization problems with separable cost functions, in the presence of coupling constraints. In this letter, we study its asynchronous counterpart. Specifically, we assume that each agent only relies on the outdated information received from some neighbors. Differently from the existing randomized and dual block-coordinate schemes, we show convergence under heterogeneous delays, communication and update frequencies. Consequently, our asynchronous dual ascent algorithm can be implemented without requiring any coordination between the agents.
Stability of Neptune’s Distant Resonances in the Presence of Planet Nine

Trans-Neptunian Objects (TNOs) in the scattered disk with 50 ≲ a ≲ 100 au are thought to cluster near Neptune’s n:1 resonances (e.g., 3:1, 4:1, and so on). While these objects spend lengthy periods of time at large heliocentric distances, if their perihelia remain less than around 40 au, their dynamical evolution is still largely coupled to Neptune’s. Conversely, around a dozen extreme TNOs with a ≳ 250 au and detached perihelia seem to exist in a regime where they are too distant to be affected by the giant planets and too close for their dynamics to be governed by external forces. Recent work suggests that the apparent alignment of these orbits in physical space is a signature of gravitational shepherding by a distant massive planet. In this paper, we investigate the evolution of TNOs in each of Neptune’s n:1 resonances between the 3:1 and 14:1. We conclude that both resonant and nonresonant objects beyond the 12:1 near ∼157 au are removed rather efficiently via perturbations from the hypothetical Planet Nine. Additionally, we uncover a population of simulated TNOs with a ≲ 100 au, 40 ≲ q ≲ 45 au, and low inclinations that experience episodes of resonant interactions with both Neptune and Planet Nine. Finally, we simulate the evolution of observed objects with a > 100 au and identify several TNOs that are potentially locked in n:1 resonances with Neptune, including the most distant known resonant candidates, 2014 JW80 and 2014 OS394,which appear to be in the 10:1 and 11:1 resonances, respectively. Our results suggest that the detection of similar remote objects might provide a useful constraint on hypotheses invoking the existence of additional distant planets.
X-ray spectral components of the blazar and binary black hole candidate OJ 287 (2005–2020)

We present a comprehensive analysis of all XMM–Newton spectra of OJ 287 spanning 15 yr of X-ray spectroscopy of this bright blazar. We also report the latest results from our dedicated Swift UVOT and XRT monitoring of OJ 287, which started in 2015, along with all earlier public Swift data since 2005. During this time interval, OJ 287 was caught in extreme minima and outburst states. Its X-ray spectrum is highly variable and encompasses all states seen in blazars from very flat to exceptionally steep. The spectrum can be decomposed into three spectral components: Inverse Compton (IC) emission dominant at low-state, supersoft synchrotron emission that becomes increasingly dominant as OJ 287 brightens, and an intermediately-soft (Γx = 2.2) additional component seen at outburst. This last component extends beyond 10 keV and plausibly represents either a second synchrotron/IC component and/or a temporary disc corona of the primary supermassive black hole (SMBH). Our 2018 XMM–Newton observation, quasi-simultaneous with the Event Horizon Telescope observation of OJ 287, is well described by a two-component model with a hard IC component of Γx = 1.5 and a soft synchrotron component. Low-state spectra limit any long-lived accretion disc/corona contribution in X-rays to a very low value of Lx/LEdd < 5.6 × 10−4 (for MBH, primary = 1.8 × 1010 M⊙). Some implications for the binary SMBH model of OJ 287 are discussed.
Holographic MIMO Communications

Imagine a MIMO communication system that fully exploits the propagation characteristics offered by an electromagnetic channel and ultimately approaches the limits imposed by wireless communications. This is the concept of Holographic MIMO communications. Accurate and tractable channel modeling is critical to understanding its full potential. Classical stochastic models used by communications theorists are derived under the electromagnetic far-ﬁeld assumption. However, such assumption breaks down when large (compared to the wavelength) antenna arrays are considered - as envisioned in future wireless communications. In this paper, we start from the ﬁrst principles of wave propagation and provide a Fourier plane-wave series expansion of the channel response, which fully captures the essence of electromagnetic propagation in arbitrary scattering and is also valid in the (radiative) near-ﬁeld. The expansion is based on the Fourier spectral representation and has an intuitive physical interpretation, as it statistically describes the angular coupling between source and receiver. When discretized, it leads to a low-rank semi-unitarily equivalent approximation of the spatial electromagnetic channel in the angular domain. The developed channel model is used to compute the ergodic capacity of a Holographic MIMO system with different degrees of channel state information.
Canonical Saliency Maps: Decoding Deep Face Models

As Deep Neural Network models for face processing tasks approach human-like performance, their deployment in critical applications such as law enforcement and access control has seen an upswing, where any failure may have far-reaching consequences. We need methods to build trust in deployed systems by making their working as transparent as possible. Existing visualization algorithms are designed for object recognition and do not give insightful results when applied‘ to the face domain. In this work, we present ‘Canonical Saliency Maps’, a new method which highlights relevant facial areas by projecting saliency maps onto a canonical face model. We present two kinds of Canonical Saliency Maps: image-level maps and model-level maps. Image-level maps highlight facial features responsible for the decision made by a deep face model on a given image, thus helping to understand how a DNN made a prediction on the image. Model-level maps provide an understanding of what the entire DNN model focuses on in each task, and thus can be used to detect biases in the model. Our qualitative and quantitative results show the usefulness of the proposed canonical saliency maps, which can be used on any deep face model regardless of the architecture.
Scalar Adjective Identification and Multilingual Ranking

The intensity relationship that holds between scalar adjectives (e.g., nice < great < wonderful) is highly relevant for natural language inference and common-sense reasoning. Previous research on scalar adjective ranking has focused on English, mainly due to the availability of datasets for evaluation. We introduce a new multilingual dataset in order to promote research on scalar adjectives in new languages. We perform a series of experiments and set performance baselines on this dataset, using monolingual and multilingual contextual language models. Additionally, we introduce a new binary classification task for English scalar adjective identification which examines the models’ ability to distinguish scalar from relational adjectives. We probe contextualised representations and report baseline results for future comparison on this task.
High-resolution observations of prominence plume formation with the new vacuum solar telescope

Prominence plumes are evacuated upflows that emerge from bubbles below prominences, whose formation mechanism is still unclear. Here we present a detailed study of plumes in a quiescent prominence using the high-resolution Hα filtergrams at the line center as well as line wing at ± 0.4 Å from the New Vacuum Solar Telescope. Enhancements of brightening, blue shifts, and turbulence at the fronts of plumes are found during their formation. Some large plumes split at their heads and finger-shaped structures are formed between them. Blue-shifted flows along the bubble-prominence interface are found before and during the plume formation. Our observations are consistent with the hypothesis that prominence plumes are related to coupled Kelvin-Helmholtz and Rayleigh-Taylor (KH/RT) instabilities. Plume splittings and fingers are evidence of RT instability, and the flows may increase the growth rate of KH/RT instabilities. However, the significant turbulence at plume fronts may suggest that the RT instability is triggered by the plumes penetrating into the prominence. In this scenario, extra mechanisms are necessary to drive the plumes.
Weak Alfvénic turbulence in relativistic plasmas II: current sheets and dissipation

Alfvén waves as excited in black hole accretion disks and neutron star magnetospheres are the building blocks of turbulence in relativistic, magnetized plasmas. A large reservoir of magnetic energy is available in these systems, such that the plasma can be heated significantly even in the weak turbulence regime. We perform high-resolution three-dimensional simulations of counter-propagating Alfvén waves, showing that an $E_{B_{\perp }}(k_{\perp }) \propto k_{\perp }^{-2}$ energy spectrum develops as a result of the weak turbulence cascade in relativistic magnetohydrodynamics and its infinitely magnetized (force-free) limit. The plasma turbulence ubiquitously generates current sheets, which act as locations where magnetic energy dissipates. We show that current sheets form as a natural result of nonlinear interactions between counter-propagating Alfvén waves. These current sheets form owing to the compression of elongated eddies, driven by the shear induced by growing higher-order modes, and undergo a thinning process until they break-up into small-scale turbulent structures. We explore the formation of current sheets both in overlapping waves and in localized wave packet collisions. The relativistic interaction of localized Alfvén waves induces both Alfvén waves and fast waves, and efficiently mediates the conversion and dissipation of electromagnetic energy in astrophysical systems. Plasma energization through reconnection in current sheets emerging during the interaction of Alfvén waves can potentially explain X-ray emission in black hole accretion coronae and neutron star magnetospheres.
NeuralLog: a Neural Logic Language

Application domains that require considering relationships among objects which have real-valued attributes are becoming even more important. In this paper we propose NeuralLog, a first-order logic language that is compiled to a neural network. The main goal of NeuralLog is to bridge logic programming and deep learning, allowing advances in both fields to be combined in order to obtain better machine learning models. The main advantages of NeuralLog are: to allow neural networks to be defined as logic programs; and to be able to handle numeric attributes and functions. We compared NeuralLog with two distinct systems that use first-order logic to build neural networks. We have also shown that NeuralLog can learn link prediction and classification tasks, using the same theory as the compared systems, achieving better results for the area under the ROC curve in four datasets: Cora and UWCSE for link prediction; and Yelp and PAKDD15 for classification; and comparable results for link prediction in the WordNet dataset.
Personalized Algorithm Generation: A Case Study in Meta-Learning ODE Integrators

We study the meta-learning of numerical algorithms for scientiﬁc computing, which combines the mathematically driven, handcrafted design of general algo-rithm structure with a data-driven adaptation to speciﬁc classes of tasks. This represents a departure from the classical approaches in numerical analysis, which typically do not feature such learning-based adaptations. As a case study, we develop a machine learning approach that automatically learns eﬀective solvers for initial value problems in the form of ordinary diﬀerential equations (ODEs), based on the Runge-Kutta (RK) integrator architecture. By combining neural network approximations and meta-learning, we show that we can obtain high-order integrators for targeted families of diﬀerential equations without the need for computing integrator coeﬃcients by hand. Moreover, we demonstrate that in certain cases we can obtain superior performance to classical RK methods. This can be attributed to certain properties of the ODE families being identiﬁed and exploited by the approach. Overall, this work demonstrates an eﬀective, learning-based approach to the design of algorithms for the numerical solution of diﬀerential equations, an approach that can be readily extended to other numerical tasks.
VQCPC-GAN: Variable-Length Adversarial Audio Synthesis Using Vector-Quantized Contrastive Predictive Coding

Influenced by the field of Computer Vision, Generative Adversarial Networks (GANs) are often adopted for the audio domain using fixed-size two-dimensional spectrogram representations as the “image data”. However, in the (musical) audio domain, it is often desired to generate output of variable duration. This paper presents VQCPC-GAN, an adversarial framework for synthesizing variable-length audio by exploiting Vector-Quantized Contrastive Predictive Coding (VQCPC). A sequence of VQCPC tokens extracted from real audio data serves as conditional input to a GAN architecture, providing step-wise time-dependent features of the generated content. The input noise $z$ (characteristic in adversarial architectures) remains fixed over time, ensuring temporal consistency of global features. We evaluate the proposed model by comparing a diverse set of metrics against various strong baselines. Results show that, even though the baselines score best, VQCPC-GAN achieves comparable performance even when generating variable-length audio. Numerous sound examples are provided in the accompanying website,11sonycslparis.github. io/vqcpc-gan.io and we release the code for reproducibility.22github. com/SonyCSLParis/vqcpc-gan
Towards favorable landscapes in quantum combinatorial optimization

The performance of variational quantum algorithms relies on the success of us-ing quantum and classical computing resources in tandem. Here, we study how these quantum and classical components interrelate. In particular, we focus on algorithms for solving the combinatorial optimization problem MaxCut, and study how the structure of the classical optimization landscape relates to the quantum circuit used to evaluate the MaxCut objective function. In order to analytically characterize the impact of quantum features on the landscape critical point structure, we consider a family of quantum circuit ans¨atze composed of mutually commuting elements. We identify multi-qubit operations as a key resource, and show that overparameterization allows for obtaining favorable landscapes.
Global 21cm absorption signal from superconducting cosmic strings

Superconducting cosmic strings emit electromagnetic waves between the times of recombination and reionization. Hence, they have an effect on the global 21cm signal. We compute the resulting absorption features, focusing on strings with critical current, study their dependence on the string tension μ, and compare with observational results. For string tensions in the range of Gμ = 10-10, where G is Newton's gravitational constant, there is an interesting amplification of the two characteristic absorption features, one during the cosmic dawn, z ≲ 30, and the other during the cosmic dark age, z ∼ 80, the former being comparable in amplitude to what was observed by the EDGES experiment.
WORKING PAPER SERIES MANDATED FINANCIAL REPORTING AND CORPORATE INNOVATION

We gratefully acknowledge helpful comments and suggestions from Ray Ball, Benjamin Balsmeier, Phil Berger, Jannis Bischof, Hans Christensen, Wouter Dessein, Stephen Glaeser, Laurence van Lent, Alexander Wagner (discussant), and seminar participants at the University of Chicago, University of Mannheim, the 2019 EASYS, the 2019 Conference on Regulating Financial Markets at Goethe University Frankfurt. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Mid-to late Holocene morphological and hydrological changes in the south 1 Taihu area of the Yangtze delta plain , China 2 3

20 The Taihu Plain of the Lower Yangtze valley, China was a centre of rice 21 agriculture during the Neolithic period. Reasons for the rapid development of rice 22 cultivation during this period, however, have not been fully understood for this coastal 23 lowland, which is highly sensitive to sea-level change. To improve understanding of 24 the morphological and hydrological context for evolution of prehistoric rice 25