Miracle-dz/UniFAR-FTUs huggingface dataset:
Graph-based Approximate Message Passing Iterations

Approximate-message passing (AMP) algorithms have become an important element of high-dimensional statistical inference, mostly due to their adaptability and concentration properties, the state evolution (SE) equations. This is demonstrated by the growing number of new iterations proposed for increasingly complex problems, ranging from multi-layer inference to low-rank matrix estimation with elaborate priors. In this paper, we address the following questions: is there a structure underlying all AMP iterations that unifies them in a common framework? Can we use such a structure to give a modular proof of state evolution equations, adaptable to new AMP iterations without reproducing each time the full argument ? We propose an answer to both questions, showing that AMP instances can be generically indexed by an oriented graph. This enables to give a unified interpretation of these iterations, independent from the problem they solve, and a way of composing them arbitrarily. We then show that all AMP iterations indexed by such a graph admit rigorous SE equations, extending the reach of previous proofs, and proving a number of recent heuristic derivations of those equations. Our proof naturally includes non-separable functions and we show how existing refinements, such as spatial coupling or matrix-valued variables, can be combined with our framework.
Learning-Based Noise Component Map Estimation for Image Denoising

A problem of image denoising, when images are corrupted by a non-stationary noise, is considered in this paper. Since, in practice, no a priori information on noise is available, noise statistics should be pre-estimated prior to image denoising. In this paper, deep convolutional neural network (CNN) based method for estimation of a map of local, patch-wise, standard deviations of noise (so-called sigma-map) is proposed. It achieves the state-of-the-art performance in accuracy of estimation of sigma-map for the case of non-stationary noise, as well as estimation of a noise variance for the case of an additive white Gaussian noise. Extensive experiments on image denoising using estimated sigma-maps demonstrate that our method outperforms recent CNN-based blind image denoising methods by up to 6 dB in PSNR, as well as other state-of-the-art methods based on sigma-map estimation by up to 0.5 dB, providing, at the same time, better usage flexibility. A comparison with the ideal case, when denoising is applied using ground-truth sigma-map, shows that a difference of corresponding PSNR values for the most of noise levels is within 0.1-0.2 dB, and does not exceed 0.6 dB.
Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images

We propose a novel deep reinforcement learning-based approach for 3D object reconstruction from monocular images. Prior works that use mesh representations are template based. Thus, they are limited to the reconstruction of objects that have the same topology as the template. Methods that use volumetric grids as intermediate representations are computationally expensive, which limits their application in real-time scenarios. In this paper, we propose a novel end-to-end method that reconstructs 3D objects of arbitrary topology from a monocular image. It is composed of of (1) a Vertex Generation Network (VGN), which predicts the initial 3D locations of the object's vertices from an input RGB image, (2) a differentiable triangulation layer, which learns in a non-supervised manner, using a novel reinforcement learning algorithm, the best triangulation of the object's vertices, and finally, (3) a hierarchical mesh refinement network that uses graph convolutions to refine the initial mesh. Our key contribution is the learnable triangulation process, which recovers in an unsupervised manner the topology of the input shape. Our experiments on ShapeNet and Pix3D benchmarks show that the proposed method outperforms the state-of-the-art in terms of visual quality, reconstruction accuracy, and computational time.
Unsupervised Translation of German–Lower Sorbian: Exploring Training and Novel Transfer Methods on a Low-Resource Language

This paper describes the methods behind the systems submitted by the University of Groningen for the WMT 2021 Unsupervised Machine Translation task for German–Lower Sorbian (DE–DSB): a high-resource language to a low-resource one. Our system uses a transformer encoder-decoder architecture in which we make three changes to the standard training procedure. First, our training focuses on two languages at a time, contrasting with a wealth of research on multilingual systems. Second, we introduce a novel method for initializing the vocabulary of an unseen language, achieving improvements of 3.2 BLEU for DE->DSB and 4.0 BLEU for DSB->DE.Lastly, we experiment with the order in which offline and online back-translation are used to train an unsupervised system, finding that using online back-translation first works better for DE->DSB by 2.76 BLEU. Our submissions ranked first (tied with another team) for DSB->DE and third for DE->DSB.
Parameter-Free Deterministic Reduction of the Estimation Bias in Continuous Control

—Approximation of the value functions in value-based deep reinforcement learning systems induces overes-timation bias, resulting in suboptimal policies. We show that when the reinforcement signals received by the agents have a high variance, deep actor-critic approaches that overcome the overestimation bias lead to a substantial underestimation bias. We introduce a parameter-free, novel deep Q-learning variant to reduce this underestimation bias for continuous control. By obtaining ﬁxed weights in computing the critic objective as a linear combination of the approximate critic functions, our Q-value update rule integrates the concepts of Clipped Double Q-learning and Maxmin Q-learning. We test the performance of our improvement on a set of MuJoCo and Box2D continuous control tasks and ﬁnd that it improves the state-of-the-art and outperforms the baseline algorithms in the majority of the environments.
Saturation mechanism of the fluctuation dynamo in supersonic turbulent plasmas

Magnetic fields in several astrophysical objects are amplified and maintained by a dynamo mechanism, which is the conversion of the turbulent kinetic energy to magnetic energy. A dynamo that amplifies magnetic fields at scales less than the driving scale of turbulence is known as the fluctuation dynamo. We aim to study the properties of the fluctuation dynamo in supersonic turbulent plasmas, which is of relevance to the interstellar medium of star-forming galaxies, structure formation in theUniverse, and laboratory experiments of laser-plasma turbulence. Using numerical simulations of driven turbulence, we explore the global and local properties of the exponentially growing and saturated (statistically steady) state of the fluctuation dynamo for subsonic and supersonic turbulent flows. First, we confirm that the fluctuation dynamo efficiency decreases with compressibility. Then, we show that the fluctuation dynamo generated magnetic fields are spatially intermittent and the intermittency is higher for supersonic turbulence, but in both cases, the level of intermittency decreases as the field saturates. We also find a stronger back reaction of the magnetic field on the velocity for the subsonic case as compared to the supersonic case. Locally, we find that the level of alignment between vorticity and velocity, velocity and magnetic field, and current density and magnetic field in the saturated stage is enhanced in comparison to the exponentially growing phase for the subsonic case, but only the current density and magnetic field alignment is enhanced for the supersonic case. Finally, we show that both the magnetic field amplification (mainly due to weaker stretching of magnetic field lines) and diffusion decreases when the field saturates, but the diffusion is enhanced relative to amplification. This occurs throughout the volume in the subsonic turbulence, but primarily in the strong-field regions for the supersonic case. This leads to the saturation of the fluctuation dynamo. Overall, both the amplification and diffusion of magnetic fields are affected by the exponentially growing magnetic fields and thus a drastic change in either of them is not required for the saturation of the fluctuation dynamo.
Dynamic Parallel Spin Stripes from the 1/8 anomaly to the End of Superconductivity in La$_{1.6-x}$Nd$_{0.4}$Sr$_x$CuO$_4$

Qianli Ma,1 Evan M. Smith,1 Zachary W. Cronkwright,1 Mirela Dragomir,2, 3 Gabrielle Mitchell,1 Alexander I. Kolesnikov,4 Matthew B. Stone,4 and Bruce D. Gaulin1, 2, 5 1Department of Physics and Astronomy, McMaster University, Hamilton, Ontario, L8S 4M1, Canada 2Brockhouse Institute for Materials Research, Hamilton, Ontario, L8S 4M1, Canada 3Electronic Ceramics Department, Jožef Stefan Institute, 1000 Ljubljana, Slovenia 4Neutron Scattering Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830, United States 5Canadian Institute for Advanced Research, MaRS Centre, West Tower 661 University Ave., Suite 505, Toronto, ON, M5G 1M1, Canada (Dated: January 27, 2022)
AraT5: Text-to-Text Transformers for Arabic Language Understanding and Generation

Transfer learning with a uniﬁed Transformer framework (T5) that converts all language problems into a text-to-text format has recently been proposed as a simple, yet effective, transfer learning approach. Although a multilingual version of the T5 model (mT5) has been introduced, it is not clear how well it can fare on non-English tasks involving diverse data. To investigate this question, we apply mT5 on a language with a wide variety of dialects–Arabic. For evaluation, we use an existing benchmark for Arabic language understanding and introduce a new benchmark for Arabic language generation (ARGEN). We also pre-train three powerful Arabic-speciﬁc text-to-text Transformer based models and evaluate them on the two benchmarks. Our new models perform signiﬁcantly better than mT5 and exceed MAR-BERT, the current state-of-the-art Arabic BERT-based model, on Arabic language understanding. The models also set new SOTA on the generation benchmark. Our new models and are publicly released at https://github.com/UBC-NLP/araT5 and ARGEN will be released through the same repository.
Multiphoton Emission

G. Dı́az Camacho, E. Zubizarreta Casalengua, J. C. López Carreño, S. Khalid, C. Tejedor, E. del Valle, 4, 2 and F. P. Laussy 5 Departamento de F́ısica Teórica de la Materia Condensada & IFIMAC, Universidad Autónoma de Madrid, 28049 Madrid, Spain Faculty of Science and Engineering, University of Wolverhampton, Wulfruna St, Wolverhampton WV1 1LY, UK Institute of Theoretical Physics, University of Warsaw, ul. Pasteura 5, 02-093, Warsaw, Poland Institute for Advanced Study, Technical University of Munich, Lichtenbergstrasse 2a, D-85748 Garching, Germany. Russian Quantum Center, Novaya 100, 143025 Skolkovo, Moscow Region, Russia∗ (Dated: September 27, 2021)
Morita equivalence for operator systems

We define ∆-equivalence for operator systems and show that it is identical to stable isomorphism. We define ∆-contexts and bihomomorphism contexts and show that two operator systems are ∆-equivalent if and only if they can be placed in a ∆-context, equivalently, in a bihomomorphism context. We show that nuclearity for a variety of tensor products is an invariant for ∆-equivalence and that function systems are ∆-equivalent precisely when they are order isomorphic. We prove that ∆-equivalent operator systems have equivalent categories of representations. As an application, we characterise ∆-equivalence of graph operator systems in combinatorial terms.
A Decomposition Augmented Lagrangian Method for Low-Rank Semidefinite Programming

We develop a decomposition method based on the augmented Lagrangian framework to solve a broad family of semidefinite programming problems, possibly with nonlinear objective functions, nonsmooth regularization, and general linear equality/inequality constraints. In particular, the positive semidefinite variable along with a group of linear constraints can be transformed into a variable on a smooth manifold via matrix factorization. The nonsmooth regularization and other general linear constraints are handled by the augmented Lagrangian method. Therefore, each subproblem can be solved by a semismooth Newton method on a manifold. Theoretically, we show that the first and second-order necessary optimality conditions for the factorized subproblem are also sufficient for the original subproblem under certain conditions. Convergence analysis is established for the Riemannian subproblem and the augmented Lagrangian method. Extensive numerical experiments on large-scale semidefinite programming problems such as max-cut, nearest correlation estimation, clustering, and sparse principal component analysis demonstrate the strength of our proposed method compared to other state-of-the-art methods.
Bleaching, brightness stability and chemical characteristics of Eucalyptus grandis-bio-TMP pulps prepared in a biopulping pilot plant

Biopulping is a technology that can be applied advantageously to mechanical pulping. It presents benefits such as the creation of stronger pulp, as well as energy savings. Unfortunately, biomechanical pulps are darker than conventional TMP pulps. In this work, E. grandis wood was biotreated in a 45-ton biopulping pilot plant and used to produce bio-TMP pulps. Brightness values for unbleached and peroxide bleached pulps were 44% and 70% ISO for these bio-TMP pulps, compared with 60% and 72% ISO for control-TMP produced from untreated wood, respectively. Bleached bio-TMP was more stable to thermal-reversion than the control, while the stability to photo-reversion was similar in both pulps. Milled wood lignins (MWL) were extracted from studied samples and characterized by gel permeation chromatography and spectroscopic methods. MWL from bio-TMP presented lower average molecular weight (6390 g/mol) than that from control-TMP (8264 g/mol). Data on spectroscopic characterization of MWLs indicated that the chromophores responsible for the lower initial brightness of bio-TMP pulps were quinone structures, which were partially destroyed during peroxide bleaching.
Renormalization approach to bound states

A new semi-analytical approach to bound states based on a unitary transformation of the Hamiltonian is presented. The method is applied to study the interaction between a superconductor and a quantum spin. Known results from the t-matrix method and numerical studies are reproduced. A uniform picture of the interplay between the Yu-Shiba-Rusinov (YSR) bound states and the Kondo singlet state is presented by revealing the structure of the quasiparticles in both phases. The method can straightforwardly be extended to study the topological properties of combined bound states in chains of magnetic impurities.
Towards A Measure Of General Machine Intelligence

To build general-purpose artiﬁcial intelligence systems that can deal with unknown variables across unknown domains, we need benchmarks that measure how well these systems perform on tasks they have never seen before. A prerequisite for this is a measure of a task’s generalization difﬁculty, or how dissimilar it is from the system’s prior knowledge and experience. If the skill of an intelligence system in a particular domain is deﬁned as it’s ability to consistently generate a set of instructions (or programs) to solve tasks in that domain, current benchmarks do not quantitatively measure the efﬁciency of acquiring new skills, making it possible to brute-force skill acquisition by training with unlimited amounts of data and compute power. With this in mind, we ﬁrst propose a common language of instruction, a programming language that allows the expression of programs in the form of directed acyclic graphs across a wide variety of real-world domains and computing platforms. Using programs generated in this language, we demonstrate a match-based method to both score performance and calculate the generalization difﬁculty of any given set of tasks. We use these to deﬁne a numeric benchmark called the generalization index, or the g-index , to measure and compare the skill-acquisition efﬁciency of any intelligence system on a set of real-world tasks. Finally, we evaluate the suitability of some well-known models as general intelligence systems by calculating their g-index scores.
Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment

Algorithmic recommendations and decisions have become ubiquitous in today’s society. Many of these and other data-driven policies, especially in the realm of public policy, are based on known, deterministic rules to ensure their transparency and interpretability. For example, algorithmic pre-trial risk assessments, which serve as our motivating application, provide relatively simple, deterministic classification scores and recommendations to help judges make release decisions. How can we use the data based on existing deterministic policies to learn new and better policies? Unfortunately, prior methods for policy learning are not applicable because they require existing policies to be stochastic rather than deterministic. We develop a robust optimization approach that partially identifies the expected utility of a policy, and then finds an optimal policy by minimizing the worst-case regret. The resulting policy is conservative but has a statistical safety guarantee, allowing the policy-maker to limit the probability of producing a worse outcome than the existing policy. We extend this approach to common and important settings where humans make decisions with the aid of algorithmic recommendations. Lastly, we apply the proposed methodology to a unique field experiment on pre-trial risk assessment instruments. We derive new classification and recommendation rules that retain the transparency and interpretability of the existing instrument while potentially leading to better overall outcomes at a lower cost.
Local tensor-network codes

Tensor-network codes enable the construction of large stabilizer codes out of tensors describing smaller stabilizer codes. An application of tensor-network codes was an efficient and exact decoder for holographic codes. Here, we show how to write some topological codes, including the surface code and colour code, as simple tensor-network codes. We also show how to calculate distances of stabilizer codes by contracting a tensor network. The algorithm actually gives more information, including a histogram of all logical coset weights. We prove that this method is efficient in the case of stabilizer codes encoded via local log-depth circuits in one dimension and holographic codes. Using our tensor-network distance calculator, we find a modification of the rotated surface code that has the same distance but fewer minimum-weight logical operators by ‘doping’ the tensor network, i.e., we break the homogeneity of the tensor network by locally replacing tensors. For this example, this corresponds to an improvement in successful error correction of almost 2% against depolarizing noise (in the perfect-measurement setting), but comes at the cost of introducing three higher-weight stabilizers. Our general construction lets us pick a network geometry (e.g., a Euclidean lattice in the case of the surface code), and, using only a small set of seed codes (constituent tensors), build extensive codes with the potential for optimisation.
The equivariant coarse Baum–Connes conjecture for metric spaces with proper group actions

The equivariant coarse Baum-Connes conjecture interpolates between the Baum-Connes conjecture for a discrete group and the coarse Baum-Connes conjecture for a proper metric space. In this paper, we study this conjecture under certain assumptions. More precisely, assume that a countable discrete group $\Gamma$ acts properly and isometrically on a discrete metric space $X$ with bounded geometry, not necessarily cocompact. We show that if the quotient space $X/\Gamma$ admits a coarse embedding into Hilbert space and $\Gamma$ is amenable, and that the $\Gamma$-orbits in $X$ are uniformly equivariantly coarsely equivalent to each other, then the equivariant coarse Baum-Connes conjecture holds for $(X, \Gamma)$. Along the way, we prove a $K$-theoretic amenability statement for the $\Gamma$-space $X$ under the same assumptions as above, namely, the canonical quotient map from the maximal equivariant Roe algebra of $X$ to the reduced equivariant Roe algebra of $X$ induces an isomorphism on $K$-theory.
Transformers Generalize Linearly

Natural language exhibits patterns of hierarchically governed dependencies, in which relations between words are sensitive to syntactic structure rather than linear ordering. While re-current network models often fail to generalize in a hierarchically sensitive way (McCoy et al.,2020) when trained on ambiguous data, the improvement in performance of newer Trans-former language models (Vaswani et al., 2017)on a range of syntactic benchmarks trained on large data sets (Goldberg, 2019; Warstadtet al., 2019) opens the question of whether these models might exhibit hierarchical generalization in the face of impoverished data.In this paper we examine patterns of structural generalization for Transformer sequence-to-sequence models and find that not only do Transformers fail to generalize hierarchically across a wide variety of grammatical mapping tasks, but they exhibit an even stronger preference for linear generalization than comparable recurrent networks
Recurrent Neural Networks for Partially Observed Dynamical Systems

Complex nonlinear dynamics are ubiquitous in many fields. Moreover, we rarely have access to all of the relevant state variables governing the dynamics. Delay embedding allows us, in principle, to account for unobserved state variables. Here we provide an algebraic approach to delay embedding that permits explicit approximation of error. We also provide the asymptotic dependence of the first-order approximation error on the system size. More importantly, this formulation of delay embedding can be directly implemented using a recurrent neural network (RNN). This observation expands the interpretability of both delay embedding and the RNN and facilitates principled incorporation of structure and other constraints into these approaches.
SDSS-IV MaNGA: drivers of stellar metallicity in nearby galaxies

The distribution of stellar metallicities within and across galaxies is an excellent relic of the chemical evolution across cosmic time. We present a detailed analysis of spatially resolved stellar populations based on >2.6 million spatial bins from 7439 nearby galaxies in the SDSS-IV MaNGA survey. To account for accurate inclination corrections, we derive an equation for morphology dependent determination of galaxy inclinations. Our study goes beyond the well-known global mass-metallicity relation and radial metallicity gradients by providing a statistically sound exploration of local relations between stellar metallicity [Z/H], stellar surface mass density Σ⋆ and galactocentric distance in the global mass-morphology plane. We find a significant resolved mass density-metallicity relation $\rm r\Sigma _\star ZR$ for galaxies of all types and masses above 109.8 M⊙. Different radial distances make an important contribution to the spread of the relation. Particularly, in low and intermediate mass galaxies, we find that at fixed Σ⋆ metallicity increases with radius independently of morphology. For high masses, this radial dependence is only observed in high Σ⋆ regions of spiral galaxies. This result calls for a driver of metallicity, in addition to Σ⋆ that promotes chemical enrichment in the outer parts of galaxies more strongly than in the inner parts. We discuss gas accretion, outflows, recycling and radial migration as possible scenarios.