Miracle-dz/UniFAR-FTUs huggingface dataset:
Scalable Approximate Inference and Some Applications

Approximate inference in probability models is a fundamental task in machine learning. Approximate inference provides powerful tools to Bayesian reasoning, decision making, and Bayesian deep learning. The main goal is to estimate the expectation of interested functions w.r. t. a target distribution. When it comes to high dimensional probability models and large datasets, efficient approximate inference becomes critically important. In this thesis, we propose a new framework for approximate inference, which combines the advantages of these three frameworks and overcomes their limitations. Our proposed four algorithms are motivated by the recent computational progress of Stein's method. Our proposed algorithms are applied to continuous and discrete distributions under the setting when the gradient information of the target distribution is available or unavailable. Theoretical analysis is provided to prove the convergence of our proposed algorithms. Our adaptive IS algorithm iteratively improves the importance proposal by functionally decreasing the KL divergence between the updated proposal and the target. When the gradient of the target is unavailable, our proposed sampling algorithm leverages the gradient of a surrogate model and corrects induced bias with importance weights, which significantly outperforms other gradient-free sampling algorithms. In addition, our theoretical results enable us to perform the goodness-of-fit test on discrete distributions. At the end of the thesis, we propose an importance-weighted method to efficiently aggregate local models in distributed learning with one-shot communication. Results on simulated and real datasets indicate the statistical efficiency and wide applicability of our algorithm.
Follow-up Ground-based Observations of the Dwarf Nova KZ Gem

We present spectroscopy of stars in the immediate vicinity of the dwarf nova (DN) KZ Gem to confirm its identification, which is ambiguous in the literature. Analysis of 73 radial velocities spanning from 2014 to 2019 provides a high-precision orbital period of 0.2224628(2) days (∼5.34 hr) and shows KZ Gem to be a double-lined DN. Time series photometry taken from 2016 to 2018 shows a variable double-hump modulation with a full amplitude of ∼0.3 mag, along with five Gaussian-like transient events lasting ∼30 minutes or more. Using the light-curve code XRBinary and nonlinear fitting code NMfit, we obtain an optimized binary model of the dwarf nova (DN) KZ Gem, from time series photometry, consisting of a Roche-lobe-filling K-type dwarf with a mass transfer rate of (2.7–7.9) × 10−10 M⊙ yr−1 to a large, cool, and thick disk surrounding a white dwarf, in an orbit with an inclination of 51. °6(±1.°4). Two hotspots on the disk are demonstrated to cause the observed variations in the ellipsoidal modulations from the secondary star. This physical model is compatible with the Gaia distance of KZ Gem.
Photospheric magnetic topology of a north polar region

Aims. We aim to characterise the magnetism of a large fraction of the north polar region close to a maximum of activity, when the polar regions are reversing their dominant polarity. Methods. We make use of full spectropolarimetric data from the CRisp Imaging Spectro-Polarimeter installed at the Swedish Solar Telescope. The data consist of a photospheric spectral line, which is used to infer the various physical parameters of different quiet Sun regions by means of the solution of the radiative transfer equation. We focus our analysis on the properties found for the north polar region and their comparison to the same analysis applied to data taken at disc centre and low-latitude quiet Sun regions for reference. We also analyse the spatial distribution of magnetic structures throughout the north polar region. Results. We find that the physical properties of the polar region (line-of-sight velocity, magnetic flux, magnetic inclination and magnetic azimuth) are compatible with those found for the quiet Sun at disc centre and are similar to the ones found at low latitudes close to the limb. Specifically, the polar region magnetism presents no specific features. The structures for which the transformation from a line-of-sight to a local reference frame was possible harbour large magnetic fluxes (>1017 Mx) and are in polarity imbalance with a dominant positive polarity, the largest ones (>1019 Mx) being located below 73° latitude.
Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networks

In this paper, we introduce interpretable Siamese Neural Networks (SNN) for similarity detection to the field of theoretical physics. More precisely, we apply SNNs to events in special relativity, the transformation of electromagnetic fields, and the motion of particles in a central potential. In these examples, the SNNs learn to identify datapoints belonging to the same events, field configurations, or trajectory of motion. It turns out that in the process of learning which datapoints belong to the same event or field configuration, these SNNs also learn the relevant symmetry invariants and conserved quantities. These SNNs are highly interpretable, which enables us to reveal the symmetry invariants and conserved quantities without prior knowledge.
Discovering linguistic (ir)regularities in word embeddings through max-margin separating hyperplanes

We experiment with new methods for learning how related words are positioned relative to each other in word embedding spaces. Previous approaches learned constant vector offsets: vectors that point from source tokens to target tokens with an assumption that these offsets were parallel to each other. We show that the offsets between related tokens are closer to orthogonal than parallel, and that they have low cosine similarities. We proceed by making a different assumption; target tokens are linearly separable from source and un-labeled tokens. We show that a max-margin hyperplane can separate target tokens and that vectors orthogonal to this hyperplane represent the relationship between source and targets. We find that this representation of the relationship obtains the best results in dis-covering linguistic regularities. We experiment with vector space models trained by a variety of algorithms (Word2vec: CBOW/skip-gram, fastText, or GloVe), and various word context choices such as linear word-order, syntax dependency grammars, and with and without knowledge of word position. These experiments show that our model, SVMCos, is robust to a range of experimental choices when training word embeddings.
Spanish National dietary survey in adults, elderly and pregnant women

ENALIA2 is a dietary survey conducted in Spain to collect food consumption data and other information about eating habits and physical activity on adults (18 – 64 years old), elderly (65 to 74 years old) and pregnant women. Data on 1033 individuals were obtained in line with the EU Menu guideline to be included in the EFSA Comprehensive European Food Consumption Database. In this sense the survey followed the EFSA “Guidance recommendations on the General principles for the collection of national food consumption data in the view of a pan-European dietary survey”. The project was funded through an EFSA procurement and was included in the EU Menu Project. The participation rate was 53.4% in general population and 80.1% in pregnant women. No major deviation from the guidance or technical specifications was required.
Prospects for pre-supernova neutrino observation in future large liquid-scintillator detectors

Before massive stars heavier than (8 ⋯ 10) solar masses evolve to the phase of a gravitational core collapse, they will emit a huge number of MeV-energy neutrinos that are mainly produced in the thermal processes and nuclear weak interactions. The detection of such pre-supernova (pre-SN) neutrinos could provide an important and independent early warning for the optical observations of core-collapse SNe. In this paper, we investigate the prospects of future large liquid-scintillator detectors for the observation of pre-SN neutrinos in both ν̄e + p → e+n and ν (ν̄) + e− → ν (ν̄) + e− reaction channels, where ν (ν̄) denotes neutrinos (antineutrinos) of all three flavors. We propose a quantitative assessment of the capability in terms of three working criteria, namely, how far the SN distance can be covered, how long the early warning before the core collapse can be sent out, and how well the direction pointing to the SN can be determined. The dependence of the final results on the different models of progenitor stars, neutrino flavor conversions and the relevant backgrounds is also discussed.
P–nets: Deep Polynomial Neural Networks

Deep Convolutional Neural Networks (DCNNs) is currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose $\Pi$-Nets, a new class of DCNNs. $\Pi$-Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. $\Pi$-Nets can be implemented using special kind of skip connections and their parameters can be represented via high-order tensors. We empirically demonstrate that $\Pi$-Nets have better representation power than standard DCNNs and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, $\Pi$-Nets produce state-of-the-art results in challenging tasks, such as image generation. Lastly, our framework elucidates why recent generative models, such as StyleGAN, improve upon their predecessors, e.g., ProGAN.
Deep Inverse Feature Learning: A Representation Learning of Error

This paper introduces a novel perspective about error in machine learning and proposes inverse feature learning (IFL) as a representation learning approach that learns a set of high-level features based on the representation of error for classification or clustering purposes. The proposed perspective about error representation is fundamentally different from current learning methods, where in classification approaches they interpret the error as a function of the differences between the true labels and the predicted ones or in clustering approaches, in which the clustering objective functions such as compactness are used. Inverse feature learning method operates based on a deep clustering approach to obtain a qualitative form of the representation of error as features. The performance of the proposed IFL method is evaluated by applying the learned features along with the original features, or just using the learned features in different classification and clustering techniques for several data sets. The experimental results show that the proposed method leads to promising results in classification and especially in clustering. In classification, the proposed features along with the primary features improve the results of most of the classification methods on several popular data sets. In clustering, the performance of different clustering methods is considerably improved on different data sets. There are interesting results that show some few features of the representation of error capture highly informative aspects of primary features. We hope this paper helps to utilize the error representation learning in different feature learning domains.
A T ] 9 M ar 2 02 0 INVARIANTS FOR TAME PARAMETRISED CHAIN COMPLEXES

We set the foundations for a new approach to Topological Data Analysis (TDA) based on homotopical methods at chain complexes level. We present the category of tame parametrised chain complexes as a comprehensive environment that includes several cases that usually TDA handles separately, such as persistence modules, zigzag modules, and commutative ladders. We extract new invariants in this category using a model structure and various minimal cofibrant approximations. Such approximations and their invariants retain some of the topological, and not just homological, aspects of the objects they approximate. Introduction Data analysis is often about simplifying, ignoring most of the information available and extracting what might be meaningful for the task at hand. The same strategy of extracting summaries is also at the core of topology. In recent years, these two branches have merged, giving rise to Topological Data Analysis (TDA) [7]. TDA can benefit from a broad spectrum of existing homotopical tools for extracting such summaries. Currently, the most popular is persistent homology. The first step in persistence is to transform data into spatial information via, for example, the VietorisRips construction. The second step is typically the extraction of the homology of the obtained spaces, resulting in a so-called persistence module, effectively studied by enumerating its indecomposables [20]. Despite its success, TDA based on persistent homology has limitations. Firstly, one is limited to objects for which it is possible to list their indecomposable summands [20], and whose decompositions can be computed algorithmically. For example, the class of commutative ladders cannot be analysed using its indecomposables because it is of wild representation type [6, 12]. On the other hand, the indecomposables of the class of zigzag modules are fully described, but so far there is no efficient software to analyse them [8, 9]. Secondly, applying homology might be too drastic, disregarding a large amount of geometric information. The main goal of this paper is to show how to use homotopy theory not only to overcome the previous issues but also to open the way towards new invariants. The same strategy of disregarding some information and focusing on aspects that might be relevant is also at the core of homotopy theory, with colocalization being an example of such a process. In colocalization, the simplification is achieved by approximating arbitrary objects by other objects that are simpler and more manageable, such as the class of cofibrant objects in a model category. Our work is based on the realisation that 2020 Mathematics Subject Classification: 55PXX, 55N31.
When Person Re-identification Meets Changing Clothes

Person re-identification (ReID) is now an active research topic for AI-based video surveillance applications such as specific person search, but the practical issue that the target person(s) may change clothes (clothes inconsistency problem) has been overlooked for long. For the first time, this paper systematically studies this problem. We first overcome the difficulty of lack of suitable dataset, by collecting a small yet representative real dataset for testing whilst building a large realistic synthetic dataset for training and deeper studies. Facilitated by our new datasets, we are able to conduct various interesting new experiments for studying the influence of clothes inconsistency. We find that changing clothes makes ReID a much harder problem in the sense of bringing difficulties to learning effective representations and also challenges the generalization ability of previous ReID models to identify persons with unseen (new) clothes. Representative existing ReID models are adopted to show informative results on such a challenging setting, and we also provide some preliminary efforts on improving the robustness of existing models on handling the clothes inconsistency issue in the data. We believe that this study can be inspiring and helpful for encouraging more researches in this direction. The dataset is available on the project website: https://wanfb.github. io/dataset.html.
Amenability and weak containment for actions of locally compact groups on $C^*$-algebras

In this work we introduce and study a new notion of amenability for actions of locally compact groups on $C^*$-algebras. Our definition extends the definition of amenability for actions of discrete groups due to Claire Anantharaman-Delaroche. We show that our definition has several characterizations and permanence properties analogous to those known in the discrete case. We also give new characterizations of amenability even in the discrete case: in particular, we show that amenability is equivalent to the so-called quasi-central approximation property, a strong approximation property that was recently used by Suzuki in equivariant classification theory. We use our new notion of amenability to study when the maximal and reduced crossed products agree. One of our main results generalizes a theorem of Matsumura: we show that for an action of an exact locally compact group $G$ on a locally compact space $X$ the full and reduced crossed products $C_0(X)\rtimes_\max G$ and $C_0(X)\rtimes_{\mathrm{red}} G$ coincide if and only if the action of $G$ on $X$ is measurewise amenable. We also show that the analogue of this theorem does not hold for actions on noncommutative $C^*$-algebras. Finally, we study amenability as it relates to more detailed structure in the case of $C^*$-algebras that fibre over an appropriate $G$-space $X$, and the interaction of amenability with various regularity properties such as nuclearity and exactness, and the equivariant versions of injectivity and the WEP.
Percolation phase transition in weight-dependent random connection models

Abstract We investigate spatial random graphs defined on the points of a Poisson process in d-dimensional space, which combine scale-free degree distributions and long-range effects. Every Poisson point is assigned an independent weight. Given the weight and position of the points, we form an edge between any pair of points independently with a probability depending on the two weights of the points and their distance. Preference is given to short edges and connections to vertices with large weights. We characterize the parameter regime where there is a non-trivial percolation phase transition and show that it depends not only on the power-law exponent of the degree distribution but also on a geometric model parameter. We apply this result to characterize robustness of age-based spatial preferential attachment networks.
Tightly-coupled Fusion of Global Positional Measurements in Optimization-based Visual-Inertial Odometry

Motivated by the goal of achieving robust, drift-free pose estimation in long-term autonomous navigation, in this work we propose a methodology to fuse global positional information with visual and inertial measurements in a tightly-coupled nonlinear-optimization–based estimator. Differently from previous works, which are loosely-coupled, the use of a tightly-coupled approach allows exploiting the correlations amongst all the measurements. A sliding window of the most recent system states is estimated by minimizing a cost function that includes visual re-projection errors, relative inertial errors, and global positional residuals. We use IMU preintegration to formulate the inertial residuals and leverage the outcome of such algorithm to efficiently compute the global position residuals. The experimental results show that the proposed method achieves accurate and globally consistent estimates, with negligible increase of the optimization computational cost. Our method consistently outperforms the loosely-coupled fusion approach. The mean position error is reduced up to 50% with respect to the loosely-coupled approach in outdoor Unmanned Aerial Vehicle (UAV) flights, where the global position information is given by noisy GPS measurements. To the best of our knowledge, this is the first work where global positional measurements are tightly fused in an optimization-based visual-inertial odometry algorithm, leveraging the IMU preintegration method to define the global positional factors.
Improved Baselines with Momentum Contrastive Learning

Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR's design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.
On the de Rham–Witt Complex over Perfectoid Rings

Fix an odd prime $p$. The results in this paper are modeled after work of Hesselholt and Hesselholt–Madsen on the $p$-typical absolute de Rham–Witt complex in mixed characteristic. We have two primary results. The 1st result is an exact sequence that describes the kernel of the restriction map on the de Rham–Witt complex over $A$, where $A$ is the ring of integers in an algebraic extension of $\textbf{Q}_p$ or where $A$ is a $p$-torsion-free perfectoid ring. The 2nd result is a description of the $p$-power torsion (and related objects) in the de Rham–Witt complex over $A$, where $A$ is a $p$-torsion-free perfectoid ring containing a compatible system of $p$-power roots of unity. Both of these results are analogous to the results of Hesselholt and Madsen. Our main contribution is the extension of their results to certain perfectoid rings. We also provide algebraic proofs of these results, whereas the proofs of Hesselholt and Madsen used techniques from topology.
Large-scale benchmark study of survival prediction methods using multi-omics data

Abstract Multi-omics data, that is, datasets containing different types of high-dimensional molecular variables, are increasingly often generated for the investigation of various diseases. Nevertheless, questions remain regarding the usefulness of multi-omics data for the prediction of disease outcomes such as survival time. It is also unclear which methods are most appropriate to derive such prediction models. We aim to give some answers to these questions through a large-scale benchmark study using real data. Different prediction methods from machine learning and statistics were applied on 18 multi-omics cancer datasets (35 to 1000 observations, up to 100 000 variables) from the database ‘The Cancer Genome Atlas’ (TCGA). The considered outcome was the (censored) survival time. Eleven methods based on boosting, penalized regression and random forest were compared, comprising both methods that do and that do not take the group structure of the omics variables into account. The Kaplan–Meier estimate and a Cox model using only clinical variables were used as reference methods. The methods were compared using several repetitions of 5-fold cross-validation. Uno’s C-index and the integrated Brier score served as performance metrics. The results indicate that methods taking into account the multi-omics structure have a slightly better prediction performance. Taking this structure into account can protect the predictive information in low-dimensional groups—especially clinical variables—from not being exploited during prediction. Moreover, only the block forest method outperformed the Cox model on average, and only slightly. This indicates, as a by-product of our study, that in the considered TCGA studies the utility of multi-omics data for prediction purposes was limited. Contact: moritz.herrmann@stat.uni-muenchen.de, +49 89 2180 3198 Supplementary information: Supplementary data are available at Briefings in Bioinformatics online. All analyses are reproducible using R code freely available on Github.