nils-herrmann/clean_scirepeval huggingface dataset:
Balancing Reconstruction Quality and Regularisation in ELBO for VAEs

A trade-off exists between reconstruction quality and the prior regularisation in the Evidence Lower Bound (ELBO) loss that Variational Autoencoder (VAE) models use for learning. There are few satisfactory approaches to deal with a balance between the prior and reconstruction objective, with most methods dealing with this problem through heuristics. In this paper, we show that the noise variance (often set as a fixed value) in the Gaussian likelihood p(x|z) for real-valued data can naturally act to provide such a balance. By learning this noise variance so as to maximise the ELBO loss, we automatically obtain an optimal trade-off between the reconstruction error and the prior constraint on the posteriors. This variance can be interpreted intuitively as the necessary noise level for the current model to be the best explanation of the observed dataset. Further, by allowing the variance inference to be more flexible it can conveniently be used as an uncertainty estimator for reconstructed or generated samples. We demonstrate that optimising the noise variance is a crucial component of VAE learning, and showcase the performance on MNIST, Fashion MNIST and CelebA datasets. We find our approach can significantly improve the quality of generated samples whilst maintaining a smooth latent-space manifold to represent the data. The method also offers an indication of uncertainty in the final generative model.
Bergman kernel on Riemann surfaces and Kähler metric on symmetric products

Let  see text] be a compact hyperbolic Riemann surface equipped with the Poincaré metric. For any integer  see text], we investigate the Bergman kernel associated to the holomorphic Hermitian line bundle  see text], where  see text] is the holomorphic cotangent bundle of . Our first main result estimates the corresponding Bergman metric on  see text] in terms of the Poincaré metric. We then consider a certain natural embedding of the symmetric product of  see text] into a Grassmannian parametrizing subspaces of fixed dimension of the space of all global holomorphic sections of . The Fubini–Study metric on the Grassmannian restricts to a Kähler metric on the symmetric product of  see text]. The volume form for this restricted metric on the symmetric product is estimated in terms of the Bergman kernel of  see text] and the volume form for the orbifold Kähler form on the symmetric product given by the Poincaré metric on
Tehran Stock Exchange Prediction Using Sentiment Analysis of Online Textual Opinions

In this paper, we investigate the impact of the social media data in predicting the Tehran Stock Exchange (TSE) variables for the first time. We consider the closing price and daily return of three different stocks for this investigation. We collected our social media data from this http URL for about three months. To extract information from online comments, we propose a hybrid sentiment analysis approach that combines lexicon-based and learning-based methods. Since lexicons that are available for the Persian language are not practical for sentiment analysis in the stock market domain, we built a particular sentiment lexicon for this domain. After designing and calculating daily sentiment indices using the sentiment of the comments, we examine their impact on the baseline models that only use historical market data and propose new predictor models using multi regression analysis. In addition to the sentiments, we also examine the comments volume and the users' reliabilities. We conclude that the predictability of various stocks in TSE is different depending on their attributes. Moreover, we indicate that for predicting the closing price only comments volume and for predicting the daily return both the volume and the sentiment of the comments could be useful. We demonstrate that Users' Trust coefficients have different behaviors toward the three stocks.
Composing Knowledge Graph Embeddings via Word Embeddings

Learning knowledge graph embedding from an existing knowledge graph is very important to knowledge graph completion. For a fact $(h,r,t)$ with the head entity $h$ having a relation $r$ with the tail entity $t$, the current approaches aim to learn low dimensional representations $(\mathbf{h},\mathbf{r},\mathbf{t})$, each of which corresponds to the elements in $(h, r, t)$, respectively. As $(\mathbf{h},\mathbf{r},\mathbf{t})$ is learned from the existing facts within a knowledge graph, these representations can not be used to detect unknown facts (if the entities or relations never occur in the knowledge graph). 
This paper proposes a new approach called TransW, aiming to go beyond the current work by composing knowledge graph embeddings using word embeddings. Given the fact that an entity or a relation contains one or more words (quite often), it is sensible to learn a mapping function from word embedding spaces to knowledge embedding spaces, which shows how entities are constructed using human words. More importantly, composing knowledge embeddings using word embeddings makes it possible to deal with the emerging new facts (either new entities or relations). Experimental results using three public datasets show the consistency and outperformance of the proposed TransW.
Subjectivity Learning Theory towards Artificial General Intelligence

The construction of artificial general intelligence (AGI) was a long-term goal of AI research aiming to deal with the complex data in the real world and make reasonable judgments in various cases like a human. However, the current AI creations, referred to as "Narrow AI", are limited to a specific problem. The constraints come from two basic assumptions of data, which are independent and identical distributed samples and single-valued mapping between inputs and outputs. We completely break these constraints and develop the subjectivity learning theory for general intelligence. We assign the mathematical meaning for the philosophical concept of subjectivity and build the data representation of general intelligence. Under the subjectivity representation, then the global risk is constructed as the new learning goal. We prove that subjectivity learning holds a lower risk bound than traditional machine learning. Moreover, we propose the principle of empirical global risk minimization (EGRM) as the subjectivity learning process in practice, establish the condition of consistency, and present triple variables for controlling the total risk bound. The subjectivity learning is a novel learning theory for unconstrained real data and provides a path to develop AGI.
HoughNet: Neural Network Architecture for Vanishing Points Detection

In this paper we introduce a novel neural network architecture based on Fast Hough Transform layer. The layer of this type allows our neural network to accumulate features from linear areas across the entire image instead of local areas. We demonstrate its potential by solving the problem of vanishing points detection in the images of documents. Such problem occurs when dealing with camera shots of the documents in uncontrolled conditions. In this case, the document image can suffer several specific distortions including projective transform. To train our model, we use MIDV-500 dataset and provide testing results. Strong generalization ability of the suggested method is proven with its applying to a completely different ICDAR 2011 dewarping contest. In previously published papers considering this dataset authors measured quality of vanishing point detection by counting correctly recognized words with open OCR engine Tesseract. To compare with them, we reproduce this experiment and show that our method outperforms the state-of-the-art result.
Learning Concepts Definable in First-Order Logic with Counting

We study classification problems over relational background structures for hypotheses that are defined using logics with counting. The aim of this paper is to find learning algorithms running in time sublinear in the size of the background structure. We show that hypotheses defined by FOCN(P)-formulas over structures of polylogarithmic degree can be learned in sublinear time. Furthermore, we prove that for structures of unbounded degree there is no sublinear learning algorithm for first-order formulas.
Comparing galaxy clustering in Horizon-AGN simulated light-cone mocks and VIDEO observations

Hydrodynamical cosmological simulations have recently made great advances in reproducing galaxy mass assembly over cosmic time – as often quantified from the comparison of their predicted stellar mass functions to observed stellar mass functions from data. In this paper, we compare the clustering of galaxies from the hydrodynamical cosmological simulated light-cone Horizon-AGN to clustering measurements from the VIDEO survey observations. Using mocks built from a VIDEO-like photometry, we first explore the bias introduced into clustering measurements by using stellar masses and redshifts derived from spectral energy distribution fitting, rather than the intrinsic values. The propagation of redshift and mass statistical and systematic uncertainties in the clustering measurements causes us to underestimate the clustering amplitude. We then find that clustering and halo occupation distribution (HOD) modelling results are qualitatively similar in Horizon-AGN and VIDEO. However, at low stellar masses, Horizon-AGN underestimates the observed clustering by up to a factor of ∼3, reflecting the known excess stellar mass to halo mass ratio for Horizon-AGN low-mass haloes, already discussed in previous works. This reinforces the need for stronger regulation of star formation in low-mass haloes in the simulation. Finally, the comparison of the stellar mass to halo mass ratio in the simulated catalogue, inferred from angular clustering, to that directly measured from the simulation validates HOD modelling of clustering as a probe of the galaxy–halo connection.
Distributed creation of Machine learning agents for Blockchain analysis

Creating efficient deep neural networks involves repetitive manual optimization of the topology and the hyperparameters. This human intervention significantly inhibits the process. Recent publications propose various Neural Architecture Search (NAS) algorithms that automate this work. We have applied a customized NAS algorithm with network morphism and Bayesian optimization to the problem of cryptocurrency predictions, where it achieved results on par with our best manually designed models. This is consistent with the findings of other teams, while several known experiments suggest that given enough computing power, NAS algorithms can surpass state-of-the-art neural network models designed by humans. In this paper, we propose a blockchain network protocol that incentivises independent computing nodes to run NAS algorithms and compete in finding better neural network models for a particular task. If implemented, such network can be an autonomous and self-improving source of machine learning models, significantly boosting and democratizing the access to AI capabilities for many industries.
The sigma function over a family of cyclic trigonal curves with a singular fiber

In this paper we investigate the behavior of the sigma function over the family of cyclic trigonal curves $X_s$ defined by the equation $y^3 =x(x-s)(x-b_1)(x-b_2)$ in the affine $(x,y)$ plane, for $s\in D_\varepsilon:=\{s \in \mathbb{C} | |s|<\varepsilon\}$. We compare the sigma function over the punctured disc $D_\varepsilon^*:=D_\varepsilon\setminus\{0\}$ with the extension over $s=0$ that specializes to the sigma function of the normalization $X_{\hat{0}}$ of the singular curve $X_{s=0}$ by investigating explicitly the behavior of a basis of the first algebraic de Rham cohomology group and its period integrals. We demonstrate, using modular properties, that sigma, unlike the theta function, has a limit. In particular, we obtain the limit of the theta characteristics and an explicit description of the theta divisor translated by the Riemann constant.
Bayesian Design of Experiments: Implementation, Validation and Application to Chemical Kinetics

Bayesian experimental design (BED) is a tool for guiding experiments founded on the principle of expected information gain. I.e., which experiment design will inform the most about the model can be predicted before experiments in a laboratory are conducted. BED is also useful when specific physical questions arise from the model which are answered from certain experiments but not from other experiments. BED can take two forms, and these two forms are expressed in three example models in this work. The first example takes the form of a Bayesian linear regression, but also this example is a benchmark for checking numerical and analytical solutions. One of two parameters is an estimator of the synthetic experimental data, and the BED task is choosing among which of the two parameters to inform (limited experimental observability). The second example is a chemical reaction model with a parameter space of informed reaction free energy and temperature. The temperature is an independent experimental design variable explored for information gain. The second and third examples are of the form of adjusting an independent variable in the experimental setup. The third example is a catalytic membrane reactor similar to a plug-flow reactor. For this example, a grid search over the independent variables, temperature and volume, for the greatest information gain is conducted. Also, maximum information gain is conducted is optimized with two algorithms: the differential evolution algorithm and steepest ascent, both of which benefitted in terms of initial guess from the grid search.
Symmetry in Multiple Self-Consistent-Field Solutions of Transition-Metal Complexes.

We use a method based on metadynamics to locate multiple low-energy Unrestricted Hartree-Fock (UHF) self-consistent-field (SCF) solutions of two model octahedral d1 and d2 transition-metal complexes, [MF6]3- (M = Ti, V). By giving a group-theoretical definition of symmetry breaking, we classify these solutions in the framework of representation theory and observe that a number of them break spin or spatial symmetry, if not both. These solutions seem unphysical at first, but we show that they can be used as bases for Non-Orthogonal Configuration Interaction (NOCI) to yield multi-determinantal wavefunctions that have the right symmetry to be assigned to electronic terms. Furthermore, by examining the natural orbitals and occupation numbers of these NOCI wavefunctions, we gain insight into the amount of static correlation that they incorporate. We then investigate the behaviors of the most low-lying UHF and NOCI wavefunctions when the octahedral symmetry of the complexes is lowered and deduce that the symmetry-broken UHF solutions must first have their symmetry restored by NOCI before they can describe any vibronic stabilization effects dictated by the Jahn-Teller theorem.
SE-SLAM: Semi-Dense Structured Edge-Based Monocular SLAM

Vision-based Simultaneous Localization And Mapping (VSLAM) is a mature problem in Robotics. Most VSLAM systems are feature based methods, which are robust and present high accuracy, but yield sparse maps with limited application for further navigation tasks. Most recently, direct methods which operate directly on image intensity have been introduced, capable of reconstructing richer maps at the cost of higher processing power. In this work, an edge-based monocular SLAM system (SE-SLAM) is proposed as a middle point: edges present good localization as point features, while enabling a structural semidense map reconstruction. However, edges are not easy to associate, track and optimize over time, as they lack descriptors and biunivocal correspondence, unlike point features. To tackle these issues, this paper presents a method to match edges between frames in a consistent manner; a feasible strategy to solve the optimization problem, since its size rapidly increases when working with edges; and the use of non-linear optimization techniques. The resulting system achieves comparable precision to state of the art feature-based and dense/semi-dense systems, while inherently building a structural semi-dense reconstruction of the environment, providing relevant structure data for further navigation algorithms. To achieve such accuracy, state of the art non-linear optimization is needed, over a continuous feed of 10000 edgepoints per frame, to optimize the full semi-dense output. Despite its heavy processing requirements, the system achieves near to real-time operation, thanks to a custom built solver and parallelization of its key stages. In order to encourage further development of edge-based SLAM systems, SE-SLAM source code will be released as open source.
Search for Dark Matter Annihilation to Neutrinos from the Sun

Weakly interacting massive particles (WIMPs) can be gravitationally captured by the Sun and trapped in its core. The annihilation of those WIMPs into Standard Model particles produces a spectrum of neutrinos whose energy distribution is related to the dark matter mass. In this work, we present the theoretical framework for relating an observed neutrino flux to the WIMP-nucleon cross section and summarize a previous solar WIMP search carried out by IceCube. We then outline an ongoing updated solar WIMP search, focusing on improvements over the previous search.
GAN-Leaks: A Taxonomy of Membership Inference Attacks against GANs

In recent years, the success of deep learning has carried over from discriminative models to generative models. In particular, generative adversarial networks (GANs) have facilitated a new level of performance ranging from media manipulation to dataset re-generation. Despite the success, the potential risks of privacy breach stemming from GANs are less well explored. In this paper, we focus on membership inference attack against GANs that has the potential to reveal information about victim models' training data. Specifically, we present the first taxonomy of membership inference attacks, which encompasses not only existing attacks but also our novel ones. We also propose the first generic attack model that can be instantiated in various settings according to adversary's knowledge about the victim model. We complement our systematic analysis of attack vectors with a comprehensive experimental study, that investigates the effectiveness of these attacks w.r.t. model type, training configurations, and attack type across three diverse application scenarios ranging from images, over medical data to location data. We show consistent effectiveness in all the setups, which bridges the assumption gap and performance gap in previous study with a complete spectrum of performance across settings. We conclusively remind users to think over before publicizing any part of their models.
Differentially Private Algorithms for Learning Mixtures of Separated Gaussians

Learning the parameters of Gaussian mixture models is a fundamental and widely studied problem with numerous applications. In this work, we give new algorithms for learning the parameters of a high-dimensional, well separated, Gaussian mixture model subject to the strong constraint of differential privacy. In particular, we give a differentially private analogue of the algorithm of Achlioptas and McSherry. Our algorithm has two key properties not achieved by prior work: (1) The algorithm’s sample complexity matches that of the corresponding non-private algorithm up to lower order terms in a wide range of parameters. (2) The algorithm does not require strong a priori bounds on the parameters of the mixture components.
HELP: a catalogue of 170 million objects, selected at 0.36–4.5 μm, from 1270 deg2 of prime extragalactic fields

We present an optical to near-infrared (NIR) selected astronomical catalogue covering 1270 deg2. This is the first attempt to systematically combine data from 23 of the premier extragalactic survey fields – the product of a vast investment of telescope time. The fields are those imaged by the Herschel Space Observatory that form the Herschel Extragalactic Legacy Project (HELP). Our catalogue of 170 million objects is constructed by a positional cross-match of 51 public surveys. This high-resolution optical, NIR, and mid-infrared catalogue is designed for photometric redshift estimation, extraction of fluxes in lower resolution far-infrared maps, and spectral energy distribution modelling. It collates, standardizes, and provides value added derived quantities including corrected aperture magnitudes and astrometry correction over the Herschel extragalactic wide fields for the first time. $grizy$ fluxes are available on all fields with g-band data reaching $5\sigma$ point-source depths in a 2 arcsec aperture of 23.5, 24.4, and 24.6 (AB) mag at the 25th, 50th, and 75th percentiles, by area covered, across all HELP fields. It has K or $K_s$ coverage over 1146 deg2 with depth percentiles of 20.2, 20.4, and 21.0 mag, respectively. The IRAC Ch 1 band is available over 273 deg2 with depth percentiles of 17.7, 21.4, and 22.2 mag, respectively. This paper defines the ‘masterlist’ objects for the first data release (DR1) of HELP. This large sample of standardized total and corrected aperture fluxes, uniform quality flags, and completeness measures provides large well-understood statistical samples over the full Herschel extragalactic sky.
Forecaster: A Graph Transformer for Forecasting Spatial and Time-Dependent Data

Spatial and time-dependent data is of interest in many applications. This task is difficult due to its complex spatial dependency, long-range temporal dependency, data non-stationarity, and data heterogeneity. To address these challenges, we propose Forecaster, a graph Transformer architecture. Specifically, we start by learning the structure of the graph that parsimoniously represents the spatial dependency between the data at different locations. Based on the topology of the graph, we sparsify the Transformer to account for the strength of spatial dependency, long-range temporal dependency, data non-stationarity, and data heterogeneity. We evaluate Forecaster in the problem of forecasting taxi ride-hailing demand and show that our proposed architecture significantly outperforms the state-of-the-art baselines.
Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum

ImageNet pre-training has been regarded as essential for training accurate object detectors for a long time. Recently, it has been shown that object detectors trained from randomly initialized weights can be on par with those fine-tuned from ImageNet pre-trained models. However, the effects of pre-training and the differences caused by pre-training are still not fully understood. In this paper, we analyze the eigenspectrum dynamics of the covariance matrix of each feature map in object detectors. Based on our analysis on ResNet-50, Faster R-CNN with FPN, and Mask R-CNN, we show that object detectors trained from ImageNet pre-trained models and those trained from scratch behave differently from each other even if both object detectors have similar accuracy. Furthermore, we propose a method for automatically determining the widths (the numbers of channels) of object detectors based on the eigenspectrum. We train Faster R-CNN with FPN from randomly initialized weights, and show that our method can reduce ~27% of the parameters of ResNet-50 without increasing Multiply-Accumulate operations and losing accuracy. Our results indicate that we should develop more appropriate methods for transferring knowledge from image classification to object detection (or other tasks).