nils-herrmann/clean_scirepeval huggingface dataset:
Differentiable Supervector Extraction for Encoding Speaker and Phrase Information in Text Dependent Speaker Verification

In this paper, we propose a new differentiable neural network alignment mechanism for text-dependent speaker verification which uses alignment models to produce a supervector representation of an utterance. Unlike previous works with similar approaches, we do not extract the embedding of an utterance from the mean reduction of the temporal dimension. Our system replaces the mean by a phrase alignment model to keep the temporal structure of each phrase which is relevant in this application since the phonetic information is part of the identity in the verification task. Moreover, we can apply a convolutional neural network as front-end, and thanks to the alignment process being differentiable, we can train the whole network to produce a supervector for each utterance which will be discriminative with respect to the speaker and the phrase simultaneously. As we show, this choice has the advantage that the supervector encodes the phrase and speaker information providing good performance in text-dependent speaker verification tasks. In this work, the process of verification is performed using a basic similarity metric, due to simplicity, compared to other more elaborate models that are commonly used. The new model using alignment to produce supervectors was tested on the RSR2015-Part I database for text-dependent speaker verification, providing competitive results compared to similar size networks using the mean to extract embeddings.
Modified Causal Forests for Estimating Heterogeneous Causal Effects

Uncovering the heterogeneity of causal effects of policies and business decisions at various levels of granularity provides substantial value to decision makers. This paper develops new estimation and inference procedures for multiple treatment models in a selection-on-observables framework by modifying the Causal Forest approach suggested by Wager and Athey (2018). The new esti-mators have desirable theoretical and computational properties for various aggregation levels of the causal effects. An Empirical Monte Carlo study shows that they may outperform previously suggested estimators. Inference tends to be accurate for effects relating to larger groups and conservative for effects relating to fine levels of granularity. An application to the evaluation of an active labour mar-ket programme shows the value of the new methods for applied research.
Random Projection in Deep Neural Networks

This work investigates the ways in which deep learning methods can benefit from random projection (RP), a classic linear dimensionality reduction method. We focus on two areas where, as we have found, employing RP techniques can improve deep models: training neural networks on high-dimensional data and initialization of network parameters. Training deep neural networks (DNNs) on sparse, high-dimensional data with no exploitable structure implies a network architecture with an input layer that has a huge number of weights, which often makes training infeasible. We show that this problem can be solved by prepending the network with an input layer whose weights are initialized with an RP matrix. We propose several modifications to the network architecture and training regime that makes it possible to efficiently train DNNs with learnable RP layer on data with as many as tens of millions of input features and training examples. In comparison to the state-of-the-art methods, neural networks with RP layer achieve competitive performance or improve the results on several extremely high-dimensional real-world datasets. The second area where the application of RP techniques can be beneficial for training deep models is weight initialization. Setting the initial weights in DNNs to elements of various RP matrices enabled us to train residual deep networks to higher levels of performance.
Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions

VAE requires the standard Gaussian distribution as a prior in the latent space. Since all codes tend to follow the same prior, it often suffers the so-called ”posterior collapse”. To avoid this, this paper introduces the class specific distribution for the latent code. But different from CVAE, we present a method for disentangling the latent space into the label relevant and irrelevant dimensions, zs and zu, for a single input. We apply two separated encoders to map the input into zs and zu respectively, and then give the concatenated code to the decoder to reconstruct the input. The label irrelevant code zu represent the common characteristics of all inputs, hence they are constrained by the standard Gaussian, and their encoder is trained in amortized variational inference way, like VAE. While zs is assumed to follow the Gaussian mixture distribution in which each component corresponds to a particular class. The parameters for the Gaussian components in zs encoder are optimized by the label supervision in a global stochastic way. In theory, we show that our method is actually equivalent to adding a KL divergence term on the joint distribution of zs and the class label c, and it can directly increase the mutual information between zs and the label c. Our model can also be extended to GAN by adding a discriminator in the pixel domain so that it produces high quality and diverse images.
Upper Triangular Linear Relations on Mmultiplicities and the Stanley-Stembridge Conjecture

In 2015, Brosnan and Chow, and independently Guay-Paquet, proved the Shareshian-Wachs conjecture, which links the Stanley-Stembridge conjecture in combinatorics to the geometry of Hessenberg varieties through Tymoczko's permutation group action on the cohomology ring of regular semisimple Hessenberg varieties. In previous work, the authors exploited this connection to prove a graded version of the Stanley-Stembridge conjecture in a special case. In this manuscript, we derive a new set of linear relations satisfied by the multiplicities of certain permutation representations in Tymoczko's representation. We also show that these relations are upper-triangular in an appropriate sense, and in particular, they uniquely determine the multiplicities. As an application of these results, we prove an inductive formula for the multiplicity coefficients corresponding to partitions with a maximal number of parts.
Flat so(p,q)-connections for manifolds of non-Euclidean signature

The well-known fact that S1, S3 and S7 are parallelizable manifolds admitting flat connections is revisited. The role of torsion in the construction of those flat connections is made explicit, and the possibilities allowed by different metric signatures are examined. A necessary condition for parallelizability in an open region is that the torsion tensor must be covariantly constant. This property can be used to obtain a relation between a torsion-free and flat connections. Our treatment covers Riemannian and pseudo-Riemannian (non-Euclidean signature) hyperbolic manifolds of dimensions three and seven. Apart from the spherical cases mentioned above, the explicit flat connections with and p   −  q  =  1 are constructed for the coset manifolds .
Enumeration on Trees with Tractable Combined Complexity and Efficient Updates

We give an algorithm to enumerate the results on trees of monadic second-order (MSO) queries represented by nondeterministic tree automata. After linear time preprocessing (in the input tree), we can enumerate answers with linear delay (in each answer). We allow updates on the tree to take place at any time, and we can then restart the enumeration after logarithmic time in the tree. Further, all our combined complexities are polynomial in the automaton. Our result follows our previous circuit-based enumeration algorithms based on deterministic tree automata, and is also inspired by our earlier result on words and nondeterministic sequential extended variable-set automata in the context of document spanners. We extend these results and combine them with a recent tree balancing scheme by Niewerth, so that our enumeration structure supports updates to the underlying tree in logarithmic time (with leaf insertions, leaf deletions, and node relabelings). Our result implies that, for MSO queries with free first-order variables, we can enumerate the results with linear preprocessing and constant-delay and update the underlying tree in logarithmic time, which improves on several known results for words and trees. Building on lower bounds from data structure research, we also show unconditionally that up to a doubly logarithmic factor the update time of our algorithm is optimal. Thus, unlike other settings, there can be no algorithm with constant update time.
Universal Supervised Learning for Individual Data

Universal supervised learning is considered from an information theoretic point of view following the universal prediction approach, see Merhav and Feder (1998). We consider the standard supervised "batch" learning where prediction is done on a test sample once the entire training data is observed, and the individual setting where the features and labels, both in the training and test, are specific individual quantities. The information theoretic approach naturally uses the self-information loss or log-loss. Our results provide universal learning schemes that compete with a "genie" (or reference) that knows the true test label. In particular, it is demonstrated that the main proposed scheme, termed Predictive Normalized Maximum Likelihood (pNML), is a robust learning solution that outperforms the current leading approach based on Empirical Risk Minimization (ERM). Furthermore, the pNML construction provides a pointwise indication for the learnability of the specific test challenge with the given training examples
On Functional Aggregate Queries with Additive Inequalities

Motivated by fundamental applications in databases and relational machine learning, we formulate and study the problem of answering functional aggregate queries (FAQ) in which some of the input factors are defined by a collection of additive inequalities between variables. We refer to these queries as FAQ-AI for short. To answer FAQ-AI in the Boolean semiring, we define relaxed tree decompositions and relaxed submodular and fractional hypertree width parameters. We show that an extension of the InsideOut algorithm using Chazelle's geometric data structure for solving the semigroup range search problem can answer Boolean FAQ-AI in time given by these new width parameters. This new algorithm achieves lower complexity than known solutions for FAQ-AI. It also recovers some known results in database query answering. Our second contribution is a relaxation of the set of polymatroids that gives rise to the counting version of the submodular width, denoted by #subw. This new width is sandwiched between the submodular and the fractional hypertree widths. Any FAQ and FAQ-AI over one semiring can be answered in time proportional to #subw and respectively to the relaxed version of #subw. We present three applications of our FAQ-AI framework to relational machine learning: k-means clustering, training linear support vector machines, and training models using non-polynomial loss. These optimization problems can be solved over a database asymptotically faster than computing the join of the database relations.
Quantized $SL(2)$ representations of knot groups

For a braided Hopf algebra $A$ with braided commutativity, we introduce the space of $A$ representations of a knot $K$ as a generalization of the $G$ representation space of $K$ defined for a group $G$. By rebuilding the $G$ representation space from the view point of Hopf algebras, it is extended to any braided Hopf algebra with braided commutativity. Applying this theory to $\mathrm{BSL}(2)$ which is the braided quantum $\mathrm{SL}(2)$ introduced by S. Majid, we get the space of $\mathrm{BSL}(2)$ representations. It is a non-commutative algebraic scheme which provides quantized $\mathrm{SL}(2)$ representations of $K$.
Learning Dynamical Demand Response Model in Real-Time Pricing Program

Price responsiveness is a major feature of end use customers (EVCs) that participate in demand response (DR) programs, and has been conventionally modeled with static demand functions, which take the electricity price as the input and the aggregate energy consumption as the output. This, however, neglects the inherent temporal correlation of the EUC behaviors, and may result in large errors when predicting the actual responses of EUCs in real-time pricing (RTP) programs. In this paper, we propose a dynamical DR model so as to capture the temporal behavior of the EUCs. The states in the proposed dynamical DR model can be explicitly chosen, in which case the model can be represented by a linear function or a multi-layer feedforward neural network, or implicitly chosen, in which case the model can be represented by a recurrent neural network or a long short-term memory unit network. In both cases, the dynamical DR model can be learned from historical price and energy consumption data. Numerical simulation illustrated how the states are chosen and also showed the proposed dynamical DR model significantly outperforms the static ones.
The algorithm of formation of a training set for an artificial neural network for image segmentation

This article suggests an algorithm of formation a training set for artificial neural network in case of image segmentation. The distinctive feature of this algorithm is that it using only one image for segmentation. The segmentation performs using three-layer perceptron. The main method of the segmentation is a method of region growing. Neural network is using for get a decision to include pixel into an area or not. Impulse noise is using for generation of a training set. Pixels damaged by noise are not related to the same region. Suggested method has been tested with help of computer experiment in automatic and interactive modes.
The homotopy Leray spectral sequence

In this work, we build a spectral sequence in motivic homotopy that is analogous to both the Serre spectral sequence in algebraic topology and the Leray spectral sequence in algebraic geometry. Here, we focus on laying the foundations necessary to build the spectral sequence and give a convenient description of its $E_2$-page. Our description of the $E_2$-page is in terms of homology of the local system of fibers, which is given using a theory similar to Rost's cycle modules. We close by providing some sample applications of the spectral sequence and some hints at future work.
Uncovering Urban Mobility and City Dynamics from Large-Scale Taxi Origin-Destination (O-D) Trips: Case Study in Washington DC Area

We perform a systematic analysis on the large-scale taxi trip data to uncover urban mobility and city dynamics in multimodal urban transportation environments. As a case study, we use the taxi origin-destination trip data and some additional data sources in Washington DC area. We first study basic characteristics of taxi trips, then focus on five important aspects. Three of them concern urban mobility, which are respectively mobility and cost including effect of traffic congestion, trip safety, and multimodal connectivity; the other two pertain to city dynamics, which are respectively transportation resilience and the relation between trip patterns and land use. For these aspects, we use appropriate statistical methods and geographic techniques to mine patterns and characteristics from taxi trip data for better understanding qualitative and quantitative impacts of the inputs from key stakeholders on available measures of effectiveness on urban mobility and city dynamics, where key stakeholders include road users, system operators, and city. Finally, we briefly summarize our findings and discuss some critical roles and implications of the uncovered patterns and characteristics from the relation between taxi system and key stakeholders. The results can support road users by providing evidence-based information of trip cost, mobility, safety, multimodal connectivity and transportation resilience, can assist taxi drivers and operators to deliver transportation services in a higher quality of mobility, safety and operational efficiency, and can also help city planners and policy makers to transform multimodal transportation and to manage urban resources in a more effective and better way.
Inference and Sampling of K33-free Ising Models

We call an Ising model tractable when it is possible to compute its partition function value (statistical inference) in polynomial time. The tractability also implies an ability to sample configurations of this model in polynomial time. The notion of tractability extends the basic case of planar zero-field Ising models. Our starting point is to describe algorithms for the basic case computing partition function and sampling efficiently. To derive the algorithms, we use an equivalent linear transition to perfect matching counting and sampling on an expanded dual graph. Then, we extend our tractable inference and sampling algorithms to models, whose triconnected components are either planar or graphs of $O(1)$ size. In particular, it results in a polynomial-time inference and sampling algorithms for $K_{33}$ (minor) free topologies of zero-field Ising models - a generalization of planar graphs with a potentially unbounded genus.
Sum-Free Sets of Integers with a Forbidden Sum

A set of integers is sum-free if it contains no solution to the equation $x+y=z$. We study sum-free subsets of the set of integers $[n]=\{1,\ldots,n\}$ for which the integer $2n+1$ cannot be represented as a sum of their elements. We prove a bound of $O(2^{n/3})$ on the number of these sets, which matches, up to a multiplicative constant, the lower bound obtained by considering all subsets of $B_n = \{ \lceil \frac{2}{3}(n+1) \rceil, \ldots, n \}$. A main ingredient in the proof is a stability theorem saying that if a subset of $[n]$ of size close to $|B_n|$ contains only a few subsets that contradict the sum-freeness or the forbidden sum, then it is almost contained in $B_n$. Our results are motivated by the question of counting symmetric complete sum-free subsets of cyclic groups of prime order. The proofs involve Freiman's $3k-4$ theorem, Green's arithmetic removal lemma, and structural results on independent sets in hypergraphs.
Search-Guided, Lightly-supervised Training of Structured Prediction Energy Networks

In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching through the entire output space to find the best output with respect to this reward function is typically intractable. In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction. In particular, this truncated randomized search in the reward function yields previously unknown local improvements, providing effective supervision to SPENs, avoiding their traditional need for labeled training data.
Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification

Text classification must sometimes be applied in a low-resource language with no labeled training data. However, training data may be available in a related language. We investigate whether character-level knowledge transfer from a related language helps text classification. We present a cross-lingual document classification framework (CACO) that exploits cross-lingual subword similarity by jointly training a character-based embedder and a word-based classifier. The embedder derives vector representations for input words from their written forms, and the classifier makes predictions based on the word vectors. We use a joint character representation for both the source language and the target language, which allows the embedder to generalize knowledge about source language words to target language words with similar forms. We propose a multi-task objective that can further improve the model if additional cross-lingual or monolingual resources are available. Experiments confirm that character-level knowledge transfer is more data-efficient than word-level transfer between related languages.
Can VAEs Generate Novel Examples?

An implicit goal in works on deep generative models is that such models should be able to generate novel examples that were not previously seen in the training data. In this paper, we investigate to what extent this property holds for widely employed variational autoencoder (VAE) architectures. VAEs maximize a lower bound on the log marginal likelihood, which implies that they will in principle overfit the training data when provided with a sufficiently expressive decoder. In the limit of an infinite capacity decoder, the optimal generative model is a uniform mixture over the training data. More generally, an optimal decoder should output a weighted average over the examples in the training data, where the magnitude of the weights is determined by the proximity in the latent space. This leads to the hypothesis that, for a sufficiently high capacity encoder and decoder, the VAE decoder will perform nearest-neighbor matching according to the coordinates in the latent space. To test this hypothesis, we investigate generalization on the MNIST dataset. We consider both generalization to new examples of previously seen classes, and generalization to the classes that were withheld from the training set. In both cases, we find that reconstructions are closely approximated by nearest neighbors for higher-dimensional parameterizations. When generalizing to unseen classes however, lower-dimensional parameterizations offer a clear advantage.
Estimating Rationally Inattentive Utility Functions with Deep Clustering for Framing - Applications in YouTube Engagement Dynamics

We consider a framework involving behavioral economics and machine learning. Rationally inattentive Bayesian agents make decisions based on their posterior distribution, utility function and information acquisition cost Renyi divergence which generalizes Shannon mutual information). By observing these decisions, how can an observer estimate the utility function and information acquisition cost? Using deep learning, we estimate framing information (essential extrinsic features) that determines the agent's attention strategy. Then we present a preference based inverse reinforcement learning algorithm to test for rational inattention: is the agent an utility maximizer, attention maximizer, and does an information cost function exist that rationalizes the data? The test imposes a Renyi mutual information constraint which impacts how the agent can select attention strategies to maximize their expected utility. The test provides constructive estimates of the utility function and information acquisition cost of the agent. We illustrate these methods on a massive YouTube dataset for characterizing the commenting behavior of users.
Mixed Membership Recurrent Neural Networks

Models for sequential data such as the recurrent neural network (RNN) often implicitly model a sequence as having a fixed time interval between observations and do not account for group-level effects when multiple sequences are observed. We propose a model for grouped sequential data based on the RNN that accounts for varying time intervals between observations in a sequence by learning a group-level base parameter to which each sequence can revert. Our approach is motivated by the mixed membership framework, and we show how it can be used for dynamic topic modeling in which the distribution on topics (not the topics themselves) are evolving in time. We demonstrate our approach on a dataset of 3.4 million online grocery shopping orders made by 206K customers.