nils-herrmann/clean_scirepeval huggingface dataset:
Régularisation du prix des options : Stacking

The non-parametric modelization of the stock options and other derivatives generated an increased interest over the past years. The goal of this paper is to predict the market price of an option from the same information as needed by the Black-Scholes formula. This is a continuation of more recent papers based on the modelization of these prices by the use of neural networks with a structure inspired by our economic knowledge of option pricing. Our contribution, with this paper, is the successful use of the stacking algorithm to improve the generalization of these models. This algorithm combines two training levels for the models, the second being used to improve the out-of-sample deficits of the first one. The obtained results are very interesting, and span the call options of the S&P 500 between 1987 and 1993. La modelisation non-parametrique du prix des options et autres produits derives a connu un interet croissant au cours des dernieres annees. Ce rapport se situe dans la perspective de predire le prix de l'option au marche a partir des memes informations utilisees dans la formule de Black-Scholes. Il se situe dans la continuation de travaux recents sur la modelisation de ces prix par des reseaux de neurones avec une structure inspiree des connaissances economiques sur la valorisation d'options. La contribution de la recherche presentee ici est l'utilisation avec succes de l'algorithme de Stacking pour ameliorer la generalisation de ces modeles. Cet algorithme combine deux niveaux d'entrainement des modeles, le deuxieme cherchant a combler les deficits hors-echantillon du premier. Les resultats obtenus sont tres interessants et portent sur des options d'achat du S&P 500 entre 1987 et 1993.
Segmentation en thèmes de conversations téléphoniques : traitement en amont pour l’extraction d’information

Nous présentons une approche de découpage thématique que nous utiliserons pour faciliter l’extraction d’information à partir de conversations téléphoniques transcrites. Nous expérimentons avec un modèle de Markov caché utilisant des informations de différents niveaux linguistiques, des marques d’extra-grammaticalités et les entités nommées comme source additionnelle d’information. Nous comparons le modèle obtenu avec notre modèle de base utilisant uniquement les marques linguistiques et les extra-grammaticalités. Les résultats montrent l’efficacité de l’approche utilisant les entités nommées.
Multi-Task Learning For Option Pricing

Multi-task learning is a process used to learn domain-specific bias. It consists in simultaneously training models on different tasks derived from the same domain and forcing them to exchange domain information. This transfer of knowledge is performed by imposing constraints on the parameters defining the models and can lead to improved generalization performance. In this paper, we explore a particular multi-task learning method that forces the parameters of the models to lie on an affine manifold defined in parameter space and embedding domain information. We apply this method to the prediction of the prices of call options on the S&P index for a period of time ranging from 1987 to 1993. An analysis of variance of the results is presented that shows significant improvements of the generalization performance. L'apprentissage multi-tâches est une maniere d'apprendre des particularites d'un domaine (le biais) qui comprend plusieurs tâches possibles. On entraine simultanement plusieurs modeles, un par tâche, en imposant des contraintes sur les parametres de maniere a capturer ce qui est en commun entre les tâches, afin d'obtenir une meilleure generalisation sur chaque tâche, et pour pouvoir rapidement generaliser (avec peu d'exemples) sur une nouvelle tâche provenant du meme domaine. Ici cette commonalite est definie par une variete affine dans l'espace des parametres. Dans cet article, nous appliquons ces methodes a la prediction du prix d'options d'achat de l'indice S&P 500 entre 1987 et 1993. Une analyse de la variance des resultats est presentee, demontrant des ameliorations significatives de la prediction hors-echantillon.
Experiments on the application of IOHMMs to model financial returns series

Input-output hidden Markov models (IOHMM) are conditional hidden Markov models in which the emission (and possibly the transition) probabilities can be conditioned on an input sequence. For example, these conditional distributions can be linear, logistic, or nonlinear (using for example multilayer neural networks). We compare the generalization performance of several models which are special cases of input-output hidden Markov models on financial time-series prediction tasks: an unconditional Gaussian, a conditional linear Gaussian, a mixture of Gaussians, a mixture of conditional linear Gaussians, a hidden Markov model, and various IOHMMs. The experiments compare these models on predicting the conditional density of returns of market and sector indices. Note that the unconditional Gaussian estimates the first moment with the historical average. The results show that, although for the first moment the historical average gives the best results, for the higher moments, the IOHMMs yielded significantly better performance, as estimated by the out-of-sample likelihood.
EVALUATING LONG-TERM DEPENDENCYBENCHMARK PROBLEMS BY RANDOM GUESSINGJ

Numerous recent papers focus on standard recurrent nets' problems with tasks involving long-term dependencies. We solve such tasks by random weight guessing (RG). Although RG cannot be viewed as a reasonable learning algorithm we nd that it often outperforms previous, more complex methods on widely used benchmark problems. One reason for RG's success is that the solutions to many of these benchmarks are dense in weight space. An analysis of cases in which RG works well versus those in which it does not can serve to improve the quality of benchmarks for novel recurrent net algorithms.
Estimating Car Insurance Premia: a Case Study in High-Dimensional Data Inference

Estimating insurance premia from data is a difficult regression problem for several reasons: the large number of variables, many of which are discrete, and the very peculiar shape of the noise distribution, asymmetric with fat tails, with a large majority zeros and a few unreliable and very large values. We compare several machine learning methods for estimating insurance premia, and test them on a large data base of car insurance policies. We find that function approximation methods that do not optimize a squared loss, like Support Vector Machines regression, do not work well in this context. Compared methods include decision trees and generalized linear models. The best results are obtained with a mixture of experts, which better identifies the least and most risky contracts, and allows to reduce the median premium by charging more to the most risky customers.
Cost functions and model combination for VaR-based asset allocation using neural networks

We introduce an asset-allocation framework based on the active control of the value-at-risk of the portfolio. Within this framework, we compare two paradigms for making the allocation using neural networks. The first one uses the network to make a forecast of asset behavior, in conjunction with a traditional mean-variance allocator for constructing the portfolio. The second paradigm uses the network to directly make the portfolio allocation decisions. We consider a method for performing soft input variable selection, and show its considerable utility. We use model combination (committee) methods to systematize the choice of hyperparameters during training. We show that committees using both paradigms are significantly outperforming the benchmark market performance.
Input decay: simple and effective soft variable selection

To deal with the overfitting problems that occur when there are not enough examples compared to the number of input variables in supervised learning, traditional approaches are weight decay and greedy variable selection. An alternative that has recently started to attract attention is to keep all the variables but to put more emphasis on the "most useful" ones. We introduce a new regularization method called input decay that exerts more relative penalty on the parameters associated with the inputs that contribute less to the learned function. This method, like weight decay and variable selection, still requires to perform a kind of model selection. Successful comparative experiments with this new method were performed both on a simulated regression task and a real-world financial prediction task.
Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies

D3EGF(FIH)J KMLONPEGQSRPETN UCV.WYX(Z R.[ V R6\M[ X N@]_^O\`JaNcb V RcQ W d EGKeL(^(QgfhKeLOE?i)^(QSj ETNPfPQkRl[ V R)m"[ X ^(KeLOEG^ npo qarpo m"[ X ^(KeLOEG^tsAu EGNPb V ^ v wyx zlwO{(|(}<~OC}((xp{ay.~A}_~ Cl3#|<Azw#|l6 (|  JpfhL XV EG^O QgJ  ETFOR] ^O\JNPb V RcQ X E)ETR 6EGKeLOETNcKMLOE F ETN V RcQgJp^(^OE ZgZ E i ^(Qkj EGNPfhQSRO E OE2m1Jp^ RcNY E VZ sO! ¡ q.n sCD X KGKa8¢EG^ RPNhE¤£ ¥¦Q ZgZ Es m§J^ RPNO E VZ s( ̈ X  EG©#EKas# V ^ V  V s(H a «a¬3­ ®#|.Y ̄y} xa°OC}l{x  yxlY~3{|  ±2Pz   V J Z J U N V fhKTJp^(Q  ETFOR J\ D vYf3RPEGb ́f V ^(§JpbF X RPETN@D KTQEG^(KTE i ^(QSjpEGNPfhQSR4vμJ\ U¶Z JaNPEG^(K·E jYQ V (Q ̧D V ^ R V m V N3R V aOs#1 o ¡Ga r U QNhE^OoTE1⁄4»,] R VZ vC1⁄2 3⁄4  x ± x  #¿ }À 3t}lC}2P}<~ ¬t[ X NPE^§D KeL(b ́Qg(L X
K-Local Hyperplane and Convex Distance Nearest Neighbor Algorithms

Guided by an initial idea of building a complex (non linear) decision surface with maximal local margin in input space, we give a possible geometrical intuition as to why K-Nearest Neighbor (KNN) algorithms often perform more poorly than SVMs on classification tasks. We then propose modified K-Nearest Neighbor algorithms to overcome the perceived problem. The approachis similar in spirit to Tangent Distance, but with invariances inferred from the local neighborhood rather than prior knowledge. Experimental results on real world classification tasks suggest that the modified KNN algorithms often give a dramatic improvement over standard KNN and perform as well or better than SVMs.
Taking on the curse of dimensionality in joint distributions using neural networks

The curse of dimensionality is severe when modeling high-dimensional discrete data: the number of possible combinations of the variables explodes exponentially. In this paper, we propose a new architecture for modeling high-dimensional data that requires resources (parameters and computations) that grow at most as the square of the number of variables, using a multilayer neural network to represent the joint distribution of the variables as the product of conditional distributions. The neural network can be interpreted as a graphical model without hidden random variables, but in which the conditional distributions are tied through the hidden units. The connectivity of the neural network can be pruned by using dependency tests between the variables (thus reducing significantly the number of parameters). Experiments on modeling the distribution of several discrete data sets show statistically significant improvements over other methods such as naive Bayes and comparable Bayesian networks and show that significant improvements can be obtained by pruning the network.
Probabilistic neural network models for sequential data

Artificial neural networks (ANN) can be incorporated into probabilistic models. In this paper we review some of the approaches which have been proposed to incorporate them into probabilistic models of sequential data, such as hidden Markov models (HMM). We also discuss new developments and new ideas in this area, in particular how ANN can be used to model high-dimensional discrete and continuous data to deal with the curse of dimensionality and how the ideas proposed in these models could be applied to statistical language modeling to represent longer-term context than allowed by trigram models, while keeping word-order information.
A Neural Support Vector Network architecture with adaptive kernels

In the Support Vector Machines (SVM) framework, the positive-definite kernel can be seen as representing a fixed similarity measure between two patterns, and a discriminant function is obtained by taking a linear combination of the kernels computed at training examples called support vectors. We investigate learning architectures in which the kernel functions can be replaced by more general similarity measures that can have arbitrary internal parameters. The training criterion used in SVMs is not appropriate for this purpose so we adopt the simple criterion that is generally used when training neural networks for classification tasks. Several experiments are performed which show that such Neural Support Vector Networks perform similarly to SVMs while requiring significantly fewer support vectors, even when the similarity measure has no internal parameters.
Continuous optimization of hyper-parameters

Many machine learning algorithms can be formulated as the minimization of a training criterion which involves a hyper-parameter. This hyper-parameter is usually chosen by trial and error with a model selection criterion. In this paper we present a methodology to optimize several hyper-parameters, based on the computation of the gradient of a model selection criterion with respect to the hyper-parameters. In the case of a quadratic training criterion, the gradient of the selection criterion with respect to the hyper-parameters is efficiently computed by back-propagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyper-parameter gradient involving second derivatives of the training criterion.
A Neural Probabilistic Language Model

A goal of statistical language modeling is to learn the joint probability function of sequences of words in a language. This is intrinsically difficult because of the curse of dimensionality: a word sequence on which the model will be tested is likely to be different from all the word sequences seen during training. Traditional but very successful approaches based on n-grams obtain generalization by concatenating very short overlapping sequences seen in the training set. We propose to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. The model learns simultaneously (1) a distributed representation for each word along with (2) the probability function for word sequences, expressed in terms of these representations. Generalization is obtained because a sequence of words that has never been seen before gets high probability if it is made of words that are similar (in the sense of having a nearby representation) to words forming an already seen sentence. Training such large models (with millions of parameters) within a reasonable time is itself a significant challenge. We report on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach significantly improves on state-of-the-art n-gram models, and that the proposed approach allows to take advantage of longer contexts.
Boosting Neural Networks

Boosting is a general method for improving the performance of learning algorithms. A recently proposed boosting algorithm,  Boost, has been applied with great success to several benchmark machine learning problems using mainly decision trees as base classifiers. In this article we investigate whether Ada Boost also works as well with neural networks, and we discuss the advantages and drawbacks of different versions of the Ada Boost algorithm. In particular, we compare training methods based on sampling the training set and weighting the cost function. The results suggest that random resampling of the training data is not the main explanation of the success of the improvements brought by Ada Boost. This is in contrast to bagging, which directly aims at reducing variance and for which random resampling is essential to obtain the reduction in generalization error. Our system achieves about 1.4 error on a data set of on-line handwritten digits from more than 200 writers. A boosted multilayer network achieved 1.5 error on the UCI letters and 8.1 error on the UCI satellite data set, which is significantly better than boosted decision trees.
Incorporating Second-Order Functional Knowledge for Better Option Pricing

Incorporating prior knowledge of a particular task into the architecture of a learning algorithm can greatly improve generalization performance. We study here a case where we know that the function to be learned is non-decreasing in two of its arguments and convex in one of them. For this purpose we propose a class of functions similar to multi-layer neural networks but (1) that has those properties, (2) is a universal approximator of continuous functions with these and other properties. We apply this new class of functions to the task of modeling the price of call options. Experiments show improvements on regressing the price of call options using the new types of function classes that incorporate the a priori constraints.
Gradient-Based Optimization of Hyperparameters

Many machine learning algorithms can be formulated as the minimization of a training criterion that involves a hyperparameter. This hyperparameter is usually chosen by trial and error with a model selection criterion. In this article we present a methodology to optimize several hyper-parameters, based on the computation of the gradient of a model selection criterion with respect to the hyperparameters. In the case of a quadratic training criterion, the gradient of the selection criterion with respect to the hyperparameters is efficiently computed by backpropagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyper-parameter gradient involving second derivatives of the training criterion.
Bias learning, knowledge sharing

Biasing the hypothesis space of a learner has been shown to improve generalisation performances. Methods for achieving this goal are proposed, that range from deriving and introducing a bias into a learner to automatically learning the bias. In the latter case, most methods learn the bias by simultaneously training several related tasks derived from the same domain and imposing constraints on their parameters. We extend some of the ideas presented in this field and describe a new model that parametrizes the parameters of each task as a function of an affine manifold defined in parameter space and a point lying on the manifold. An analysis of variance on a class of learning tasks is performed that shows some significantly improved performances when using the model.
Modeling High-Dimensional Discrete Data with Multi-Layer Neural Networks

The curse of dimensionality is severe when modeling high-dimensional discrete data: the number of possible combinations of the variables explodes exponentially. In this paper we propose a new architecture for modeling high-dimensional data that requires resources (parameters and computations) that grow only at most as the square of the number of variables, using a multi-layer neural network to represent the joint distribution of the variables as the product of conditional distributions. The neural network can be interpreted as a graphical model without hidden random variables, but in which the conditional distributions are tied through the hidden units. The connectivity of the neural network can be pruned by using dependency tests between the variables. Experiments on modeling the distribution of several discrete data sets show statistically significant improvements over other methods such as naive Bayes and comparable Bayesian networks, and show that significant improvements can be obtained by pruning the network.
Stochastic Learning of Strategic Equilibria for Auctions

This article presents a new application of stochastic adaptive learning algorithms to the computation of strategic equilibria in auctions. The proposed approach addresses the problems of tracking a moving target and balancing exploration (of action space) versus exploitation (of better modeled regions of action space). Neural networks are used to represent a stochastic decision model for each bidder. Experiments confirm the correctness and usefulness of the approach.
Object Recognition with Gradient-Based Learning

Finding an appropriate set of features is an essential problem in the design of shape recognition systems. This paper attempts to show that for recognizing simple objects with high shape variability such as handwritten characters, it is possible, and even advantageous, to feed the system directly with minimally processed images and to rely on learning to extract the right set of features. Convolutional Neural Networks are shown to be particularly well suited to this task. We also show that these networks can be used to recognize multiple objects without requiring explicit segmentation of the objects from their surrounding. The second part of the paper presents the Graph Transformer Network model which extends the applicability of gradient-based learning to systems that use graphs to represents features, objects, and their combinations.
Binary pseudowavelets and applications to bilevel image processing

This paper shows the existence of binary pseudowavelets, bases on the binary domain that exhibit some of the properties of wavelets, such as multiresolution reconstruction and compact support. The binary pseudowavelets are defined on B/sup n/ (binary vectors of length n) and are operated upon with the binary operators logical and, and exclusive or. The forward transform, or analysis, is the decomposition of a binary vector into its constituent binary pseudowavelets. Binary pseudowavelets allow multiresolution, progressive reconstruction of binary vectors by using progressively more coefficients in the inverse transform. Binary pseudowavelets bases, being sparse matrices, also provide for fast transforms; moreover pseudowavelets rely on hardware-friendly operations for efficient software and hardware implementation.
Learning Simple Non Stationarities with Hyper Parameters

We consider sequential data that is sampled from an unknown process so that the data are not necessarily i i d Most approaches to machine learning assume that data points are i i d Instead we consider a measure of generalization that does not make this assumption and we consider in this context a recently proposed approach to opti mizing hyper parameters based on the computation of the gradient of a model selection criterion with respect to hyper parameters Here we use hyper parameters that con trol a function that gives di erent weights to di erent time steps in the historical data sequence The approach is successfully applied to modeling the volatility of stock re turns one month ahead Comparative experiments with more traditional methods are presented