title: Correlation-based feature selection for machine learning

authors: Hall, Mark A.

abstract: A central problem in machine learning is identifying a representative set of features from which to construct a classification model for a particular task. This thesis addresses the problem of feature selection for machine learning through a correlation based approach. The central hypothesis is that good feature sets contain features that are highly correlated with the class, yet uncorrelated with each other. A feature evaluation formula, based on ideas from test theory, provides an operational definition of this hypothesis. CFS (Correlation based Feature Selection) is an algorithm that couples this evaluation formula with an appropriate correlation measure and a heuristic search strategy. 

CFS was evaluated by experiments on artificial and natural datasets. Three machine learning algorithms were used: C4.5 (a decision tree learner), IB 1 (an instance based learner), and naive Bayes. Experiments on artificial datasets showed that CFS quickly identifies and screens irrelevant, redundant, and noisy features, and identifies relevant features as long as their relevance does not strongly depend on other features. On natural domains, CFS typically eliminated well over half the features. In most cases, classification accuracy using the reduced feature set equaled or bettered accuracy using the complete feature set. Feature selection degraded machine learning performance in cases where some features were eliminated which were highly predictive of very small areas of the instance space. 

Further experiments compared CFS with a wrapper - a well known approach to feature selection that employs the target learning algorithm to evaluate feature sets. In many cases CFS gave comparable results to the wrapper, and in general, outperformed the wrapper on small datasets. CFS executes many times faster than the wrapper, which allows it to scale to larger datasets.

Two methods of extending CFS to handle feature interaction are presented and experimentally evaluated. The first considers pairs of features and the second incorporates feature weights calculated by the RELIEF algorithm. Experiments on artificial domains showed that both methods were able to identify interacting features. On natural domains, the pairwise method gave more reliable results than using weights provided by RELIEF.

text: Correlation-based feature selection for machine learning A central problem in machine learning is identifying a representative set of features from which to construct a classification model for a particular task. This thesis addresses the problem of feature selection for machine learning through a correlation based approach. The central hypothesis is that good feature sets contain features that are highly correlated with the class, yet uncorrelated with each other. A feature evaluation formula, based on ideas from test theory, provides an operational definition of this hypothesis. CFS (Correlation based Feature Selection) is an algorithm that couples this evaluation formula with an appropriate correlation measure and a heuristic search strategy. 

CFS was evaluated by experiments on artificial and natural datasets. Three machine learning algorithms were used: C4.5 (a decision tree learner), IB 1 (an instance based learner), and naive Bayes. Experiments on artificial datasets showed that CFS quickly identifies and screens irrelevant, redundant, and noisy features, and identifies relevant features as long as their relevance does not strongly depend on other features. On natural domains, CFS typically eliminated well over half the features. In most cases, classification accuracy using the reduced feature set equaled or bettered accuracy using the complete feature set. Feature selection degraded machine learning performance in cases where some features were eliminated which were highly predictive of very small areas of the instance space. 

Further experiments compared CFS with a wrapper - a well known approach to feature selection that employs the target learning algorithm to evaluate feature sets. In many cases CFS gave comparable results to the wrapper, and in general, outperformed the wrapper on small datasets. CFS executes many times faster than the wrapper, which allows it to scale to larger datasets.

Two methods of extending CFS to handle feature interaction are presented and experimentally evaluated. The first considers pairs of features and the second incorporates feature weights calculated by the RELIEF algorithm. Experiments on artificial domains showed that both methods were able to identify interacting features. On natural domains, the pairwise method gave more reliable results than using weights provided by RELIEF.

year: 1999

title: Estimating the size of the cosmic-ray halo using particle distribution moments

authors: Litvinenko, Yuri E.

subjects: cosmic rays – galaxies

abstract: Context: Particle transport in many astrophysical problems can be described either by the Fokker–Planck equation or by an equivalent system of stochastic differential equations. Aims: It is shown that the latter method can be applied to the problem of defining the size of the cosmic-ray galactic halo. Methods: Analytical expressions for the leading moments of the pitch-angle distribution of relativistic particles are determined. Particle scattering and escape are analyzed in terms of the moments. Results: In the case of an anisotropic distribution, the first moment leads to an expression for the halo size, identified with the particle escape from the region of strong scattering. Previous studies are generalized by analyzing the case of a strictly isotropic initial distribution. A new expression for the variance of the distribution is derived, which illustrates the anisotropization of the distribution. Conclusions: Stochastic calculus tools allow one to analyze physically motivated forms for the scattering rate, so that a detailed realistic model can be developed.

text: Estimating the size of the cosmic-ray halo using particle distribution moments Context: Particle transport in many astrophysical problems can be described either by the Fokker–Planck equation or by an equivalent system of stochastic differential equations. Aims: It is shown that the latter method can be applied to the problem of defining the size of the cosmic-ray galactic halo. Methods: Analytical expressions for the leading moments of the pitch-angle distribution of relativistic particles are determined. Particle scattering and escape are analyzed in terms of the moments. Results: In the case of an anisotropic distribution, the first moment leads to an expression for the halo size, identified with the particle escape from the region of strong scattering. Previous studies are generalized by analyzing the case of a strictly isotropic initial distribution. A new expression for the variance of the distribution is derived, which illustrates the anisotropization of the distribution. Conclusions: Stochastic calculus tools allow one to analyze physically motivated forms for the scattering rate, so that a detailed realistic model can be developed.

year: 2009

title: The Debate over 'Kai Tangata' (Maori Cannibalism): New Perspectives from the Correspondence of the Marists

authors: Jennings, William

subjects: cannibalism

abstract: French Marist missionaries working in New Zealand from 1838 sometimes mentioned Maori cannibalism in their correspondence. This article finds that the Marists considered cannibalism to be a cultural practice of minor interest, not a defining point of difference between Maori and Europeans. The article also finds some evidence of ritual cannibalism, but not of mass battlefield cannibalism or of the eating of people for food.

text: The Debate over 'Kai Tangata' (Maori Cannibalism): New Perspectives from the Correspondence of the Marists French Marist missionaries working in New Zealand from 1838 sometimes mentioned Maori cannibalism in their correspondence. This article finds that the Marists considered cannibalism to be a cultural practice of minor interest, not a defining point of difference between Maori and Europeans. The article also finds some evidence of ritual cannibalism, but not of mass battlefield cannibalism or of the eating of people for food.

year: 2011