title: Reduced-error pruning with significance tests

authors: Frank, Eibe

subjects: computer science

abstract: When building classification models, it is common practice to prune them to counter spurious effects of the training data: this often improves performance and reduces model size. “Reduced-error pruning” is a fast pruning procedure for decision trees that is known to produce small and accurate trees. Apart from the data from which the tree is grown, it uses an independent “pruning” set, and pruning decisions are based on the model’s error rate on this fresh data. Recently it has been observed that reduced-error pruning overfits the pruning data, producing unnecessarily large decision trees. This paper investigates whether standard statistical significance tests can be used to counter this phenomenon. 
The problem of overfitting to the pruning set highlights the need for significance testing. We investigate two classes of test, “parametric” and “non-parametric.” The standard chi-squared statistic can be used both in a parametric test and as the basis for a non-parametric permutation test. In both cases it is necessary to select the significance level at which pruning is applied. We show empirically that both versions of the chi-squared test perform equally well if their significance levels are adjusted appropriately. Using a collection of standard datasets, we show that significance testing improves on standard reduced error pruning if the significance level is tailored to the particular dataset at hand using cross-validation, yielding consistently smaller trees that perform at least as well and sometimes better.

text: Reduced-error pruning with significance tests When building classification models, it is common practice to prune them to counter spurious effects of the training data: this often improves performance and reduces model size. “Reduced-error pruning” is a fast pruning procedure for decision trees that is known to produce small and accurate trees. Apart from the data from which the tree is grown, it uses an independent “pruning” set, and pruning decisions are based on the model’s error rate on this fresh data. Recently it has been observed that reduced-error pruning overfits the pruning data, producing unnecessarily large decision trees. This paper investigates whether standard statistical significance tests can be used to counter this phenomenon. 
The problem of overfitting to the pruning set highlights the need for significance testing. We investigate two classes of test, “parametric” and “non-parametric.” The standard chi-squared statistic can be used both in a parametric test and as the basis for a non-parametric permutation test. In both cases it is necessary to select the significance level at which pruning is applied. We show empirically that both versions of the chi-squared test perform equally well if their significance levels are adjusted appropriately. Using a collection of standard datasets, we show that significance testing improves on standard reduced error pruning if the significance level is tailored to the particular dataset at hand using cross-validation, yielding consistently smaller trees that perform at least as well and sometimes better.

year: 1999-06

title: Complexes of η⁵, κP-cyclopentadienyl-phosphine ligands by chelate-assisted dehydrofluorinative coupling; the coupling of bis{bis(pentafluorophenyl)phosphino}methane to η⁵-pentamethylcyclopentadienyl in a rhodium complex

authors: Marr, Andrew C.

subjects: Cyclopentadienyl ligands

abstract: The reaction between [{(η⁵-C₅Me₅)RhCl(μ-Cl)}₂] and (C₆F₅)₂PCH₂P(C₆F₅)₂ in refluxing benzene yields the singly linked complex [{(η⁵,κP-C₅Me₄ CH₂C₆F₄-2-P(C₆F₅)CH₂P(C₆F₅)₂)}RhCl₂] in which only one phosphorus atom is coordinated to the rhodium.

text: Complexes of η⁵, κP-cyclopentadienyl-phosphine ligands by chelate-assisted dehydrofluorinative coupling; the coupling of bis{bis(pentafluorophenyl)phosphino}methane to η⁵-pentamethylcyclopentadienyl in a rhodium complex The reaction between [{(η⁵-C₅Me₅)RhCl(μ-Cl)}₂] and (C₆F₅)₂PCH₂P(C₆F₅)₂ in refluxing benzene yields the singly linked complex [{(η⁵,κP-C₅Me₄ CH₂C₆F₄-2-P(C₆F₅)CH₂P(C₆F₅)₂)}RhCl₂] in which only one phosphorus atom is coordinated to the rhodium.

year: 2006

title: Analysis of errors in the writing of first year university students of Māori

authors: Earnshaw, Lorene

subjects: Te reo Māori

abstract: This paper presents a survey of the range of grammatical and lexical errors made in written Māori by University students taking an introductory course in Māori language. We begin by introducing, and discussing the motivation for, an error classification system which accommodates different classes of error. We then provide an analysis of errors in three different types of student writing: homework assignments, impromptu tests and examination transcripts. We conclude with some remarks about the patterns of error which we found.

text: Analysis of errors in the writing of first year university students of Māori This paper presents a survey of the range of grammatical and lexical errors made in written Māori by University students taking an introductory course in Māori language. We begin by introducing, and discussing the motivation for, an error classification system which accommodates different classes of error. We then provide an analysis of errors in three different types of student writing: homework assignments, impromptu tests and examination transcripts. We conclude with some remarks about the patterns of error which we found.

year: 2004-09