title: Naive Bayes for regression

authors: Frank, Eibe

subjects: Science & Technology

abstract: Despite its simplicity, the naive Bayes learning scheme performs well on most classification tasks, and is often significantly more accurate than more sophisticated methods. Although the probability estimates that it produces can be inaccurate, it often assigns maximum probability to the correct class. This suggests that its good performance might be restricted to situations where the output is categorical. It is therefore interesting to see how it performs in domains where the predicted value is numeric, because in this case, predictions are more sensitive to inaccurate probability estimates.
This paper shows how to apply the naive Bayes methodology to numeric prediction (i.e., regression) tasks by modeling the probability distribution of the target value with kernel density estimators, and compares it to linear regression, locally weighted linear regression, and a method that produces “model trees”—decision trees with linear regression functions at the leaves. Although we exhibit an artificial dataset for which naive Bayes is the method of choice, on real-world datasets it is almost uniformly worse than locally weighted linear regression and model trees. The comparison with linear regression depends on the error measure: for one measure naive Bayes performs similarly, while for another it is worse. We also show that standard naive Bayes applied to regression problems by discretizing the target value performs similarly badly.We then present empirical evidence that isolates naive Bayes’ independence assumption as the culprit for its poor performance in the regression setting. These results indicate that the simplistic statistical assumption that naive Bayes makes is indeed more restrictive for regression than for classification.

text: Naive Bayes for regression Despite its simplicity, the naive Bayes learning scheme performs well on most classification tasks, and is often significantly more accurate than more sophisticated methods. Although the probability estimates that it produces can be inaccurate, it often assigns maximum probability to the correct class. This suggests that its good performance might be restricted to situations where the output is categorical. It is therefore interesting to see how it performs in domains where the predicted value is numeric, because in this case, predictions are more sensitive to inaccurate probability estimates.
This paper shows how to apply the naive Bayes methodology to numeric prediction (i.e., regression) tasks by modeling the probability distribution of the target value with kernel density estimators, and compares it to linear regression, locally weighted linear regression, and a method that produces “model trees”—decision trees with linear regression functions at the leaves. Although we exhibit an artificial dataset for which naive Bayes is the method of choice, on real-world datasets it is almost uniformly worse than locally weighted linear regression and model trees. The comparison with linear regression depends on the error measure: for one measure naive Bayes performs similarly, while for another it is worse. We also show that standard naive Bayes applied to regression problems by discretizing the target value performs similarly badly.We then present empirical evidence that isolates naive Bayes’ independence assumption as the culprit for its poor performance in the regression setting. These results indicate that the simplistic statistical assumption that naive Bayes makes is indeed more restrictive for regression than for classification.

year: 2000-10-01

title: Why size maters: Investigating the drivers of innovation and economic performance in New Zealand using the business operation survey

authors: Hong, Shangqin

subjects: innovation

abstract: The economic performance of the New Zealand economy is something of an enigma. Although ranked number one (of 144 countries) for four important 'growth fundamentals' New Zealand is 'middle of the pack' when it comes to economic growth, productivity and innovation. So what is missing in this story of New Zealand performance? Using three iterations (2005, 2007 and 2009) of the Business Operations Survey, the paper seeks to answer the question using a bivariate probit regression (biprobit) approach applied to samples in excess of 2,000 unit record observations of New Zealand firms. The results suggest that factors such as firm size, high perceived quality product, investment/R&D capability, major technology change, application of formal IP protection and new export markets are systematically and positively related to innovation; while many external issues such as those related to geography, market structure, business environment, appear to have little influence. At the firm level, innovations in New Zealand are highly dependent on the firms’ internal ability to develop new technologies and market demand. (Small) size does matter in New Zealand where ultimately government may need to be involved to maintain a viable (minimum) scale for domestic R&D.

text: Why size maters: Investigating the drivers of innovation and economic performance in New Zealand using the business operation survey The economic performance of the New Zealand economy is something of an enigma. Although ranked number one (of 144 countries) for four important 'growth fundamentals' New Zealand is 'middle of the pack' when it comes to economic growth, productivity and innovation. So what is missing in this story of New Zealand performance? Using three iterations (2005, 2007 and 2009) of the Business Operations Survey, the paper seeks to answer the question using a bivariate probit regression (biprobit) approach applied to samples in excess of 2,000 unit record observations of New Zealand firms. The results suggest that factors such as firm size, high perceived quality product, investment/R&D capability, major technology change, application of formal IP protection and new export markets are systematically and positively related to innovation; while many external issues such as those related to geography, market structure, business environment, appear to have little influence. At the firm level, innovations in New Zealand are highly dependent on the firms’ internal ability to develop new technologies and market demand. (Small) size does matter in New Zealand where ultimately government may need to be involved to maintain a viable (minimum) scale for domestic R&D.

year: 2013-09

title: Te Puea Herangi: an ethno-biographical study

authors: King, Michael

abstract: Early explanations of Maori leadership (for example Buck 1950) tended to emphasize the principle of male primogeniture - the expectation that community leadership would in the normal course of affairs pass to the eldest sons of lines of first-born males. Later studies of Western-educated leaders and those with a non-traditional power base (see Butterworth 1969, Cody 1953, Condliffe 1971, and Henderson 1972) continued to emphasize male primacy. Against the tenor of these studies, however, is the fact that in the first half of the twentieth century the Maori King Movement was dominated and welded together by a woman who preached male primogeniture though her life seemed to contradict the principle; Te Puea Herangi was not of a senior descent line and neither compensated for its absence by a substantial degree of European education nor consolidated her position by marriage. Nor was she simply a forceful leader in Maori terms only: Professor J.G.A. Pocock (in The Maori and New Zealand Politics) called her “possibly the most influential woman in our political history”.

This study examines Te Puea's leadership role: how she came to assume it, how she exercised it, and its consequences. Analytical concepts like Max Weber's distinctions between charismatic, traditional and rational-legal authority, have been avoided. The danger of adopting them is that the themes and emphases of research material can be easily and unintentionally warped to fit the model, in the interests of tidiness and intelligibility. Instead, this study proceeds by way of a detailed and chronological examination of the major events of Te Puea's life, and an evaluation of actions and statements as they occur and in the context in which they occur. It is, in other words, a consideration of Te Puea's life in terms of her own concepts, and in relation to the Maori and wider public affairs of her day. Explanations for her behaviour are sought within the boundaries of her own life and culture rather than from the imposition of alien models.

text: Te Puea Herangi: an ethno-biographical study Early explanations of Maori leadership (for example Buck 1950) tended to emphasize the principle of male primogeniture - the expectation that community leadership would in the normal course of affairs pass to the eldest sons of lines of first-born males. Later studies of Western-educated leaders and those with a non-traditional power base (see Butterworth 1969, Cody 1953, Condliffe 1971, and Henderson 1972) continued to emphasize male primacy. Against the tenor of these studies, however, is the fact that in the first half of the twentieth century the Maori King Movement was dominated and welded together by a woman who preached male primogeniture though her life seemed to contradict the principle; Te Puea Herangi was not of a senior descent line and neither compensated for its absence by a substantial degree of European education nor consolidated her position by marriage. Nor was she simply a forceful leader in Maori terms only: Professor J.G.A. Pocock (in The Maori and New Zealand Politics) called her “possibly the most influential woman in our political history”.

This study examines Te Puea's leadership role: how she came to assume it, how she exercised it, and its consequences. Analytical concepts like Max Weber's distinctions between charismatic, traditional and rational-legal authority, have been avoided. The danger of adopting them is that the themes and emphases of research material can be easily and unintentionally warped to fit the model, in the interests of tidiness and intelligibility. Instead, this study proceeds by way of a detailed and chronological examination of the major events of Te Puea's life, and an evaluation of actions and statements as they occur and in the context in which they occur. It is, in other words, a consideration of Te Puea's life in terms of her own concepts, and in relation to the Maori and wider public affairs of her day. Explanations for her behaviour are sought within the boundaries of her own life and culture rather than from the imposition of alien models.

year: 1977