Talk:Random forest

message: First time hearing about this.  Reads a bit like an advertisement, and sort of short on info.  I suspect that the claim of "most accurate classifier" would depend on the data being classified.  I intend to read the link and see if I can contribute a bit. Dsol 09:34, 30 July 2005 (UTC)
replies: text: I agree, the choice of prediction algorithm depends on the nature of the data among many other factors, and claiming random forests are "most accurate" isn't really true. I've updated the article to make a less strong claim. Neilc 22:40, 2 April 2006 (UTC)
thread_title: First

message: The learning process is very well documented, and in the end of it we do get a 'forest' or collection off trees. But what is the decision process, which allows us to classify a new data point once we have the forest?  —Preceding unsigned comment added by Hirak 99 (talk • contribs) 14:21, 15 October 2010 (UTC)
thread_title:  Decision process 

message: The following two bullets were removed from the "disadvantages" section on 10 Sep, 2011 by user 78.42.107.187 with the comment "The statements were unrelated to the given publications.":
* Random forests do not handle large numbers of irrelevant features as well as ensembles of entropy-reducing decision trees.
* It is more efficient to select a random decision boundary than an entropy-reducing decision boundary, thus making larger ensembles more feasible. Although this may seem to be an advantage at first, it has the effect of shifting the computation from training time to evaluation time, which is actually a disadvantage for most applications.
The point expressed in the first bullet is made in Section 4, and is clearly illustrated in Figure 5. Is anyone aware of a better paper for making this point? Even if no suitable reference can be found, I think the bullet should remain since it is demonstrably true. Also, it appears that the second bullet was removed by mistake, since it was not connected to that reference. Thus, I propose restoring both bullets to the disadvantages section. Any objections? --Headlessplatter (talk) 17:29, 12 September 2011 (UTC)
thread_title:  Disadvantages 

message: If one reads the Bayes classifier article, you then have a reasonable chance of banging out a working Bayesan algorithm. I do not believe this is the case for the Random Forest article. I'm not sure one can glean enough information from the description presented here to write code that incorporates the many ideas from the lead paragraph. DavesPlanet (talk) 15:03, 3 January 2012 (UTC)
thread_title:  Algorithm 

message: Random Forests do not overfit. The testing performance of Random Forests does not decrease (due to overfitting) as the number of trees increases. Hence after certain number of trees the performance tend to stay in a certain value. There are tons of examples about that. 
http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#mislabel  — Preceding unsigned comment added by 78.181.156.92 (talk) 12:01, 26 August 2012 (UTC)
replies: text: This is completely false. No algorithm that trains its model is immune to overfit. (That which is considered noise in one problem could be considered signal in another problem, so only a magical algorithm could always discern what the user wants.) A Random Forest with few trees is quite prone to overfit to noise. This is easily demonstrated because RF with just one tree is the same as a single tree. As more trees are added, the tendency to overfit generally decreases. It never, however, approaches zero. No number of trees will ever remove overfit. The RF page here on Wikipedia gives a visual demonstration of this behavior. It is also well documented in published literature. Segal et al. showed that Brieman's choice of datasets lacked significant noise, and that RF does in fact overfit (http://escholarship.org/uc/item/35x3v9t4.pdf). Gashler et al. showed empirically that Random Forest does not handle noise injected into arbitrary datasets as well as entropy-reducing decision trees (http://axon.cs.byu.edu/papers/gashler2008icmla.pdf). This is precisely because RF overfits to the noise, whereas the entropy-reducing trees overfit less. The method of randomly choosing divisions is inherently more susceptible to overfit because it gives equal probability to all attributes. Breiman has a conflict-of-interest in promoting RF, and repeatable experiments show that he is wrong to suggest that RF never overfits. Further, the notion of an algorithm that does not overfit is absurd in context of the No Free Lunchn Theorem. Adding more trees to the ensemble does in fact reduce variance, but it does not correct any bias. Please stop repeating Breiman's false marketing efforts.--Headlessplatter (talk) 13:45, 28 August 2012 (UTC)

text: I am sorry. I thought you were saying RF cannot overfit anything. I see now that you were really saying that additional trees do not promote overfit. I believe that is true. (That is probably what Breiman meant too.) --Headlessplatter (talk) 23:03, 31 August 2012 (UTC)
replies: text: The fact that random forests have proven in practical applications to be often the most successful of all available techniques indicates that this article is providing a biased view by only presenting a toy example where it does very badly against a simpler method. The article urgently needs better examples in order to better inform readers. 31.185.205.225 (talk) 18:38, 26 February 2013 (UTC)
thread_title:  Overfitting 

message: The random naive Bayes article really only has a single reference for the idea of RNB, Prinzie and V.d. Poel 2007, which only has 11 citations according to GScholar. (The other references discuss somewhat similar ideas, such as AODE.) QVVERTYVS (hm?) 10:33, 9 December 2013 (UTC)
replies: text: I'd be in favor of simply deleting the random naive bayes article without merging.  There's no reason to have an article for every different type of model that someone has applied bagging to. Maybe add a sentence here that says something like "Random feature selection and bagging have been applied to other models than trees." Nippashish (talk) 14:02, 19 January 2014 (UTC)
thread_title:  Proposed merge with [[Random naive Bayes]] 

message: The following is a section added by a contributor. I removed it since it omitted some of the algorithmic elements already available in the article. Also, it adds references but without giving the exact source. I am pasting the edit below so others may revise it, and re-insert it to the article:
Tal Galili (talk) 19:34, 14 December 2013 (UTC)
replies: text: Looks like a word-for-word copy of part of the introduction to Breiman's paper on Random Forest.  It's not him, since he's passed away. --C S (talk) 14:58, 16 December 2013 (UTC)
thread_title:  Section to add to the article 

message: The article Kernel random forest discusses some results published on arXiv and in tech reports (not peer-reviewed). I don't think the topic is notable enough to stand on its own, and that it should be discussed in a subsection of random forest, just like the relationship to nearest neighbors. QVVERTYVS (hm?) 15:45, 27 May 2015 (UTC)
replies: text: Agree - happy to do this in the near future Situphobos (talk) 20:48, 10 July 2016 (UTC)
replies: text:  seems that there are no objections to the merge, and that you have the expertise to do it; still interested? Klbrain (talk) 21:43, 18 October 2017 (UTC)
thread_title:  Proposed merge with [[Kernel random forest]] 

message: This article seems to indicate that the bias (more categories -> more important) disappears if you use a "one-hot" instead of "integer" encoding.
:WP:SPS. Wait till it gets published and peer reviewed. QVVERTYVS (hm?) 14:57, 12 August 2015 (UTC)
thread_title:  Variable Importance 

message: The unsupervised learning section is almost entirely copy/pasted from the abstract of reference 21:
Shi, T., Horvath, S. (2006). "Unsupervised Learning with Random Forest Predictors".
Is that an issue? Besides that, the section seems pretty unclear to me. So the method is based on distinguishing observed vs. synthetic data, but how does that give a dissimilarity metric?  — Preceding unsigned comment added by 173.71.96.250 (talk) 12:33, 26 May 2016 (UTC)
thread_title:  The unsupervised learning section 

message: This article has been targeted by an (apparent) campaign to insert "Decision Stream" into various Wikipedia pages about Machine Learning. "Decision Stream" refers to a recently published paper that currently has zero academic citations.  The number of articles that have been specifically edited to include "Decision Stream" within the last couple of months suggests  conflict-of-interest editing by someone who wants to advertise this paper.  They are monitoring these pages and quickly reverting any edits to remove this content.
Known articles targeted:
* Artificial intelligence
* Statistical classification
* Deep learning
* Random forest
* Decision tree learning
* Decision tree
* Pruning (decision trees)
* Predictive analytics
* Chi-square automatic interaction detection 
* MNIST database   — Preceding unsigned comment added by ForgotMyPW (talk • contribs) 17:39, 2 September 2018 (UTC)
thread_title:  "Decision Stream" Editing Campaign 

message: Some very general statements regarding performance comparison of random forest performance versus other classifiers have been recently added by a single user. Those statements are always followed by multiple citations to works by Piryonesi and co-authors related to Pavement Deterioration Modeling. One common characteristic of all these three works is that they are recent and not published in mainstream machine learning journals. As such, they do not seem to represent authoritative references, and rather seem to be an attempt of self-promotion (see Wikipedia:Conflict_of_interest/Noticeboard#Refspam_across_many_articles). I already removed some of these references scattered in the article without success, as the same user who inserted those works in the first place systematically reverted my changes. Unfortunately, there is not much that I can do in the current state, such that further review by a third party is needed.93.13.173.200 (talk) 23:04, 2 March 2021 (UTC)
thread_title:  Statements not supported by sources, possible  [[WP:CITESPAM]]