Information gain ratio - wikipedia Information gain ratio This article does not cite any sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. ( November 2008 ) In decision tree learning, Information gain ratio is a ratio of information gain to the intrinsic information. It is used to reduce a bias towards multi-valued attributes by taking the number and size of branches into account when choosing an attribute. Information Gain is also known as Mutual Information. Contents 1 Information gain calculation 2 Intrinsic value calculation 3 Information gain ratio calculation 4 Advantages 5 References Information gain calculation Let A t t r ( displaystyle Attr ) be the set of all attributes and E x ( displaystyle Ex ) the set of all training examples, v a l u e ( x, a ) ( displaystyle value ( x, a ) ) with x ∈ E x ( displaystyle x in Ex ) defines the value of a specific example x ( displaystyle x ) for attribute a ∈ A t t r ( displaystyle a in Attr ), H ( displaystyle H ) specifies the entropy. The v a l u e s ( a ) ( \textstyle values ( a ) ) function denotes the set of all possible values of attribute a ∈ A t t r ( \textstyle a in Attr ). The information gain for an attribute a ∈ A t t r ( displaystyle a in Attr ) is defined as follows : I G ( E x, a ) = H ( E x ) − ∑ v ∈ v a l u e s ( a ) ( ( x ∈ E x v a l u e ( x, a ) = v ) E x ⋅ H ( ( x ∈ E x v a l u e ( x, a ) = v ) ) ) ( displaystyle IG ( Ex, a ) = H ( Ex ) - sum _ ( v in values ( a ) ) left ( ( \frac ( ( x in Ex value ( x, a ) = v ) ) ( Ex ) ) cdot H ( ( x in Ex value ( x, a ) = v ) ) \right ) ) The information gain is equal to the total entropy for an attribute if for each of the attribute values a unique classification can be made for the result attribute. In this case the relative entropies subtracted from the total entropy are 0. Intrinsic value calculation The intrinsic value for a test is defined as follows : I V ( E x, a ) = − ∑ v ∈ v a l u e s ( a ) ( x ∈ E x v a l u e ( x, a ) = v ) E x ⋅ log 2 ⁡ ( ( x ∈ E x v a l u e ( x, a ) = v ) E x ) ( displaystyle IV ( Ex, a ) = - sum _ ( v in values ( a ) ) ( \frac ( ( x in Ex value ( x, a ) = v ) ) ( Ex ) ) cdot log _ ( 2 ) left ( ( \frac ( ( x in Ex value ( x, a ) = v ) ) ( Ex ) ) \right ) ) Information gain ratio calculation The information gain ratio is just the ratio between the information gain and the intrinsic value : I G R ( E x, a ) = I G / I V ( displaystyle IGR ( Ex, a ) = IG / IV ) Advantages Information gain ratio biases the decision tree against considering attributes with a large number of distinct values. So it solves the drawback of information gain -- namely, information gain applied to attributes that can take on a large number of distinct values might learn the training set too well. For example, suppose that we are building a decision tree for some data describing a business 's customers. Information gain is often used to decide which of the attributes are the most relevant, so they can be tested near the root of the tree. One of the input attributes might be the customer 's credit card number. This attribute has a high information gain, because it uniquely identifies each customer, but we do not want to include it in the decision tree : deciding how to treat a customer based on their credit card number is unlikely to generalize to customers we have n't seen before. References Jump up ^ http://www.ke.tu-darmstadt.de/lehre/archiv/ws0809/mldm/dt.pdf Jump up ^ http://stats.stackexchange.com/questions/13389/information-gain-mutual-information-and-related-measures Retrieved from `` https://en.wikipedia.org/w/index.php?title=Information_gain_ratio&oldid=816275805 '' Categories : Decision trees Classification algorithms Entropy and information Statistical ratios Hidden categories : Articles lacking sources from November 2008 All articles lacking sources Talk Contents About Wikipedia Add links This page was last edited on 20 December 2017, at 09 : 59. About Wikipedia

Question: when is gain ratio required to be calculated
Answer: to reduce a bias towards multi-valued attributes by taking the number and size of branches into account when choosing an attribute