<figure><img src="image_1.png" /></figure>

HAL
open science

# A simple introduction to programming and statistics with decision trees in R

Marc Tang

<figure><img src="image_2.png" /></figure>

## To cite this version:

Marc Tang. A simple introduction to programming and statistics with decision trees in R. Teaching Statistics, 2020, 10.1111/test.12210. hal-02529129

HAL Id: hal-02529129

https://hal.science/hal-02529129v1

Submitted on 16 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

ORIGINAL ARTICLE

<figure><img src="image_5.png" /></figure>

# A simple introduction to programming and statistics with decision trees in R

Marc Tang

<figure><img src="image_6.png" /></figure>

Lab Dynamics of Language UMR 5596, CNRS/University of Lyon, France

**Correspondence**

Marc Tang, Lab Dynamics of Language UMR 5596, CNRS/University of Lyon, France.

Email: marc.tang@univ-lyon2.fr

**Summary**

University students in other disciplines without prior knowledge in statistics and/or programming language are introduced to the statistical method of decision trees in the programming language R during a 45-minute teaching and practice session. Statistics and programming skills are now frequently required within a wide variety of research fields and private industries. However, students unfamiliar with these subjects may be reluctant to join a full course because of time or student workloads or other commitments or a belief it is not for them. The proposed session is short and can be used as an ice-breaker to let students have a basic understanding of running statistical models in programming language.

**KEYWORDS**

Teaching, introductory session, R, decision trees, r, teaching statistics

## 1 | INTRODUCTION

Students in other disciplines are generally reluctant to invest time in a full course on programming and/or statistics as they already have an extensive workload (Buckley, Brown, Thomson, Olsen, & Carter,⁵; MacInnes,¹¹). We propose a 45-minute teaching activity that gives university students practical knowledge of complex statistics and programming language. This activity is developed for students in nonquantitative disciplines, either undergraduate or as a small part in research methods, but could be used as an innovative ice-breaker in many student situations. No prior knowledge of statistics and/or programming is required. The teaching materials are based on the programming language R (R Core Team,¹³), which is free and available under an open-access license.

This session can be beneficial for both students and instructors. On the one hand, students can explore programming language and statistics. If they consider it relevant and useful, they can join full courses. On the other hand, instructors may assess the reaction of students and

decide whether it is possible to have regular courses in programming language and statistics at the local institution.

## 2 | WHY R AND DECISION TREES

The use of statistical models has steadily increased in scientific fields such as linguistics, phylogenetics, statistics, among others (Nichols & Warnow,¹², p. 760). Within this novel trend of learning, the most common difficulties of teaching statistics involve the complexity of the task (Gould,⁸) and the fact that statistics is often perceived as a “difficult and unpleasant subject” (Albarracin, Aymerich, & Gorgorio,¹, p. 100; Ben-Zvi & Garfield,²; Carnell,⁶).

To overcome these difficulties, the nonparametric statistical method of decision trees (Breiman,³) is selected. This method requires few theoretical assumptions about the distribution of the population. That is to say, the method can be applied directly on the data (Tagliamonte
<table><caption>**TABLE 1 Schedule of the teaching session**</caption><thead><tr><th>Subject</th><th>Time</th><th>Content</th><th>Functions in R</th></tr></thead><tbody><tr><td>Software</td><td>Presession</td><td>Installation of R</td><td></td></tr><tr><td>Introduction</td><td>3 minutes</td><td>Outline</td><td></td></tr><tr><td rowspan="2">Background</td><td>3 minutes</td><td>Set the context</td><td></td></tr><tr><td>3 minutes</td><td>Introduce the data</td><td></td></tr><tr><td rowspan="6">Practice</td><td>5 minutes</td><td>Load data</td><td>data, str, head</td></tr><tr><td>2 minutes</td><td>Install packages</td><td>install.Packages, library</td></tr><tr><td>12 minutes</td><td>Run decision trees</td><td>ctree</td></tr><tr><td>5 minutes</td><td>Visualize the results</td><td>plot</td></tr><tr><td>3 minutes</td><td>Evaluate decision trees</td><td>predict, table, diag, sum</td></tr><tr><td>4 minutes</td><td>Report the results</td><td></td></tr><tr><td>Conclusion</td><td>5 minutes</td><td>Summary and feedback</td><td></td></tr></tbody></table>

& Baayen,¹⁴, p. 135), which is expected to smooth the teaching and understanding process. Moreover, the output of the method is intuitive to visualize and interpret. R (R Core Team,¹³) is chosen as the programming language in our study since it is free and is commonly considered as one of the “good options for introductory statistic courses” (Howington,¹⁰, p. 86). For instance, complex algorithms can be loaded automatically to avoid the need for students to write overtly complex code themselves.

## 3 | STRUCTURE OF THE TEACHING SESSION

The suggested timetable of the session is displayed in Table 1. The students are provided with a set of data that they load directly in R and use as a sample for statistical analysis decision trees. While an overview of the theoretical framework of decision trees is included, the majority of the session (69%, 31/45 minutes) is dedicated to real-time practice. During the practice period, the instructor shares his/her screen via projector to the students and writes down the necessary code for each step of the process. The students can replicate the content on their own computer. Brief explanation of each function is provided, so that students can understand the main purpose of every symbol in the code. For instance, when writing the function *str*, the instructor explains to students that it is used to display the general structure of a dataset.

As a short overview of the content, students are asked to install R and R studio in their computer before the session. An example of the instructions is available at https://github.com/marctang/ResearchMethod/blob/master/Materials/InstallRandRstudio.pdf. In class, they

start by loading the data that are already installed in R by default and install the *party* package (Hothorn et al.,⁹). Then, the instructor goes through the process of generating a decision tree (function *ctree*) to analyze how the variables included in the data interact with each other. The *plot* function is used to visualize the decision tree. Finally, the performance of the model is evaluated according to its predictive power, ie, how accurate could one get by using the decision tree on the data.

## 4 | CONTENT OF THE INTRODUCTORY SESSION

This section contains the information needed by the instructor to run the session. First, a brief explanation of the decision trees is provided and the data used in this session is presented. Then, the R code for running the decision trees is written and explained. Finally, the output and its summary are given.

### 4.1 | General concept and data

Sets of algorithms that have been tested and published in the R working environment are commonly referred to as packages. The *party* package consists of several algorithms of decision trees (Breiman, Friedman, Stone, & Olshen,⁴). Decisions trees show how the variables of a dataset interact with each other and whether this interaction is statistically significant or not. With this method, the algorithm re-runs several times. Each time, the algorithm selects randomly a subset of the data, (eg, random columns and rows) and tries to find regularities in the data. These regularities are represented as a set of binary splits, which create a decision tree as a final output. If a regularity is found in most of the data subsets, it is
considered to be statistically significant. That is to say, this regularity does not occur by chance.

To demonstrate a real application of decision trees, the information of petal and sepal length/width is used to identify different types of flowers. The concept can be explained to the students as follows: To facilitate the working progress of biologists at the university, different species of flowers need to be identified based on quick observations on the sepal length/size and petal length/size. The students are working as a group of consultants and have to find a statistical explanation to this question. The *iris* data that are by default loaded in R are ideal for this exercise. The dataset was introduced by the British statistician and biologist Ronald Fisher (Fisher,⁷) and is commonly used as a practice dataset for statistics nowadays. The data consist of five features related to flowers: petal length, petal width, sepal length, sepal width, and the identification of species. In this data, three species are included (Setosa, Virginica, and Versicolor) with 50 samples each.

## 4.2 | R code and output

The following code should be written in the console of R studio (following the > sign) and implemented individually by typing each line separately and then pressing the Enter key. The output of some lines is shown and highlighted with the "##" signs in this paper, so that the instructor can check that the code generates the expected results. Students should also be warned that the code is case-sensitive and that wrong input results in error messages in the console, eg, `Error: object 'iri' not found.`

First, the data are loaded by using a function. A function can be explained as a command given to the computer. Functions are written in R by writing the name of the function, followed by a parenthesis that includes details about the command. For instance, the function *data* are used to load data that are installed in R. In this session, the *iris* data are required. Thus, the name of the data is specified within the parenthesis of the function, ie, *data*

(iris). If this process is successful, R should be able to summarize the raw data (function *str*) as 150 objects (the flowers) and 5 variables (the five columns about petal length/width, sepal length/width, and species).

```
data (iris)

str (iris).

## 'data.frame': 150 obs. of 5 variables:
## $ Sepal.Length: num 5.14.94.74.65.54.46.54.44.9...
## $ Sepal.Width: num 3.53.32.31.36.39.34.34.29.31...
## $ Petal.Length: num 1.41.41.31.51.41.71.41.51.41.5...
## $ Petal.Width: num 0.20.20.20.20.20.20.40.30.20.20.1...
```

```R
## $ Species: Factor w/3 levels "setosa", "versicolor", ...: 11111
1111...
```

The data can be considered as a table. Each column name is indicated by the "$" sign in the output of the function *str*. The table view of the data can be printed by using the function *head*, which prints out the first six rows of the table.

```
head (iris).

## Sepal. Length Sepal. Width Petal. Length Petal. Width Species
## 15.13.51.40.2 setosa
## 24.93.01.40.2 setosa
## 34.73.21.30.2 setosa
## 44.63.11.50.2 setosa
## 55.03.61.40.2 setosa
## 65.43.91.70.4 setosa.
```

Then, the algorithms for the decision trees (named *party*) are installed and loaded in R via the functions *install*.

*packages* and *library*. By using *install.packages*, the algorithms are available in the local computer. However, they are not loaded automatically to avoid extra-processing for the computer. Thus, the function *library* is used to load the algorithms in R for the analysis.

```
Install.packages("party")
library (party).
```

Now, the decision trees can be generated via the function *ctree*. As a reminder, the students want to predict the species of the flowers based on sepal length/width and petal length/width. In the function *ctree*, the response variable (the variable to be predicted, ie, species) is written before the "~" sign, while all the explanatory variables (the other variables used to predict the species of the flowers) are summarized with a point after the "~" sign. The first line can thus be read as "use all the variables in the data to predict the species of the flowers". The second line indicates that the data to be used are the *iris* data. The third line indicates a specific parameter that does not require extensive clarification due to the introductory nature of this session, but students may be informed that *MonteCarlo* refers to the random shuffling of the data that was explained at the beginning of this section. Finally, the "->" indicates that the output of the algorithm is saved as *flower_tree*.

```
ctree (Species ~., data = iris, controls = ctree_control (test-
ttype = "MonteCarlo")) - > flower_tree.
```

The output of the decision trees can then be visualized via the *plot* function.

```
plot (flower_tree).
```

As shown in Figure 1, the bars of the buckets at the bottom of the graph indicate the ratio of the predicted flowers, ie, Setosa, Virginica, and Versicolor. The decision tree can be read from the top. Depending on the answer
<figure><img src="image_7.png" /><figcaption>**FIGURE 1** Decision tree with flower species as the response variable and information on sepal length/width and petal length/width as the explanatory variables</figcaption></figure>

at each numbered node, a prediction is made about the species of the flower being analyzed. For instance, if the petal length is smaller than 1.9 cm (Nodes 1 to 2), the flower is most likely a Setosa. If the petal length is bigger than 5 cm (Node 9) and the petal width is above 1.7 cm, the flower is most likely a Virginica. The variables that are shown in the decision tree are the variables considered to have statistically significant interpretability on the data. The variables that are not included in the tree are thus considered not helpful to identify the species of the flowers. By way of illustration, information on sepal length and width is not included.

Finally, the accuracy of this model needs to be assessed. In other words, if the students predict the species of each of the 150 flowers in the data based on this tree, how often would they get it right or wrong. Thus, the predictions of the tree are extracted via the `predict` function and saved as `pred_flower`. Then, these predictions are compared with the actual data (the `Species` column in the `iris` dataset) by merging them into one table with the `table` function.

```
predict (flower_tree) -> pred_flower
table (pred_flower,iris$Species) -> accuracy_table.
```

The output of this comparison is saved as `accuracy_table` and displayed in Table 2. The columns indicate the values in the sample data, whereas the rows referred to the values predicted by the tree. For instance, the top left cell shows that all the 50 flowers from the Setosa species were predicted correctly by the decision tree.

The correct predictions are the values on the diagonal of the table, ie, 50, 49, and 45. The total of the correct predictions can be calculated by the sum (the function `sum`)

<table><caption>**TABLE 2** Predictions (rows) compared with actual data (columns)</caption><thead><tr><th>Data prediction</th><th>Setosa</th><th>Versicolor</th><th>Virginica</th></tr></thead><tbody><tr><td>Setosa</td><td>50</td><td>0</td><td>0</td></tr><tr><td>Versicolor</td><td>0</td><td>49</td><td>5</td></tr><tr><td>Virginica</td><td>0</td><td>1</td><td>45</td></tr></tbody></table>

of the values on the diagonal of the table (the function `diag`). This total divided by the total of all the values in the table is the accuracy of the predictions.

```
sum(diag(accuracy_table))/sum(accuracy_table).
```

In the exercise, the accuracy is equal to $(50 + 49 + 45)/(50 + 49 + 1 + 5 + 45) = 144/150 = 96\%$, which is good since it very close to 100%. We can thus summarize our analysis as follows: by measuring petal length and width, the biologists should be able distinguish Setosa, Versicolor, and Virginica with an accuracy of 96%. For instance, if the petal length is smaller than 1.9 cm, it is most likely a Virginica.

## 5 | ADDITIONAL NOTES FOR THE INSTRUCTORS

Students should have access to computers and internet for this session. Running the R code does not require internet, yet, sudden technical issues may require the internet to be solved. R and R studio should be installed in advance by the students to avoid wasting time in technical issues during the session. For instance, R is
available on different operating systems such as Windows, Mac, and Linux. However, the version of the operating system may require the user to select a specific version of R rather than the newest one, eg, if the user did not upgrade the operating system since a long time. With regard to the participants, the number is flexible but a group consisting of not more than 10 students is considered ideal, since the instructor may devote more time to each student in case of technical issues. Finally, the instructor should keep in mind that the purpose of this session is to encourage students to experience first-hand programming and statistics. Thus, the content and the process are kept as simple as possible. The purpose of the session is not to show that decision trees with R may replace all types of existing statistical methods and programming language. A full course should include a much larger variety of statistical methods and concepts. By way of illustration, the performance of a model is conventionally assessed based on several methods, eg, cross-validation with precision and recall. Due to the limitation of time during this introductory session, only the overall accuracy is used.

## 6 | CONCLUSION

A 45-minute session is proposed to introduce statistics and programming language to university students without prior knowledge in these two fields. The programming language R and the statistical method of decision trees are selected due to their ease of operation. On the one hand, the session can be used as a stand-alone module to engage students in further learning of programming and statistics. On the other hand, the session can also easily be included in an existing course.

## ACKNOWLEDGEMENTS

This paper is realized as a partial fulfillment of the Academic Teacher Training Course of the Division for Quality Enhancement at Uppsala University. The author is grateful for the fruitful discussion with the instructors and participants of the Academic Teacher Training Course 2018. The author would also like to thank his colleagues Klara Bertils, Marie Dubremetz, Pär Eliasson, Rima Haddad, Helena Löthman, Simon Magnusson, Linnéa Öberg, and Buket Öztekin for their comments and assistance. Special thanks to Helena Löthman from the department of linguistics and philology at Uppsala University for participating as a mentor of the project, and to Ulrika Svalfors from the Division for Quality Enhancement, Academic Teaching and Learning at Uppsala

University for providing detailed comments about this project. The author also appreciates the constructive comments from the anonymous referees. The author is fully responsible for all remaining errors.

## ORCID

Marc Tang https://orcid.org/0000-0002-9057-642X

## REFERENCES

1. L. Albarracin, A. Aymerich, and N. Gorgorio, *An open task to promote students to create statistical concepts through modelling*, Teaching Statistics, 39 (3) (2017), 100–105.

2. D. Ben-Zvi and J. Garfield, *The challenge of developing statistical literacy, reasoning, and thinking*, Springer, Dordrecht, 2004.

3. L. Breiman, *Random forests*, Machine Learning, 45 (1) (2001), 5–32.

4. L. Breiman et al., *Classification and regression trees*, Taylor & Francis, New York, 1984.

5. J. Buckley et al., *Embedding quantitative skills into the social science curriculum: Case studies from Manchester*, International Journal of Social Research Methodology, 18 (2015), 495–510.

6. L. J. Carnell, *The effect of a student-designed data collection project on attitudes toward statistics*, Journal of Statistics Education, 16 (1) (2008), 1–15.

7. Fisher, R.A. (1936). The use of multiple measurements in taxonomic problems. *Annual Eugenics*, 7(part II), 179–188.

8. R. Gould, *Statistics and the modern students*, Int Stat Rev, 78 (2010), 297–315.

9. T. Hothorn, K. Hornik, and A. Zeileis, *Unbiased recursive partitioning: A conditional inference framework*, J Comput Graph Stat, 15 (3) (2006), 651–574.

10. E. B. Howington, *Teach a confidence interval for the median in the first statistics course*, Teaching Statistics, 39 (3) (2017), 84–87.

11. J. MacInnes, *Proposals to support and improve the teaching of quantitative research methods at undergraduate level in the UK*, ESRC, Swindon, 2009.

12. J. Nichols and T. Warnow, *Tutorial on computational linguistic phylogeny*, Language and Linguistics Compass, 2 (5) (2008), 760–820, https://doi.org/10.1111/j.1749-818X.2008.00082.x.

13. R. Core Team, *R: A language and environment for statistical computing*, R Foundation for Statistical Computing, Vienna, 2018.

14. S. A. Tagliamonte and H. Baayen, *Models, forests, and trees of York English: Was/were variation as a case study for statistical practice*, Language Variation and Change, 24 (2012), 135–178.

**How to cite this article:** Tang M. A simple introduction to programming and statistics with decision trees in R. *Teaching Statistics*. 2020;42:36–40. https://doi.org/10.1111/test.12210