# Identifying Engineering Undergraduates' Learning Style Profiles Using Machine Learning Techniques
## 2 Background
### Predictive Analysis with Decision Trees  
A decision tree identifies a model that best fits the relationship between the attribute set and the class label of the input data. Specifically, decision trees have a hierarchical structure composed of a group of internal nodes and leaf nodes that classify a set of data by categorising them from the root node to some leaf node. Each internal node in the tree specifies a test condition that evaluates one or more attributes. Each descendant branch of the tree represents a sequence of decisions made by the model to determine the class membership of a new unclassified entity.  
Unlike other classification models considered black boxes, decision trees are white-box models that allows someone to see why the model classifies in one way or another or to argue such a classification.  
Different techniques have been developed to induce decision trees in the machine learning community. One of the pioneering works came from [PERSON] with the ID3 algorithm [25]. This algorithm generates decision trees based on the information obtained from training examples and then uses them to classify the test set. The dataset generally has nominal attributes to perform the classification task with non-missing values.  
The C4.5 algorithm is an extension of the ID3 algorithm [PERSON] introduced to improve certain deficiencies [26]. Since it was not intended for numerical attributes and did not use pruning to reduce overtraining, the C4.5 algorithm uses a new calculation that allows the measuring of a gain ratio. It also handles attributes with continuous values. Finally, C4.5 employs a pruning technique to reduce the error rate. This technique reduces the size of the tree by removing sections that may be based on erroneous or missing data, thus reducing the complexity of the tree and improving its classification power.  
There are several advantages in using decision trees [27]. First, the graphical representation of decision trees is intuitive when there are a reasonable number of nodes for users unfamiliar with the subject. In general, this favours transparency and decision making between professionals from different areas. Second, decision trees, unlike other techniques, are helpful for regression and classification problems. Third, the algorithms for creating decision trees are very flexible with the data. They can handle nominal, ordinal, and numeric data. Additionally, many of these algorithms can take missing and even errored values, which is useful for saving time in the data-cleaning process.  
On the other hand, using decision trees also has its disadvantages. First, the tree can become complex when the data include nominal variables with many categories or several numerical variables. As a result, it tends to overfit the data with which it was trained. However, techniques such as pruning and setting growth limits solve this problem. Second, they are sensitive to irrelevant characteristics and variability in the data. Slight variations in the data can result in a completely different tree. Cross-validation procedures are used to avoid this problem. Finally, the process of building a decision tree can take a significant amount of time. This issue usually happens when there are many characteristics of each observation due to the algorithms in each iteration that compare which best divides the data.