10.1. MBONSAI DECISION TREE GENERATED FROM SEQUENCE DATA

0.1

mbonsai Decision Tree Generated from Sequence Data

This command builds a decision tree model based on sequence patterns. Analysis of the sequence data can beapplied in a variety of applications such as order of brands purchased by customers, sales floor cyclic patterns indepartment stores and supermarkets, the onset order of injuries. The original idea of this command, BONSAI[1],was developed by a research team from Kyushu University and Kyushu Institute of Technology. This technique
is applied for the analysis of amino acid sequences in molecular biology. Similarly, this command implemented
several improvements for the application of sequence analysis in business data with the following features.
• Transform patterns from numerical and categorical sequence data into classication conditions.
• Use alphabet indexing as data reduction technique for sequence data.
• Process multiple variables of sequence data, numerical and categorical variables as predictors.
• Allow cost sensitive learning approach to account for dierential misclassification costs.
• Allow separate training and testing of decision tree models.
• Allow two or more classes in target variable for classification.
• Allow cross-validation for assessing performance of predictive model.
First, the example below illustrates how to obtain an intuitive understanding of this command. A data set
contains purchase sequences of brand a,b,c by customers from a retail store, and the corresponding churnstatus of the customer is shown in Table 1. Under the hypothesis of which purchase order of the brand isrelated to the estrangement of the customer, we will build a decision tree model with partial pattern of brandpurchasing order as explanatory variable, and the churn status as target variable (Refer to the next section
on details of the definition of patterns that is considered partial string, such as ”ab”, ”cbb”). Pattern used as
explanatory variables is referred to as candidate pattern, the pattern generated (More details on this method isavailable in the next section) will contribute to the accuracy of the model, afterwards, the patterns are converted
to 0-1 variable (Table 2). Decision tree is constructed in a conventional manner from this data set consisting ofcandidate patterns. The actual decision tree created with this command is shown in Figure 3.
Table 2: The BrandSequence column is extracted from
Table 1: Example of sequence data. Each line corre- Table 1 and matched against partial purchase patterns
sponds to one customer, with the target variable indi- as explanatory variables (referred to as candidate patcating the status of continual purchase of the customer. terns). All samples are matched,, the presence of patThe alphabetic characters a, b, c, shown in ”BrandSe- terns is converted to 0-1 formatted data. In this exquence” column represents the brand purchased by the ample, the first record contains the partial pattern ”a”,
customer in the respective order as shown.
”b”, ”c”, ”ab” in the sequence, however,!Haa !Iand!Hcc !I
BrandSequence Churnis not included.
bcaba
bcabcaaaaabac
caa
cca
cacbc
bcc
acca

yes
yes
yes
yes
no
no
no
no

a
1
1
1
1
1
1
0
1

b
1
1
1
0
0
1
1
0

c
1
1
1
1
1
1
1
1

aa
0
1
1
1
0
0
0
0

ab
1
1
1
0
0
0
0
0

cc
0
0
0
0
1
0
1
1

···

···

Churn
yes
yes
yes
yes
no
no
no
no

A notable feature of the BONSAI is the function to group elements (known as alphabet) that make up the
pattern automatically. The grouping of each alphabet is referred to as index.
For example, there are three ways to group the 3 brands a,b,c into two index (a and bc, b and ac, c and ab),each grouping is replaced with the matching index. Three decision trees are built with the above described
procedure, the best decision tree is selected based on classification accuracy.

2

Table 3: Example of Building Customer Churn Modelwith BONSAI
Table 4: Example of Building Customer Churn Modelwith BONSAI
Decision tree built in this manner is shown in Figure 4. The result of grouping with index increases the accuracyof the classification model, further more, useful knowledge could also be obtained from the result.

0.1.1

Details

The details concerning the construction of the decision tree are explained as follows.
Regular pattern
Given, n character string constants π1 , π2 , ..., πn on alphabet Σ, with any of the n+1 character string x0 , x1 , ..., xn ,regular pattern (also referred to as sequence pattern) is represented in the format x0 π1 x1 π2 x2 · · · πn xn . This
technique is easier to understand when the concept of wildcard is applied to the field of data retrieval (“*” isused instead of xi as wildcard). In this command, besides the definition of regular pattern described above, thatis x0 πx1 , the regular pattern can be handled by restricting to the substring π (hereinafter referred to as ”string
pattern”). The use of string pattern and sequence pattern is specified in the second parameter at p=.
Begin / End Match
It is possible to specify the matching of beginning and ending string as matching rule for regular pattern ofeach sample of sequence data. This can be specified in the fourth (match beginning), and fifth (match ending)parameters of p=. Based on this specification, the position of the alphabet of the beginning (or ending) of regularpattern, the beginning (or ending) of sequence data is matched within the specified number of characters. For
example, given the sequence data aabccd, when matched with character substring pattern ab from the left, if
the matching sequence only contains 1 character, it is not considered as match, if the beginning of the sequence
contains first two characters of the substring pattern, it is considered as match. If the sequence pattern is madeup of ab, it is possible to match the first charter in the beginning. If this parameter is not specified, it operates
without restriction of matching as described above.
Alphabetical Order
This command can deal with order structure that contains alphabets. The key difference of order structure is
the rules for creating the index. Given the alphabet set {a1 , a2 , . . . , an }, with the relationship of a1 ≺ a2 ≺. . . ≺ an for any three consecutive alphabet ai , ai+1 , ai+2 , an index is generated to satisfy the relationship of
ψ(ai ) = ψ(ai+2 ) ⇒ ψ(ai ) = ψ(ai+1 ). Here, ψ(a) represents an index to be associated with the alphabet a.
This means that the alphabets belonging to a group for indexing must always follow consecutive order. For

0.1. MBONSAI DECISION TREE GENERATED FROM SEQUENCE DATA

3

example, given three consecutive alphabet a ≺ b ≺ c, {a, b}and{c} is grouped together, but {a, c}and{b} is not.
Consecutive alphabet can be specified in the third parameter of p=.

Local Search and Search Space of Optimal Alphabet Index
Given n number of alphabet(s), the number of cases to be grouped into m or less indexes is not certain, butwhen m = 2 is set as limit, the number can be presented in mn−1 ways. If both the value of m, n are small, it ispossible to construct a decision tree for all cases, and if the value is large, the local search method of searching in
subspace is applied. Initially, the random index is set to corresponding alphabet, the corresponding relationshipchanges bit by bit when building the decision tree, and continue to change until there is no improvements inclassification accuracy (Refer to literature [1] for details). Thus, sometimes the results may differ depending on
the initial value of the corresponding relationship between the index and the alphabet. In order to obtain thesame results from the initial value and select the best model (multi-start), the initial value can be specified atiter= in this command. The default value is iter=1.

Method of Generating Candidate Patterns
Candidate patterns as shown in Table 2 are generated from the updated alphabet index by local search. The
generation of the classification rules for the candidate patterns is based on the nodes of the decision tree. In thiscommand, candidate patterns are enumerated by the heuristics method described below. First, the index forthe regular pattern with length 1 is constructed, and is stored in the priority queue. Precedence in the priority
queue is determined by ascending order (see below) of the entropy of a regular pattern Afterwards, select theregular pattern with the lowest entropy in the priority queue, an index is added to the selected regular pattern,
regular pattern with length of 2 is created, and stored in priority queue again. Repeat the above steps suchthat, regular pattern with length n is selected, and regular pattern with length n + 1 is stored in the priority
queue. However, the index cannot be added when n is greater than 5 (the upper size limit of the regular pattern
can be specified in the sixth parameter of p=). In addition, the process will terminate if the size is greaterthan the number of candidates (specified by cand=) specified by the user. Entropy is used to evaluate regularpatterns and is also used in the selection of the splitting rules at the nodes of the decision tree, the value ofentropy becomes smaller as the regular patterns are classified into extreme distribution classes. Entropy ent(π)
of regular pattern π is defined in Equation 1.

ent(π) = −q m(π)

c
c
X
X
u(π)
u(π)
m(π)
m(π)
pi log pi− q u(π)
log pi
pi

m(π)

(1)

i=1

i=1

u(π)

represents the composition ratio of class i that matches(piHere, c represents the number of classes, pi
P m(π)= 1). In addition, q m(π) (q u(π) )(or unmatch) with the sample in the regular pattern π expressed as ( ci pi
represents the composition ratio of matching (or unmatch) with regular patterns π out of all samples (q m(π) +q u(π) = 1).

Other Types of Variables
In this command, besides sequence data, numerical and categorical variables can be specified as an explanatory
variable. By doing so, it is possible to construct decision tree rule that includes regular patterns with categoricaland numerical rules. The creation of branch rules based on category and numbers is similar to that in C4.5[2]. In this command, columns with sequence, numeric, and categorical data can be defined at p=,n=,d=parameters.

4
Selection of Splitting Rule
This command adopts a top down greedy method for the splitting of branches. In other words, splitting rules
at the node of the tree is determined according to the evaluation criteria from the information in the node. The
evaluation criteria is based on the selection of branching rules which maximizes the entropy gain. For samples
that have been classified into certain nodes, thePc probability (composition ratio) of class i is represented by pi ,the entropy at node is calculated by ent = − i=1 pi log pi . At node pi , number of samples n are classified intoclass i, the ratio of classified sample ni can be calculated by ni /n. Equation 1 shows the entropy gain from thedifference of entropy ent(π) after splitting the regular pattern π. It means by splitting the regular pattern π,
how much entropy is reduced. The procedure is repeated until all samples are classified into respective classes
at the node, and the tree can no longer be grown bigger. This type of decision tree is referred to as maximaltree.
Pruning
Construct a maximum tree Tmax according to the method shown in the previous section, but since the tree sizeis larger than normal tree, the training data tends to overfit the model, where the classification accuracy of
training data is high (misclassification rate decreases), but prediction accuracy of test data decreases. In order
to avoid this problem, a small section (including the root node) of the maximal tree is removed during pruningthat may be based on noisy or erroneous data.
In consideration to the set of nodes t from decision tree T , where the maximal tree is denoted by Tmax ={t1 , t2 , . . . , tk }, node t and root node of the subtree is represented by Tt , the subtree
of node t ∈ P to root nodeS
is pruned (replaced with leaf node) from the decision tree expressed as Tmax − t∈P Tt +P . The misclassification
rate of decision tree T is denoted by R(T ), the pruning is based on the selection of subtree T ∗ with the lowest
misclassification rate on unknown data. It is not possible to determine the actual misclassification rate forunknown data of T , instead, it is estimated based on the training data. Given the prediction amount is C(T ),the formula of pruning is expressed in the formula 2.
P ⊆Tmax C(Tmax −[Tt + P )(2)t∈P

To solve this problem, this command applies the cost-complexity pruning method [3]. This method is comprisedof two key phases. First, a series of subtrees T1 ⊃ T2 ⊃ . . . ⊃ Tk is selected which is nested from the maximal treebuild from the training data (here T1 is the maximal tree, Tk is the root node). Next, the accuracy of these trees
is estimated by test sample using cross-validation, the tree with highest prediction accuracy is selected.
When selecting the series of subtree, the evaluation function of decision tree T measuring the cost complexity
is defined as Rα (T ) = R(T ) + α|T̃ |, the decision tree model is considered better if the value is smaller. Thisequation is obtained by balancing the misclassification rate of decision tree R(T ), and the tradeoffs of decision
tree complexity |T̃ | (number of leaf nodes in T ) where the complexity parameter is α(≥ 0). In the training data,when the tree size grows larger, the misclassification rate R(T ) decreases monotonically, and yet, the complexity|T̃ | increases monotonically. When α is adjusted, the priority of misclassification rate and complexity is also
adjusted accordingly. Here, α is fixed as a single value, the pruning process find the subtree T (α) that minimizes
the function Rα (T ). Further, T (α) continues to minimize the tree when α increases, it is possible to enumerate
α corresponding to T (α), as a result, the nested subtree T1 ⊃ T2 ⊃ . . . ⊃ Tk is created (Refer to [3] formore details). Here, the subtree Ti consist of the smallest size decision tree with the minimal cost complexityα ∈ (αi , αi+1 ] . In the above series of nested subtree, the corresponding cost complexity parameter α1 , α2 , . . . , αkcan be obtained.
Next, among the series of subtree obtained from the above method, select the optimal subtree T̂ ∗ with theminimum prediction misclassification rate when applied to unknown data. In this command, users can selectbetween test sample method (specified by ts=) or cross-validation (specified by cv=). In test sample method,training data D is partitioned into two sets D1 , D2 at a ratio of1 : 2. Then, the maximal tree based on D2
training data is built, based on the complexity parameter α1 , α2 , . . . , αk obtained from the previous phase, the0.1. MBONSAI DECISION TREE GENERATED FROM SEQUENCE DATA

5

corresponding subtree is selected. From this subtree, D1 is used as the set of unknown data to predict thevalue of misclassification rate. In cross-validation method, training data D is partitioned equally into n sets as
D1 , D2 , . . . , Dn . First, D1 is treated as unknown data for prediction, similarly, this applies to other training datausing the test sample method. Similar process is applied n times to the remaining test data D1 , D2 , . . . , Dn , then
all training data D is applied as one set for validation. Thus, the average misclassification rate is obtained by thismethod. From the results obtained above, complexity parameter α1 , α2 , . . . , αk for decision tree T1 , T2 , . . . , Tk ,the optimal decision tree with the lowest misclassification rate is selected. In addition, the smallest tree whose
estimated mean error rate is within one standard error of the estimated mean error rate of the best tree selected(this is called !H 1 SE rule !I).
Nevertheless, pruning can be done by directly specifying the α value without using the cross-validation or test
sample method. A decision tree can be built at high speed without having to process calculations for prediction,
but the drawback is that the specified α value is arbitrary.
Actual Pruning
In the model building mode, pruning can be done by specifying any of the parameters ts=, cv=, alpha= in
this command. When cv= or ts= is specified, test data is used for prediction, the model with the minimum
misclassification rate is selected. In addition, when alpha= is specified, the pruned model corresponding to thespecified α value is selected. The selection of the model based on the evaluation of the decision tree model issaved in model.txt and model_info.txt. However, regardless of any specification, all α corresponding to the
maximal tree is calculated internally, at prediction mode (-predict), if alpha= is specified, it is possible thatdifferent models are used for prediction.
Learning Cost Considerations
When applying classification model, instead of increasing classification accuracy (percentage of correct answers),
there are many instances when it is useful to minimize the cost of misclassification. Cost is used when predicting
customer churn, there are various considerations when implementing cost sensitivity, such as consideration of
the cost of customer who stayed who is predicted as customer who left. Construction of model that takes misclassification costs into consideration is known as cost sensitive learning. Various methods have been proposed,
the method proposed by Breiman et al [3] is used in this case. This method is used for the calculation of the
branch rule, probability pi of class i in the sample is therefore modified by assigning cost calculation. Now, thecost of class j that is predicted as class i expressed as c(i|j), given the sum of costs of class i is expressed asPj c(i|j), weight is added to class i and probability pi is updated. If the total cost of class i is large, it means
the number of samples are inflated (oversampling), sensitive model based on information of class i is built. The
class file in CSV format is shown in Table 5, actual class (real), the corresponding predicted class (predict)and its cost (cost) is displayed in the same row. Given the combination of actual class and predicted class is
not specified, and the cost file is not defined, when actual class and predicted class is the same, the cost is 0,
otherwise, 1 is set.
Table 5: Example of specifying the cost in churn model. In first row, the churnclass yes in the actual sample is predicted as no, the cost is set to 2. In thesecond row, when the churn class no is predicted as yes, the cost is set to 5.
The column name can be pre-defined, but must be arranged in the order ofactual class, predicted class, and cost.
real
yes
no

predict
no
yes

cost
2
5

6

0.1.2

Output Data

The output of various data files from this command are summarized in Table 6.
Table 6: List of output data from model building mode in mbonasi
File namemodel.pmml

Content
The decision tree model represented by PMMLa

alpha_list.csv

Other model information of the complexity parameter αSummary of pruned model with minimum classification prediction error
Summary of pruned model with the same 1SE rule
Summary of pruned model for the specified αvalue
Various information of pruned model with minimum classification prediction error
Various information of pruned model with thesame 1SE rule
Summary of pruned model for the specified α
The prediction information of pruned model withminimum classification prediction error
The prediction information of pruned model withthe same 1SE rule
The prediction pruned model for the specified α
List of execution parametersmodel_min.txt
model_1se.txt
model.txt
model_info_min.csv
model_info_1se.csv
model_info.csv
predict_min.csv
predict_1se.csv
predict.csv
param.csv

Remarks
Records pruning information for maximum tree.
Prediction mode is selected when -predict is
specified.
Series of α corresponding to the model such as thesize and accuracy of the model.
Created when cv= or ts= is specified.
Created when cv= or ts= is specified.

Created when cv= or ts= is specified.
Created when cv= or ts= is specified.

Created when cv= or ts= is specified.
Created when cv= or ts= is specified.

Returns the pair of keyword-value for the specified
parameters

a Predictive Model Markup Language is an industry standard to describe data mining and mathematical models represented in
XML-based file format. Note that this specific command uses an extended tag of the PMML standard.

model.pmml Based on the maximal tree of the decision tree created, complexity penalty attribute is shownfor each node, the branch will be pruned when α is greater than the value of complexity penalty. Since themaximal tree and pruning information is recorded in PMML, α is specified for the execution of prediction model,
it is possible that the corresponding value will be used to predict the decision tree model.
:<Node id="0" score="yes" recordCount="8" ><Extension extender="KGMOD" name="complexity penalty" value="0.500000"/>:

model min.txt,model 1se.txt,model.txt Unlike PMML data, a summary of pruned model is saved intext format. The section [alphabet-index] shows the alphabet corresponding to the index. In the following
example, alphabet c corresponds to index 1, while b,a corresponds to index 2. The branching rule of thepattern shown in the decision tree is indicated by index symbol.
[alphabet-index]
Field Name: BrandSequence
Index[1]={c}Index[2]={b,a}The section [decision tree] shows the decision tree in text format, information such as model size (thenumber of leaf nodes) and number of layers of the deepest leaf is shown below the tree.
[decision tree]
if($BrandSequence has 22)

0.1. MBONSAI DECISION TREE GENERATED FROM SEQUENCE DATA

7

then $Churn=yes (hit/sup)=(4/4)
else $Churn=no (hit/sup)=(4/4)
number of leaves: 2
deepest level: 1
The section [Confusion Matrix by Training] shows the prediction classification results of decision tree modelusing training data. In addition, classification table by cost, prediction accuracy by class, overall prediction
accuracy is also shown.
The section [Confusion Matrix by Estimation] is shown when ts= or cv= is specified. The same format
applies to [Confusion Matrix by Training] section, the result of test data will be shown differently.
[Confusion Matrix]## TRAINING DATA #### By countPredicted As ...
yes
no
Total
yes
4
0
4
no
0
4
4
Total
4
4
8
## By cost
Predicted As ...
yes
no
Total
yes
0
0
0
no
0
0
0
Total
0
0
0
## Detailed accuracy by classclass,recall,precision,FPrate,F-measure
yes,1,1,0,1
no,1,1,0,1## Summary
accuracy=1
totalCost=0
Finally, the section [Selected Alpha] shows the value of the complexity parameters used for pruning.
[Selected Alpha]alpha: 0

predict min.txt,predict 1se.txt,predict.txt The prediction results is added to the training data used forbuilding the decision tree model in CSV format. The prediction results, as described below, output the highest
prediction probability in the predict column, and the prediction accuracy for each class (yes and no as shown
below). When ts= is specified, it returns the prediction results of test data, when cv= is specified, it returns theprediction results by all test data using cross validation. In addition, when alpha= is specified, the prediction
result of the training data is returned.
Brand Sequence,Churn,predict,yes,nobcaba,yes,yes,1,0
bcabcaa,yes,yes,1,0
aaabac,yes,yes,1,0caa,yes,yes,1,0
cca,no,no,0,1
cacbc,no,no,0,1
bcc,no,no,0,1
acca,no,no,0,1

model info min.csv,model info 1se.csv,model info.csv These files store the model information in CSV
format. The column nobs refer to the number of records in training data, alpha= refers to the value of pruning
complexity parameter, accuracy,totalCost refer to the percentage of correct answers in the test model andthe total cost.

8

nobs,alpha,accuracy,totalCost
8,0,1,0alpha list.csv Display the error rate, standard error, error rate !^standard error corresponding to the α valueof pruning complexity parameter for the pruned decision tree.
alpha,leafSize,errorRate,SE
,up
,lo
0
,102
,0.0082,0.0015 ,0.0098 ,0.0066
8.15e-05 , 98
,0.0085,0.0016 ,0.0101 ,0.0069
9.41e-05 , 91
,0.0091,0.0016 ,0.0108 ,0.0074
0.000124 , 82
,0.0100,0.0017 ,0.0118 ,0.0083
:
:
:
:
:
:
0.035669 , 2,0.0878,0.0049 ,0.0927 ,0.0828
1.79e+308, 1,0.1399,0.0060 ,0.1460 ,0.1339

param.csv

0.1.3

Various parameter values used when building the model is stored in CSV format.

Format 1: Model building mode

mbonsai i= [p=] [n=] [d=] c= O= [delim=] [cost=] [seed=] [cand=] [iter=] [cv=|ts=][leafSize=] [--help]
i=
p=

n=
d=
c=
O=
delim=
cost=
seed=
cand=
iter=
leafSize=alpha=
ts=
cv=--help: Training data file name: Column name of pattern (multiple fields can be specified): Specify up to five parameters after the column name, each separated by a colon.
: p=column_name:is:seq:ordered:head:tail:rs: is: Size of the index: When the parameter is not specified, an index is not generated, instead, original alphabet of the pattern is used.
: seq: Type of pattern: true: Partial sequence pattern: false: Partial string pattern (default): ordered: Arrangement in alphabetical order when generating the index.
:(this parameter is ignored when is is not specified): true: Ordered, group alphabet above / below the threshold value.
: false: Unordered(default): head: Match string or numeric characters from the beginning (default: start of string is not considered for matching): tail: Match string or numeric characters from the end (default: end of string is not considered for matching): rs: Upper size limit of regular pattern (default: 5): Column name with numerical data (multiple fields can be specified): Column name with categorical data (multiple fields can be specified): Column name of class: Output directory name (text, PMML model, and model statistics): Delimiter character of pattern (default: empty character, that is 1 byte character is regarded as 1 alphabet): Name of cost file
: Seed of random number (default=-1: time dependent): Number of patterns as explanatory variable (default=30,range:1!A256): Iterations of local search (default=1): Lower limit of the number of samples in one leaf (default: no limit): Specify the degree of pruning. However, when cv= or ts= is specified,
this parameter is disabled. Default=0.01.
: Specify the percentage of test data partitioned using the test sample method. When ”ts=” is not specified, the default v: Specify partition of data by cross-validation method. When ”cv=” is not specified, the default value is set as 10.
: If either ts=,cv= is not specified, default value of alpha=0.01 will be applied.
: Even when alpha=,ts=,cv=, is specified, pruning degree of maximum tree is recorded in PMML,: the value of alpha could change in prediction mode.
: Show help

0.1. MBONSAI DECISION TREE GENERATED FROM SEQUENCE DATA

0.1.4

9

Format 2: Prediction mode
mbonsai -predict i= I= o= [alpha=] [--help]-predicti=I=o=alpha=

delim=--help0.1.5: Operate in prediction mode. This parameter is required for prediction mode.
: Input data [required]: The column names must be the same as the columns that was used for building the model.
: Destination directory path for model building mode [required]: File required are as follows.
: bonsai.pmml: pmml containing the decision model: Output file name containing prediction result: The ”predict” column is added to the input data in output.
: Columns must be the same as columns that was used in building the model.
: Specify pruning complexity parameter.
: Accepts real number greater than 0, as well as the following two symbols with special functions.
: min: α value that corresponds to pruned model which minimizes the estimated misclassification rate.
: 1se: Alpha value that corresponds to the pruned model with the same 1SE rule.
: Designation of the two symbols is effective only when ts= or cv= is specified when building the model.
: Default behavior:: If ts= or cv= is specified when building the model, min is used.
: If you specify alpha= when building the model, the specified value is applied.
: Delimiter character of pattern (default: empty character, that is 1 byte character is regarded as 1 alphabet): Show help

ExamplesExample 1 An example of building a model
In this example, since ts=,cv=,alpha= parameters are not specified, pruning is carried out when alpha=0.01,
the result is saved to the file model.txt.
$ more input.csv
BrandSequence,Churnbcaba,yes
bcabcaa,yes
aaabac,yes
caa,yes
cca,nocacbc,nobcc,noacca,no$ mbonsai p=BrandSequence:2 c=Churn i=input.csv O=result1$ more result1/model.txt[alphabet-index]
Field Name: BrandSequence
Index[1]={a}Index[2]={b,c}[decision tree]
if($BrandSequence has 11)
then $Churn=yes (hit/sup)=(3/3)
else if($BrandSequence has 2212)
then $Churn=yes (hit/sup)=(1/1)
else $Churn=no (hit/sup)=(4/4)
numberOfLeaves=3
deepestLevel&=& 2[Confusion Matrix by Training]## By countPredicted As \ldots

10

yes
no
Total

yes
4
0

4

no
0
4

4

4

Total
4
8

## By cost
Predicted As \ldots
yes
no
Total
yes
0
0
0
no
0
0
0
Total
0
0
0
## Detailed accuracy by classclass,recall,precision,FPrate,F-measure
yes,1,1,0,1
no,1,1,0,1## Summary
accuracy=1
totalCost=0$ more result1/model.pmml<?xml version="1.0" encoding="UTF-8"?><PMML version="4.0"><Header copyright="KGMOD"><Application name="mclassify" version="1.0"/><Timestamp>2014/07/20 23:00:13</Timestamp></Header><DataDictionary numberOfFields="2"><DataField name="BrandSequence" optype="categorical" dataType="string"><Value value="b"/><Value value="c"/><Value value="a"/></DataField><DataField name="Churn" optype="categorical" dataType="string"><Value value="yes"/><Value value="no"/></DataField></DataDictionary><TreeModel functionName="classification" splitCharacteristic="binarySplit"><MiningSchema><MiningField name="BrandSequence"><Extension extender="KGMOD" name="alphabetIndex"><alphabetIndex alphabet="b" index="2"/><alphabetIndex alphabet="c" index="1"/><alphabetIndex alphabet="a" index="2"/></Extension></MiningField><MiningField name="Churn" usageType="predicted"/></MiningSchema><Node id="0" score="yes" recordCount="8"><Extension extender="KGMOD" name="pruning" value="0.000000"/><True/><ScoreDistribution value="yes" recordCount="4"/><ScoreDistribution value="no" recordCount="4"/><Node id="1" score="yes" recordCount="4"><Extension extender="KGMOD" name="pruning" value="0.000000"/><Extension extender="KGMOD" name="patternPredicate" value="substring"><SimplePredicate field="BrandSequence" operator="contain"><index seqNo="1" value="2"/><index seqNo="2" value="2"/></SimplePredicate></Extension><ScoreDistribution value="yes" recordCount="4"/><ScoreDistribution value="no" recordCount="0"/></Node><Node id="2" score="no" recordCount="4"><Extension extender="KGMOD" name="pruning" value="0.000000"/><Extension extender="KGMOD" name="patternPredicate" value="substring"><SimplePredicate field="BrandSequence" operator="notcontain"><index seqNo="1" value="1"/><index seqNo="2" value="1"/></SimplePredicate>0.1. MBONSAI DECISION TREE GENERATED FROM SEQUENCE DATA

11</Extension><ScoreDistribution value="yes" recordCount="0"/><ScoreDistribution value="no" recordCount="4"/></Node></Node></TreeModel></PMML>

Example 2 Predict unknown data with the model from example 1
This example predict unknown data (using training data and unknown data in the following) based on thedecision tree model constructed. The prediction result includes prediction accuracy of each class (yes,nocolumn), class name with the highest accuracy (predict column) is returned.
$ more unknown.csv
BrandSequence
bcaba
bcabcaaaaabac
caa
cca
cacbc
bcc
acca
$ mbonsai -predict i=unknown.csv I=result1 o=predict.csv$ more predict.csv
BrandSequence,predict,yes,nobcaba,yes,1,0
bcabcaa,yes,1,0
aaabac,yes,1,0caa,yes,1,0
cca,no,0,1
cacbc,no,0,1
bcc,no,0,1
acca,no,0,1

Bibliography[1] S. Shimozono, A. Shinohara, T. Shinohara, S. Miyano, S. Kuhara and S. Arikawa, Knowledge Acquisition
from Amino Acid Sequences by Machine Learning System BONSAI, Trans. Information Processing Societyof Japan, Vol. 35, pp. 2009-2018, 1994.
[2] J. R. Quinlan, C4.5: Programs for Machine Learning, San Meteo: Morgan Kaufmann, 1993.
[3] L. Breiman, J. Friedman, R. Olshen, C. Stone Classification and regression trees, Wadsworth: Belmont, CA,
1984.

13