# Presentation: 160885

## ADaM TutorialDr. John Rushing

- Information Technology and Systems Center
- University of Alabama in Huntsville
- National Space Science and Technology Center
- 256-824-6064
- dhardin_[@itsc.uah.edu](mailto:s@itsc.uah.edu)_
- _[http://](http://www.itsc.uah.edu/)__[www.itsc.uah.edu](http://www.itsc.uah.edu/)_/
- http://datamining.itsc.uah.edu/

**Notes:**
Hello Everyone. Welcome to the data mining tutorial. I’m Danny Hardin of the Information Technology and Systems Center at the University of Alabama in Huntsville. Today I will go through a PowerPoint presentation that covers some basics of Data Mining in general and then illustrate through example how the UAH Algorithm Development and Mining system is used. There will be no demonstration as such. However, you may go through all the steps I am presenting today by downloading the ADaM application and sample files from our website at datamining.itsc.uah.edu. You will also find this tutorial posted there next week.

## Data Mining – What it is

- Extracting knowledge from large amounts of data
- Motivation
- Our ability to collect data has expanded rapidly
- It is impossible to analyze all of the data manually
- Data contains valuable information that can aid in decision making
- Uses techniques from:
- Pattern Recognition
- Machine Learning
- Statistics
- High Performance Database Systems
- On-Line Analytical Processing (OLAP)
- Plus other techniques unique to data mining (Association rules)
- Data mining methods must be efficient and scalable

**Notes:**
This slide describes what data mining is, why it is necessary, and where its influences are derived.
Data mining attempts to extract knowledge from large amounts of data. 

The reason we need this technique is because:
Our ability to collect data has expanded rapidly
It is impossible to analyze all of the data manually
The data of highest interest is typically a small subset of the whole

Data mining employs common techniques such as pattern recognition, machine learning, statistics, database tools and on-line analytical processing (OLAP).  There are some techniques however, that are unique to data mining: Association rules for example.
Since data mining is normally applied to large volumes of data it must be efficient and scalable

## Data Mining – What it isn’t

- Small Scale
- Data mining methods are designed for large data sets
- Scale is one of the characteristics that distinguishes data mining applications from traditional machine learning applications
- Foolproof
- Data mining techniques will discover patterns in any data
- The patterns discovered may be meaningless
- It is up to the user to determine how to interpret the results
- Magic
- Data mining techniques cannot generate information that is not present in the data
- They can only find the patterns that are already there

**Notes:**
This slide presents some common misconceptions about data mining. 
Data mining is not small scale. It’s methods are designed for large data sets. 
Data mining is not foolproof. 
Patterns will be discovered in any data. 
The patterns may be meaningless.
The user must be able to ascertain the value of the results.
Data mining is not magic. It cannot generate information that is not present in the data. 
It can only find existing patterns.

## Data Mining – Types of Mining

- Association Rule Mining
- Initially developed for market basket analysis
- Goal is to discover relationships between attributes
- Uses include decision support, classification and clustering
- Classification and Prediction (Supervised Learning)
- Classifiers are created using labeled training samples
- Training samples created by ground truth / experts
- Classifier later used to classify unknown samples
- Clustering (Unsupervised Learning)
- Grouping objects into classes so that similar objects are in the same class and dissimilar objects are in different classes
- Discover overall distribution patterns and relationships between attributes
- Other Types of Mining
- Outlier Analysis
- Concept / Class Description
- Time Series Analysis

**Notes:**
In general mining operations are performed to figure out characteristics of the existing data or to figure out ways to infer some prediction of the future. Here is an overview of data mining types. 

Association Rule Mining is often used for market basket or transactional data analysis. It seeks to discover rules used to describe conditions where items occur together. 

Classification and Prediction  involves the identification of characteristics that can be used to generate a model for prediction of similar occurrences in similar data sets. 

Cluster Analysis attempts to look for groups of data items that have a strong similarity to other objects in the group, but are dissimilar to objects in other groups. 

Outlier Mining  uses statistical, distance and deviation-based methods to look for rare events (or outliers) in datasets, things that are not the norm. 

Concept/Class Description uses data characterization and/or data discrimination to summarize and compare data with target concepts or classes. This technique is useful in data warehousing applications. 

Time Series Analysis  can include analyses of similarity, periodicity, sequential patterns, trends and deviations. It is useful for modeling how data events change with time. 

## ADaM – System Overview

- Developed by the Information Technology and Systems Center at the University of Alabama in Huntsville
- Consists of over 75 interoperable mining and image processing components
- Each component is provided with a C++ application programming interface (API), an executable in support of scripting tools (e.g. Perl, Python, Tcl, Shell)
- ADaM components are lightweight and autonomous, and have been used successfully in a grid environment
- ADaM has several translation components that provide data level interoperability with other mining systems (such as WEKA and Orange), and point tools (such as libSVM and svmLight)
- Future versions will include Python wrappers and possible web service interfaces

**Notes:**
ADaM is primarily a collection of tools for data mining and image processing. 
It was originally conceived for the analysis of imagery, such as satellite data. 
Because of this, many of the modules are optimized to enable handling very large datasets typically associated with Earth science domains. 
ADam has over 75 interoperable mining and image processing components. 
Many of them are general purpose.
Each component comes with with a C++ application programming interface - an executable that supports scripting tools like Perl, Python, Tcl, and Unix Shell. 
ADaM components are lightweight and autonomous. 
They have been used successfully in a grid environment.
ADaM has several translation components that provide data level interoperability with other mining systems such as WEKA and Orange, and point tools such as libSVM and svmLight.
Future versions will include Python wrappers and web service interfaces.

## ADaM 4.0 Components

**Notes:**
This slide shows some of the pattern recognition and image processing components.

## ADaM Classification

- Definition: Classification is a way to find and develop a procedure for identifying some phenomenon of interest from a known set of training data in a way that can be used in the future to find similar patterns in similar data sets.
- Method:
- Define or categorize classes of data objects.
- Create a model represented by training samples which taken together form a training data set.
- Represent the model mathematically in the form of classification rules, decision trees or formulae.
- Use the model to categorize similar data sets.

**Notes:**
Here is a general description of classification. 

It is a way to find and develop a procedure for identifying some phenomenon of interest from a known set of training data in a way that can be used in the future to find similar patterns in similar data sets.

The approach is to define or categorize classes of data objects. 
Then Create a model represented by training samples which taken together form a training data set. 
Represent the model mathematically in the form of classification rules, decision trees or formulae. 
Use the model to categorize similar data sets. 

## ADaM Classification - Process

- Identify potential features which may characterize the phenomenon of interest
- Generate a set of training instances where each instance consists of a set of feature values and the corresponding class label
- Describe the instances using ARFF file format
- Preprocess the data as necessary (normalize, sample etc.)
- Split the data into training / test set(s) as appropriate
- Train the classifier using the training set
- Evaluate classifier performance using test set
- K-Fold cross validation, leave one out or other more sophisticated methods may also be used for evaluating classifier performance

**Notes:**
Here is more detail about classification with respect to ADaM. I will be showing an example of this approach in a few slides.
The first three steps are problem/domain specific.

First you identify potential features that characterize the phenomenon of interest.
Then generate a set of training instances where each instance consists of a set of feature values and a corresponding class label.
Describe the instances using ARFF file format. A format that uses ... (fill in later after checking the website notes)

The remaining steps are pretty much domain independent.

Preprocess the data as necessary (normalize, sample etc.)
Split the data into training / test sets as appropriate
Train the classifier using the training set
Evaluate the classifier performance using the test set

Evaluation may utilize K-Fold cross validation, leave one out, or other more sophisticated methods. 
K-Fold cross validation involves partitioning the data into K sets, then creating and evaluating a classifier K times.
At each iteration, all but one of the K sets is used for training, with the remaining set used for classification.

## Naïve Bayes Classification - 1

- Classification problem with m classes C1, C2, ... Cm
- Given an unknown sample X, the goal is to choose a class that is most likely based on statistics from training data

- P(Ci | X) can be computed using Bayes’ Theorem:

- [1] Equations from J. Han and M. Kamber, “Data Mining: Concepts and Techniques”, Morgan Kaufmann, 2001.

**Notes:**
Bayesian classification is based on Bayes Theorem. 
These classifiers are useful in predicting the probability that a sample belongs to a particular class or grouping. 
This technique tends to be highly accurate and fast, making it useful on large databases. 

This chart shows the simplest form of Bayes classifier – the Naive Bayes Classification.
This classification problem has m classes.
Given an unknown sample X, the goal is to choose a class that is most likely based on statistics from training data.

[Point out the parts of the equations and tell their purpose from the below.]

P(Ci | X) This is the probability of class Ci given sample X. 
P(X | Ci) This is the probability of vector X given that it is of class Ci
P(Ci) This is the prior probability of class Ci
P(X) This is the prior probability of vector X 

## Naïve Bayes Classification - 2

- P(X) is constant for all classes, so finding the most likely class amounts to maximizing P(X | Ci) P(Ci)
- P(Ci ) is the prior probability of class i. If the probabilities are not known, equal probabilities can be assumed.
- Assuming attributes are conditionally independent:

- P(xk | Ci) is the probability density function for attribute k

- [1] Equation from J. Han and M. Kamber, “Data Mining: Concepts and Techniques”, Morgan Kaufmann, 2001.

**Notes:**
Here is the derivation of the equation for the naive Bayesian Classification. 
The formula multiplies the component probabilities together, which only works if the attributes are class conditionally independent.

P(X) is constant for all classes, so finding the most likely class amounts to maximizing P(X | Ci) P(Ci) 
P(Ci ) is the prior probability of class i. If the probabilities are not known, equal probabilities can be assumed. 
Assuming attributes are conditionally independent the equation shown here holds.

## Naïve Bayes Classification - 3

- P(xk | Ci) is estimated from the training samples
- Categorical Attributes (non-numeric attributes)
- Estimate P(xk | Ci) as percentage of samples of class i with value xk
- Training involves counting percentage of occurrence of each possible value for each class
- Numeric attributes
- Also use statistics of the sample data to estimate P(xk | Ci)
- Actual form of density function is generally not known, so Gaussian density is often assumed
- Training involves computation of mean and variance for each attribute for each class

**Notes:**
P(xk | Ci) is the probability density function for attribute k.  It is estimated from the training samples. 

Categorical attributes have discrete values, while numeric attributes have continuous values.
Categorical Attributes (non-numeric attributes) 
Estimate P(xk | Ci) as percentage of samples of class i with value xk 
Training involves counting percentage of occurrence of each possible value for each class 
Numeric attributes 
Also use statistics of the sample data to estimate P(xk | Ci) 
Actual form of density function is generally not known, so Gaussian density is often assumed 
Training involves computation of mean and variance for each attribute for each class 

## Naïve Bayes Classification - 4

- Gaussian distribution for numeric attributes:

- Where        is the mean of attribute k observed in samples of class Ci
- And       is the standard deviation of attribute k observed in samples of class Ci

- [1] Equation from J. Han and M. Kamber, “Data Mining: Concepts and Techniques”, Morgan Kaufmann, 2001.

**Notes:**
Since the actual form of density function is generally not known, a Gaussian density, as shown here, is often assumed.

The mu and sigma values are learned during training and are based on the samples.
Note that there is one mu and sigma value per class Ci

## ADaM Classification - Example

- Starting with an ARFF file, the ADaM system will be used to create a Naïve Bayes classifier and evaluate it
- The source data will be an ARFF version of the Wisconsin breast cancer data from the University of California Irvine (UCI) Machine Learning Database:
- 		http://www.ics.uci.edu/~mlearn/MLRepository.html
- The Naïve Bayes classifier will be trained to distinguish malignant vs. benign tumors based on nine characteristics

**Notes:**
Now for an example of ADaM classification. The Naïve Bayes classifier will be trained to distinguish malignant vs. benign tumors. 

This example uses a real-world dataset involving information collected about breast cancer patients. 
This is a classic data set provided by the University of California Irvine (UCI) Machine Learning group.
 In order to use the data with ADaM the original data set was converted to ARFF data format. 
ARFF is also the format commonly used by the WEKA data mining tools. You will see this format on the next slide.
A description of the original data set used for this example is also available.

## Sample Data Set – ARFF Format

**Notes:**
Here is a screenshot of the data set that will be used in the example.
It is in ARFF format – note the attribute name and value at the top.
The Naïve Bayes classifier will be trained to distinguish malignant vs. benign tumors based on the nine characteristics shown from left to right. The tenth column on the far right is the class attribute variable that is to be predicted from the others.
2 = Benign
4 = Malignant

The data are arranged so that one row is a pattern. 
Each attribute is a column in the data
Attribute values range from 1-10 (except for the class which is 2 or 4)

## Preparing the Training Samples

- ADaM has utilities for splitting data sets into disjoint groups for training and testing classifiers
- The simplest is ITSC_Sample, which splits the source data set into two disjoint subsets

**Notes:**
It is necessary to split the data set into disjoint groups for training and testing classifiers.
The ADaM utility - ITSC_Sample - will accomplish this splitting.
ITSC_Sample splits the data into two sets. 
The user can control the size of the sets.
Random sampling is important to avoid artifacts in the original arrangement of the data.

Here is a screenshot showing the help information that is printed when no arguments are given on the command line.
For a complete list of the options for this module or any other ADaM modules simply type the command name at a console window, with no arguments.

## Splitting the Samples

- We will split the breast cancer data set into two groups, one with 2/3 of the patterns and another with 1/3 of the patterns:
-         ITSC_Sample -c class -i bcw.arff -o trn.arff -t tst.arff –p 0.66
- The –i argument specifies the input file name
- The –o and –t arguments specify the names of the two output files (-o = output one, -t = output two)
- The –p argument specifies the portion of data that goes into output one (trn.arff), the remainder goes to output two (tst.arff)
- The –c argument tells the sample program which attribute is the class attribute

**Notes:**
The ADaM module ITSC_Sample breaks the data set into training and test cases. 
A 2/3 split like we have used here is common.

The ADaM modules are invoked through the command line as shown here with the ITSC_Sample module.
This makes it easy to create scripts that access ADaM components, as we will see in a few more slides.

In this command line invocation for ITSC_Sample the following arguments are:
-c is a name for the class – just named class here
-i indicates the input data set file – the breast cancer test file bcw.arff
-o specifies the output training data file name – trn.arff
-t the output test data file name – tst.arff
and -p indicates what portion of the samples are in set one. 

The outputs of the ITSC_Sample module are also formatted in ARFF and are just subsets of the original data set file. 

C:\> ITSC_Sample -c class -i bcw.arff -o trn.arff -t tst.arff -p 0.66
For a complete list of the options for this module or any other ADaM modules simply keyin the command name at a console window, with no arguments, and the module provides usage statement output - see example below.

## Training the Classifier - 1

- Training involves computation of the mean vector and covariance matrix for each class
- The mean of an attribute for a class is the sum of the attribute values divided by number of instances
- The mean vector is the vector of attribute means
- The variance of an attribute is the sum of the squared deviation from the mean for that attribute over patterns in the class
- The covariance of a pair of attributes is the sum of the deviation from the mean of attribute one * the deviation from the mean of attribute two over class patterns

**Notes:**
Training methods build a model from the training set.

Training involves computation of the mean vector and covariance matrix for each class 
The mean of an attribute for a class is the sum of the attribute values divided by number of instances 
The mean vector is the vector of attribute means 
The variance of an attribute is the sum of the squared deviation from the mean for that attribute over patterns in the class 
The covariance of a pair of attributes is the sum of the deviation from the mean of attribute one * the deviation from the mean of attribute two over class patterns

## Training the Classifier - 2

- ADaM has several different types of classifiers
- Each classifier has a training method and an application method
- Here we show one – the Naïve Bayes classifier with the following syntax:

**Notes:**
Each ADaM classifier has a training method and an application method.
The application method will be used to evaluate the accuracy of the model, and again when actually classifying unknown patterns.
Here you see the ADaM training module ITSC_NaiveBayesTrain.

It uses the input pattern from training data to train the Naive Bayes classifier. 
The resulting parameters of the training operation are written to an output file for later use. 
The screenshot shows the help information that is printed when no arguments are given on the command line.

---------------
This partition of functions is common for PR tools, including 3d party ones like libSVM and svmLight. 
In other software, the functions reside in a single program which is run with different options for the 2 modes

## Training the Classifier - 3

- Training the Naïve Bayes classifier:
-         ITSC_NaiveBayesTrain -c class -i trn.arff –b bayes.txt
- The –i argument specifies the input file name
- The –c argument specifies the name of the class attribute
- The –b argument specifies the name of the classifier file:
- The output of the ITSC_NaiveBayesTrain module is shown here.

**Notes:**
This slide shows the invocation of NaiveBayesTrain.

-c provides the name of the class attribute
-i indicates the input training data set file – trn.arff
-b specifies the name of the output Bayes classifier file – bayes.txt
The file bayes.txt is produced by the training method
There are 3 sections: Means, Sdev and Priors
The Mean section has one vector per class (the mean of each attribute over samples of the class)
The Sdev section has one vector per class (the standard deviation for each attribute over samples of the class)
The priors section has the prior probability of each class based on the training sample
The attributes are arranged in the same order as they appear in the ARFF file

C:\> ITSC_NaiveBayesTrain -c class -i trn.arff -b bayes.txt

## Applying the Classifier - 1

- Use mean and covariance to compute P(X | Ci), the probability that pattern X occurs in class Ci for all classes
- The prior probabilities P(Ci) are already known
- Assign the pattern X to the class for which P(X | Ci) P(Ci) is maximal
- Note that P(X | Ci) is high when X is close to the mean of the class.
- The variance is used to adjust for how much patterns within the class typically vary.
- A pattern may be assigned to a class for which the mean is further away than the mean of some other class!

**Notes:**
The next step is to apply the new classifier to the data set.

Use mean and covariance to compute P(X | Ci), the probability that pattern X occurs in class Ci for all classes 
The prior probabilities P(Ci) are already known 
Assign the pattern X to the class for which P(X | Ci) P(Ci) is maximal 
Note that P(X | Ci) is high when X is close to the mean of the class. 
The variance is used to adjust for how much patterns within the class typically vary. 
A pattern may be assigned to a class for which the mean is further away than the mean of some other class!

## Applying the Classifier - 2

- Once trained, the Naïve Bayes classifier can be used to classify unknown instances
- The syntax for ADaM’s Naïve Bayes classifier is as follows:

**Notes:**
The ADaM module ITSC_NaiveBayesApply applies a trained classifier to a specified data file. 
Each pattern in the set is classified and the resulting classified data file is written to an output file. 
The screenshot shows the help information that is printed when no arguments are given on the command line.

## Applying the Classifier - 3

- The classifier is run as follows:
-    ITSC_NaiveBayesApply -c class -i trn.arff –b bayes.txt -o res_trn.arff
- The –i argument specifies the input file name
- The –c argument specifies the name of the class attribute
- The –b argument specifies the name of the classifier file
- The –o argument specifies the name of the result file:

**Notes:**
Applying the Naïve Bayes Classifier to the data file. 
Here we use the bcw training dataset as input where:
-c specifies the name of the class attribute.
-i indicates the input training data set file – trn.arff
-b specifies the name of the input Bayes classifier file – bayes.txt
	The –b option should specify the name of the file created by the training procedure
-o specifies the name of the result file – res_trn.arff

The result file is in the same format as the original input file, with the value of the class attribute replaced by the classifier.

## Evaluating Classifier Performance

- By applying the classifier to a test set where the correct class is known in advance, it is possible to compare the expected output to the actual output.
- The ITSC_Accuracy utility performs this function:

**Notes:**
ITSC_Accuracy is an ADaM module that will compute the overall accuracy of a classifier. It compares the classes of a truth set to those produced by a classifier. It generates a confusion matrix and an overall accuracy. 

## Evaluating Classifier Performance

- ITSC_Accuracy is run as follows:
-      ITSC_Accuracy -c class -t res_tst.arff –v tst.arff –o acc_tst.txt
- -c provides the name of the class attribute.
- -t  indicates the input classified data set file,
- -v specifies the name of a valid input file,
- -o is the output filename that holds the results of the comparison

**Notes:**
ITSC_Accuracy is run as follows:

     ITSC_Accuracy -c class -t res_tst.arff –v tst.arff –o acc_tst.txt

-c provides the name of the class attribute.
-t  indicates the input classified data set file, 
-v specifies the name of a valid input file, 
-o is the output filename that holds the results of the comparison

## Evaluating Classifier Performance

- Output from example

**Notes:**
The output of the ITSC_Accuracy module is illustrated here.
Note that the accuracy of this example classification was 96.2%. 

POD = probability of detection
FAR = false alarm rate
CSI = critical success index
HSS = Heideke skill score

Note that numbers on the diagonal are correct classifications, while those off diagonal are errors
Size of the confusion matrix is N by N where N is the number of classes

## Python Script for Workflow

**Notes:**
Using Python to Script the Workflow 

Using Python, or some other scripting language, the ADaM modules can easily be combined into a workflow that performs all the steps necessary to sample, train, classify and evaluate the Naive Bayes Classifier for a data set. 
Here is an example Python script for the previous bcw data classification example. 

This script expects one input - the base name (without the extension) of the input data file. The script generates a usage statement if the input name is not provided. 

C:\>NaiveBayes.py Usage: NaiveBayes.py "basename" 

The script assumes that the location of the ADaM module executables is specified in the local system's PATH variable. 

## ADaM Image Classification

- Classification of image data is a bit more involved, as there is an additional set of steps that must be performed to extract useful features from the images before classification can be performed.
- In addition, it is also useful to transform the data back into image format for visualization purposes.
- As an example problem, we will consider detection of cumulus cloud fields in GOES satellite images
- GOES satellites produce a 5 channel image every 15 minutes
- The classifier must label each pixel as either belonging to a cumulus cloud field or not based on the GOES data
- Algorithms based on spectral properties often miss cumulus clouds because of the low resolution of the IR channels and the small size of clouds
- Texture features computed from the GOES visible image provide a means to detect cumulus cloud fields.

**Notes:**
Here a feature = an attribute used by a classifier
IR channels have resolution of 4km, while visible channel has resolution of 1km
Since small scale cumulus clouds can be smaller than 4km, the spectral methods miss them
Also, texture is needed to identify fields (rather than individual clouds) 

## GOES Images - Preprocessing

- Segmentation is based only on the high resolution (1km) visible channel.
- In order to remove the effects of the light reflected from the Earth’s surface, a visible reference background image is constructed for each time of the day.
- The reference image is subtracted from the visible image before it is segmented.
- GOES image patches containing cumulus cloud regions, other cloud regions, and background were selected
- Independent experts labeled each pixel of the selected image patches as cumulus cloud or not
- The expert labels were combined to form a single “truth” image for each of the original image patches. In cases where the experts disagreed, the truth image was given a “don’t know” value

**Notes:**
Preprocessing is done to remove effect of highly reflective objects on the ground that might look like small cumulus clouds

## GOES Images - Example

- GOES Visible Image

- Expert Labels

**Notes:**
Red = not cumulus cloud field
Blue = cumulus cloud field
Black = areas where experts disagreed

## Image Quantization

- Some texture features perform better when the image is quantized to some small number of levels before the features are computed.
- ITSC_RelLevel performs local image quantization

**Notes:**
Statistical texture features like gray level cooccurrence, gray level run length, and association rules require quantization to work well

## Image Quantization

- For this demo, we will reduce the number of levels from 256 to just three using local image statistics:
-         ITSC_RelLevel –d  -s 30 –i src.bin –o q4.bin –k
- The –i argument specifies the input file name
- The –o argument specifies the output file name
- The –d argument tells the program to use standard deviation to set the cutoffs instead of a fixed value
- The –k option tells the program to keep values in the range 0, 1, 2 rather than normalizing to 0..1.
- The –s argument indicates the size of the local area used to compute statistics

**Notes:**
The area should be large enough to avoid noise in the estimate but small enough to avoid DC effects (gradual changes in background over the image)

## Computing Texture Features

- ADaM is currently able to compute five different types of texture features: gray level cooccurrence, gray level run length, association rules, Gabor filters, and MRF models
- The syntax for gray level run length computation is:

**Notes:**
GLRL is a statistical texture feature that generally performs better than GLCM and is very simple to compute

## Computing Texture Features

- For this demo, we will compute gray level run length features using a tile size of 25:
-         ITSC_Glrl –i q4.bin –o glrl.arff –l 3 –B –t 25
- The –i argument specifies the input file name
- The –o argument specifies the output file name
- The –l argument tells the program the number of levels in the input image
- The –B option tells the program to write a binary version of the ARFF file (default is ASCII)
- The –t argument indicates the size of the tiles used to compute the gray level run length features

**Notes:**
Tile size is the area used to compute the statistics
The statistics are computed at each pixel using a tile centered on the pixel

## Converting the Label Images

- Since the labels are in the form of images, it is necessary to convert them to vector form
- ITSC_CvtImageToArff will do this:

**Notes:**
The object is to get the data into a form where a data mining or pattern recognition tool can handle it
DM tools expect pattern vectors

## Converting the Label Images

- The labels can be converted to vector form using:
-         ITSC_CvtImageToArff –i lbl.bin –o lbl.arff -B
- The –i argument specifies the input file name
- The –o argument specifies the output file name
- The –B argument tells the program to write the output file in binary form (default is ASCII)

**Notes:**
The –B option produces a binary file which is smaller than the equivalent ASCII file but not human readable

## Labeling the Patterns

- Once the labels are in vector form, they can be appended to the patterns produced by ITSC_Glrl
- ITSC_LabelPatterns will do this:

**Notes:**
This simply assigns a class label to each feature vector produced by GLRL
Note that there is one pattern vector per pixel in the original image

## Labeling the Patterns

- The labels are assigned to patterns as follows:
- ITSC_LabelPatterns –i glrl.arff –c class –l lbl.bin –L lbl.arff –o all.arff –B
- The –i argument specifies the input file name (patterns)
- The –o argument specifies the output file name The –c argument
- The –c argument specifies the name of the class attribute in the pattern set
- The –l argument specifies the name of the label attribute in the label set
- The –L argument specifies the name of the input label file
- The –B argument tells the program to write the output file in binary form (default is ASCII)

**Notes:**
The class labels are the pixel values in the expert labeled image 

## Eliminating “Don’t Know” Patterns

- Some of the original pixels were classified differently by different experts and marked as “don’t know”
- The corresponding patterns can be removed from the training set using ITSC_Subset:

**Notes:**
Don’t know pixels cannot be used in training or evaluation
This is a subset by value function

## Eliminating “Don’t Know” Patterns

- ITSC_Subset is used to remove patterns with unclear class assignment. The subset is generated based on the value of the class attribute:
-         ITSC_Subset –i all.arff –o subset.arff –a class –r 0 1 -B
- The –i argument specifies the input file name
- The –o argument specifies the output file name
- The –a argument tells which attribute to test
- The –r argument tells the legal range of the attribute
- The –B argument tells the program to write the output file in binary form (default is ASCII)

**Notes:**
The –r option specifies the legal range for the attributes
The program could be used to validate data points based on more than one attribute, but in this case only the class attribute is of interest

## Selecting Random Samples

- Random samples are selected from the original training data using the same ITSC_Sample program shown in the previous demo
- The program is used in a slightly different way:
-         ITSC_Sample –i subset.arff –c class –o s1.arff –n 2000
- The –i argument specifies the input file name
- The –o argument specifies the output file name
- The –c argument specifies the name of the class attribute
- The –n option tells the program to select an equal number of random samples (in this case 2000) from each class.

**Notes:**
The original samples may be heavily skewed with many more samples from one class than another
That can bias the Bayes classifier (especially if the Bayes classifier actually uses the priors)
Also, we want to make sure that we have a statistically valid sample of each class

## Python Script for Sample Creation

**Notes:**
This script takes the original image and expert labels and produces a data set usable by a classifier (as in the previous demo)

## Merging Samples / Multiple Images

- The procedure up to this point has created a random subset of points from a particular image. Subsets from multiple images can be combined using ITSC_MergePatterns:

**Notes:**
Since we want to used multiple image patches for training and testing, we have to merge the samples produced from multiple images
The ITSC_MergePatterns program just concatenates the patterns from the source sets into one destination set

## Merging Samples / Multiple Images

- Multiple pattern sets are merged using the following command:
-         ITSC_MergePatterns –c class –o merged.arff –i s1.arff s2.arff
- The –i argument specifies the input file names
- The –o argument specifies the output file name
- The –c argument specifies the name of the class attribute

**Notes:**
This sample just uses 2 images
When we did this for real, we used 30

## Python Script for Training

**Notes:**
This script merges the samples, partitions them into training and test sets, trains a classifier, applies it, and evaluates the result
It is quite similar to the one in the previous sample

## Results of Classifier Evaluation

- The results of running this procedure using five sample images of size 500x500 is as follows:

**Notes:**
Note that the confusion matrix is of the same form as before
Accuracy of close to 90% is good for this application, especially since this is a pixel by pixel comparison, and the experts agreed with one another at about the same level
Most errors are at the borders of regions where the samples the classifier is seeing are mixed

## Applying the Classifier to Images

- Once the classifier is trained, it can be applied to segment images. One further program is required on the end to convert the classified patterns back into an image:

**Notes:**
This converts the classified patterns back into image form
Only the class label is used to produce the image

## Python Function for Segmentation

**Notes:**
This script does end to end image classification and segmentation based on the classifier previously derived
This is the part of the application that would be operational

## Sample Image Results

- Expert Labels

- Segmentation Result

**Notes:**
Looks pretty good 
Similar results are achieved using Gabor filter and association rule features
GLCM features did not work as well for this application

## Remarks 

- The procedure illustrated here is one specific example of ADaM’s capabilities
- There are many other classifiers, texture features and other tools that could be used for this problem
- Since all of the algorithms of a particular type work in more or less the same way, the same general procedure could be used with other tools

## Data Mining Examples From ITSC

## Mining on Data Ingest: Tropical Cyclone Detection

**Advanced Microwave Sounding Unit (AMSU-A) Data**

**Calibration/**
**Limb Correction/**
**Converted to Tb**

**ADaM Mining **
**Environment**

**Data Archive**

**Result**

***Results are placed on the web, made available to ***
***National Hurricane Center & Joint Typhoon Warning Center,***
***and stored for further analysis***

***Mining Plan:***
**Water cover mask to eliminate land**
**Laplacian filter to compute temperature gradients**
**Science Algorithm to estimate wind speed**
**Contiguous regions with wind speeds above a desired threshold identified**
**Additional test to eliminate false positives**
**Maximum wind speed and location produced**

**Hurricane Floyd**

**Further Analysis**

**Knowledge**
**Base**

- _[http://pm-esip.msfc.nasa.gov/cyclone](http://pm-esip.msfc.nasa.gov/cyclone)__  _

**Notes:**
This is the Roy Spencer procedure

## Mesocyclone Signature Detection Using ADaM

_*Problem*_:  Detecting mesocyclone signatures in Radar data
_*Science Rationale*_:  Improved accuracy and reduced false alarm rate for indicators of tornadic activity
_*Technique*_:  Developing an algorithm based on wind velocity shear signatures

**Notes:**
Weather forecasters can be overwhelmed with data volumes during severe weather events.  ITSC is pursuing automating methods of detecting mesocyclone signatures to assist human analysis of severe weather conditions.

## Data Mining / Earth Science Collaboration: Classification Based on Texture Features

- Cumulus cloud fields have a very characteristic texture signature in the GOES visible imagery

- Science Rationale:  Man-made changes to land use cause changes in weather patterns, especially cumulus clouds
- Comparison based on
- Accuracy of detection
- Amount of time required to classify

## Data Mining / Space Science Collaboration: Boundary Detection and Quantification

- Analysis of polar cap auroras in  large volumes of spacecraft UV images
- Scientific Rationale:
- Indicators to predict geomagnetic storm
- Damage satellites
- Disrupt radio connection
- Developing different mining algorithms to detect and quantify polar cap boundary

- Polar Cap Boundary

## Data Mining / BioInformatics Collaboration: Genome Patterns

**Data Mining / BioInformatics Collaboration: ****Genome Patterns**

**Mining**
**Results:**
**MCSs**

**Genome DB**

- Scientists

_**Text Pattern Recognition: **_
**Used to search for text patterns in bioscience data as well as other text documents.**

## Additional Information

- Website
- www.itsc.uah.edu
- Dr. Sara Graves
- sgraves@itsc.uah.edu
- Danny Hardin
- dhardin@itsc.uah.edu