Load a ML model into InterSystems IRIS
Tags: analytics, ai, beginners, machinelearning
Published: 2022-01-25 19:02:56

Hi all. Today we are going to upload a ML model into IRIS Manager and test it.

Note: I have done the following on Ubuntu 18.04, Apache Zeppelin 0.8.0, Python 3.6.5.

### **Introduction**

These days many available different tools for Data Mining enable you to develop predictive models and analyze the data you have with unprecedented ease. InterSystems IRIS Data Platform provide a stable foundation for your big data and fast data applications, providing interoperability with modern DataMining tools. 

In this series of articles we explore Data mining capabilities available with InterSystems IRIS. In the first [article](https://community.intersystems.com/post/way-launch-apache-spark-apache-zeppelin-intersystems-iris) we configured our infrastructure and got ready to start. In the second [article](https://community.intersystems.com/post/k-means-clustering-iris-dataset) we built our first predictive model that predicts species of flowers using instruments from Apache Spark and Apache Zeppelin. In this article we will build a KMeans PMML model and test it in InterSystems IRIS.

Intersystems IRIS provides PMML execution capabilities. So, you can upload your model and test it against any data using SQL queries. It will show accuracy, precision, F-score and more.

### **Check requirements**

First, download [jpmml](https://github.com/jpmml/pyspark2pmml#configuration-and-usage) (look at the table and select suitable version) and move it to any directory. If you use Scala, it will be enough.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fd8khxhftwk495mdxgxf.png)

If you use Python, run the following in the terminal

> pip3 install --user --upgrade git+https://github.com/jpmml/pyspark2pmml.git

After success message go to **Spark Dependencies** and add dependence to downloaded jpmml:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/837dztsf367hugsbr126.png)

### **Create KMeans model** 

PMML builder uses pipelines, so I changed the code written in the previous [article](https://community.intersystems.com/post/k-means-clustering-iris-dataset) a bit. Run the following code in Zeppelin:

> %pyspark  
> from pyspark.ml.linalg import Vectors  
> from pyspark.ml.feature import VectorAssembler  
> from pyspark.ml.clustering import KMeans  
> from pyspark.ml import Pipeline  
> from pyspark.ml.feature import RFormula  
> from pyspark2pmml import PMMLBuilder
> 
> dataFrame=spark.read.format("com.intersystems.spark").\  
> option("url", "IRIS://localhost:51773/NEWSAMPLE").option("user", "dev").\  
> option("password", "123").\  
> option("dbtable", "DataMining.IrisDataset").load() # load iris dataset
> 
> (trainingData, testData) = dataFrame.randomSplit([0.7, 0.3]) # split the data into two sets  
> assembler = VectorAssembler(inputCols = ["PetalLength", "PetalWidth", "SepalLength", "SepalWidth"], outputCol="features") # add a new column with features
> 
> kmeans = KMeans().setK(3).setSeed(2000) # clustering algorithm that we use
> 
> pipeline = Pipeline(stages=[assembler, kmeans]) # First, passed data will run against assembler and after will run against kmeans.  
> modelKMeans = pipeline.fit(trainingData) # pass training data
> 
> pmmlBuilder = PMMLBuilder(sc, dataFrame, modelKMeans)  
> pmmlBuilder.buildFile("KMeans.pmml") # create pmml model

It will create a model, that predicts Species using PetalLength, PetalWidth, SepalLength, SepalWidth as features. It uses PMML format. 

PMML is an XML-based predictive model interchange format that provides a way for analytic applications to describe and exchange predictive models produced by data mining and machine learning algorithms. It allows us to separate model building from model execution.

In the output, you will see a path to the PMML model.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r3vc518vzwz7fjf3k560.png)

### **Upload and test the PMML model**

Open **IRIS manager** -> **Menu** -> **Manage Web Applications** -\`> click on your **namespace** -> enable **Analytics** -> **Save**.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x9tmfxfqwounyi44runl.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/edt1vl0igbrwggplnj90.png)

Now, go to **Analytics** -> **Tools **-> **PMML Model Tester**


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5t7g516nw5w3x5x36fa0.png)

You should see something like the image below:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wl9aqmrtrs3s7fsu9lwx.png)

Click on **New** -> write a class name, upload PMML file (the path was in the output), and click on **Import** . Paste the following SQL querie in **Custom data source** :

> SELECT PetalLength, PetalWidth, SepalLength, SepalWidth, Species,  
>  CASE Species  
>   WHEN 'Iris-setosa' THEN 0  
>   WHEN 'Iris-versicolor' THEN 2  
>   ELSE 1  
>  END  
> As prediction  
> FROM DataMining.IrisDataset

We use CASE here because KMeans clustering returns clusters as numbers (0, 1, 2)  and if we do not replace species to numbers it will count it incorrectly. Please comment if you know how can I replace сluster number with a species name.

My result is below:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hie9pqqdou7doqs2ziyi.png)

There you can look at detailed analytics:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ue8nhxv6d1pe3nnjsqmk.png) 

If you want to know better what is **true positive**, **false negative**, **etc**, read [Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall).

### **Conclusion**

We have found out that PMML Model Tester is very useful tool to test your model against data. It provides detailed analytics, graphs, and SQL executor. So, you can test your model without any extended tool.

### **Links**

[Previous article](https://community.intersystems.com/post/k-means-clustering-iris-dataset)

[PySpark2PMML](https://github.com/jpmml/pyspark2pmml)

[JPMML](https://github.com/jpmml/jpmml-sparkml)

[ML Pipelines](https://spark.apache.org/docs/latest/ml-pipeline.html)

[Apache Spark documentation](https://spark.apache.org/docs/latest/)