ibm-research/WatsonxDocsQARetrieval huggingface dataset:
# Using an Analysis node #

You can assess the accuracy of the model using an Analysis node\. From the Palette, under Outputs, place an Analysis node on the canvas and attach it to the C5\.0 model nugget\. Then right\-click the Analysis node and select  Run\.

Figure 1\. Analysis node

![Analysis node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_analysis.png)The Analysis node output shows that with this artificial dataset, the model correctly predicted the choice of drug for every record in the dataset\. With a real dataset you are unlikely to see 100% accuracy, but you can use the Analysis node to help determine whether the model is acceptably accurate for your particular application\.

Figure 2\. Analysis node output

![Analysis node output](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_analysis_output.png)
<!-- </article "role="article" "> -->
# Browsing the model #

When the C5\.0 node runs, its model nugget is added to the flow\. To browse the model, right\-click the model nugget and choose  View Model\.

The Tree Diagram displays the set of rules generated by the C5\.0 node in a tree format\. Now you can see the missing pieces of the puzzle\. For people with an `Na-to-K` ratio less than `14.829` and high blood pressure, age determines the choice of drug\. For people with low blood pressure, cholesterol level seems to be the best predictor\.

Figure 1\. Tree diagram

![Tree diagram](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_browse_tree.png)

You can hover over the nodes in the tree to see more details such as the number of cases for each blood pressure category and the confidence percentage of cases\.

<!-- </article "role="article" "> -->
# Building a model #

By exploring and manipulating the data, you have been able to form some hypotheses\. The ratio of sodium to potassium in the blood seems to affect the choice of drug, as does blood pressure\. But you cannot fully explain all of the relationships yet\. This is where modeling will likely provide some answers\. In this case, you will try to fit the data using a rule\-building model called  C5\.0\.

Since you're using a derived field, `Na_to_K`, you can filter out the original fields, `Na` and `K`, so they're not used twice in the modeling algorithm\. You can do this by using a Filter node\.

<!-- <ol> -->

1.  Place a Filter node on the canvas and connect it to the Derive node\.
    
    Figure 1. Filter node
    
    ![Derive node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_build_flow.png)
2.  Double\-click the Filter node to edit its properties\. Name it  Discard Fields\.
3.  For Mode, make sure  Filter the selected fields is selected\. Then select the `K` and `Na` fields\. Click `Save`\.
4.  Place a Type node on the canvas and connect it to the Filter node\. With the Type node, you can indicate the types of fields you're using and how they're used to predict the outcomes\.
    
    Figure 2. Type node
    
    ![Type node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_build_flow2.png)
5.  Double\-click the Type node to edit its properties\. Name it  Define Types\.
6.  Set the role for the `Drug` field to  Target, indicating that `Drug` is the field you want to predict\. Leave the role for the other fields set to  Input so they'll be used as predictors\. Click  Save\.
7.  To estimate the model, place a C5\.0 node on the canvas and attach it to the end of the flow\. Then click the  Run button on the toolbar to run the flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Reading in text data #

<!-- <ol> -->

1.  You can read in delimited text data using a Data Asset import node\. From the  Palette, under  Import, add a Data Asset node to your flow\.
2.  Double\-click the node to display its properties and select the data file  drug1n\.csv\.
3.  Now that you've added the data file, you may want to glance at the values for some of the records\. One way to do this is by building a flow that includes a Table node\. An easier way is to simply right\-click the Data Asset node you just added and select  Preview\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Creating a distribution chart #

During data mining, it is often useful to explore the data by creating visual summaries\.  Watson Studio offers many different types of charts to choose from, depending on the kind of data you want to summarize\. For example, to find out what proportion of the patients responded to each drug, use a Distribution node\.

Figure 1\. Distribution node

![Distribution node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_distribution.png)

<!-- <ol> -->

1.  Under  Graphs on the Palette, add a  Distribution node to the flow and connect it to the  drug1n\.csv Data Asset node\. Then double\-click the node to edit its options\.
2.  Select  Drug as the target field whose distribution you want to show\. Then click  Save, right\-click the Distribution node, and select  Run\. A distribution chart is added to the  Outputs panel\.

<!-- </ol> -->

The chart helps you see the shape of the data\. It shows that patients responded to drug `Y` most often and to drugs `B` and `C` least often\.

Alternatively, you can attach and run a Data Audit node for a quick glance at distributions and histograms for all fields at once\. The Data Audit node is available under Outputs on the Palette\.

<!-- </article "role="article" "> -->
# Deriving a new field #

Figure 1\. Scatterplot of drug distribution

![Scatterplot of drug distribution](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_scatterplot.png)

Since the ratio of sodium to potassium seems to predict when to use drug `Y`, you can derive a field that contains the value of this ratio for each record\. This field might be useful later when you build a model to predict when to use each of the five drugs\.

<!-- <ol> -->

1.  To simplify your flow layout, start by deleting all the nodes except the  drug1n\.csv Data Asset node\.
2.  Place a Derive node on the canvas and connect it to the  drug1n\.csv Data Asset node\.
    
    Figure 2. Derive node
    
    ![Derive node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_newfield_flow.png)
3.  Double\-click the Derive node to edit its properties\.
4.  Name the new field Na\_to\_K\. Since you obtain the new field by dividing the sodium value by the potassium value, enter Na/K for the expression\. You can also create an expression by clicking the calculator icon\. This opens the Expression Builder, a way to interactively create expressions using built\-in lists of functions, operands, and fields and their values\.
5.  You can check the distribution of your new field by attaching a Histogram node to the Derive node\. In the Histogram node properties, specify `Na_to_K` as the field to be plotted and `Drug` as the color overlay field\.
    
    Figure 3. Histogram node
    
    ![Histogram node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_newfield_histogram_flow.png)
6.  Right\-click the Histogram node and select  Run\. A histogram chart is added to the Outputs pane\. Based on the chart, you can conclude that when the `Na_to_K` value is around 15 or more, drug `Y` is the drug of choice\.
    
    Figure 4. Histogram chart output
    
    ![Histogram chart output](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_histogram.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Creating a scatterplot #

Now let's take a look at what factors might influence `Drug`, the target variable\. As a researcher, you know that the concentrations of sodium and potassium in the blood are important factors\. Since these are both numeric values, you can create a scatterplot of sodium versus potassium, using the drug categories as a color overlay\.

Figure 1\. Plot node

![Plot node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_scatterplot_flow.png)

<!-- <ol> -->

1.  Place a Plot node on the canvas and connect it to the  drug1n\.csv Data Asset node\. Then double\-click the Plot node to edit its properties\.
2.  Select `Na` as the X field, `K` as the Y field, and `Drug` as the Color (overlay) field\. Click  Save, then right\-click the Plot node and select  Run\. A plot chart is added to the Outputs pane\.
    
    The plot clearly shows a threshold above which the correct drug is always drug `Y` and below which the correct drug is never drug `Y`. This threshold is a ratio -- the ratio of sodium (`Na`) to potassium (`K`).
    
    Figure 2. Scatterplot of drug distribution
    
    ![Scatterplot of drug distribution](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_scatterplot.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Creating advanced visualizations #

The previous three sections use different types of graph nodes\. Another way to explore data is with the advanced visualizations feature\.

You can use the Charts node to launch the chart builder and create advanced charts to explore your data from different perspectives and identify patterns, connections, and relationships within your data\.

Figure 1\. Advanced visualizations

![Advanced visualizations](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_viz.png)
<!-- </article "role="article" "> -->
# Creating a web chart #

Since many of the data fields are categorical, you can also try plotting a web chart, which maps associations between different categories\.

Figure 1\. Web node

![Web node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_web_flow.png)

<!-- <ol> -->

1.  Place a Web node on the canvas and connect it to the  drug1n\.csv Data Asset node\. Then double\-click the Web node to edit its properties\.
2.  Select the fields `BP` (for blood pressure) and `Drug`\. Click  Save, then right\-click the Web node and select  Run\. A web chart is added to the Outputs pane\.
    
    Figure 2. Web graph of drugs vs. blood pressure
    
    ![Web graph of drugs vs. blood pressure](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_web.png)

<!-- </ol> -->

From the plot, it appears that drug `Y` is associated with all three levels of blood pressure\. This is no surprise; you have already determined the situation in which drug `Y` is best\.

But if you ignore drug `Y` and focus on the other drugs, you can see that drugs `A` and `B` are also associated with high blood pressure\. And drugs `C` and `X` are associated with low blood pressure\. And normal blood pressure is associated with drug `X`\. At this point, though, you still don't know how to choose between drugs `A` and `B` or between drugs `C` and `X`, for a given patient\. This is where modeling can help\.

<!-- </article "role="article" "> -->
# Forecasting catalog sales #

A catalog company is interested in forecasting monthly sales of its men's clothing line, based on 10 years of their sales data\.

This example uses the flow  Forecasting Catalog Sales, available in the example project \. The data file is  catalog\_seasfac\.csv\.

We've seen in an earlier tutorial how you can let the Expert Modeler decide which is the most appropriate model for your time series\. Now it's time to take a closer look at the two methods that are available when choosing a model yourself—exponential smoothing and ARIMA\.

To help you decide on an appropriate model, it's a good idea to plot the time series first\. Visual inspection of a time series can often be a powerful guide in helping you choose\. In particular, you need to ask yourself:

<!-- <ul> -->

 *  Does the series have an overall trend? If so, does the trend appear constant or does it appear to be dying out with time?
 *  Does the series show seasonality? If so, do the seasonal fluctuations seem to grow with time or do they appear constant over successive periods?

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Examining the data #

The series shows a general upward trend; that is, the series values tend to increase over time\. The upward trend is seemingly constant, which indicates a linear trend\.

Figure 1\. Actual sales of men's clothing

![Actual sales of men's clothing](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_forecast_series.png)

The series also has a distinct seasonal pattern with annual highs in December, as indicated by the vertical lines on the graph\. The seasonal variations appear to grow with the upward series trend, which suggests multiplicative rather than additive seasonality\.

Now that you've identified the characteristics of the series, you're ready to try modeling it\. The exponential smoothing method is useful for forecasting series that exhibit trend, seasonality, or both\. As we've seen, this data exhibits both characteristics\.

<!-- </article "role="article" "> -->
# Creating the flow #

<!-- <ol> -->

1.  Create a new flow and add a Data Asset node that points to  catalog\_seasfac\.csv\.
2.  Connect a Type node to the Data Asset node and double\-click it to open its properties\.
3.  Click  Read Values\. For the `men` field, set the role to  Target\.
    
    Figure 1. Specifying the target field
    
    ![Specifying the target field](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_forecast_fields.png)
4.  Set the role for all other fields to `None` and click `Save`\.
5.  Attach a Time Plot graph node to the Type node and double\-click it\.
    
    Figure 2. Plotting the time series
    
    ![Plotting the time series](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_forecast_plot.png)
6.  For the Plot, add the field `men` to the  Series list\.
7.  Select  Use custom x axis field label and select `date`\.
8.  Deselect the  Normalize option and click  Save\.
9.  Run the flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Summary #

You've successfully modeled a complex time series, incorporating not only an upward trend but also seasonal and other variations\. You've also seen how, through trial and error, you can get closer and closer to an accurate model, which you can then use to forecast future sales\.

In practice, you would need to reapply the model as your actual sales data are updated—for example, every month or every quarter—and produce updated forecasts\.

<!-- </article "role="article" "> -->
# Introduction to modeling #

A model is a set of rules, formulas, or equations that can be used to predict an outcome based on a set of input fields or variables\. For example, a financial institution might use a model to predict whether loan applicants are likely to be good or bad risks, based on information that is already known about past applicants\.

Video disclaimer: Some minor steps and graphical elements in these videos might differ from your platform\.

[https://video\.ibm\.com/embed/recorded/131116287](https://video.ibm.com/embed/recorded/131116287)

The ability to predict an outcome is the central goal of predictive analytics, and understanding the modeling process is the key to using flows in  Watson Studio\.

Figure 1\. A decision tree model

![A decision tree model](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/spss-tree-diagram-Jun2023.png)

This example uses a decision tree model, which classifies records (and predicts a response) using a series of decision rules\. For example:

    IF income = Medium 
    AND cards <5
    THEN -> 'Good'

While this example uses a CHAID (Chi\-squared Automatic Interaction Detection) model, it is intended as a general introduction, and most of the concepts apply broadly to other modeling types in  Watson Studio\.

To understand any model, you first need to understand the data that goes into it\. The data in this example contains information about the customers of a bank\. The following fields are used:

<!-- <table "summary="" class="defaultstyle" "> -->

| Field name     | Description                                                   |
| -------------- | ------------------------------------------------------------- |
| Credit\_rating | Credit rating: 0=Bad, 1=Good, 9=missing values                |
| Age            | Age in years                                                  |
| Income         | Income level: 1=Low, 2=Medium, 3=High                         |
| Credit\_cards  | Number of credit cards held: 1=Less than five, 2=Five or more |
| Education      | Level of education: 1=High school, 2=College                  |
| Car\_loans     | Number of car loans taken out: 1=None or one, 2=More than two |

<!-- </table "summary="" class="defaultstyle" "> -->

The bank maintains a database of historical information on customers who have taken out loans with the bank, including whether or not they repaid the loans (Credit rating = Good) or defaulted (Credit rating = Bad)\. Using this existing data, the bank wants to build a model that will enable them to predict how likely future loan applicants are to default on the loan\.

Using a decision tree model, you can analyze the characteristics of the two groups of customers and predict the likelihood of loan defaults\.

This example uses the flow named  Introduction to Modeling, available in the example project \. The data file is  tree\_credit\.csv\.

Let's take a look at the flow\.

<!-- <ol> -->

1.  Open the  Example Project\.
2.  Scroll down to the  Modeler flows section, click  View all, and select the  Introduction to Modeling flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Scoring records #

Earlier, we scored the same records used to estimate the model so we could evaluate how accurate the model was\. Now we'll score a different set of records from the ones used to create the model\. This is the goal of modeling with a target field: Study records for which you know the outcome, to identify patterns that will allow you to predict outcomes you don't yet know\.

Figure 1\. Attaching new data for scoring

![Attaching new data for scoring](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_intro_score.png)

You could update the data asset Import node to point to a different data file, or you could add a new Import node that reads in the data you want to score\. Either way, the new dataset must contain the same input fields used by the model (`Age`, `Income level`, `Education` and so on), but not the target field `Credit rating`\.

Alternatively, you could add the model nugget to any flow that includes the expected input fields\. Whether read from a file or a database, the source type doesn't matter as long as the field names and types match those used by the model\.

<!-- </article "role="article" "> -->
# Summary #

This example  Introduction to Modeling flow demonstrates the basic steps for creating, evaluating, and scoring a model\.

<!-- <ul> -->

 *  The modeling node estimates the model by studying records for which the outcome is known, and creates a model nugget\. This is sometimes referred to as training the model\.
 *  The model nugget can be added to any flow with the expected fields to score records\. By scoring the records for which you already know the outcome (such as existing customers), you can evaluate how well it performs\.
 *  After you're satisfied that the model performs acceptably well, you can score new data (such as prospective customers) to predict how they will respond\.
 *  The data used to train or estimate the model may be referred to as the analytical or historical data; the scoring data may also be referred to as the operational data\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Reducing input data string length #

For binomial logistic regression, and auto classifier models that include a binomial logistic regression model, string fields are limited to a maximum of eight characters\. Where strings are more than eight characters, you can recode them using a Reclassify node\.

This example uses the flow named  Reducing Input Data String Length, available in the example project \. The data file is  drug\_long\_name\.csv\.

This example focuses on a small part of a flow to show the type of errors that may be generated with overlong strings, and explains how to use the Reclassify node to change the string details to an acceptable length\. Although the example uses a binomial Logistic Regression node, it is equally applicable when using the Auto Classifier node to generate a binomial Logistic Regression model\.

<!-- </article "role="article" "> -->
# Reclassifying the data #

Figure 1\. Example flow showing string reclassification for binomial logistic regression

![Example flow showing string reclassification for binomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_reducing.png)

<!-- <ol> -->

1.  Add a Data Asset node that points to  drug\_long\_name\.csv\.
2.  Add a Type node after the Data Asset node\. Double\-click the Type node to open its properties, and select `Cholesterol_long` as the target\.
3.  Add a Logistic Regression node after the Type node\. Double\-click the node and select the  Binomial procedure (instead of the default  Multinomial procedure)\.
4.  Right\-click the Logistic Regression node and run it\. An error message warns you that the `Cholesterol_long` string values are too long\. When you encounter this type of message, follow the procedure described in the rest of this example to modify your data\.
    
    Figure 2. Error message displayed when running the binomial logistic regression node
    
    ![Error message displayed when running the binomial logistic regression node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_reducing_error.png)
5.  Add a Reclassify node after the Type node and double\-click it to open its properties\.
6.  For the  Reclassify Field, select `Cholesterol_long` and type Cholesterol for the new field name\.
7.  Click  Get values to add the `Cholesterol_long` values to the original value column\.
8.  In the new value column, type High next to the original value of `High level of cholesterol` and Normal next to the original value of `Normal level of cholesterol`\.
    
    Figure 3. Reclassifying long strings
    
    ![Reclassifying long strings](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_reducing_reclassify.png)
9.  Add a Filter node after the Reclassify node\. Double\-click the node, choose  Filter the selected fields, and select the `Cholesterol_long` field\.
    
    Figure 4. Filtering the "Cholesterol\_long" field from the data
    
    ![Filtering the "Cholesterol\_long" field from the data](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_reducing_filter.png)
10. Add a Type node after the Filter node\. Double\-click the node and select `Cholesterol` as the target\.
    
    Figure 5. Short string details in the "Cholesterol" field
    
    ![Short string details in the "Cholesterol" field](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_reducing_type.png)
11. Add a Logistic node after the Type node\. Double\-click the node and select the  Binomial procedure\.

<!-- </ol> -->

You can now run the binomial Logistic node and generate a model without encountering the error as you did before\.

This example only shows part of a flow\. For more information about the types of flows in which you might need to reclassify long strings, see the following example:

<!-- <ul> -->

 *  Auto Classifier node\. See [Automated modeling for a flag target](https://dataplatform.cloud.ibm.com/docs/content/wsd/tutorials/tut_autoflag.html)\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Retail sales promotion #

This example deals with fictitious data that describes retail product lines and the effects of promotion on sales\.

Your goal in this example is to predict the effects of future sales promotions\. Similar to the [condition monitoring example](https://dataplatform.cloud.ibm.com/docs/content/wsd/tutorials/tut_condition.html), the data mining process consists of the exploration, data preparation, training, and test phases\.

This example uses the flow named  Retail Sales Promotion, available in the example project \. The data files are  goods1n\.csv and  goods2n\.csv\.

<!-- <ol> -->

1.  Open the  Example Project\.
2.  Scroll down to the  Modeler flows section, click  View all, and select the  Retail Sales Promotion flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Examining the data #

Each record contains:

<!-- <ul> -->

 *  `Class`\. Product type\.
 *  `Cost`\. Unit price\.
 *  `Promotion`\. Index of amount spent on a particular promotion\.
 *  `Before`\. Revenue before promotion\.
 *  `After`\. Revenue after promotion\.

<!-- </ul> -->

The flow is simple\. It displays the data in a table\. The two revenue fields (`Before` and `After`) are expressed in absolute terms\. However, it seems likely that the increase in revenue after the promotion (and presumably as a result of it) would be a more useful figure\.

Figure 1\. Effects of promotion on product sales

![Effects of promotion on product sales](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_retail_data_effects.png)

The flow also contains a node to derive this value, expressed as a percentage of the revenue before the promotion, in a field called `Increase`\. A table shows this field\.

Figure 2\. Increase in revenue after promotion

![Increase in revenue after promotion](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_retail_data_increase.png)

For each class of product, and almost linear relationship exists between the increase in revenue and the cost of the promotion\. Therefore, it seems likely that a decision tree or neural network could predict, with reasonable accuracy, the increase in revenue from the other available fields\.

<!-- </article "role="article" "> -->
# Learning and testing #

The flow trains a neural network and a decision tree to make this prediction of revenue increase\.

Figure 1\. Retail Sales Promotion example flow

![Retail Sales Promotion example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_retail.png)

After you run the flow to generate the model nuggets, you can test the results of the learning process\. You do this by connecting the decision tree and network in series between the Type node and a new Analysis node, changing the Data Asset import node to point to  goods2n\.csv, and running the Analysis node\. From the output of this node, in particular from the linear correlation between the predicted increase and the correct answer, you will find that the trained systems predict the increase in revenue with a high degree of success\.

Further exploration might focus on the cases where the trained systems make relatively large errors\. These could be identified by plotting the predicted increase in revenue against the actual increase\. Outliers on this graph could be selected using the interactive graphics within  SPSS Modeler, and from their properties, it might be possible to tune the data description or learning process to improve accuracy\.

<!-- </article "role="article" "> -->
# Screening predictors #

The Feature Selection node helps you identify the fields that are most important in predicting a certain outcome\. From a set of hundreds or even thousands of predictors, the Feature Selection node screens, ranks, and selects the predictors that may be most important\. Ultimately, you may end up with a quicker, more efficient model—one that uses fewer predictors, runs more quickly, and may be easier to understand\.

The data used in this example represents a data warehouse for a hypothetical telephone company and contains information about responses to a special promotion by 5,000 of the company's customers\. The data includes many fields that contain customers' age, employment, income, and telephone usage statistics\. Three "target" fields show whether or not the customer responded to each of three offers\. The company wants to use this data to help predict which customers are most likely to respond to similar offers in the future\.

This example uses the flow named  Screening Predictors, available in the example project \. The data file is  customer\_dbase\.csv\.

This example focuses on only one of the offers as a target\. It uses the CHAID tree\-building node to develop a model to describe which customers are most likely to respond to the promotion\. It contrasts two approaches:

<!-- <ul> -->

 *  Without feature selection\. All predictor fields in the dataset are used as inputs to the CHAID tree\.
 *  With feature selection\. The Feature Selection node is used to select the best 10 predictors\. These are then input into the CHAID tree\.

<!-- </ul> -->

By comparing the two resulting tree models, we can see how feature selection can produce effective results\.

<!-- </article "role="article" "> -->
# Building the flow #

Figure 1\. Feature Selection example flow

![Feature Selection example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_screening.png)

<!-- <ol> -->

1.  Add a Data Asset node that points to  customer\_dbase\.csv\.
2.  Add a Type node after the Data Asset node\.
3.  Double\-click the Type node to open its properties, and change the role for `response_01` to  Target\. Change the role to  None for the other response fields (`response_02` and `response_03`) and for the customer ID (`custid`) field\. Leave the role set to  Input for all other fields\.
    
    Figure 2. Adding a Type node
    
    ![Adding a Type node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_screening_target.png)
4.  Click  Read Values and then click  Save\.
5.  Add a Feature Selection modeling node after the Type node\. In the node properties, the rules and criteria used for screening or disqualifying fields are defined\.
    
    Figure 3. Adding a Feature Selection node
    
    ![Adding a Feature Selection node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_screening_criteria.png)
6.  Run the flow to generate the Feature Selection model nugget\.
7.  To look at the results, right\-click the model nugget and choose  View Model\. The results show the fields found to be useful in the prediction, ranked by importance\. By examining these fields, you can decide which ones to use in subsequent modeling sessions\.
8.  To compare results without feature selection, you must add two CHAID modeling nodes to the flow: one that uses feature selection and one that doesn't\. Add two CHAID nodes, one connected to the Type node and the other connected to the Feature Selection model nugget, as shown in the example flow at the beginning of this section\.
9.  Double\-click each CHAID node to open its properties\. Under  Objectives, make sure that  Build new model and  Create a standard model are selected\. Under , select  Custom and set it to 5\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Building the models #

<!-- <ol> -->

1.  Run the CHAID node that uses all the predictors in the dataset (the one connected to the Type node)\. As it runs, notice how long it takes to finish\.
2.  Right\-click the generated model nugget, select  View Model, and look at the tree diagram\.
3.  Now run the other CHAID model, which uses less predictors\. Again, look at its tree diagram\.
    
    It might be hard to tell, but the second model ran faster than the first one. Because this dataset is relatively small, the difference in run times is probably only a few seconds; but for larger real-world datasets, the difference might be very noticeable—minutes or even hours. Using feature selection may speed up your processing times dramatically.
    
    The second tree also contains fewer tree nodes than the first. It's easier to comprehend. Using fewer predictors is less expensive. It means that you have less data to collect, process, and feed into your models. Computing time is improved. In this example, even with the extra feature selection step, model building was faster with the smaller set of predictors. With a larger real-world dataset, the time savings should be greatly amplified.
    
    Using fewer predictors results in simpler scoring. For example, you might identify only four profiles of customers who are likely to respond to the promotion. Note that with larger numbers of predictors, you run the risk of overfitting your model. The simpler model may generalize better to other datasets (although you would need to test this to be sure).
    
    You could instead use a tree-building algorithm to do the feature selection work, allowing the tree to identify the most important predictors for you. In fact, the CHAID algorithm is often used for this purpose, and it's even possible to grow the tree level-by-level to control its depth and complexity. However, the Feature Selection node is faster and easier to use. It ranks all of the predictors in one fast step, allowing you to identify the most important fields quickly.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Making offers to customers (self\-learning) #

The Self\-Learning Response Model (SLRM) node generates and enables the updating of a model that allows you to predict which offers are most appropriate for customers and the probability of the offers being accepted\. These sorts of models are most beneficial in customer relationship management, such as marketing applications or call centers\.

This example is based on a fictional banking company\. The marketing department wants to achieve more profitable results in future campaigns by matching the appropriate offer of financial services to each customer\. Specifically, the example uses a Self\-Learning Response model to identify the characteristics of customers who are most likely to respond favorably based on previous offers and responses and to promote the best current offer based on the results\.

This example uses the flow named  Making Offers to Customers \- Self\-Learning, available in the example project \. The data files are  pm\_customer\_train1\.csv,  pm\_customer\_train2\.csv, and  pm\_customer\_train3\.csv\.

<!-- <ol> -->

1.  Open the  Example Project\.
2.  Scroll down to the  Modeler flows section, click  View all, and select the  Making Offers to Customers \- Self\-Learning flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Building the flow #

<!-- <ol> -->

1.  Add a Data Asset node that points to  pm\_customer\_train1\.csv\.
    
    Figure 1. SLRM example flow
    
    ![SLRM example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_selflearn_slrm.png)
2.  Attach a Filler node to the Data Asset node\. Double\-click the node to open its properties and, under  Fill in fields, select `campaign`\.
3.  Select a  Replace type of  Always\.
4.  In the  Replace with text box, enter to\_string(campaign) and click  Save\.
    
    Figure 2. Derive a campaign field
    
    ![Derive a campaign field](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_selflearn_derive.png)
5.  Add a Type node and set the  Role to `None` for the following fields:
    
    <!-- <ul> -->
    
     *  `customer_id`
     *  `response_date`
     *  `purchase_date`
     *  `product_id`
     *  `Rowid`
     *  `X_random`
    
    <!-- </ul> -->
    
6.  Set the  Role to `Target` for the `campaign` and `response` fields\. These are the fields on which you want to base your predictions\. Set the  Measurement to `Flag` for the `response` field\.
7.  Click  Read Values then click  Save\. Because the campaign field data shows as a list of numbers (1, 2, 3, and 4), you can reclassify the fields to have more meaningful titles\.
8.  Add a Reclassify node after the Type node and open its properties\.
9.  Under  Reclassify Into, select  Existing field\.
10. Under  Reclassify Field, select `campaign`\.
11. Click  Get values\. The campaign values are added to the `ORIGINAL VALUE` column\.
12. In the `NEW VALUE` column, enter the following campaign names in the first four rows:
    
    <!-- <ul> -->
    
     *  Mortgage
     *  Car loan
     *  Savings
     *  Pension
    
    <!-- </ul> -->
    
13. Click  Save\.
    
    Figure 3. Reclassify the campaign names
    
    ![Reclassify the campaign names](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_selflearn_reclassify.png)
14. Attach an SLRM modeling node to the Reclassify node\. Select `campaign` for the  Target field, and `response` for the  Target response field\.
15. Under  MODEL OPTIONS, for  Maximum number of predictions per record, reduce the number to 2\. This means that for each customer there will be two offers identified that have the highest probability of being accepted\.
16. Make sure  Take account of model reliability is selected, then click  Save and run the flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Hotel satisfaction example for Text Analytics #

SPSS Modeler offers nodes that are specialized for handling text\.

In this example, a hotel manager is interested in learning what customers think about the hotel\.

Figure 1\. Chart of positive opinions

![Chart of positive opinions](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_positive.png)

Figure 2\. Chart of negative opinions

![Chart of negative opinions](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_negative.png)

This example uses the flow named  Hotel Satisfaction, available in the example project \. The data files are  hotelSatisfaction\.csv and  hotelSatisfaction\.xlsx\. The flow uses Text Analytics nodes to analyze fictional text data about hotel personnel, comfort, cleanliness, price, etc\.

This flow illustrates two ways of analyzing data with a Text Mining node and a Text Link Analysis node\. It also illustrates how you can deploy a text model and score current or new data\.

Let's take a look at the flow\.

<!-- <ol> -->

1.  Open the \.
2.  Scroll down to the  Modeler flows section and select the  Hotel Satisfaction flow\.
    
    Figure 3. Completed flow
    
    ![Completed flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Building and deploying the model #

<!-- <ol> -->

1.  When your model is ready, click  Generate a model to generate a text nugget\.
    
    Figure 1. Generate a new model
    
    ![Generate a new model](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_build.png)
    
    Figure 2. Build a category model
    
    ![Build a category model](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_buildcat.png)
2.  If you want to save the Text Analytics Workbench session, instead click  Return to flow and then  Save and exit\.
    
    Figure 3. Saving your session
    
    ![Saving your session](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_build_save.png)The generated text nugget appears on your flow canvas.
    
    Figure 4. Generated text nugget
    
    ![Generated text nugget](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_build_nugget.png)After the category model has been validated and generated in the Text Analytics Workbench, you can deploy it in your flow and score the same data set or score a new one.
    
    Figure 5. Example flow with two modes for scoring
    
    ![Example flow with two modes for scoring](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_build_ex.png)This example flow illustrates the two modes for scoring:
    
    <!-- <ul> -->
    
     *  Categories as fields. With this option, there are just as many output records as there were in the input. However, each record now contains one new field for every category that was selected on the  Model tab. For each field, enter a flag value for true and for false, such as `True/False`, or `1/0`. In this flow, values are set to `1` and `0` to aggregate results and count the number of positive, negative, mixed (both positive and negative), or no score (no opinion) answers.
        
        Figure 6. Model results - categories as fields
        
        ![Model results - categories as fields](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_build_excats.png)
     *  Categories as records. With this option, a new record is created for each `category, document` pair. Typically, there are more records in the output than there were in the input. Along with the input fields, new fields are also added to the data depending on what kind of model it is.
        
        Figure 7. Model results - categories as records
        
        ![Model results - categories as records](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_build_exrecs.png)
    
    <!-- </ul> -->
    
3.  You can add a Select node after the DeriveSentiment SuperNode, include `Sentiments=Pos`, and add a Charts node to gain quick insight about what guests appreciate about the hotel:
    
    Figure 8. Chart of positive opinions
    
    ![Chart of positive opinions](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_positive.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Text Link Analysis node #

In some cases, you may not need to create a category model to score\. The Text Link Analysis (TLA) node adds a pattern\-matching technology to text mining's concept extraction\. This identifies relationships between the concepts in the text data based on known patterns\. These relationships can describe how a customer feels about a product, which companies are doing business together, or even the relationships between genes or pharmaceutical agents\.

Figure 1\. Text Link Analysis node

![Text Link Analysis node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tla.png)

<!-- <ol> -->

1.  Add a Text Link Analysis node to your canvas and connect it to the Data Asset node that points to  hotelSatisfaction\.csv\. Double\-click the node to open its properties\.
2.  Select `id` for the  ID field and `Comments` for the  Text field\. Note that only the  Text field is required\.
3.  For  Copy resources from, select the  Hotel Satisfaction (English) template\.
    
    Figure 2. Text Link Analysis node FIELD properties
    
    ![Text Link Analysis node FIELD properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tlaprops.png)
4.  Under  Expert, select  Accommodate spelling for a minimum word character length of\.
    
    Figure 3. Text Link Analysis node Expert properties
    
    ![Text Link Analysis node Expert properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tlaexpert.png)The resulting output is a table (or the result of an Export node).
    
    Figure 4. Raw TLA output
    
    ![Raw TLA output](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tlaraw.png)
    
    Figure 5. Counting sentiments on a TLA node
    
    ![Counting sentiments on a TLA node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tlacount.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Text Mining node #

Figure 1\. Text Mining node to analyze comments from hotel guests

![Text Mining node to analyze comments from hotel guests](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tm.png)

<!-- <ol> -->

1.  Add a Data Asset node that points to  hotelSatisfaction\.csv\.
2.  From the  Text Analytics category on the node palette, add a Text Mining node, connect it to the Data Asset node you added in the previous step, and double\-click it to open its properties\.
3.  Under  Fields, select `Comments` for the  Text field and select `id` for the  ID field\. Note that only the  Text field is required\.
    
    Figure 2. Text Mining node properties
    
    ![Text Mining node build properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tm_props1.png)
4.  Under  Copy resources from, select  Text analysis package, click  Select Resources, and then load  Hotel Satisfaction (English)\.tap (with `Current category set(s) = Topic + Opinion`)\.A text analysis package (TAP) is a predefined set of libraries and advanced linguistic and nonlinguistic resources bundled with one or more sets of predefined categories\. If no text analysis package is relevant for your application, you can instead start by selecting  Resource template under  Copy resources from\. A resource template is a predefined set of libraries and advanced linguistic and nonlinguistic resources that have been fine\-tuned for a particular domain or usage\.
    
    Figure 3. Text Mining node properties
    
    ![Text Mining node build properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tm_props2.png)
5.  Under  Build models, make sure  Build interactively (category model nugget) is selected\. Later when you run the node, this option will launch an interactive interface (known as the Text Analytics Workbench) in which you can extract concepts and patterns, explore and fine\-tune the extracted results, build and refine categories, and build category model nuggets\.
6.  Under  Begin session by, select  Extracting concepts and text links\. The option  Extracting concepts extracts only concepts, whereas TLA extraction outputs both concepts and text links that are connections between topics (service, personnel, food, etc\.) and opinions\.
7.  Under  Expert, select  Accommodate spelling for a minimum word character length of\. This option applies a fuzzy grouping technique that helps group commonly misspelled words or closely spelled words under one concept\. The fuzzy grouping algorithm temporarily strips all vowels (except the first one) and strips double/triple consonants from extracted words and then compares them to see if they're the same (so, for example, `location` and `locatoin` are grouped together)\.
    
    Figure 4. Text Mining node properties
    
    ![Text Mining node expert properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_ta_hotel_tm_props3.png)
8.  Click  Save\. Right\-click the Text Mining node and run it to open the Text Analytics Workbench and proceed to the next section of this tutorial\.

<!-- </ol> -->

<!-- </article "role="article" "> -->