ibm-research/WatsonxDocsQARetrieval huggingface dataset:
# Flow and SuperNode parameters #

You can define parameters for use in CLEM expressions and in scripting\. They are, in effect, user\-defined variables that are saved and persisted with the current flow or SuperNode and can be accessed from the user interface as well as through scripting\.

If you save a flow, for example, any parameters you set for that flow are also saved\. (This distinguishes them from local script variables, which can be used only in the script in which they are declared\.) Parameters are often used in scripting to control the behavior of the script, by providing information about fields and values that don't need to be hard coded in the script\.

You can set flow parameters in a flow script or in a flow's properties (right\-click the canvas in your flow and select  Flow properties), and they're available to all nodes in the flow\. They're displayed in the  Parameters list in the Expression Builder\.

You can also set parameters for SuperNodes, in which case they're visible only to nodes encapsulated within that SuperNode\.

Tip: For complete details about scripting, see the [Scripting and automation](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/scripting_overview.html) guide\.
<!-- </article "role="article" "> -->
<!-- <article "role="article" "> -->

You may need to describe a flow to others in your organization\. To help you do this, you can attach explanatory comments to nodes, and model nuggets\.

Others can then view these comments on\-screen, or you might even print out an image of the flow that includes the comments\. You can also add notes in the form of text annotations to nodes and model nuggets by means of the Annotations tab in a node's properties\. These annotations are visible only when the Annotations tab is open\.

<!-- </article "role="article" "> -->
# Supported data sources for  SPSS Modeler #

In  SPSS Modeler, you can connect to your data no matter where it lives\.

<!-- <ul> -->

 *  [Connectors](https://dataplatform.cloud.ibm.com/docs/content/wsd/spss-connections.html?context=cdpaas&locale=en#sql_overview__ibm-data-src-spss)
 *  [Data files](https://dataplatform.cloud.ibm.com/docs/content/wsd/spss-connections.html?context=cdpaas&locale=en#sql_overview__file-types-spss)

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Creating  SPSS Modeler flows #

With  SPSS Modeler flows, you can quickly develop predictive models using business expertise and deploy them into business operations to improve decision making\. Designed around the long\-established  SPSS Modeler client software and the industry\-standard CRISP\-DM model it uses, the flows interface supports the entire data mining process, from data to better business results\.

SPSS Modeler offers a variety of modeling methods taken from machine learning, artificial intelligence, and statistics\. The methods available on the node palette allow you to derive new information from your data and to develop predictive models\. Each method has certain strengths and is best suited for particular types of problems\.

Data format
:   Relational: Tables in relational data sources
:   Tabular:  \.xls,  \.xlsx,  \.csv,  \.sav,  \.json,  \.xml, or  \.sas\. For Excel files, only the first sheet is read\.
:   Textual: In the supported relational tables or files

Data size
:   Any

How can I prepare data?
:   Use automatic data preparation functions
:   Write SQL statements to manipulate data
:   Cleanse, shape, sample, sort, and derive data

How can I analyze data?
:   Visualize data with many chart options
:   Identify the natural language of a text field

How can I build models?
:   Build predictive models
:   Choose from over 40 modeling algorithms, and many other nodes
:   Use automatic modeling functions
:   Model time series or geospatial data
:   Classify textual data
:   Identify relationships between the concepts in textual data

Getting started
:   To create an  SPSS Modeler flow from the project's  Assets tab, click \.

Note: Watsonx\.ai doesn't include SPSS functionality in Peru, Ecuador, Colombia, or Venezuela\.
<!-- </article "role="article" "> -->
# SPSS algorithms #

Many of the nodes available in  SPSS Modeler are based on statistical algorithms\.

If you're interested in learning more about the underlying algorithms used in your flows, you can read the SPSS Modeler Algorithms Guide available in PDF format\. The guide is for advanced users, and the information is provided by a team of SPSS statisticians\.

[Download the SPSS Modeler Algorithms Guide<br><br>![SPSS Modeler Algorithms Guide](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/spss_algorithms.png)](https://public.dhe.ibm.com/software/analytics/spss/documentation/modeler/new/AlgorithmsGuide.pdf)
<!-- </article "role="article" "> -->
# How does SQL pushback work? #

The initial fragments of a flow leading from the data import nodes are the main targets for SQL generation\. When a node is encountered that can't be compiled to SQL, the data is extracted from the database and subsequent processing is performed\.

During flow preparation and prior to running, the SQL generation process happens as follows:

<!-- <ul> -->

 *  The software reorders flows to move downstream nodes into the “SQL zone” where it can be proven safe to do so\.
 *  Working from the import nodes toward the terminal nodes, SQL expressions are constructed incrementally\. This phase stops when a node is encountered that can't be converted to SQL or when the terminal node (for example, a Table node or a Graph node) is converted to SQL\. At the end of this phase, each node is labeled with an SQL statement if the node and its predecessors have an SQL equivalent\.
 *  Working from the nodes with the most complicated SQL equivalents back toward the import nodes, the SQL is checked for validity\. The SQL that was successfully validated is chosen for execution\.
 *  Nodes for which all operations have generated SQL are highlighted with a  SQL icon next to the node on the flow canvas\. Based on the results, you may want to further reorganize your flow where appropriate to take full advantage of database execution\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Generating SQL from model nuggets #

When using data from a database, SQL code can be pushed back to the database for execution, providing superior performance for many operations\. For some nodes, SQL for the model nugget can be generated, pushing back the model scoring stage to the database\. This allows flows containing these nuggets to have their full SQL pushed back\.

For a generated model nugget that supports SQL pushback:

<!-- <ol> -->

1.  Double\-click the model nugget to open its settings\.
2.  Depending on the node type, one or more of the following options is available\. Choose one of these options to specify how SQL generation is performed\.
    
    Generate SQL for this model
    
    <!-- <ul> -->
    
     *  Default: Score using Server Scoring Adapter (if installed) otherwise in process. This is the default option. If connected to a database with a scoring adapter installed, this option generates SQL using the scoring adapter and associated user defined functions (UDF) and scores your model within the database. When no scoring adapter is available, this option fetches your data back from the database and scores it in  SPSS Modeler.
     *  Score by converting to native SQL without Missing Value Support. This option generates native SQL to score the model within the database, without the overhead of handling missing values. This option simply sets the prediction to null (`$null$`) when a missing value is encountered while scoring a case.
     *  Score by converting to native SQL with Missing Value Support. For CHAID, QUEST, and C&R Tree models, you can generate native SQL to score the model within the database with full missing value support. This means that SQL is generated so that missing values are handled as specified in the model. For example, C&R Trees use surrogate rules and biggest child fallback.
     *  Score outside of the Database. This option fetches your data back from the database and scores it in  SPSS Modeler.
    
    <!-- </ul> -->
    
<!-- </ol> -->

<!-- </article "role="article" "> -->
# Tutorials #

These tutorials use the assets that are available in the sample project, and they provide brief, targeted introductions to specific modeling methods and techniques\.

You can build the example flows provided by following the steps in the tutorials\.

Some of the simple flows are already completed in the projects, but you can still walk through them using their accompanying tutorials\. Some of the more complicated flows must be completed by following the steps in the tutorials\.

Important: Before you begin the tutorials, complete the following steps to create the sample projects\.
<!-- </article "role="article" "> -->
# Automated modeling for a continuous target #

You can use the Auto Numeric node to automatically create and compare different models for continuous (numeric range) outcomes, such as predicting the taxable value of a property\. With a single node, you can estimate and compare a set of candidate models and generate a subset of models for further analysis\. The node works in the same manner as the Auto Classifier node, but for continuous rather than flag or nominal targets\.

<!-- </article "role="article" "> -->
# Building the flow #

<!-- <ol> -->

1.  Add a Data Asset node that points to  property\_values\_train\.csv\.
2.  Add a Type node, and select `taxable_value` as the target field (Role =  Target)\. Other fields will be used as predictors\.
    
    Figure 1. Setting the measurement level and role
    
    ![Setting the role](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont_build_target.png)
3.  Attach an Auto Numeric node, and select  Correlation as the metric used to rank models (under  BASICS in the node properties)\.
4.  Set the  Number of models to use to  3\. This means that the three best models will be built when you run the node\.
    
    Figure 2. Auto Numeric node BASICS
    
    ![Setting BASIC options](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont_build_basics.png)
5.  Under  EXPERT, leave the default settings in place\. The node will estimate a single model for each algorithm, for a total of six models\. (Alternatively, you can modify these settings to compare multiple variants for each model type\.)
    
    Because you set  Number of models to use to  3 under  BASICS, the node will calculate the accuracy of the six algorithms and build a single model nugget containing the three most accurate.
    
    Figure 3. Auto Numeric node EXPERT options
    
    ![Setting EXPERT options](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont_build_expert.png)
6.  Under  ENSEMBLE, leave the default settings in place\. Since this is a continuous target, the ensemble score is generated by averaging the scores for the individual models\.![ENSEMBLE options](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont_build_ensemble.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Comparing the models #

<!-- <ol> -->

1.  Run the flow\. A generated model nugget is built and placed on the canvas, and results are added to the Outputs panel\. You can view the model nugget, or save or deploy it in a number of ways\.

<!-- </ol> -->

Right\-click the model nugget and select  View Model\. You'll see details about each of the models created during the run\. (In a real situation, in which hundreds of models are estimated on a large dataset, this could take many hours\.)

Figure 1\. Auto numeric example flow with model nugget

![Auto numeric sample flow with model nugget](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont.png)

If you want to explore any of the individual models further, you can click a model name in the  ESTIMATOR column to drill down and explore the individual model results\.

Figure 2\. Auto Numeric results

![Auto Numeric results](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont_compare_view.png)

By default, models are sorted by accuracy (correlation) because correlation this was the measure you selected in the Auto Numeric node's properties\. For purposes of ranking, the absolute value of the accuracy is used, with values closer to 1 indicating a stronger relationship\.

You can sort on a different column by clicking the header for that column\.

Based on these results, you decide to use all three of these most accurate models\. By combining predictions from multiple models, limitations in individual models may be avoided, resulting in a higher overall accuracy\.

In the  USE column, make sure all three models are selected\.

Attach an Analysis node (from the Outputs palette) after the model nugget\. Right\-click the Analysis node and choose  Run to run the flow again\.

Figure 3\. Auto Numeric sample flow

![Auto numeric sample flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont.png)

The averaged score generated by the ensembled model is added in a field named `$XR-taxable_value`, with a correlation of 0\.934, which is higher than those of the three individual models\. The ensemble scores also show a low mean absolute error and may perform better than any of the individual models when applied to other datasets\.

Figure 4\. Auto Numeric sample flow analysis results

![Auto numeric sample flow analysis results](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autocont_compare_results.png)
<!-- </article "role="article" "> -->
# Summary #

With this example  Automated Modeling for a Flag Target flow, you used the Auto Numeric node to compare a number of different models, selected the three most accurate models, and added them to the flow within an ensembled Auto Numeric model nugget\.

The ensembled model showed performance that was better than two of the individual models and may perform better when applied to other datasets\. If your goal is to automate the process as much as possible, this approach allows you to obtain a robust model under most circumstances without having to dig deeply into the specifics of any one model\.

<!-- </article "role="article" "> -->
# Training data #

The data file includes a field named `taxable_value`, which is the target field, or value, that you want to predict\. The other fields contain information such as neighborhood, building type, and interior volume, and may be used as predictors\.

<!-- <table "summary="" id="tut_autocont_train__table_yhn" class="defaultstyle" "> -->

| Field name        | Label                                |
| ----------------- | ------------------------------------ |
| `property_id`     | Property ID                          |
| `neighborhood`    | Area within the city                 |
| `building_type`   | Type of building                     |
| `year_built`      | Year built                           |
| `volume_interior` | Volume of interior                   |
| `volume_other`    | Volume of garage and extra buildings |
| `lot_size`        | Lot size                             |
| `taxable_value`   | Taxable value                        |

<!-- </table "summary="" id="tut_autocont_train__table_yhn" class="defaultstyle" "> -->

<!-- </article "role="article" "> -->
# Automated data preparation #

Preparing data for analysis is one of the most important steps in any data\-mining project—and traditionally, one of the most time consuming\. The Auto Data Prep node handles the task for you, analyzing your data and identifying fixes, screening out fields that are problematic or not likely to be useful, deriving new attributes when appropriate, and improving performance through intelligent screening techniques\.

You can use the Auto Data Prep node in fully automated fashion, allowing the node to choose and apply fixes, or you can preview the changes before they're made and accept or reject them as desired\. With this node, you can ready your data for data mining quickly and easily, without the need for prior knowledge of the statistical concepts involved\. If you run the node with the default settings, models will tend to build and score more quickly\.

This example uses the flow named  Automated Data Preparation, available in the example project \. The data file is  telco\.csv\. This example demonstrates the increased accuracy you can find by using the default Auto Data Prep node settings when building models\.

Let's take a look at the flow\.

<!-- <ol> -->

1.  Open the  Example Project\.
2.  Scroll down to the  Modeler flows section, click  View all, and select the  Automated Data Preparation flow\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Building the flow #

<!-- <ol> -->

1.  Add a Data Asset node that points to  telco\.csv\.
    
    Figure 1. Auto Data Prep example flow
    
    ![Auto Data Prep example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata.png)
2.  Attach a Type node to the Data Asset node\. Set the measure for the `churn` field to  Flag, and set the role to  Target\. Make sure the role for all other fields is set to  Input\.
    
    Figure 2. Setting the measurement level and role
    
    ![Setting the measurement level and role](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_build_target.png)
3.  Attach a Logistic node to the Type node\.
4.  In the Logistic node's properties, under  MODEL SETTINGS, select the  Binomial procedure\. For  Model Name, select  Custom and enter No ADP \- churn\.
    
    Figure 3. Choosing model options
    
    ![Choosing model options](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_binomial_default.png)
5.  Attach an Auto Data Prep node to the Type node\. Under  OBJECTIVES, leave the default settings in place to analyze and prepare your data by balancing both speed and accuracy\.
6.  Run the flow to analyze and process your data\. Other Auto Data Prep node properties allow you to specify that you want to concentrate more on accuracy, more on the speed of processing, or to fine tune many of the data preparation processing steps\. Note: If you want to adjust the node properties and run the flow again in the future, since the model already exists, you must first click Clear Analysis, under  OBJECTIVES before running the flow again\.
    
    Figure 4. Auto Data Prep default objectives
    
    ![Auto Data Prep default objectives](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_objectives.png)
7.  Attach a Logistic node to the Auto Data Prep node\.
8.  In the Logistic node's properties, under  MODEL SETTINGS, select the  Binomial procedure\. For  Model Name, select  Custom and enter After ADP \- churn\.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Comparing the models #

<!-- <ol> -->

1.  Right\-click each Logistic node and run it to create the model nuggets, which are added to the flow\. Results are also added to the  Outputs panel\.
    
    Figure 1. Attaching the model nuggets
    
    ![Attaching the model nuggets](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_partial_flow.png)
2.  Attach Analysis nodes to the model nuggets and run the Analysis nodes (using their default settings)\.
    
    Figure 2. Attaching the Analysis nodes
    
    ![Attaching the Analysis nodes](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_partial_flow2.png)The Analysis of the non Auto Data Prep-derived model shows that just running the data through the Logistic Regression node with its default settings gives a model with low accuracy - just 10.6%.
    
    Figure 3. Non ADP-derived model results
    
    ![Non ADP-derived model results](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_analysis_non_adp.png)The Analysis of the Auto-Data Prep-derived model shows that by running the data through the default Auto Data Prep settings, you have built a much more accurate model that's 78.3% correct.
    
    Figure 4. ADP-derived model results
    
    ![ADP-derived model results](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_analysis_adp.png)

<!-- </ol> -->

In summary, by just running the Auto Data Prep node to fine tune the processing of your data, you were able to build a more accurate model with little direct data manipulation\.

Obviously, if you're interested in proving or disproving a certain theory, or want to build specific models, you may find it beneficial to work directly with the model settings\. However, for those with a reduced amount of time, or with a large amount of data to prepare, the Auto Data Prep node may give you an advantage\.

Note that the results in this example are based on the training data only\. To assess how well models generalize to other data in the real world, you would use a Partition node to hold out a subset of records for purposes of testing and validation\.

<!-- </article "role="article" "> -->
# Building the flow #

<!-- <ol> -->

1.  Add a Data Asset node that points to  pm\_customer\_train1\.csv\.
2.  Add a Type node, and select `response` as the target field (Role =  Target)\. Set the measure for this field to  Flag\.
    
    Figure 1. Setting the measurement level and role
    
    ![Setting the measurement level and role](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_target.png)
3.  Set the role to  None for the following fields: `customer_id`, `campaign`, `response_date`, `purchase`, `purchase_date`, `product_id`, `Rowid`, and `X_random`\. These fields will be ignored when you are building the model\.
4.  Click  Read Values in the Type node to make sure that values are instantiated\.
    
    As we saw earlier, our source data includes information about four different campaigns, each targeted to a different type of customer account. These campaigns are coded as integers in the data, so to make it easier to remember which account type each integer represents, let's define labels for each one.
    
    Figure 2. Choosing to specify values for a field
    
    ![Choosing to specify values for a field](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_value.png)
5.  On the row for the  campaign field, click the entry in the  Value mode column\.
6.  Choose  Specify from the drop\-down\.
    
    Figure 3. Defining labels for the field values
    
    ![Defining labels for the field values](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_labels.png)
7.  Click the  Edit icon in the column for the  campaign field\. Type the labels as shown for each of the four values\.
8.  Click  OK\. Now the labels will be displayed in output windows instead of the integers\.
9.  Attach a Table node to the Type node\.
10. Right\-click the Table node and select  Run\.
11. In the  Outputs panel, double\-click the table output to open it\.
12. Click  OK to close the output window\.

<!-- </ol> -->

Although the data includes information about four different campaigns, you will focus the analysis on one campaign at a time\. Since the largest number of records fall under the Premium account campaign (coded `campaign=2` in the data), you can use a Select node to include only these records in the flow\.

Figure 4\. Selecting records for a single campaign

![Selecting records for a single campaign](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_select.png)
<!-- </article "role="article" "> -->
# Historical data #

This example uses the data file  pm\_customer\_train1\.csv, which contains historical data that tracks the offers made to specific customers in past campaigns, as indicated by the value of the `campaign` field\. The largest number of records fall under the `Premium account` campaign\.

The values of the `campaign` field are actually coded as integers in the data (for example `2 = Premium account`)\. Later, you'll define labels for these values that you can use to give more meaningful output\.

Figure 1\. Data about previous promotions

![Data about previous promotions](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_historical.png)

The file also includes a `response` field that indicates whether the offer was accepted (`0 = no`, and `1 = yes`)\. This will be the target field, or value, that you want to predict\. A number of fields containing demographic and financial information about each customer are also included\. These can be used to build or "train" a model that predicts response rates for individuals or groups based on characteristics such as income, age, or number of transactions per month\.

<!-- </article "role="article" "> -->
# Summary #

With this example  Automated Modeling for a Flag Target flow, you used the Auto Classifier node to compare a number of different models, used the three most accurate models, and added them to the flow within an ensembled Auto Classifier model nugget\.

<!-- <ul> -->

 *  Based on overall accuracy, the XGBoost Tree, C5\.0, and C&R Tree models performed best on the training data\.
 *  The ensembled model performed nearly as well as the best of the individual models and may perform better when applied to other datasets\. If your goal is to automate the process as much as possible, this approach allows you to obtain a robust model under most circumstances without having to dig deeply into the specifics of any one model\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Forecasting with the Time Series node #

This example uses the flow  Forecasting Bandwidth Utilization, available in the example project \. The data file is  broadband\_1\.csv\.

In SPSS Modeler, you can produce multiple time series models in a single operation\. The  broadband\_1\.csv data file has monthly usage data for each of 85 local markets\. For the purposes of this example, only the first five series will be used; a separate model will be created for each of these five series, plus a total\.

The file also includes a date field that indicates the month and year for each record\. This field will be used to label records\. The date field reads into SPSS Modeler as a string, but to use the field in SPSS Modeler you will convert the storage type to numeric Date format using a Filler node\.

Figure 1\. Example flow to show Time Series modeling

![Example flow to show Time Series modeling](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast.png)

The Time Series node requires that each series be in a separate column, with a row for each interval\.  Watson Studio provides methods for transforming data to match this format if necessary\.

Figure 2\. Monthly subscription data for broadband local markets

![Monthly subscription data for broadband local markets](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_table.png)
<!-- </article "role="article" "> -->
# Creating the flow #

<!-- <ol> -->

1.  Add a Data Asset node that points to  broadband\_1\.csv\.
2.  To simplify the model, use a Filter node to filter out the `Market_6` to `Market_85` fields and the `MONTH_` and `YEAR_` fields\.

<!-- </ol> -->

Figure 1\. Example flow to show Time Series modeling

![Example flow to show Time Series modeling](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_build.png)
<!-- </article "role="article" "> -->
# Examining the data #

It's always a good idea to have a feel for the nature of your data before building a model\.

Does the data exhibit seasonal variations? Although  Watson Studio can automatically find the best seasonal or nonseasonal model for each series, you can often obtain faster results by limiting the search to nonseasonal models when seasonality is not present in your data\. Without examining the data for each of the local markets, we can get a rough picture of the presence or absence of seasonality by plotting the total number of subscribers over all five markets\.

Figure 1\. Plotting the total number of subscribers

![Plotting the total number of subscribers](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_data1.png)

<!-- <ol> -->

1.  From the Graphs palette, attach a Time Plot node to the Filter node\.
2.  Add the `Total` field to the Series list\.
3.  Deselect the  Display series in separate panel and  Normalize options\. Save the changes\.
4.  Right\-click the Time Plot node and run it, then open the output that was generated\.
    
    Figure 2. Time plot of the Total field
    
    ![Time plot of the Total field](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_data2.png)
    
    The series exhibits a very smooth upward trend with no hint of seasonal variations. There might be individual series with seasonality, but it appears that seasonality isn't a prominent feature of the data in general.
    
    Of course, you should inspect each of the series before ruling out seasonal models. You can then separate out series exhibiting seasonality and model them separately.
    
    Watson Studio makes it easy to plot multiple series together.
5.  Double\-click the Time Plot node to open its properties again\.
6.  Remove the `Total` field from the Series list\.
7.  Add the `Market_1` through `Market_5` fields to the list\.
8.  Run the Time Plot node again\.
    
    Figure 3. Time plot of multiple fields
    
    ![Time plot of multiple fields](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_data3.png)
    
    Inspection of each of the markets reveals a steady upward trend in each case. Although some markets are a little more erratic than others, there's no evidence of seasonality.

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Defining the dates #

Now you need to change the storage type of the `DATE_` field to date format\.

<!-- <ol> -->

1.  Attach a Filler node to the Filter node, then double\-click the Filler node to open its properties
2.  Add the `DATE_` field, set the  Replace option to  Always, and set the  Replace with value to `to_date(DATE_)`\.
    
    Figure 1. Setting the date storage type
    
    ![Setting the date storage type](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_date1.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Creating the model #

<!-- <ol> -->

1.  Double\-click the Time Series node to open its properties\.
2.  Under  FIELDS, add all 5 of the markets to the  Candidate Inputs lists\. Also add the `Total` field to the  Targets list\.
3.  Under  BUILD OPTIONS \- GENERAL, make sure the  Expert Modeler method is selected using all default settings\. Doing so enables the Expert Modeler to decide the most appropriate model to use for each time series\.
    
    Figure 1. Choosing the Expert Modeler method for Time Series
    
    ![Choosing the Expert Modeler method for Time Series](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_expert.png)
4.  Save the settings and then run the flow\. A Time Series model nugget is generated\. Attach it to the Time Series node\.
5.  Attach a Table node to the Time Series model nugget and run the flow again\.
    
    Figure 2. Example flow showing Time Series modeling
    
    ![Example flow showing Time Series modeling](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_flow.png)

<!-- </ol> -->

There are now three new rows appended to the end of the original data\. These are the rows for the forecast period, in this case January to March 2004\.

Several new columns are also present now\. The `$TS-` columns are added by the Time Series node\. The columns indicate the following for each row (that is, for each interval in the time series data):

<!-- <table "summary="" class="defaultstyle" "> -->

| Column            | Description                                                                       |
| ----------------- | --------------------------------------------------------------------------------- |
| $TS\-*colname*    | The generated model data for each column of the original data\.                   |
| $TSLCI\-*colname* | The lower confidence interval value for each column of the generated model data\. |
| $TSUCI\-*colname* | The upper confidence interval value for each column of the generated model data\. |
| $TS\-Total        | The total of the $TS\-*colname* values for this row\.                             |
| $TSLCI\-Total     | The total of the $TSLCI\-*colname* values for this row\.                          |
| $TSUCI\-Total     | The total of the $TSUCI\-*colname* values for this row\.                          |

<!-- </table "summary="" class="defaultstyle" "> -->

The most significant columns for the forecast operation are the `$TS-Market_n`, `$TSLCI-Market_n`, and `$TSUCI-Market_n` columns\. In particular, these columns in the last three rows contain the user subscription forecast data and confidence intervals for each of the local markets\.

<!-- </article "role="article" "> -->
# Defining the targets #

<!-- <ol> -->

1.  Add a Type node after the Filler node, then double\-click the Type node to open its properties\.
2.  Set the role to  None for the `DATE_` field\. Set the role to  Target for all other fields (the `Market_n` fields plus the `Total` field)\.
3.  Click  Read Values to populate the  Values column\.
    
    Figure 1. Setting the role for fields
    
    ![Setting the role for fields](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_targets.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Setting the time intervals #

<!-- <ol> -->

1.  Add a Time Series node and attach it to the Type node\. Double\-click the node to edit its properties\.
2.  Under  OBSERVATIONS AND TIME INTERVAL, select `DATE_` as the  Time/Date field\.
3.  Select  Months as the time interval\.
    
    Figure 1. Setting the time interval
    
    ![Setting the time interval](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_interval.png)
4.  Under  MODEL OPTIONS, select the  Extend records into the future option and set the value to  3\.
    
    Figure 2. Setting the forecast period
    
    ![Setting the forecast period](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_period.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Telecommunications churn #

Logistic regression is a statistical technique for classifying records based on values of input fields\. It is analogous to linear regression, but takes a categorical target field instead of a numeric one\.

For example, suppose a telecommunications provider is concerned about the number of customers it's losing to competitors\. If service usage data can be used to predict which customers are liable to transfer to another provider, offers can be customized to retain as many customers as possible\.

This example uses the flow named  Telecommunications Churn, available in the example project \. The data file is  telco\.csv\.

This example focuses on using usage data to predict customer loss (churn)\. Because the target has two distinct categories, a binomial model is used\. In the case of a target with multiple categories, a multinomial model could be created instead\. See [Classifying telecommunications customers](https://dataplatform.cloud.ibm.com/docs/content/wsd/tutorials/tut_classify.html#tut_classify) for more information\.

<!-- </article "role="article" "> -->
# Browsing the model #

<!-- <ul> -->

 *  Right\-click the Logistic node and run it to generate its model nugget\. Right\-click the nugget and select  View Model\.The Parameter Estimates page shows the target (churn) and inputs (predictor fields) used by the model\. These are the fields that were actually chosen based on the Forwards Stepwise method, not the complete list submitted for consideration\.
    
    Figure 1. Parameter estimates showing input fields
    
    ![Parameter estimates showing input fields](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_churn_estimates.png)
    
    To assess how well the model actually fits your data, a number of diagnostics are available in the expert node settings when you're building the flow.
    
    Note also that these results are based on the training data only. To assess how well the model generalizes to other data in the real world, you would use a Partition node to hold out a subset of records for purposes of testing and validation.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Classifying telecommunications customers #

Logistic regression is a statistical technique for classifying records based on values of input fields\. It is analogous to linear regression, but takes a categorical target field instead of a numeric one\.

For example, suppose a telecommunications provider has segmented its customer base by service usage patterns, categorizing the customers into four groups\. If demographic data can be used to predict group membership, you can customize offers for individual prospective customers\.

This example uses the flow named  Classifying Telecommmunications Customers, available in the example project \. The data file is  telco\.csv\.

The example focuses on using demographic data to predict usage patterns\. The target field `custcat` has four possible values that correspond to the four customer groups, as follows:

<!-- <table "summary="" class="defaultstyle" "> -->

Table 1\. Possible values for the target field

| Value | Label         |
| ----- | ------------- |
| 1     | Basic Service |
| 2     | E\-Service    |
| 3     | Plus Service  |
| 4     | Total Service |

<!-- </table "summary="" class="defaultstyle" "> -->

Because the target has multiple categories, a multinomial model is used\. In the case of a target with two distinct categories, such as yes/no, true/false, or churn/don't churn, a binomial model could be created instead\. See [Telecommunications churn](https://dataplatform.cloud.ibm.com/docs/content/wsd/tutorials/tut_churn.html#tut_churn) for more information\.

<!-- </article "role="article" "> -->
# Building the flow #

Figure 1\. Example flow to classify customers using multinomial logistic regression

![Example flow to classify customers using multinomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify.png)

<!-- <ol> -->

1.  Add a Data Asset node that points to  telco\.csv\.
2.  Add a Type node, double\-click it to open its properties, and click  Read Values\. Make sure all measurement levels are set correctly\. For example, most fields with values of `0.0` and `1.0` can be regarded as flags\.
    
    Figure 2. Measurement levels
    
    ![Measurement levels](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_measurement.png)
    
    Notice that `gender` is more correctly considered as a field with a set of two values, instead of a flag, so leave its measurement value as  Nominal.
3.  Set the role for the `custcat` field to  Target\. Leave the role for all other fields set to  Input\.
4.  Since this example focuses on demographics, use a Filter node to include only the relevant fields: `region`, `age`, `marital`, `address`, `income`, `ed`, `employ`, `retire`, `gender`, `reside`, and `custcat`)\. Other fields will be excluded for the purpose of this analysis\. To filter them out, in the Filter node properties, click  Add Columns and select the fields to exclude\.
    
    Figure 3. Filtering on demographic fields
    
    ![Filtering on demographic fields](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_filter.png)
    
    (Alternatively, you could change the role to  None for these fields rather than excluding them, or select the fields you want to use in the modeling node.)
5.  In the Logistic node properties, under  MODEL SETTINGS, select the  Stepwise method\. Also select  Multinomial,  Main Effects, and  Include constant in equation\.
    
    Figure 4. Example flow to classify customers using multinomial logistic regression
    
    ![Example flow to classify customers using multinomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_logistic.png)
6.  Under  EXPERT OPTIONS, select  Expert mode, expand the  Output section, and select  Classification table\.
    
    Figure 5. Example flow to classify customers using multinomial logistic regression
    
    ![Example flow to classify customers using multinomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_output.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Browsing the model #

<!-- <ul> -->

 *  Run the Logistic node to generate the model\. Right\-click the model nugget and select  View Model\.
    
    Figure 1. Browsing the model results
    
    ![Browsing the model results](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_model.png)
    
    You can then explore the model information, feature (predictor) importance, and parameter estimates information.
    
    Note that these results are based on the training data only. To assess how well the model generalizes to other data in the real world, you can use a Partition node to hold out a subset of records for purposes of testing and validation.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Examining the data #

For the first part of the process, imagine you have a flow that plots a number of graphs\. If the time series of temperature or power contains visible patterns, you could differentiate between impending error conditions or possibly predict their occurrence\. For both temperature and power, the flow plots the time series associated with the three different error codes on separate graphs, yielding six graphs\. Select nodes separate the data associated with the different error codes\.

The graphs clearly display patterns distinguishing 202 errors from 101 and 303 errors\. The 202 errors show rising temperature and fluctuating power over time; the other errors don't\. However, patterns distinguishing 101 from 303 errors are less clear\. Both errors show even temperature and a drop in power, but the drop in power seems steeper for 303 errors\.

Based on these graphs, it appears that the presence and rate of change for both temperature and power, as well as the presence and degree of fluctuation, are relevant to predicting and distinguishing faults\. These attributes should therefore be added to the data before applying the learning systems\.

<!-- </article "role="article" "> -->
# Learning #

Running the flow trains the C5\.0 rule and neural network (net)\. The network may take some time to train, but training can be interrupted early to save a net that produces reasonable results\. After the learning is complete, model nuggets are generated: one represents the neural net and one represents the rule\.

Figure 1\. Generated model nuggets

![Generated model nuggets](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_condition_nuggets.png)

These model nuggets enable us to test the system or export the results of the model\. In this example, we will test the results of the model\.

<!-- </article "role="article" "> -->
# Data preparation #

Based on the results of exploring the data, the following flow derives the relevant data and learns to predict faults\.

This example uses the flow named  Condition Monitoring, available in the example project installed with the product\. The data files are  cond1n\.csv and  cond2n\.csv\.

<!-- <ol> -->

1.  On the  My Projects screen, click  Example Project\.
2.  Scroll down to the  Modeler flows section, click  View all, and select the  Condition Monitoring flow\.

<!-- </ol> -->

Figure 1\. Condition Monitoring example flow

![Condition Monitoring example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_condition.png)The flow uses a number of Derive nodes to prepare the data for modeling\.

<!-- <ul> -->

 *  Data Asset import node\. Reads data file  cond1n\.csv\.
 *  Pressure Warnings (Derive)\. Counts the number of momentary pressure warnings\. Reset when time returns to 0\.
 *  TempInc (Derive)\. Calculates momentary rate of temperature change using `@DIFF1`\.
 *  PowerInc (Derive)\. Calculates momentary rate of power change using `@DIFF1`\.
 *  PowerFlux (Derive)\. A flag, true if power varied in opposite directions in the last record and this one; that is, for a power peak or trough\.
 *  PowerState (Derive)\. A state that starts as `Stable` and switches to `Fluctuating` when two successive power fluxes are detected\. Switches back to `Stable` only when there hasn't been a power flux for five time intervals or when `Time` is reset\.
 *  PowerChange (Derive)\. Average of `PowerInc` over the last five time intervals\.
 *  TempChange (Derive)\. Average of `TempInc` over the last five time intervals\.
 *  Discard Initial (Select)\. Discards the first record of each time series to avoid large (incorrect) jumps in `Power` and `Temperature` at boundaries\.
 *  Discard fields (Filter)\. Cuts records down to `Uptime`, `Status`, `Outcome`, `Pressure Warnings`, `PowerState`, `PowerChange`, and `TempChange`\.
 *  Type\. Defines the role of `Outcome` as Target (the field to predict)\. In addition, defines the measurement level of `Outcome` as Nominal, `Pressure Warnings` as Continuous, and `PowerState` as Flag\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Testing #

Both of the generated model nuggets are connected to the Type node\.

<!-- <ol> -->

1.  Reposition the nuggets as shown, so the Type node connects to the neural net nugget, which connects to the C5\.0 nugget\.
2.  Attach an Analysis node to the C5\.0 nugget\.
3.  Edit the Data Asset node to use the file  cond2n\.csv (instead of  cond1n\.csv), which contains unseen test data\.
4.  Right\-click the Analysis node and select  Run\. Doing so yields figures reflecting the accuracy of the trained network and rule\.
    
    Figure 1. Testing the trained network
    
    ![Testing the trained network](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_condition_analysis.png)

<!-- </ol> -->

<!-- </article "role="article" "> -->
# Drug treatment \- exploratory graphs #

In this example, imagine you're a medical researcher compiling data for a study\. You've collected data about a set of patients, all of whom suffered from the same illness\. During their course of treatment, each patient responded to one of five medications\. Part of your job is to use data mining to find out which drug might be appropriate for a future patient with the same illness\.

This example uses the flow named  Drug Treatment \- Exploratory Graphs, available in the example project \. The data file is  drug1n\.csv\.

Figure 1\. Drug treatment example flow

![Drug treatment example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_data.png)

The data fields used in this example are:

<!-- <table "summary="" class="defaultstyle" "> -->

| Data field    | Description                                    |
| ------------- | ---------------------------------------------- |
| `Age`         | Age of patient (number)                        |
| `Sex`         | `M` or `F`                                     |
| `BP`          | Blood pressure: `HIGH`, `NORMAL`, or `LOW`     |
| `Cholesterol` | Blood cholesterol: `NORMAL` or `HIGH`          |
| `Na`          | Blood sodium concentration                     |
| `K`           | Blood potassium concentration                  |
| `Drug`        | Prescription drug to which a patient responded |

<!-- </table "summary="" class="defaultstyle" "> -->

<!-- </article "role="article" "> -->