ibm-research/WatsonxDocsQARetrieval huggingface dataset:
# Feature Selection node #

Data mining problems may involve hundreds, or even thousands, of fields that can potentially be used as inputs\. As a result, a great deal of time and effort may be spent examining which fields or variables to include in the model\. To narrow down the choices, the Feature Selection algorithm can be used to identify the fields that are most important for a given analysis\. For example, if you are trying to predict patient outcomes based on a number of factors, which factors are the most likely to be important?

Feature selection consists of three steps:

<!-- <ul> -->

 *  Screening\. Removes unimportant and problematic inputs and records, or cases such as input fields with too many missing values or with too much or too little variation to be useful\.
 *  Ranking\. Sorts remaining inputs and assigns ranks based on importance\.
 *  Selecting\. Identifies the subset of features to use in subsequent models—for example, by preserving only the most important inputs and filtering or excluding all others\.

<!-- </ul> -->

In an age where many organizations are overloaded with too much data, the benefits of feature selection in simplifying and speeding the modeling process can be substantial\. By focusing attention quickly on the fields that matter most, you can reduce the amount of computation required; more easily locate small but important relationships that might otherwise be overlooked; and, ultimately, obtain simpler, more accurate, and more easily explainable models\. By reducing the number of fields used in the model, you may find that you can reduce scoring times as well as the amount of data collected in future iterations\.

Example\. A telephone company has a data warehouse containing information about responses to a special promotion by 5,000 of the company's customers\. The data includes a large number of fields containing customers' ages, employment, income, and telephone usage statistics\. Three target fields show whether or not the customer responded to each of three offers\. The company wants to use this data to help predict which customers are most likely to respond to similar offers in the future\.

Requirements\. A single target field (one with its role set to `Target`), along with multiple input fields that you want to screen or rank relative to the target\. Both target and input fields can have a measurement level of `Continuous` (numeric range) or `Categorical`\.

<!-- </article "role="article" "> -->
# Filler node #

Filler nodes are used to replace field values and change storage\. You can choose to replace values based on a specified CLEM condition, such as `@BLANK(FIELD)`\. Alternatively, you can choose to replace all blanks or null values with a specific value\. Filler nodes are often used in conjunction with the Type node to replace missing values\.

Fill in fields\. Select fields from the dataset whose values will be examined and replaced\. The default behavior is to replace values depending on the specified  Condition and  Replace with expressions\. You can also select an alternative method of replacement using the  Replace options\.

Note: When selecting multiple fields to replace with a user\-defined value, it is important that the field types are similar (all numeric or all symbolic)\.

Replace\. Select to replace the values of the selected field(s) using one of the following methods:

<!-- <ul> -->

 *  Based on condition\. This option activates the Condition field and Expression Builder for you to create an expression used as a condition for replacement with the value specified\.
 *  Always\. Replaces all values of the selected field\. For example, you could use this option to convert the storage of income to a string using the following CLEM expression: `(to_string(income))`\.
 *  Blank values\. Replaces all user\-specified blank values in the selected field\. The standard condition `@BLANK(@FIELD)` is used to select blanks\. *Note*: You can define blanks using the Types tab of the source node or with a Type node\.
 *  Null values\. Replaces all system null values in the selected field\. The standard condition `@NULL(@FIELD)` is used to select nulls\.
 *  Blank and null values\. Replaces both blank values and system nulls in the selected field\. This option is useful when you are unsure whether or not nulls have been defined as missing values\.

<!-- </ul> -->

Condition\. This option is available when you have selected the  Based on condition option\. Use this text box to specify a CLEM expression for evaluating the selected fields\. Click the calculator button to open the Expression Builder\.

Replace with\. Specify a CLEM expression to give a new value to the selected fields\. You can also replace the value with a null value by typing undef in the text box\. Click the calculator button to open the Expression Builder\.

Note: When the field(s) selected are string, you should replace them with a string value\. Using the default 0 or another numeric value as the replacement value for string fields will result in an error\.Note that use of the following may change row order:

<!-- <ul> -->

 *  Running in a database via SQL pushback
 *  Deriving a list
 *  Calling any of the CLEM spatial functions

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Filter node #

You can rename or exclude fields at any point in a flow\. For example, as a medical researcher, you may not be concerned about the potassium level (field\-level data) of patients (record\-level data); therefore, you can filter out the `K` (potassium) field\. This can be done using a separate Filter node or using the Filter tab on an import or output node\. The functionality is the same regardless of which node it's accessed from\.

<!-- <ul> -->

 *  From import nodes, you can rename or filter fields as the data is read in\.
 *  Using a Filter node, you can rename or filter fields at any point in the flow\.
 *  You can use the Filter tab in various nodes to define or edit multiple response sets\.
 *  Finally, you can use a Filter node to map fields from one import node to another\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# GenLin node #

The generalized linear model expands the general linear model so that the dependent variable is linearly related to the factors and covariates via a specified link function\. Moreover, the model allows for the dependent variable to have a non\-normal distribution\. It covers widely used statistical models, such as linear regression for normally distributed responses, logistic models for binary data, loglinear models for count data, complementary log\-log models for interval\-censored survival data, plus many other statistical models through its very general model formulation\.

Examples\. A shipping company can use generalized linear models to fit a Poisson regression to damage counts for several types of ships constructed in different time periods, and the resulting model can help determine which ship types are most prone to damage\.

A car insurance company can use generalized linear models to fit a gamma regression to damage claims for cars, and the resulting model can help determine the factors that contribute the most to claim size\.

Medical researchers can use generalized linear models to fit a complementary log\-log regression to interval\-censored survival data to predict the time to recurrence for a medical condition\.

Generalized linear models work by building an equation that relates the input field values to the output field values\. After the model is generated, you can use it to estimate values for new data\. For each record, a probability of membership is computed for each possible output category\. The target category with the highest probability is assigned as the predicted output value for that record\.

Requirements\. You need one or more input fields and exactly one target field (which can have a measurement level of `Continuous` or `Flag`) with two or more categories\. Fields used in the model must have their types fully instantiated\.

Strengths\. The generalized linear model is extremely flexible, but the process of choosing the model structure is not automated and thus demands a level of familiarity with your data that is not required by "black box" algorithms\.

<!-- </article "role="article" "> -->
# GLE node #

The GLE model identifies the dependent variable that is linearly related to the factors and covariates via a specified link function\. Moreover, the model allows for the dependent variable to have a non\-normal distribution\. It covers widely used statistical models, such as linear regression for normally distributed responses, logistic models for binary data, loglinear models for count data, complementary log\-log models for interval\-censored survival data, plus many other statistical models through its very general model formulation\.

Examples\. A shipping company can use generalized linear models to fit a Poisson regression to damage counts for several types of ships constructed in different time periods, and the resulting model can help determine which ship types are most prone to damage\.

A car insurance company can use generalized linear models to fit a gamma regression to damage claims for cars, and the resulting model can help determine the factors that contribute the most to claim size\.

Medical researchers can use generalized linear models to fit a complementary log\-log regression to interval\-censored survival data to predict the time to recurrence for a medical condition\.

GLE models work by building an equation that relates the input field values to the output field values\. After the model is generated, you can use it to estimate values for new data\.

For a categorical target, for each record, a probability of membership is computed for each possible output category\. The target category with the highest probability is assigned as the predicted output value for that record\.

Requirements\. You need one or more input fields and exactly one target field (which can have a measurement level of `Continuous`, `Categorical`, or `Flag`) with two or more categories\. Fields used in the model must have their types fully instantiated\.

Note: When first creating a flow, you select which runtime to use\. By default, flows use the  IBM SPSS Modeler runtime\. If you want to use native Spark algorithms instead of SPSS algorithms, select the  Spark runtime\. Properties for this node will vary depending on which runtime option you choose\.
<!-- </article "role="article" "> -->
# Gaussian Mixture node #

A Gaussian Mixture© model is a probabilistic model that assumes all the data points are generated from a mixture of a finite number of Gaussian distributions with unknown parameters\.

One can think of mixture models as generalizing k\-means clustering to incorporate information about the covariance structure of the data as well as the centers of the latent Gaussians\.^1^

The Gaussian Mixture node in  watsonx\.ai exposes the core features and commonly used parameters of the Gaussian Mixture library\. The node is implemented in Python\.

For more information about Gaussian Mixture modeling algorithms and parameters, see [Gaussian Mixture Models](http://scikit-learn.org/stable/modules/mixture.html) and [Gaussian Mixture](https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html)\. ^2^

^1^ [User Guide\.](https://scikit-learn.org/stable/modules/mixture.html)*Gaussian mixture models*\. Web\. © 2007 \- 2017\. scikit\-learn developers\.

^2^ [Scikit\-learn: Machine Learning in Python](http://jmlr.csail.mit.edu/papers/v12/pedregosa11a.html), Pedregosa *et al\.*, JMLR 12, pp\. 2825\-2830, 2011\.

<!-- </article "role="article" "> -->
# HDBSCAN node #

Hierarchical Density\-Based Spatial Clustering (HDBSCAN)© uses unsupervised learning to find clusters, or dense regions, of a data set\.

The HDBSCAN node in  watsonx\.ai exposes the core features and commonly used parameters of the HDBSCAN library\. The node is implemented in Python, and you can use it to cluster your dataset into distinct groups when you don't know what those groups are at first\. Unlike most learning methods in  watsonx\.ai, HDBSCAN models do *not* use a target field\. This type of learning, with no target field, is called unsupervised learning\. Rather than trying to predict an outcome, HDBSCAN tries to uncover patterns in the set of input fields\. Records are grouped so that records within a group or cluster tend to be similar to each other, but records in different groups are dissimilar\. The HDBSCAN algorithm views clusters as areas of high density separated by areas of low density\. Due to this rather generic view, clusters found by HDBSCAN can be any shape, as opposed to k\-means which assumes that clusters are convex shaped\. Outlier points that lie alone in low\-density regions are also marked\. HDBSCAN also supports scoring of new samples\.^1^

To use the HDBSCAN node, you must set up an upstream Type node\. The HDBSCAN node will read input values from the Type node (or from the Types of an upstream import node)\.

For more information about HDBSCAN clustering algorithms, see the [HDBSCAN documentation](http://hdbscan.readthedocs.io/en/latest/)\. ^1^

^1^ "User Guide / Tutorial\." *The hdbscan Clustering Library*\. Web\. © 2016, Leland McInnes, John Healy, Steve Astels\.

<!-- </article "role="article" "> -->
# Isotonic\-AS node #

Isotonic Regression belongs to the family of regression algorithms\. The Isotonic\-AS node in  watsonx\.ai is implemented in Spark\.

For details, see [Isotonic regression](https://spark.apache.org/docs/2.2.0/mllib-isotonic-regression.html)\. ^1^

^1^ "Regression \- RDD\-based API\." *Apache Spark*\. MLlib: Main Guide\. Web\. 3 Oct 2017\.

<!-- </article "role="article" "> -->
# KDE node #

Kernel Density Estimation (KDE)© uses the Ball Tree or KD Tree algorithms for efficient queries, and walks the line between unsupervised learning, feature engineering, and data modeling\.

Neighbor\-based approaches such as KDE are some of the most popular and useful density estimation techniques\. KDE can be performed in any number of dimensions, though in practice high dimensionality can cause a degradation of performance\. The KDE Modeling node and the KDE Simulation node in  watsonx\.ai expose the core features and commonly used parameters of the KDE library\. The nodes are implemented in Python\. ^1^

To use a KDE node, you must set up an upstream Type node\. The KDE node will read input values from the Type node (or from the Types of an upstream import node)\.

The  KDE Modeling node is available under the Modeling node palette\. The KDE Modeling node generates a model nugget, and the nugget's scored values are kernel density values from the input data\.

The  KDE Simulation node is available under the Outputs node palette\. The KDE Simulation node generates a KDE Gen source node that can create some records that have the same distribution as the input data\. In the KDE Gen node properties, you can specify how many records the node will create (default is 1) and generate a random seed\.

For more information about KDE, including examples, see the [KDE documentation](http://scikit-learn.org/stable/modules/density.html#kernel-density-estimation)\. ^1^

^1^ "User Guide\." *Kernel Density Estimation*\. Web\. © 2007\-2018, scikit\-learn developers\.

<!-- </article "role="article" "> -->
# KDE Simulation node #

Kernel Density Estimation (KDE)© uses the Ball Tree or KD Tree algorithms for efficient queries, and walks the line between unsupervised learning, feature engineering, and data modeling\.

Neighbor\-based approaches such as KDE are some of the most popular and useful density estimation techniques\. KDE can be performed in any number of dimensions, though in practice high dimensionality can cause a degradation of performance\. The KDE Modeling node and the KDE Simulation node in  watsonx\.ai expose the core features and commonly used parameters of the KDE library\. The nodes are implemented in Python\. ^1^

To use a KDE node, you must set up an upstream Type node\. The KDE node will read input values from the Type node (or from the Types of an upstream import node)\.

The  KDE Modeling node is available under the Modeling node palette\. The KDE Modeling node generates a model nugget, and the nugget's scored values are kernel density values from the input data\.

The  KDE Simulation node is available under the Outputs node palette\. The KDE Simulation node generates a KDE Gen source node that can create some records that have the same distribution as the input data\. In the KDE Gen node properties, you can specify how many records the node will create (default is 1) and generate a random seed\.

For more information about KDE, including examples, see the [KDE documentation](http://scikit-learn.org/stable/modules/density.html#kernel-density-estimation)\. ^1^

^1^ "User Guide\." *Kernel Density Estimation*\. Web\. © 2007\-2018, scikit\-learn developers\.

<!-- </article "role="article" "> -->
# K\-Means node #

The K\-Means node provides a method of cluster analysis\. It can be used to cluster the dataset into distinct groups when you don't know what those groups are at the beginning\. Unlike most learning methods in  SPSS Modeler, K\-Means models do not use a target field\. This type of learning, with no target field, is called unsupervised learning\. Instead of trying to predict an outcome, K\-Means tries to uncover patterns in the set of input fields\. Records are grouped so that records within a group or cluster tend to be similar to each other, but records in different groups are dissimilar\.

K\-Means works by defining a set of starting cluster centers derived from data\. It then assigns each record to the cluster to which it is most similar, based on the record's input field values\. After all cases have been assigned, the cluster centers are updated to reflect the new set of records assigned to each cluster\. The records are then checked again to see whether they should be reassigned to a different cluster, and the record assignment/cluster iteration process continues until either the maximum number of iterations is reached, or the change between one iteration and the next fails to exceed a specified threshold\.

Note: The resulting model depends to a certain extent on the order of the training data\. Reordering the data and rebuilding the model may lead to a different final cluster model\.

Requirements\. To train a K\-Means model, you need one or more fields with the role set to `Input`\. Fields with the role set to `Output`, `Both`, or `None` are ignored\.

Strengths\. You do not need to have data on group membership to build a K\-Means model\. The K\-Means model is often the fastest method of clustering for large datasets\.

<!-- </article "role="article" "> -->
# K\-Means\-AS node #

K\-Means is one of the most commonly used clustering algorithms\. It clusters data points into a predefined number of clusters\. The K\-Means\-AS node in  SPSS Modeler is implemented in Spark\.

See [K\-Means Algorithms](https://spark.apache.org/docs/2.2.0/ml-clustering.html) for more details\.^1^

Note that the K\-Means\-AS node performs one\-hot encoding automatically for categorical variables\.

^1^ "Clustering\." *Apache Spark*\. MLlib: Main Guide\. Web\. 3 Oct 2017\.

<!-- </article "role="article" "> -->
# KNN node #

Nearest Neighbor Analysis is a method for classifying cases based on their similarity to other cases\. In machine learning, it was developed as a way to recognize patterns of data without requiring an exact match to any stored patterns, or cases\. Similar cases are near each other and dissimilar cases are distant from each other\. Thus, the distance between two cases is a measure of their dissimilarity\.

Cases that are near each other are said to be "neighbors\." When a new case (holdout) is presented, its distance from each of the cases in the model is computed\. The classifications of the most similar cases – the nearest neighbors – are tallied and the new case is placed into the category that contains the greatest number of nearest neighbors\.

You can specify the number of nearest neighbors to examine; this value is called `k`\. The pictures show how a new case would be classified using two different values of `k`\. When `k` = 5, the new case is placed in category `1` because a majority of the nearest neighbors belong to category `1`\. However, when `k` = 9, the new case is placed in category `0` because a majority of the nearest neighbors belong to category `0`\.

Nearest neighbor analysis can also be used to compute values for a continuous target\. In this situation, the average or median target value of the nearest neighbors is used to obtain the predicted value for the new case\.

<!-- </article "role="article" "> -->
# Reading in source text #

You can use the Language Identifier node to identify the natural language of a text field within your source data\. The output of this node is a derived field that contains the detected language code\.

![Language Identifier node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/ta_languageidentifier.png)

Data for text mining can be in any of the standard formats that are used by  SPSS Modeler flows, including databases or other "rectangular" formats that represent data in rows and columns\.

<!-- <ul> -->

 *  To read in text from any of the standard data formats used by  SPSS Modeler flows, such as a database with one or more text fields for customer comments, you can use an Import node\.
 *  When you're processing large amounts of data, which might include text in several different languages, use the Language Identifier node to identify the language used in a specific field\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Linear node #

Linear regression is a common statistical technique for classifying records based on the values of numeric input fields\. Linear regression fits a straight line or surface that minimizes the discrepancies between predicted and actual output values\.

Requirements\. Only numeric fields can be used in a linear regression model\. You must have exactly one target field (with the role set to  Target) and one or more predictors (with the role set to  Input)\. Fields with a role of  Both or  None are ignored, as are non\-numeric fields\. (If necessary, non\-numeric fields can be recoded using a Derive node\.)

Strengths\. Linear regression models are relatively simple and give an easily interpreted mathematical formula for generating predictions\. Because linear regression is a long\-established statistical procedure, the properties of these models are well understood\. Linear models are also typically very fast to train\. The Linear node provides methods for automatic field selection in order to eliminate nonsignificant input fields from the equation\.

Tip: In cases where the target field is categorical rather than a continuous range, such as  yes/no or  churn/don't churn, logistic regression can be used as an alternative\. Logistic regression also provides support for non\-numeric inputs, removing the need to recode these fields\. Note: When first creating a flow, you select which runtime to use\. By default, flows use the  IBM SPSS Modeler runtime\. If you want to use native Spark algorithms instead of SPSS algorithms, select the  Spark runtime\. Properties for this node will vary depending on which runtime option you choose\.
<!-- </article "role="article" "> -->
# Linear\-AS node #

Linear regression is a common statistical technique for classifying records based on the values of numeric input fields\. Linear regression fits a straight line or surface that minimizes the discrepancies between predicted and actual output values\.

Requirements\. Only numeric fields and categorical predictors can be used in a linear regression model\. You must have exactly one target field (with the role set to  Target) and one or more predictors (with the role set to  Input)\. Fields with a role of  Both or  None are ignored, as are non\-numeric fields\. (If necessary, non\-numeric fields can be recoded using a Derive node\.)

Strengths\. Linear regression models are relatively simple and give an easily interpreted mathematical formula for generating predictions\. Because linear regression is a long\-established statistical procedure, the properties of these models are well understood\. Linear models are also typically very fast to train\. The Linear node provides methods for automatic field selection in order to eliminate non\-significant input fields from the equation\.

Note: In cases where the target field is categorical rather than a continuous range, such as  yes/no or  churn/don't churn, logistic regression can be used as an alternative\. Logistic regression also provides support for non\-numeric inputs, removing the need to recode these fields\.
<!-- </article "role="article" "> -->
# LSVM node #

With the LSVM node, you can use a linear support vector machine to classify data\. LSVM is particularly suited for use with wide datasets\-\-that is, those with a large number of predictor fields\. You can use the default settings on the node to produce a basic model relatively quickly, or you can use the build options to experiment with different settings\.

The LSVM node is similar to the SVM node, but it is linear and is better at handling a large number of records\.

After the model is built, you can:

<!-- <ul> -->

 *  Browse the model nugget to display the relative importance of the input fields in building the model\.
 *  Append a Table node to the model nugget to view the model output\.

<!-- </ul> -->

Example\. A medical researcher has obtained a dataset containing characteristics of a number of human cell samples extracted from patients who were believed to be at risk of developing cancer\. Analysis of the original data showed that many of the characteristics differed significantly between benign and malignant samples\. The researcher wants to develop an LSVM model that can use the values of similar cell characteristics in samples from other patients to give an early indication of whether their samples might be benign or malignant\.

<!-- </article "role="article" "> -->
# Means node #

The Means node compares the means between independent groups or between pairs of related fields to test whether a significant difference exists\. For example, you can compare mean revenues before and after running a promotion or compare revenues from customers who didn't receive the promotion with those who did\.

You can compare means in two different ways, depending on your data:

<!-- <ul> -->

 *  Between groups within a field\. To compare independent groups, select a test field and a grouping field\. For example, you could exclude a sample of "holdout" customers when sending a promotion and compare mean revenues for the holdout group with all of the others\. In this case, you would specify a single test field that indicates the revenue for each customer, with a flag or nominal field that indicates whether they received the offer\. The samples are independent in the sense that each record is assigned to one group or another, and there is no way to link a specific member of one group to a specific member of another\. You can also specify a nominal field with more than two values to compare the means for multiple groups\. When executed, the node calculates a one\-way ANOVA test on the selected fields\. In cases where there are only two field groups, the one\-way ANOVA results are essentially the same as an independent\-samples `t` test\.
 *  Between pairs of fields\. When comparing means for two related fields, the groups must be paired in some way for the results to be meaningful\. For example, you could compare the mean revenues from the same group of customers before and after running a promotion or compare usage rates for a service between husband\-wife pairs to see if they are different\. Each record contains two separate but related measures that can be compared meaningfully\. When executed, the node calculates a paired\-samples `t` test on each field pair selected\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Merge node #

The function of a Merge node is to take multiple input records and create a single output record containing all or some of the input fields\. This is a useful operation when you want to merge data from different sources, such as internal customer data and purchased demographic data\.

You can merge data in the following ways\.

<!-- <ul> -->

 *  Merge by  Order concatenates corresponding records from all sources in the order of input until the smallest data source is exhausted\. It is important if using this option that you have sorted your data using a Sort node\.
 *  Merge using a  Key field, such as `Customer ID`, to specify how to match records from one data source with records from the other(s)\. Several types of joins are possible, including inner join, full outer join, partial outer join, and anti\-join\.
 *  Merge by  Condition means that you can specify a condition to be satisfied for the merge to take place\. You can specify the condition directly in the node, or build the condition using the Expression Builder\.
 *  Merge by  Ranked Condition is a left sided outer join in which you specify a condition to be satisfied for the merge to take place and a ranking expression which sorts into order from low to high\. Most often used to merge geospatial data, you can specify the condition directly in the node, or build the condition using the Expression Builder\.

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Handling fields with missing values #

If the majority of missing values are concentrated in a small number of fields, you can address them at the field level rather than at the record level\. This approach also allows you to experiment with the relative importance of particular fields before deciding on an approach for handling missing values\. If a field is unimportant in modeling, it probably isn't worth keeping, regardless of how many missing values it has\.

For example, a market research company may collect data from a general questionnaire containing 50 questions\. Two of the questions address age and political persuasion, information that many people are reluctant to give\. In this case, `Age` and `Political_persuasion` have many missing values\.

<!-- </article "role="article" "> -->
# Handling records with missing values #

If the majority of missing values are concentrated in a small number of records, you can just exclude those records\. For example, a bank usually keeps detailed and complete records on its loan customers\.

If, however, the bank is less restrictive in approving loans for its own staff members, data gathered for staff loans is likely to have several blank fields\. In such a case, there are two options for handling these missing values:

<!-- <ul> -->

 *  You can use a [Select node](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/select.html) to remove the staff records
 *  If the data set is large, you can discard all records with blanks

<!-- </ul> -->

<!-- </article "role="article" "> -->
# Handling missing values #

You should decide how to treat missing values in light of your business or domain knowledge\. To ease training time and increase accuracy, you may want to remove blanks from your data set\. On the other hand, the presence of blank values may lead to new business opportunities or additional insights\.

In choosing the best technique, you should consider the following aspects of your data:

<!-- <ul> -->

 *  Size of the data set
 *  Number of fields containing blanks
 *  Amount of missing information

<!-- </ul> -->

In general terms, there are two approaches you can follow:

<!-- <ul> -->

 *  You can exclude fields or records with missing values
 *  You can impute, replace, or coerce missing values using a variety of methods

<!-- </ul> -->

<!-- </article "role="article" "> -->
# MultiLayerPerceptron\-AS node #

Multilayer perceptron is a classifier based on the feedforward artificial neural network and consists of multiple layers\.

Each layer is fully connected to the next layer in the network\. See [Multilayer Perceptron Classifier (MLPC)](https://spark.apache.org/docs/latest/ml-classification-regression.html#multilayer-perceptron-classifier) for details\.^1^

The MultiLayerPerceptron\-AS node in  watsonx\.ai is implemented in Spark\. To use a this node, you must set up an upstream Type node\. The MultiLayerPerceptron\-AS node will read input values from the Type node (or from the Types of an upstream import node)\.

^1^ "Multilayer perceptron classifier\." *Apache Spark*\. MLlib: Main Guide\. Web\. 5 Oct 2018\.

<!-- </article "role="article" "> -->
# Multiplot node #

A multiplot is a special type of plot that displays multiple `Y` fields over a single `X` field\. The `Y` fields are plotted as colored lines and each is equivalent to a Plot node with Style set to  Line and X Mode set to  Sort\. Multiplots are useful when you have time sequence data and want to explore the fluctuation of several variables over time\.

<!-- </article "role="article" "> -->
# Neural Net node #

A neural network can approximate a wide range of predictive models with minimal demands on model structure and assumption\. The form of the relationships is determined during the learning process\. If a linear relationship between the target and predictors is appropriate, the results of the neural network should closely approximate those of a traditional linear model\. If a nonlinear relationship is more appropriate, the neural network will automatically approximate the "correct" model structure\.

The trade\-off for this flexibility is that the neural network is not easily interpretable\. If you are trying to explain an underlying process that produces the relationships between the target and predictors, it would be better to use a more traditional statistical model\. However, if model interpretability is not important, you can obtain good predictions using a neural network\.

Field requirements\. There must be at least one Target and one Input\. Fields set to Both or None are ignored\. There are no measurement level restrictions on targets or predictors (inputs)\.

The initial weights assigned to neural networks during model building, and therefore the final models produced, depend on the order of the fields in the data\.  Watsonx\.ai automatically sorts data by field name before presenting it to the neural network for training\. This means that explicitly changing the order of the fields in the data upstream will not affect the generated neural net models when a random seed is set in the model builder\. However, changing the input field names in a way that changes their sort order will produce different neural network models, even with a random seed set in the model builder\. The model quality will not be affected significantly given different sort order of field names\.

<!-- </article "role="article" "> -->
# One\-Class SVM node #

The One\-Class SVM© node uses an unsupervised learning algorithm\. The node can be used for novelty detection\. It will detect the soft boundary of a given set of samples, to then classify new points as belonging to that set or not\. This One\-Class SVM modeling node is implemented in Python and requires the  scikit\-learn© Python library\.

For details about the scikit\-learn library, see [Support Vector Machines](http://scikit-learn.org/stable/modules/svm.html#svm-outlier-detection)^1^\.

The Modeling tab on the palette contains the One\-Class SVM node and other Python nodes\.

Note: One\-Class SVM is used for usupervised outlier and novelty detection\. In most cases, we recommend using a known, "normal" dataset to build the model so the algorithm can set a correct boundary for the given samples\. Parameters for the model – such as nu, gamma, and kernel – impact the result significantly\. So you may need to experiment with these options until you find the optimal settings for your situation\.

^1^Smola, Schölkopf\. "A Tutorial on Support Vector Regression\." *Statistics and Computing Archive*, vol\. 14, no\. 3, August 2004, pp\. 199\-222\. (http://citeseerx\.ist\.psu\.edu/viewdoc/summary?doi=10\.1\.1\.114\.4288)

<!-- </article "role="article" "> -->
# Partition node #

Partition nodes are used to generate a partition field that splits the data into separate subsets or samples for the training, testing, and validation stages of model building\. By using one sample to generate the model and a separate sample to test it, you can get a good indication of how well the model will generalize to larger datasets that are similar to the current data\.

The Partition node generates a nominal field with the role set to  Partition\. Alternatively, if an appropriate field already exists in your data, it can be designated as a partition using a Type node\. In this case, no separate Partition node is required\. Any instantiated nominal field with two or three values can be used as a partition, but flag fields cannot be used\.

Multiple partition fields can be defined in a flow, but if so, a single partition field must be selected in each modeling node that uses partitioning\. (If only one partition is present, it is automatically used whenever partitioning is enabled\.)

To create a partition field based on some other criterion such as a date range or location, you can also use a Derive node\. See [Derive node](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/derive.html#derive) for more information\.

Example\. When building an RFM flow to identify recent customers who have positively responded to previous marketing campaigns, the marketing department of a sales company uses a Partition node to split the data into training and test partitions\.

<!-- </article "role="article" "> -->
# QUEST node #

QUEST—or Quick, Unbiased, Efficient Statistical Tree—is a binary classification method for building decision trees\. A major motivation in its development was to reduce the processing time required for large C&R Tree analyses with either many variables or many cases\. A second goal of QUEST was to reduce the tendency found in classification tree methods to favor inputs that allow more splits, that is, continuous (numeric range) input fields or those with many categories\.

<!-- <ul> -->

 *  QUEST uses a sequence of rules, based on significance tests, to evaluate the input fields at a node\. For selection purposes, as little as a single test may need to be performed on each input at a node\. Unlike C&R Tree, all splits are not examined, and unlike C&R Tree and CHAID, category combinations are not tested when evaluating an input field for selection\. This speeds the analysis\.
 *  Splits are determined by running quadratic discriminant analysis using the selected input on groups formed by the target categories\. This method again results in a speed improvement over exhaustive search (C&R Tree) to determine the optimal split\.

<!-- </ul> -->

Requirements\. Input fields can be continuous (numeric ranges), but the target field must be categorical\. All splits are binary\. Weight fields cannot be used\. Any ordinal (ordered set) fields used in the model must have numeric storage (not string)\. If necessary, the Reclassify node can be used to convert them\.

Strengths\. Like CHAID, but unlike C&R Tree, QUEST uses statistical tests to decide whether or not an input field is used\. It also separates the issues of input selection and splitting, applying different criteria to each\. This contrasts with CHAID, in which the statistical test result that determines variable selection also produces the split\. Similarly, C&R Tree employs the impurity\-change measure to both select the input field and to determine the split\.

<!-- </article "role="article" "> -->
# Random Forest node #

Random Forest© is an advanced implementation of a bagging algorithm with a tree model as the base model\.

In random forests, each tree in the ensemble is built from a sample drawn with replacement (for example, a bootstrap sample) from the training set\. When splitting a node during the construction of the tree, the split that is chosen is no longer the best split among all features\. Instead, the split that is picked is the best split among a random subset of the features\. Because of this randomness, the bias of the forest usually slightly increases (with respect to the bias of a single non\-random tree) but, due to averaging, its variance also decreases, usually more than compensating for the increase in bias, hence yielding an overall better model\.^1^

The Random Forest node in  watsonx\.ai is implemented in Python\. The nodes palette contains this node and other Python nodes\.

For more information about random forest algorithms, see [Forests of randomized trees](https://scikit-learn.org/stable/modules/ensemble.html#forest)\.

^1^L\. Breiman, "Random Forests," Machine Learning, 45(1), 5\-32, 2001\.

<!-- </article "role="article" "> -->
# Reclassify node #

The Reclassify node enables the transformation from one set of categorical values to another\. Reclassification is useful for collapsing categories or regrouping data for analysis\.

For example, you could reclassify the values for `Product` into three groups, such as `Kitchenware`, `Bath and Linens`, and `Appliances`\.

Reclassification can be performed for one or more symbolic fields\. You can also choose to substitute the new values for the existing field or generate a new field\.

<!-- </article "role="article" "> -->
# Regression node #

Linear regression is a common statistical technique for classifying records based on the values of numeric input fields\. Linear regression fits a straight line or surface that minimizes the discrepancies between predicted and actual output values\.

Requirements\. Only numeric fields can be used in a regression model\. You must have exactly one target field (with the role set to `Target`) and one or more predictors (with the role set to `Input`)\. Fields with a role of `Both` or `None` are ignored, as are non\-numeric fields\. (If necessary, non\-numeric fields can be recoded using a Derive node\. )

Strengths\. Regression models are relatively simple and give an easily interpreted mathematical formula for generating predictions\. Because regression modeling is a long\-established statistical procedure, the properties of these models are well understood\. Regression models are also typically very fast to train\. The Regression node provides methods for automatic field selection in order to eliminate nonsignificant input fields from the equation\.

Note: In cases where the target field is categorical rather than a continuous range, such as `yes`/`no` or `churn`/`don't churn`, logistic regression can be used as an alternative\. Logistic regression also provides support for non\-numeric inputs, removing the need to recode these fields\. See [Logistic node](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/logreg.html#logreg) for more information\.
<!-- </article "role="article" "> -->
# Report node #

You can use the Report node to create formatted reports containing fixed text, data, or other expressions derived from the data\. Specify the format of the report by using text templates to define the fixed text and the data output constructions\. You can provide custom text formatting using HTML tags in the template and by setting output options\. Data values and other conditional output are included in the report using CLEM expressions in the template\.

<!-- </article "role="article" "> -->
# Restructure node #

With the Restructure node, you can generate multiple fields based on the values of a nominal or flag field\. The newly generated fields can contain values from another field or numeric flags (0 and 1)\. The functionality of this node is similar to that of the Set to Flag node\. However, it offers more flexibility by allowing you to create fields of any type (including numeric flags), using the values from another field\. You can then perform aggregation or other manipulations with other nodes downstream\. (The Set to Flag node lets you aggregate fields in one step, which may be convenient if you are creating flag fields\.)

Figure 1\. Restructure node

![Restructure node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/restructure_node.png)
<!-- </article "role="article" "> -->
# RFM Aggregate node #

The Recency, Frequency, Monetary (RFM) Aggregate node allows you to take customers' historical transactional data, strip away any unused data, and combine all of their remaining transaction data into a single row (using their unique customer ID as a key) that lists when they last dealt with you (recency), how many transactions they have made (frequency), and the total value of those transactions (monetary)\.

Before proceeding with any aggregation, you should take time to clean the data, concentrating especially on any missing values\.

After you identify and transform the data using the RFM Aggregate node, you might use an RFM Analysis node to carry out further analysis\.

Note that after the data file has been run through the RFM Aggregate node, it won't have any target values; therefore, before using the data file as input for further predictive analysis with any modeling nodes such as C5\.0 or CHAID, you need to merge it with other customer data (for example, by matching the customer IDs)\.

The RFM Aggregate and RFM Analysis nodes use independent binning; that is, they rank and bin data on each measure of recency, frequency, and monetary value, without regard to their values or the other two measures\.

<!-- </article "role="article" "> -->
# RFM Analysis node #

You can use the Recency, Frequency, Monetary (RFM) Analysis node to determine quantitatively which customers are likely to be the best ones by examining how recently they last purchased from you (recency), how often they purchased (frequency), and how much they spent over all transactions (monetary)\.

The reasoning behind RFM analysis is that customers who purchase a product or service once are more likely to purchase again\. The categorized customer data is separated into a number of bins, with the binning criteria adjusted as you require\. In each of the bins, customers are assigned a score; these scores are then combined to provide an overall RFM score\. This score is a representation of the customer's membership in the bins created for each of the RFM parameters\. This binned data may be sufficient for your needs, for example, by identifying the most frequent, high\-value customers; alternatively, it can be passed on in a flow for further modeling and analysis\.

Note, however, that although the ability to analyze and rank RFM scores is a useful tool, you must be aware of certain factors when using it\. There may be a temptation to target customers with the highest rankings; however, over\-solicitation of these customers could lead to resentment and an actual fall in repeat business\. It is also worth remembering that customers with low scores should not be neglected but instead may be cultivated to become better customers\. Conversely, high scores alone do not necessarily reflect a good sales prospect, depending on the market\. For example, a customer in bin 5 for recency, meaning that they have purchased very recently, may not actually be the best target customer for someone selling expensive, longer\-life products such as cars or televisions\.

Note: Depending on how your data is stored, you may need to precede the RFM Analysis node with an RFM Aggregate node to transform the data into a usable format\. For example, input data must be in customer format, with one row per customer; if the customers' data is in transactional form, use an RFM Aggregate node upstream to derive the recency, frequency, and monetary fields\.

The RFM Aggregate and RFM Analysis nodes in are set up to use independent binning; that is, they rank and bin data on each measure of recency, frequency, and monetary value, without regard to their values or the other two measures\.

<!-- </article "role="article" "> -->