itsjhuang/watsonx-docs-document-type huggingface dataset:
PCA/Factor node (SPSS Modeler)
PCA/Factor node The PCA/Factor node provides powerful data-reduction techniques to reduce the complexity of your data. Two similar but distinct approaches are provided. * Principal components analysis (PCA) finds linear combinations of the input fields that do the best job of capturing the variance in the entire set of fields, where the components are orthogonal (perpendicular) to each other. PCA focuses on all variance, including both shared and unique variance. * Factor analysis attempts to identify underlying concepts, or factors, that explain the pattern of correlations within a set of observed fields. Factor analysis focuses on shared variance only. Variance that is unique to specific fields is not considered in estimating the model. Several methods of factor analysis are provided by the Factor/PCA node. For both approaches, the goal is to find a small number of derived fields that effectively summarize the information in the original set of fields. Requirements. Only numeric fields can be used in a PCA-Factor model. To estimate a factor analysis or PCA, you need one or more fields with the role set to Input fields. Fields with the role set to Target, Both, or None are ignored, as are non-numeric fields. Strengths. Factor analysis and PCA can effectively reduce the complexity of your data without sacrificing much of the information content. These techniques can help you build more robust models that execute more quickly than would be possible with the raw input fields.
Troubleshooting
Troubleshooting If you encounter an issue in IBM watsonx, use the following resources to resolve the problem. * [View IBM Cloud service status](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/service-status.html) * [Troubleshoot connections](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/troubleshoot-conn.html) * [Troubleshoot Data Refinery](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/ts_index.html) * [Troubleshoot Synthetic Data Generator](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/ts_sd.html) * [Troubleshoot IBM Cloud Object Storage](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/troubleshoot-cos.html) * [Troubleshoot Watson Machine Learning](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/ml_troubleshooting.html) * [Troubleshoot Watson OpenScale](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/wos-troubleshoot.html) * [Troubleshoot Watson Studio on IBM Cloud](https://dataplatform.cloud.ibm.com/docs/content/wsj/troubleshoot/wscloud-troubleshoot.html) * [Known issues](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/known-issues.html) * [Get help](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/get-help.html)
Scatter matrix charts
Scatter matrix charts Scatter plot matrices are a good way to determine whether linear correlations exist between multiple variables.
Summary (SPSS Modeler)
Summary This example Introduction to Modeling flow demonstrates the basic steps for creating, evaluating, and scoring a model. * The modeling node estimates the model by studying records for which the outcome is known, and creates a model nugget. This is sometimes referred to as training the model. * The model nugget can be added to any flow with the expected fields to score records. By scoring the records for which you already know the outcome (such as existing customers), you can evaluate how well it performs. * After you're satisfied that the model performs acceptably well, you can score new data (such as prospective customers) to predict how they will respond. * The data used to train or estimate the model may be referred to as the analytical or historical data; the scoring data may also be referred to as the operational data.
Python and Jython
Python and Jython Jython is an implementation of the Python scripting language, which is written in the Java language and integrated with the Java platform. Python is a powerful object-oriented scripting language. Jython is useful because it provides the productivity features of a mature scripting language and, unlike Python, runs in any environment that supports a Java virtual machine (JVM). This means that the Java libraries on the JVM are available to use when you're writing programs. With Jython, you can take advantage of this difference, and use the syntax and most of the features of the Python language. As a scripting language, Python (and its Jython implementation) is easy to learn and efficient to code, and has minimal required structure to create a running program. Code can be entered interactively, that is, one line at a time. Python is an interpreted scripting language; there is no precompile step, as there is in Java. Python programs are simply text files that are interpreted as they're input (after parsing for syntax errors). Simple expressions, like defined values, as well as more complex actions, such as function definitions, are immediately executed and available for use. Any changes that are made to the code can be tested quickly. Script interpretation does, however, have some disadvantages. For example, use of an undefined variable is not a compiler error, so it's detected only if (and when) the statement in which the variable is used is executed. In this case, you can edit and run the program to debug the error. Python sees everything, including all data and code, as an object. You can, therefore, manipulate these objects with lines of code. Some select types, such as numbers and strings, are more conveniently considered as values, not objects; this is supported by Python. There is one null value that's supported. This null value has the reserved name None. For a more in-depth introduction to Python and Jython scripting, and for some example scripts, see [http://www.ibm.com/developerworks/java/tutorials/j-jython1/j-jython1.html](http://www.ibm.com/developerworks/java/tutorials/j-jython1/j-jython1.html) and [http://www.ibm.com/developerworks/java/tutorials/j-jython2/j-jython2.html](http://www.ibm.com/developerworks/java/tutorials/j-jython2/j-jython2.html).
questnode properties
questnode properties ![Quest node icon](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/images/questnodeicon.png)The Quest node provides a binary classification method for building decision trees, designed to reduce the processing time required for large C&R Tree analyses while also reducing the tendency found in classification tree methods to favor inputs that allow more splits. Input fields can be numeric ranges (continuous), but the target field must be categorical. All splits are binary. questnode properties Table 1. questnode properties questnode Properties Values Property description target field Quest models require a single target and one or more input fields. A frequency field can also be specified. See [Common modeling node properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/modelingnodeslots_common.html) for more information. continue_training_existing_model flag objective StandardBoostingBaggingpsm psm is used for very large datasets, and requires a server connection. model_output_type SingleInteractiveBuilder use_tree_directives flag tree_directives string use_max_depth DefaultCustom max_depth integer Maximum tree depth, from 0 to 1000. Used only if use_max_depth = Custom. prune_tree flag Prune tree to avoid overfitting. use_std_err flag Use maximum difference in risk (in Standard Errors). std_err_multiplier number Maximum difference. max_surrogates number Maximum surrogates. use_percentage flag min_parent_records_pc number min_child_records_pc number min_parent_records_abs number min_child_records_abs number use_costs flag costs structured Structured property. priors DataEqualCustom custom_priors structured Structured property. adjust_priors flag trails number Number of component models for boosting or bagging. set_ensemble_method VotingHighestProbabilityHighestMeanProbability Default combining rule for categorical targets. range_ensemble_method MeanMedian Default combining rule for continuous targets. large_boost flag Apply boosting to very large data sets. split_alpha number Significance level for splitting. train_pct number Overfit prevention set. set_random_seed flag Replicate results option. seed number calculate_variable_importance flag calculate_raw_propensities flag calculate_adjusted_propensities flag adjusted_propensity_partition TestValidation
Tone classification
Tone classification The Tone model in the Watson Natural Language Processing classification workflow classifies the tone in the input text. Workflow name ensemble_classification-workflow_en_tone-stock Supported languages * English and French Capabilities The Tone classification model is a pre-trained document classification model for the task of classifying the tone in the input document. The model identifies the tone of the input document and classifies it as: * Excited * Frustrated * Impolite * Polite * Sad * Satisfied * Sympathetic Unlike the Sentiment model, which classifies each individual sentence, the Tone model classifies the entire input document. As such, the Tone model works optimally when the input text to classify is no longer than 1000 characters. If you would like to classify texts longer than 1000 characters, split the text into sentences or paragraphs for example and apply the Tone model on each sentence or paragraph. A document may be classified into multiple categories or into no category. Capabilities of tone classification Capabilities Example Identifies the tone of a document and classifies it "I'm really happy with how this was handled, thank you!" --> excited, satisfied Dependencies on other blocks None Code sample import watson_nlp Load the Tone workflow model for English tone_model = watson_nlp.load('ensemble_classification-workflow_en_tone-stock') Run the Tone model tone_result = tone_model.run("I'm really happy with how this was handled, thank you!") print(tone_result) Output of the code sample: { "classes": [ { "class_name": "excited", "confidence": 0.6896854620082722 }, { "class_name": "satisfied", "confidence": 0.6570277557333078 }, { "class_name": "polite", "confidence": 0.33628806679460566 }, { "class_name": "sympathetic", "confidence": 0.17089694967744093 }, { "class_name": "sad", "confidence": 0.06880583874412932 }, { "class_name": "frustrated", "confidence": 0.010365418217209686 }, { "class_name": "impolite", "confidence": 0.002470793624966174 } ], "producer_id": { "name": "Voting based Ensemble", "version": "0.0.1" } } Parent topic:[Watson Natural Language Processing task catalog](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-catalog.html)
Hierarchical text categorization
Hierarchical text categorization The Watson Natural Language Processing Categories block assigns individual nodes within a hierarchical taxonomy to an input document. For example, in the text IBM announces new advances in quantum computing, examples of extracted categories are technology and computing/hardware/computer and technology and computing/operating systems. These categories represent level 3 and level 2 nodes in a hierarchical taxonomy. This block differs from the Classification block in that training starts from a set of seed phrases associated with each node in the taxonomy, and does not require labeled documents. Note that the Hierarchical text categorization block can only be used in a notebook that is started in an environment based on Runtime 22.2 or Runtime 23.1 that includes the Watson Natural Language Processing library. Block name categories_esa_en_stock Supported languages The Categories block is available for the following languages. For a list of the language codes and the corresponding language, see [Language codes](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-catalog.htmllang-codes). de, en Capabilities Use this block to determine the topics of documents on the web by categorizing web pages into a taxonomy of general domain topics, for ad placement and content recommendation. The model was tested on data from news reports and general web pages. For a list of the categories that can be returned, see [Category types](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-returned-categories.html). Dependencies on other blocks The following block must run before you can run the hierarchical categorization block: * syntax_izumo_<language>_stock Code sample import watson_nlp Load Syntax and a Categories model for English syntax_model = watson_nlp.load('syntax_izumo_en_stock') categories_model = watson_nlp.load('categories_esa_en_stock') Run the syntax model on the input text syntax_prediction = syntax_model.run('IBM announced new advances in quantum computing') Run the categories model on the result of syntax categories = categories_model.run(syntax_prediction) print(categories) Output of the code sample: { "categories": [ { "labels": "technology & computing", "computing" ], "score": 0.992489, "explanation": ] }, { "labels": "science", "physics" ], "score": 0.945449, "explanation": ] } ], "producer_id": { "name": "ESA Hierarchical Categories", "version": "1.0.0" } } Parent topic:[Watson Natural Language Processing task catalog](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-catalog.html)
Project collaborator roles and permissions
Project collaborator roles and permissions When you add a collaborator to a project, you specify which actions that the user can do by assigning a role. These roles provide these permissions for projects: Action Viewer Editor Admin View all information for data assets ✓ ✓ ✓ View jobs ✓ ✓ ✓ Add and read data assets ✓ ✓ View Data Refinery flows and SPSS Modeler flows ✓ ✓ View all other types of assets ✓ ✓ ✓ Create, add, modify, or delete all types of assets ✓ ✓ Submit inference requests to foundation models, including tuned foundation models ✓ ✓ Run and schedule assets that run in tools and jobs ✓ ✓ Create and modify data asset visualizations ✓ ✓ ✓ Save visualizations to your project ✓ ✓ Create and modify data asset profiles ✓ ✓ Share notebooks ✓ ✓ Promote assets to deployment spaces ✓ ✓ Edit the project readme ✓ ✓ Use project access tokens ✓ ✓ Manage environment templates ✓ ✓ Stop your own environment runtimes ✓ ✓ Export a project to desktop ✓ ✓ Manage project collaborators * ✓ Set up integrations ✓ Manage associated services ✓ Manage project access tokens ✓ Mark project as sensitive ✓ * To add collaborators or change collaborator roles, users with the Admin role in the project must also belong to the project creator's IBM Cloud account. Learn more * [Adding collaborators to a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/collaborate.html) * [Determine your roles](https://dataplatform.cloud.ibm.com/docs/content/wsj/admin/your-roles.html) Parent topic:[Administering projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/admin-project.html)
SMOTE node (SPSS Modeler)
SMOTE node The Synthetic Minority Over-sampling Technique (SMOTE) node provides an over-sampling algorithm to deal with imbalanced data sets. It provides an advanced method for balancing data. The SMOTE node in watsonx.ai is implemented in Python and requires the imbalanced-learn© Python library. For details about the imbalanced-learn library, see [imbalanced-learn documentation](https://imbalanced-learn.org/stable/index.html)^1^. The Modeling tab on the nodes palette contains the SMOTE node and other Python nodes. ^1^Lemaître, Nogueira, Aridas. "Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning." Journal of Machine Learning Research, vol. 18, no. 17, 2017, pp. 1-5. (http://jmlr.org/papers/v18/16-365.html)
reordernode properties
reordernode properties ![Field Reorder node icon](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/images/fieldreordernodeicon.png)The Field Reorder node defines the natural order used to display fields downstream. This order affects the display of fields in a variety of places, such as tables, lists, and when selecting fields. This operation is useful when working with wide datasets to make fields of interest more visible. reordernode properties Table 1. reordernode properties reordernode properties Data type Property description mode CustomAuto You can sort values automatically or specify a custom order. sort_by NameTypeStorage ascending flag start_fields [field1 field2 … fieldn] New fields are inserted after these fields. end_fields [field1 field2 … fieldn] New fields are inserted before these fields.
K-Means-AS node (SPSS Modeler)
K-Means-AS node K-Means is one of the most commonly used clustering algorithms. It clusters data points into a predefined number of clusters. The K-Means-AS node in SPSS Modeler is implemented in Spark. See [K-Means Algorithms](https://spark.apache.org/docs/2.2.0/ml-clustering.html) for more details.^1^ Note that the K-Means-AS node performs one-hot encoding automatically for categorical variables. ^1^ "Clustering." Apache Spark. MLlib: Main Guide. Web. 3 Oct 2017.
Building the flow (SPSS Modeler)
Building the flow 1. Add a Data Asset node that points to telco.csv. Figure 1. Auto Data Prep example flow ![Auto Data Prep example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata.png) 2. Attach a Type node to the Data Asset node. Set the measure for the churn field to Flag, and set the role to Target. Make sure the role for all other fields is set to Input. Figure 2. Setting the measurement level and role ![Setting the measurement level and role](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_build_target.png) 3. Attach a Logistic node to the Type node. 4. In the Logistic node's properties, under MODEL SETTINGS, select the Binomial procedure. For Model Name, select Custom and enter No ADP - churn. Figure 3. Choosing model options ![Choosing model options](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_binomial_default.png) 5. Attach an Auto Data Prep node to the Type node. Under OBJECTIVES, leave the default settings in place to analyze and prepare your data by balancing both speed and accuracy. 6. Run the flow to analyze and process your data. Other Auto Data Prep node properties allow you to specify that you want to concentrate more on accuracy, more on the speed of processing, or to fine tune many of the data preparation processing steps. Note: If you want to adjust the node properties and run the flow again in the future, since the model already exists, you must first click Clear Analysis, under OBJECTIVES before running the flow again. Figure 4. Auto Data Prep default objectives ![Auto Data Prep default objectives](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autodata_objectives.png) 7. Attach a Logistic node to the Auto Data Prep node. 8. In the Logistic node's properties, under MODEL SETTINGS, select the Binomial procedure. For Model Name, select Custom and enter After ADP - churn.
Creating a web chart (SPSS Modeler)
Creating a web chart Since many of the data fields are categorical, you can also try plotting a web chart, which maps associations between different categories. Figure 1. Web node ![Web node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_web_flow.png) 1. Place a Web node on the canvas and connect it to the drug1n.csv Data Asset node. Then double-click the Web node to edit its properties. 2. Select the fields BP (for blood pressure) and Drug. Click Save, then right-click the Web node and select Run. A web chart is added to the Outputs pane. Figure 2. Web graph of drugs vs. blood pressure ![Web graph of drugs vs. blood pressure](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_web.png) From the plot, it appears that drug Y is associated with all three levels of blood pressure. This is no surprise; you have already determined the situation in which drug Y is best. But if you ignore drug Y and focus on the other drugs, you can see that drugs A and B are also associated with high blood pressure. And drugs C and X are associated with low blood pressure. And normal blood pressure is associated with drug X. At this point, though, you still don't know how to choose between drugs A and B or between drugs C and X, for a given patient. This is where modeling can help.
Deploying Decision Optimization models
Decision Optimization When you have created and solved your Decision Optimization models, you can deploy them using Watson Machine Learning. See the [Decision Optimization experiment UI](https://dataplatform.cloud.ibm.com/docs/content/DO/DODS_Introduction/buildingmodels.htmltopic_buildingmodels) for building and solving models. The following sections describe how you can deploy your models.
C&amp;R Tree node (SPSS Modeler)
C&R Tree node The Classification and Regression (C&R) Tree node is a tree-based classification and prediction method. Similar to C5.0, this method uses recursive partitioning to split the training records into segments with similar output field values. The C&R Tree node starts by examining the input fields to find the best split, measured by the reduction in an impurity index that results from the split. The split defines two subgroups, each of which is subsequently split into two more subgroups, and so on, until one of the stopping criteria is triggered. All splits are binary (only two subgroups).
Decision Optimization Modeling Assistant scheduling tutorial
Formulating and running a model: house construction scheduling This tutorial shows you how to use the Modeling Assistant to define, formulate and run a model for a house construction scheduling problem. The completed model with data is also provided in the DO-samples, see [Importing Model Builder samples](https://dataplatform.cloud.ibm.com/docs/content/DO/DODS_Introduction/docExamples.htmlExamples__section_modelbuildersamples). In this section: * [Modeling Assistant House construction scheduling tutorial](https://dataplatform.cloud.ibm.com/docs/content/DO/DODS_Mdl_Assist/exhousebuild.html?context=cdpaas&locale=encogusercase__section_The_problem) * [More about the model view](https://dataplatform.cloud.ibm.com/docs/content/DO/DODS_Mdl_Assist/exhousebuild.html?context=cdpaas&locale=encogusercase__section_tbl_kdj_t1b) * [Generating a Python notebook from your scenario](https://dataplatform.cloud.ibm.com/docs/content/DO/DODS_Mdl_Assist/exhousebuild.html?context=cdpaas&locale=encogusercase__section_j2m_xnh_4bb)
Jython code size limits
Jython code size limits Jython compiles each script to Java bytecode, which the Java Virtual Machine (JVM) then runs. However, Java imposes a limit on the size of a single bytecode file. So when Jython attempts to load the bytecode, it can cause the JVM to crash. SPSS Modeler is unable to prevent this from happening. Ensure that you write your Jython scripts using good coding practices (such as minimizing duplicated code by using variables or functions to compute common intermediate values). If necessary, you may need to split your code over several source files or define it using modules as these are compiled into separate bytecode files.