itsjhuang/watsonx-docs-document-type huggingface dataset:
Working with your data (SPSS Modeler)
Working with your data To see a quick sample of a flow's data, right-click a node a select Preview. To more thoroughly examine your data, use a Charts node to launch the chart builder. With the chart builder, you can use advanced visualizations to explore your data from different perspectives and identify patterns, connections, and relationships within your data. You can also visualize your data with these same charts in a Data Refinery flow. Figure 1. Sample visualizations available for a flow ![Shows four example charts available in Visualizations](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/charts_thumbnail4.png) For more information, see [Visualizing your data](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/visualizations.html).
Accessing data in AWS through access points from a notebook
Accessing data in AWS through access points from a notebook In IBM watsonx you can access data stored in AWS S3 buckets through access points from a notebook. Run the notebook in an environment in IBM watsonx. Create an internet-enabled access point to connect to the S3 bucket. Connecting to AWS S3 data through an internet-enabled access point You can access data in an AWS S3 bucket through an internet-enabled access point in any AWS region. To access S3 data through an internet-enabled access point: 1. Create an access point for your S3 bucket. See [Creating access points](https://docs.aws.amazon.com/AmazonS3/latest/dev/creating-access-points.html). Set the network origin to Internet. 2. After the access point is created, make a note of the Amazon resource name (ARN) for the access point. Example: ARN: arn:aws:s3:us-east-1:675068711478:accesspoint/cust-data-bucket-internet-ap. You will need to enter the ARN in your notebook. Accessing AWS S3 data from your notebook The following sample code snippet shows you how to access AWS data from your notebook by using an access point: import boto3 import pandas as pd use an access key and a secret that has access to the bucket access_key="..." secret="..." s3_client = boto3.client('s3', aws_access_key_id=access_key, aws_secret_access_key=secret) the Amazon resource name (ARN) of the access point arn = "..." the file you want to retrieve fileName="customers.csv" response = s3_client.get_object(Bucket=arn, Key=fileName) s3FileStream = response["Body"] for other file types, change the line below to use the appropriate read_() method from pandas customerDF = pd.read_csv(s3FileStream) Parent topic:[Loading and accessing data in a notebook](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html)
Extension Export node (SPSS Modeler)
Extension Export node You can use the Extension Export node to run R scripts or Python for Spark scripts to export data.
Matrix node (SPSS Modeler)
Matrix node Use the Matrix to create a table that shows relationships between fields. It is most commonly used to show the relationship between two categorical fields (flag, nominal, or ordinal), but it can also be used to show relationships between continuous (numeric range) fields.
Getting started with IBM watsonx as a Service
Getting started with IBM watsonx as a Service You can sign up for IBM watsonx.ai or IBM watsonx.governance and explore the tutorials, resources, and tools to immediately get started working with models or governing models. If you are an administrator, follow the steps to set up watsonx for your organization. Start working To start working: 1. If you haven't already, [sign up](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/signup-wx.html) for watsonx.ai or watsonx.governance. 2. Click a task tile on the watsonx home page and start working. For example, click Experiment with foundation models and build prompts to open the Prompt Lab. Then, choose a sample prompt and start experimenting. Your first project, where you save your work, is created automatically. See [Your sandbox project](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/sandbox.html). 3. Explore your resources: * Take a [Quick start tutorial](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/quickstart-tutorials.html) * Click a category in the Samples area of the home page to try out a notebook, a prompt, or a sample project. If you are an existing Cloud Pak for Data as a Service user, you can [switch to watsonx](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/platform-switcher.html). Set up the platform as an administrator To set up the watsonx platform for your organization, see [Setting up the platform as an administrator](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/setup-platform.html). Learn about watsonx To understand watsonx, start with these resources: * [Overview of watsonx](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/overview-wx.html) * [Video library](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/videos-wx.html) * [Projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/manage-projects.html) * [Foundation models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-overview.html) * [Read blogs on Medium and the IBM Community](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/get-help.htmlcommunity) Other information: * [Get help](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/get-help.html) * [Browser support](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/browser-support.html) * [Language support](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/localization.html) * [IBM watsonx APIs](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wdp-apis.html)
Numeric functions (SPSS Modeler)
Numeric functions CLEM contains a number of commonly used numeric functions. CLEM numeric functions Table 1. CLEM numeric functions Function Result Description –NUM Number Used to negate NUM. Returns the corresponding number with the opposite sign. NUM1 + NUM2 Number Returns the sum of NUM1 and NUM2. NUM1 –NUM2 Number Returns the value of NUM2 subtracted from NUM1. NUM1 * NUM2 Number Returns the value of NUM1 multiplied by NUM2. NUM1 / NUM2 Number Returns the value of NUM1 divided by NUM2. INT1 div INT2 Number Used to perform integer division. Returns the value of INT1 divided by INT2. INT1 rem INT2 Number Returns the remainder of INT1 divided by INT2. For example, INT1 – (INT1 div INT2) INT2. BASE POWER Number Returns BASE raised to the power POWER, where either may be any number (except that BASE must not be zero if POWER is zero of any type other than integer 0). If POWER is an integer, the computation is performed by successively multiplying powers of BASE. Thus, if BASE is an integer, the result will be an integer. If POWER is integer 0, the result is always a 1 of the same type as BASE. Otherwise, if POWER is not an integer, the result is computed as exp(POWER * log(BASE)). abs(NUM) Number Returns the absolute value of NUM, which is always a number of the same type. exp(NUM) Real Returns e raised to the power NUM, where e is the base of natural logarithms. fracof(NUM) Real Returns the fractional part of NUM, defined as NUM–intof(NUM). intof(NUM) Integer Truncates its argument to an integer. It returns the integer of the same sign as NUM and with the largest magnitude such that abs(INT) <= abs(NUM). log(NUM) Real Returns the natural (base e) logarithm of NUM, which must not be a zero of any kind. log10(NUM) Real Returns the base 10 logarithm of NUM, which must not be a zero of any kind. This function is defined as log(NUM) / log(10). negate(NUM) Number Used to negate NUM. Returns the corresponding number with the opposite sign. round(NUM) Integer Used to round NUM to an integer by taking intof(NUM+0.5) if NUM is positive or intof(NUM–0.5) if NUM is negative. sign(NUM) Number Used to determine the sign of NUM. This operation returns –1, 0, or 1 if NUM is an integer. If NUM is a real, it returns –1.0, 0.0, or 1.0, depending on whether NUM is negative, zero, or positive. sqrt(NUM) Real Returns the square root of NUM. NUM must be positive. sum_n(LIST) Number Returns the sum of values from a list of numeric fields or null if all of the field values are null. mean_n(LIST) Number Returns the mean value from a list of numeric fields or null if all of the field values are null. sdev_n(LIST) Number Returns the standard deviation from a list of numeric fields or null if all of the field values are null.
Drug treatment - exploratory graphs (SPSS Modeler)
Drug treatment - exploratory graphs In this example, imagine you're a medical researcher compiling data for a study. You've collected data about a set of patients, all of whom suffered from the same illness. During their course of treatment, each patient responded to one of five medications. Part of your job is to use data mining to find out which drug might be appropriate for a future patient with the same illness. This example uses the flow named Drug Treatment - Exploratory Graphs, available in the example project . The data file is drug1n.csv. Figure 1. Drug treatment example flow ![Drug treatment example flow](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_data.png) The data fields used in this example are: Data field Description Age Age of patient (number) Sex M or F BP Blood pressure: HIGH, NORMAL, or LOW Cholesterol Blood cholesterol: NORMAL or HIGH Na Blood sodium concentration K Blood potassium concentration Drug Prescription drug to which a patient responded
hdbscannugget properties
hdbscannugget properties You can use the HDBSCAN node to generate an HDBSCAN model nugget. The scripting name of this model nugget is hdbscannugget. No other properties exist for this model nugget. For more information on scripting the modeling node itself, see [hdbscannode properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/hdbscannodeslots.htmlhdbscannodeslots).
Defining the dates (SPSS Modeler)
Defining the dates Now you need to change the storage type of the DATE_ field to date format. 1. Attach a Filler node to the Filter node, then double-click the Filler node to open its properties 2. Add the DATE_ field, set the Replace option to Always, and set the Replace with value to to_date(DATE_). Figure 1. Setting the date storage type ![Setting the date storage type](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_date1.png)
{{ document.title.text }}
Data usage rights ![icon for intellectual property risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-intellectual-property.svg)Risks associated with inputTraining and tuning phaseIntellectual propertyAmplified Description Terms of service, copyright laws, or other rules restrict the ability to use certain data for building models. Why is data usage rights a concern for foundation models? Laws and regulations concerning the use of data to train AI are unsettled and can vary from country to country, which creates challenges in the development of models. If data usage violates rules or restrictions, business entities might face fines, reputational harms, and other legal consequences. Example Text Copyright Infringement Claims According to the source article, bestselling novelists Sarah Silverman, Richard Kadrey, and Christopher Golden have sued Meta and OpenAI for copyright infringement. The article further stated that the authors had alleged the two tech companies had “ingested” text from their books into generative AI software (LLMs) and failed to give them credit or compensation. Sources: [Los Angeles Times, July 2023](https://www.latimes.com/entertainment-arts/books/story/2023-07-10/sarah-silverman-authors-sue-meta-openai-chatgpt-copyright-infringement) Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
Accessing project assets with ibm-watson-studio-lib
Accessing project assets with ibm-watson-studio-lib The ibm-watson-studio-lib library for Python and R contains a set of functions that help you to interact with IBM Watson Studio projects and project assets. You can think of the library as a programmatical interface to a project. Using the ibm-watson-studio-lib library, you can access project metadata and assets, including files and connections. The library also contains functions that simplify fetching files associated with the project. Next steps * Start using ibm-watson-studio-lib in new notebooks: * [ibm-watson-studio-lib for Python](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ws-lib-python.html) * [ibm-watson-studio-lib for R](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ws-lib-r.html) Parent topic:[Loading and accessing data in a notebook](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/load-and-access-data.html)
Creating synthetic data from a custom data schema
Creating synthetic data from a custom data schema Using the Synthetic Data Generator graphical editor flow tool, you can generate a structured synthetic data set based on meta data, automatically or with user-specified statistical distributions. You can define the data within each table column, their distributions, and any correlations. You can then export and review your synthetic data. Before you can use generate to create synthetic data, you need [to create a task](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/synthetic_data_overview_sd.htmlcreate-synthetic). 1. The Generate synthetic tabular data flow window opens. Select use case Create from custom data schema. Click Next. ![Generate synthetic tabular data flow window](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/images/sd-generate-flow.png) 2. Select Generate options. You can use the Synthetic Data Generator graphical editor flow tool to specify the number of rows and add columns. You can define properties and specify fields, storage types, statistical distributions, and distribution parameters. Click Next. ![Generate options](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/images/sd-generate-options.png) 3. Select Export data to select the export file name and type. For more information, see [Exporting data](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/export_data_sd.html). Click Next. ![Export data](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/images/sd-generate-export.png) 4. Select Review to check your selection and make any updates before generating your synthetic data. Click Save and run. ![Review data](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/images/sd-generate-review.png) Learn more [Creating synthetic data from production data](https://dataplatform.cloud.ibm.com/docs/content/wsj/synthetic/mask_mimic_data_sd.html)
Deleting a deployment
Deleting a deployment Delete your deployment when you no longer need it to free up resources. You can delete a deployment from a deployment space, or programmatically, by using the Python client or Watson Machine Learning APIs. Deleting a deployment from a space To remove a deployment: 1. Open the Deployments page of your deployment space. 2. Choose Delete from the action menu for the deployment name. ![Deleting a deployment](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/deploy-delete.png) Deleting a deployment by using the Python client Use the following method to delete the deployment. client.deployments.delete(deployment_uid) Returns a SUCCESS message. To check that the deployment was removed, you can list deployments and make sure that the deleted deployment is no longer listed. client.deployments.list() Returns: ---- ---- ----- ------- ------------- GUID NAME STATE CREATED ARTIFACT_TYPE ---- ---- ----- ------- ------------- Deleting a deployment by using the REST API Use the DELETE method for deleting a deployment. DELETE /ml/v4/deployments/{deployment_id} For more information, see [Delete](https://cloud.ibm.com/apidocs/machine-learningdeployments-delete). For example, see the following code snippet: curl --location --request DELETE 'https://us-south.ml.cloud.ibm.com/ml/v4/deployments/:deployment_id?space_id=<string>&version=2020-09-01' Parent topic:[Managing predictive deployments](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-deploy-general.html)
Viewing or selecting values (SPSS Modeler)
Viewing or selecting values You can view field values from the Expression Builder. Note that data must be fully instantiated in an Import or Type node to use this feature, so that storage, types, and values are known. To view values for a field from the Expression Builder, select the required field and then use the Value list or perform a search with the Find in column Value field to find values for the selected field. You can then double-click a value to insert it into the current expression or list. For flag and nominal fields, all defined values are listed. For continuous (numeric range) fields, the minimum and maximum values are displayed.
Common node properties
Common node properties A number of properties are common to all nodes in SPSS Modeler. Common node properties Table 1. Common node properties Property name Data type Property description use_custom_name flag name string Read-only property that reads the name (either auto or custom) for a node on the canvas. custom_name string Specifies a custom name for the node. tooltip string annotation string keywords string Structured slot that specifies a list of keywords associated with the object (for example, ["Keyword1" "Keyword2"]). cache_enabled flag node_type source_supernode <br> <br>process_supernode <br> <br>terminal_supernode <br> <br>all node names as specified for scripting Read-only property used to refer to a node by type. For example, instead of referring to a node only by name, such as real_income, you can also specify the type, such as userinputnode or filternode. SuperNode-specific properties are discussed separately, as with all other nodes. See [SuperNode properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/defining_slot_parameters_in_supernodes.htmldefining_slot_parameters_in_supernodes) for more information.
Disabling nodes in a flow (SPSS Modeler)
Disabling nodes in a flow You can disable process nodes that have a single input so that they're ignored when the flow runs. This saves you from having to remove or bypass the node and means you can leave it connected to the remaining nodes. You can still open and edit the node settings; however, any changes will not take effect until you enable the node again. For example, you might use a Filter node to filter several fields, and then build models based on the reduced data set. If you want to also build the same models without fields being filtered, to see if they improve the model results, you can disable the Filter node. When you disable the Filter node, the connections to the modeling nodes pass directly through from the Derive node to the Type node.
Building the flow (SPSS Modeler)
Building the flow 1. Add a Data Asset node that points to pm_customer_train1.csv. 2. Add a Type node, and select response as the target field (Role = Target). Set the measure for this field to Flag. Figure 1. Setting the measurement level and role ![Setting the measurement level and role](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_target.png) 3. Set the role to None for the following fields: customer_id, campaign, response_date, purchase, purchase_date, product_id, Rowid, and X_random. These fields will be ignored when you are building the model. 4. Click Read Values in the Type node to make sure that values are instantiated. As we saw earlier, our source data includes information about four different campaigns, each targeted to a different type of customer account. These campaigns are coded as integers in the data, so to make it easier to remember which account type each integer represents, let's define labels for each one. Figure 2. Choosing to specify values for a field ![Choosing to specify values for a field](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_value.png) 5. On the row for the campaign field, click the entry in the Value mode column. 6. Choose Specify from the drop-down. Figure 3. Defining labels for the field values ![Defining labels for the field values](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_labels.png) 7. Click the Edit icon in the column for the campaign field. Type the labels as shown for each of the four values. 8. Click OK. Now the labels will be displayed in output windows instead of the integers. 9. Attach a Table node to the Type node. 10. Right-click the Table node and select Run. 11. In the Outputs panel, double-click the table output to open it. 12. Click OK to close the output window. Although the data includes information about four different campaigns, you will focus the analysis on one campaign at a time. Since the largest number of records fall under the Premium account campaign (coded campaign=2 in the data), you can use a Select node to include only these records in the flow. Figure 4. Selecting records for a single campaign ![Selecting records for a single campaign](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_autoflag_build_select.png)
Creating the model (SPSS Modeler)
Creating the model 1. Double-click the Time Series node to open its properties. 2. Under FIELDS, add all 5 of the markets to the Candidate Inputs lists. Also add the Total field to the Targets list. 3. Under BUILD OPTIONS - GENERAL, make sure the Expert Modeler method is selected using all default settings. Doing so enables the Expert Modeler to decide the most appropriate model to use for each time series. Figure 1. Choosing the Expert Modeler method for Time Series ![Choosing the Expert Modeler method for Time Series](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_expert.png) 4. Save the settings and then run the flow. A Time Series model nugget is generated. Attach it to the Time Series node. 5. Attach a Table node to the Time Series model nugget and run the flow again. Figure 2. Example flow showing Time Series modeling ![Example flow showing Time Series modeling](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_flow.png) There are now three new rows appended to the end of the original data. These are the rows for the forecast period, in this case January to March 2004. Several new columns are also present now. The $TS- columns are added by the Time Series node. The columns indicate the following for each row (that is, for each interval in the time series data): Column Description $TS-colname The generated model data for each column of the original data. $TSLCI-colname The lower confidence interval value for each column of the generated model data. $TSUCI-colname The upper confidence interval value for each column of the generated model data. $TS-Total The total of the $TS-colname values for this row. $TSLCI-Total The total of the $TSLCI-colname values for this row. $TSUCI-Total The total of the $TSUCI-colname values for this row. The most significant columns for the forecast operation are the $TS-Market_n, $TSLCI-Market_n, and $TSUCI-Market_n columns. In particular, these columns in the last three rows contain the user subscription forecast data and confidence intervals for each of the local markets.