itsjhuang/watsonx-docs-document-type huggingface dataset:
Viewing jobs across projects
Viewing jobs across projects You can view the jobs that exist across projects for assets that run in tools, such as notebooks, Data Refinery flows, and SPSS Modeler flows. To view the status of jobs or job runs in projects: 1. From the navigation menu, select Projects > Jobs. 2. Select a view scope: * Jobs with finished runs: all jobs that contain finished runs * Finished runs: all job runs that have finished * Jobs with active runs: all jobs that contain that contain active runs * Active runs: all job runs that are still active 3. Click ![the Filters icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/images/edit-filters.png) from the table toolbar to further narrow down the returned search results for the view scope you selected. The filter options vary depending the view scope selection, for example, for jobs with active runs, you can filter by run state, job type and project, whereas for finished runs by time, run state, whether the runs were started manually or by a schedule, job type, run duration and project. Parent topic:[Jobs](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/jobs.html)
Apriori node (SPSS Modeler)
Apriori node The Apriori node discovers association rules in your data. Association rules are statements of the form: if antecedent(s) then consequent(s) For example, if a customer purchases a razor and after shave, then that customer will purchase shaving cream with 80% confidence. Apriori extracts a set of rules from the data, pulling out the rules with the highest information content. The Apriori node also discovers association rules in the data. Apriori offers five different methods of selecting rules and uses a sophisticated indexing scheme to efficiently process large data sets. Requirements. To create an Apriori rule set, you need one or more Input fields and one or more Target fields. Input and output fields (those with the role Input, Target, or Both) must be symbolic. Fields with the role None are ignored. Fields types must be fully instantiated before executing the node. Data can be in tabular or transactional format. Strengths. For large problems, Apriori is generally faster to train. It also has no arbitrary limit on the number of rules that can be retained and can handle rules with up to 32 preconditions. Apriori offers five different training methods, allowing more flexibility in matching the data mining method to the problem at hand.
Accessing flow run results
Accessing flow run results Many SPSS Modeler nodes produce output objects such as models, charts, and tabular data. Many of these outputs contain useful values that can be used by scripts to guide subsequent runs. These values are grouped into content containers (referred to as simply containers) which can be accessed using tags or IDs that identify each container. The way these values are accessed depends on the format or "content model" used by that container. For example, many predictive model outputs use a variant of XML called PMML to represent information about the model such as which fields a decision tree uses at each split, or how the neurons in a neural network are connected and with what strengths. Model outputs that use PMML provide an XML Content Model that can be used to access that information. For example: stream = modeler.script.stream() Assume the flow contains a single C5.0 model builder node and that the datasource, predictors, and targets have already been set up modelbuilder = stream.findByType("c50", None) results = [] modelbuilder.run(results) modeloutput = results[0] Now that we have the C5.0 model output object, access the relevant content model cm = modeloutput.getContentModel("PMML") The PMML content model is a generic XML-based content model that uses XPath syntax. Use that to find the names of the data fields. The call returns a list of strings match the XPath values dataFieldNames = cm.getStringValues("/PMML/DataDictionary/DataField", "name") SPSS Modeler supports the following content models in scripting: * Table content model provides access to the simple tabular data represented as rows and columns. * XML content model provides access to content stored in XML format. * JSON content model provides access to content stored in JSON format. * Column statistics content model provides access to summary statistics about a specific field. * Pair-wise column statistics content model provides access to summary statistics between two fields or values between two separate fields. Note that the following nodes don't contain these content models: * Time Series * Discriminant * SLRM * All Extension nodes * All Database Modeling nodes
Exporting space assets from deployment spaces
Exporting space assets from deployment spaces You can export assets from a deployment space so that you can share the space with others or reuse the assets in another space. For a list of assets that you can export from space, refer to [Assets in a deployment space](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-space-add-assets-all.html). Exporting space assets from the UI Important:To avoid problems with importing the space, export all dependencies together with the space. For more information, see [Exporting a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/export-project.html). To export space assets from the UI: 1. From your deployment space, click the import and export space (![Import or Export space icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/import-export-icon.png)) icon. From the list, select Export space. 2. Click New export file. Specify a file name and an optional description. Tip: To encrypt sensitive data in the exported archive, type the password in the Password field. 3. Select the assets that you want to export with the space. 4. Click Create to create the export file. 5. After the space is exported, click the download (![Download icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/download-icon.png)) to save the file. You can reuse this space by choosing Create a space from a file when you create a new space. Learn more * [Importing spaces and projects into existing deployment spaces](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-space-import-to-space.html). Parent topic:[Deployment spaces](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-spaces_local.html)
Extension Model node (SPSS Modeler)
Extension Model node With the Extension Model node, you can run R scripts or Python for Spark scripts to build and score models. After adding the node to your canvas, double-click the node to open its properties.
Referencing existing nodes
Referencing existing nodes A flow is often pre-built with some parameters that must be modified before the flow runs. Modifying these parameters involves the following tasks: 1. Locating the nodes in the relevant flow. 2. Changing the node or flow settings (or both).
{{ document.title.text }}
Unreliable source attribution ![icon for explainability risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-explainability.svg)Risks associated with outputExplainabilityAmplified Description Source attribution is the AI system's ability to describe from what training data it generated a portion or all its output. Since current techniques are based on approximations, these attributions might be incorrect. Why is unreliable source attribution a concern for foundation models? Low quality explanations make it difficult for users, model validators, and auditors to understand and trust the model. Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
Transpose node (SPSS Modeler)
Transpose node By default, columns are fields and rows are records or observations. If necessary, you can use a Transpose node to swap the data in rows and columns so that fields become records and records become fields. For example, if you have time series data where each series is a row rather than a column, you can transpose the data prior to analysis.
CPLEX Optimization node (SPSS Modeler)
CPLEX Optimization node With the CPLEX Optimization node, you can use complex mathematical (CPLEX) based optimization via an Optimization Programming Language (OPL) model file. For more information about CPLEX optimization and OPL, see the [IBM ILOG CPLEX Optimization Studio documentation](https://www.ibm.com/support/knowledgecenter/SSSA5P). When outputting the data generated by the CPLEX Optimization node, you can output the original data from the data sources together as single indexes, or as multiple dimensional indexes of the result. Note: * When running a flow containing a CPLEX Optimization node, the CPLEX library has a limitation of 1000 variables and 1000 constraints.
Credentials for prompting foundation models (IBM Cloud API key and IAM token)
Credentials for prompting foundation models (IBM Cloud API key and IAM token) To prompt foundation models in IBM watsonx.ai programmatically, you need an IBM Cloud API key and sometimes an IBM Cloud IAM token. IBM Cloud API key To use the [foundation models Python library](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-python-lib.html), you need an IBM Cloud API key. Python pseudo-code my_credentials = { "url" : "https://us-south.ml.cloud.ibm.com", "apikey" : <my-IBM-Cloud-API-key> } ... model = Model( ... credentials=my_credentials ... ) You can create this API key by using multiple interfaces. For full instructions, see [Creating an API key](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=uicreate_user_key) IBM Cloud IAM token When you click the View code button in the Prompt Lab, a curl command is displayed that you can call outside the Prompt Lab to submit the current prompt and parameters to the selected model and get a generated response. In the command, there is a placeholder for an IBM Cloud IAM token. For information about generating that access token, see: [Generating an IBM Cloud IAM token](https://cloud.ibm.com/docs/account?topic=account-iamtoken_from_apikey) Parent topic:[Foundation models Python library](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-python-lib.html)
Creating a distribution chart (SPSS Modeler)
Creating a distribution chart During data mining, it is often useful to explore the data by creating visual summaries. Watson Studio offers many different types of charts to choose from, depending on the kind of data you want to summarize. For example, to find out what proportion of the patients responded to each drug, use a Distribution node. Figure 1. Distribution node ![Distribution node](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_drug_distribution.png) 1. Under Graphs on the Palette, add a Distribution node to the flow and connect it to the drug1n.csv Data Asset node. Then double-click the node to edit its options. 2. Select Drug as the target field whose distribution you want to show. Then click Save, right-click the Distribution node, and select Run. A distribution chart is added to the Outputs panel. The chart helps you see the shape of the data. It shows that patients responded to drug Y most often and to drugs B and C least often. Alternatively, you can attach and run a Data Audit node for a quick glance at distributions and histograms for all fields at once. The Data Audit node is available under Outputs on the Palette.
Video library
Video library Watch short videos for data scientists, data engineers, and data stewards to learn about watsonx. The videos and accompanying tutorials are task-focused and provide hands-on experience by using the tools in watsonx. Note: These videos provides a visual method to learn the concepts and tasks in this documentation. If you are having difficulty viewing any of the videos on this page, visit the [Video playlists](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/videos-wx-docs.html) page. First watch the IBM watsonx.ai overview video. ![Watch Video](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/images/lc-video.png) Select any video from the lists below to watch here. Quick start IBM watsonx.ai overview * Classify text * Summarize large, complex documents * Generate content * Extract text from complex documents Get started * Create a project * Collaborate in projects * Tour the samples collection * Load and analyze public data sets Work with data * Prepare data with Data Refinery * Generate synthetic tabular data * Analyze data in a Jupyter notebook IBM watsonx.governance * Track a model in an AI use case * Evaluate a prompt template * Track a prompt template Work with foundation models * Prompt a foundation model using Prompt Lab * Prompt tips: Get started prompting foundation models * Introduction to the retrieval-augmented generation pattern * Tune a foundation model Build models * Build and deploy a model with AutoAI * Build and deploy a model in a Jupyter notebook * Build and deploy a model with SPSS Modeler * Build and deploy a Decision Optimization model * Create a pipeline to automate the lifecycle for a model
{{ document.title.text }}
Data provenance ![icon for transparency risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-transparency.svg)Risks associated with inputTraining and tuning phaseTransparencyAmplified Description Without standardized and established methods for verifying where data came from, there are no guarantees that available data is what it claims to be. Why is data provenance a concern for foundation models? Not all data sources are trustworthy. Data might have been unethically collected, manipulated, or falsified. Using such data can result in undesirable behaviors in the model. Business entities could face fines, reputational harms, and other legal consequences. Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
Managing and customizing report templates
Managing and customizing report templates If the default report templates that are provided with AI Factsheets do not meet your needs, you can download a default report template, customize it, and upload the new template. Customizing a custom report template Any user with at least Editor access can create a report from an AI use case that captures all the details from An AI use case. You can use reports for compliance verification, archiving, or other purposes. If the default templates for the reports do not meet the needs of your organization, you can customize the report templates, the branding file, or the default stylesheet. For example, you can replace the IBM logo with your own logo image file. You must have the Admin role for managing inventories to customize report templates. Follow these steps to customize a report template. Downloading a report To download a report template from the UI: 1. Open the AI uses cases settings and click the Report templates tab. If you do not see this tab, you might have insufficient access. 2. In the options menu for a report template, click Download. ![Downloading a report template](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/xgov-report-template.png) 3. Open the <report-name>.ftl file in an editor. 4. Edit the template by using instructions from [Apache FreeMarker](https://freemarker.apache.org/) or the API commands. To download a report template by using APIs: 1. Use the GET endpoint for /v1/aigov/report_templates in the [IBM Watson Data API](https://cloud.ibm.com/apidocs/watson-data-api) to list the available templates. Note the ID for the template that you want to download. 2. Use the GET endpoint /v1/aigov/report_templates/{template_id}/content with the template ID to download the template file. 3. Open the <report-name>.ftl file in an editor. 4. Edit the template by using instructions from [Apache FreeMarker](https://freemarker.apache.org/) or the API commands. Uploading a template 1. Open the AI uses cases settings and click the Report templates tab. If you do not see this tab, you might have insufficient access. 2. Click Add template. 3. Specify a name for the template and an optional description. 4. Choose the type of template: model or model use case. The reports are available for external models and Watson Machine Learning models. 5. Upload the updated FTL file. Restriction:The ftl file that you upload must not import any other files. Support is not yet available for import statements other than system templates in the ftl file. The custom template displays in the Report templates section and is available for creating reports. Click Edit or Delete from the action menu for a custom template to update the template details or to remove the template. Parent topic:[Creating and managing inventories](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-inventory-manage.html)
Troubleshooting connections
Troubleshooting connections Use these solutions to resolve problems that you might encounter with connections. IBM Db2 for z/OS: Error retrieving the schema list when you try to connect to a Db2 for z/OS server When you test the connection to a Db2 for z/OS server and the connection cannot retrieve the schema list, you might receive the following error: CDICC7002E: The assets request failed: CDICO2064E: The metadata for the column TABLE_SCHEM could not be obtained: Sql error: [jcc] [10300] Invalid parameter: Unknown column name TABLE_SCHEM. ERRORCODE=-4460, SQLSTATE=null Workaround: On the Db2 for z/OS server, set the DESCSTAT subsystem parameter to No. For more information, see [DESCRIBE FOR STATIC field (DESCSTAT subsystem parameter)](https://www.ibm.com/docs/SSEPEK_13.0.0/inst/src/tpc/db2z_ipf_descstat.html). Parent topic:[Supported connections](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/conn_types.html)
Creating jobs for Pipelines
Creating jobs for Pipelines You can create jobs for Pipelines. To create a Pipelines job: 1. Open your Pipelines asset from the project. 2. Click Run pipeline > Create a job. 3. On the Create a job page, you can choose the asset version that you'd like to run. The most recently saved version of the Pipelines is used by default. 4. Give a name and optional description for your job. Click next. 5. Define your IAM API key. The most recently used API key is used by default. If you'd like to use a new API key, click Generate new API key. Click next. 6. You can schedule your job by toggling Schedule off to Schedule to run. You can choose either or both options: * Start on: Choose a date for your scheduled job to run. The time zone is GMT-0400 (Eastern Daylight Time). If you do not choose a start date, the job will never run automatically and must be started manually. * Repeat: You can choose to schedule the repeated frequency (every minute to every month), exclude running the job on certain days, and choose an end date. If you do not choose to repeat the job, it runs one time if a start date is given, or does not run. 7. Review your job settings and click Create. The Pipelines job is listed under Jobs in your project. Learn more * [Viewing jobs across projects](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/job-views-projs.html) Parent topic:[Creating and managing jobs](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/jobs.html)
plotnode properties
plotnode properties ![Plot node icon](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/images/plotnodeicon.png)The Plot node shows the relationship between numeric fields. You can create a plot by using points (a scatterplot) or lines. plotnode properties Table 1. plotnode properties plotnode properties Data type Property description x_field field Specifies a custom label for the x axis. Available only for labels. y_field field Specifies a custom label for the y axis. Available only for labels. three_D flag Specifies a custom label for the y axis. Available only for labels in 3-D graphs. z_field field color_field field Overlay field. size_field field shape_field field panel_field field Specifies a nominal or flag field for use in making a separate chart for each category. Charts are paneled together in one output window. animation_field field Specifies a nominal or flag field for illustrating data value categories by creating a series of charts displayed in sequence using animation. transp_field field Specifies a field for illustrating data value categories by using a different level of transparency for each category. Not available for line plots. overlay_type NoneSmootherFunction Specifies whether an overlay function or LOESS smoother is displayed. overlay_expression string Specifies the expression used when overlay_type is set to Function. style PointLine point_type Rectangle Dot Triangle Hexagon Plus Pentagon Star BowTie HorizontalDash VerticalDash IronCross Factory House Cathedral OnionDome ConcaveTriangle OblateGlobe CatEye FourSidedPillow RoundRectangle Fan x_mode SortOverlayAsRead x_range_mode AutomaticUserDefined x_range_min number x_range_max number y_range_mode AutomaticUserDefined y_range_min number y_range_max number z_range_mode AutomaticUserDefined z_range_min number z_range_max number jitter flag records_limit number if_over_limit PlotBinsPlotSamplePlotAll x_label_auto flag x_label string y_label_auto flag y_label string z_label_auto flag z_label string use_grid flag graph_background color Standard graph colors are described at the beginning of this section. page_background color Standard graph colors are described at the beginning of this section. use_overlay_expr flag Deprecated in favor of overlay_type.
SPSS predictive analytics algorithms
SPSS predictive analytics algorithms You can use the following SPSS predictive analytics algorithms in your notebooks. Code samples are provided for Python notebooks. Notebooks must run in a Spark with Python environment runtime. To run the algorithms described in this section, you don't need the SPSS Modeler service. * [Data preparation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/datapreparation-guides.html) * [Classification and regression](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/classificationandregression-guides.html) * [Clustering](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/clustering-guides.html) * [Forecasting](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/forecasting-guides.html) * [Survival analysis](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/survivalanalysis-guides.html) * [Score](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/score-guides.html) Parent topic:[Notebooks and scripts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-and-scripts.html)