itsjhuang/watsonx-docs-document-type huggingface dataset:
KNN node (SPSS Modeler)
KNN node Nearest Neighbor Analysis is a method for classifying cases based on their similarity to other cases. In machine learning, it was developed as a way to recognize patterns of data without requiring an exact match to any stored patterns, or cases. Similar cases are near each other and dissimilar cases are distant from each other. Thus, the distance between two cases is a measure of their dissimilarity. Cases that are near each other are said to be "neighbors." When a new case (holdout) is presented, its distance from each of the cases in the model is computed. The classifications of the most similar cases – the nearest neighbors – are tallied and the new case is placed into the category that contains the greatest number of nearest neighbors. You can specify the number of nearest neighbors to examine; this value is called k. The pictures show how a new case would be classified using two different values of k. When k = 5, the new case is placed in category 1 because a majority of the nearest neighbors belong to category 1. However, when k = 9, the new case is placed in category 0 because a majority of the nearest neighbors belong to category 0. Nearest neighbor analysis can also be used to compute values for a continuous target. In this situation, the average or median target value of the nearest neighbors is used to obtain the predicted value for the new case.
Word cloud charts
Word cloud charts Word cloud charts present data as words, where the size and placement of any individual word is determined by how it is weighted.
Adding a connected folder asset to a project
Adding a connected folder asset to a project You can create a connected folder asset based on a path within an IBM Cloud Object Storage system that is accessed through a connection. You can view the files and subfolders that share the path with the connected folder asset. The files that you can view within the connected folder asset are not themselves data assets. For example, you can create a connected folder asset for a path that contains news feeds that are continuously updated. Required permissions : You must have the Admin or Editor role in the project to add a connected folder asset. Watch this video to see how to add a connected folder asset in a project, then follow the steps below the video. Video disclaimer: Some minor steps and graphical elements in this video might differ from your platform. This video provides a visual method to learn the concepts and tasks in this documentation. To add a connected folder asset from a connection to a project: 1. If necessary, [create a connection asset](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/create-conn.html). Include an Access Key and a Secret Key to your IBM Cloud Object Storage connection to enable the downloading of files within the connected folder asset. If you're using an existing IBM Cloud Object Storage connection asset that doesn't have an Access Key and Secret Key, edit the connection asset and add them. 2. Click Import assets > Connected data. 3. Select an existing connection asset as the source of the data. 4. Select the folder you want and click Import. 5. Type a name and description. 6. Click Create. The connected folder asset appears on the project Assets page in the Data assets category. Click the connected folder asset name to view the contents of the connected folder asset. Click the eye (![eye icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/console/images/visibility-on.svg)) icon next to a file name to view the contents of the files within the folder that have these formats: * CSV * JSON * Parquet You can refine the files within a connected folder asset and then save the result as a data asset. While viewing the connected folder asset, select a file and then click Prepare data. You can view the files within the connected folder asset if the IBM Cloud Object Storage connection asset that's associated with the connected folder asset has an Access Key and a Secret Key (also known as HMAC credentials). For more information about HMAC credentials, see [IBM Cloud Object Storage Service credentials](https://console.bluemix.net/docs/services/cloud-object-storage/iam/service-credentials.htmlservice-credentials). Next steps * [Refining a file within the folder](https://dataplatform.cloud.ibm.com/docs/content/wsj/refinery/refining_data.html) Parent topic:[Adding data to a project](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/add-data-project.html)
Building the flow (SPSS Modeler)
Building the flow Figure 1. Example flow to classify customers using multinomial logistic regression ![Example flow to classify customers using multinomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify.png) 1. Add a Data Asset node that points to telco.csv. 2. Add a Type node, double-click it to open its properties, and click Read Values. Make sure all measurement levels are set correctly. For example, most fields with values of 0.0 and 1.0 can be regarded as flags. Figure 2. Measurement levels ![Measurement levels](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_measurement.png) Notice that gender is more correctly considered as a field with a set of two values, instead of a flag, so leave its measurement value as Nominal. 3. Set the role for the custcat field to Target. Leave the role for all other fields set to Input. 4. Since this example focuses on demographics, use a Filter node to include only the relevant fields: region, age, marital, address, income, ed, employ, retire, gender, reside, and custcat). Other fields will be excluded for the purpose of this analysis. To filter them out, in the Filter node properties, click Add Columns and select the fields to exclude. Figure 3. Filtering on demographic fields ![Filtering on demographic fields](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_filter.png) (Alternatively, you could change the role to None for these fields rather than excluding them, or select the fields you want to use in the modeling node.) 5. In the Logistic node properties, under MODEL SETTINGS, select the Stepwise method. Also select Multinomial, Main Effects, and Include constant in equation. Figure 4. Example flow to classify customers using multinomial logistic regression ![Example flow to classify customers using multinomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_logistic.png) 6. Under EXPERT OPTIONS, select Expert mode, expand the Output section, and select Classification table. Figure 5. Example flow to classify customers using multinomial logistic regression ![Example flow to classify customers using multinomial logistic regression](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_classify_output.png)
{{ document.title.text }}
Copyright infringement ![icon for intellectual property risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-intellectual-property.svg)Risks associated with outputIntellectual propertyNew Description Generative AI output that is too similar or identical to existing work risks claims of copyright infringement. Uncertainty and variability around the ownership, copyrightability, and patentability of output generated by AI increases the risk of copyright infringement problems. Why is copyright infringement a concern for foundation models? Laws and regulations concerning the use of content that looks the same or closely similar to other copyrighted data are largely unsettled and can vary from country to country, providing challenges in determining and implementing compliance. Business entities could face fines, reputational harms, and other legal consequences. Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
Specifying values for geospatial data (SPSS Modeler)
Specifying values for geospatial data Geospatial fields display geospatial data that's in a list. For the Geospatial measurement level, you can use various options to set the measurement level of the elements within the list. Type. Select the measurement sublevel of the geospatial field. The available sublevels are determined by the depth of the list field. The defaults are: Point (zero depth), LineString (depth of one), and Polygon (depth of one). For more information about sublevels, see [Geospatial measurement sublevels](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/type_levels_geo.html). Coordinate system. This option is only available if you changed the measurement level to Geospatial from a non-geospatial level. To apply a coordinate system to your geospatial data, select this option. To use a different coordinate system, click Change.
Decision Optimization input and output data
Input and output data You can access the input and output data you defined in the experiment UI by using the following dictionaries. The data that you imported in the Prepare data view in the experiment UI is accessible from the input dictionary. You must define each table by using the syntax inputs['tablename']. For example, here food is an entity that is defined from the table called diet_food: food = inputs['diet_food'] Similarly, to show tables in the Explore solution view of the experiment UI you must specify them using the syntax outputs['tablename']. For example, outputs['solution'] = solution_df defines an output table that is called solution. The entity solution_df in the Python model defines this table. You can find this Diet example in the Model_Builder folder of the [DO-samples](https://github.com/IBMDecisionOptimization/DO-Samples). To import and run (solve) it in the experiment UI, see [Solving and analyzing a model: the diet problem](https://dataplatform.cloud.ibm.com/docs/content/DO/DODS_Notebooks/solveModel.htmltask_mtg_n3q_m1b).
Adding comments and annotations to SPSS Modeler flows
You may need to describe a flow to others in your organization. To help you do this, you can attach explanatory comments to nodes, and model nuggets. Others can then view these comments on-screen, or you might even print out an image of the flow that includes the comments. You can also add notes in the form of text annotations to nodes and model nuggets by means of the Annotations tab in a node's properties. These annotations are visible only when the Annotations tab is open.
Batch deployment input details for Scikit-learn and XGBoost models
Batch deployment input details for Scikit-learn and XGBoost models Follow these rules when you are specifying input details for batch deployments of Scikit-learn and XGBoost models. Data type summary table: Data Description Type inline, data references File formats CSV, .zip archive that contains CSV files Data source If you are specifying input/output data references programmatically: * Data source reference type depends on the asset type. Refer to the Data source reference types section in [Adding data assets to a deployment space](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-space-add-assets.html). Notes: * The environment variables parameter of deployment jobs is not applicable. * For connections of type [Cloud Object Storage](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/conn-cos.html) or [Cloud Object Storage (infrastructure)](https://dataplatform.cloud.ibm.com/docs/content/wsj/manage-data/conn-cos-infra.html), you must configure Access key and Secret key, also known as [HMAC credentials](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-uhc-hmac-credentials-main), Parent topic:[Batch deployment input details by framework](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/deploy-batch-input-by-framework.html)
{{ document.title.text }}
Jailbreaking ![icon for multi-category risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-multi-category.svg)Risks associated with inputInferenceMulti-categoryAmplified Description An attack that attempts to break through the guardrails established in the model is known as jailbreaking. Why is jailbreaking a concern for foundation models? Jailbreaking attacks can be used to alter model behavior and benefit the attacker. If not properly controlled, business entities can face fines, reputational harm, and other legal consequences. Example Bypassing LLM guardrails Cited in a [study](https://arxiv.org/abs/2307.15043) from researchers at Carnegie Mellon University, The Center for AI Safety, and the Bosch Center for AI, claims to have discovered a simple prompt addendum that allowed the researchers to trick models into answering dangerous or sensitive questions and is simple enough to be automated and used for a wide range of commercial and open-source products, including ChatGPT, Google Bard, Meta’s LLaMA, Vicuna, Claude, and others. According to the paper, the researchers were able to use the additions to reliably coax forbidden answers for Vicuna (99%), ChatGPT 3.5 and 4.0 (up to 84%), and PaLM-2 (66%). Sources: [SC Magazine, July 2023](https://www.scmagazine.com/news/researchers-find-universal-jailbreak-prompts-for-multiple-ai-chat-models) [The New York Times, July 2023](https://www.nytimes.com/2023/07/27/business/ai-chatgpt-safety-research.html) Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
{{ document.title.text }}
Decision bias ![icon for fairness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-fairness.svg)Risks associated with outputFairnessNew Description Decision bias occurs when one group is unfairly advantaged over another due to decisions of the model. This bias can result from bias in the training data or as an unintended consequence of how the model was trained. Why is decision bias a concern for foundation models? Bias can harm persons affected by the decisions of the model. Business entities could face fines, reputational harms, and other legal consequences. Example Unfair health risk assignment for black patients A study on racial bias in health algorithms estimated that racial bias reduces the number of black patients identified for extra care by more than half. The study found that bias occurred because the algorithm used health costs as a proxy for health needs. Less money is spent on black patients who have the same level of need, and the algorithm thus falsely concludes that black patients are healthier than equally sick white patients. Sources: [Science, October 2019](https://www.science.org/doi/10.1126/science.aax2342) [American Civil Liberties Union, 2022](https://www.aclu.org/news/privacy-technology/algorithms-in-health-care-may-worsen-medical-racism::text=In%202019%2C%20a%20bombshell%20study,recommended%20for%20the%20same%20care) Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
Multiple series charts
Multiple series charts Multiple series charts are similar to line charts, with the exception that you can chart multiple variables on the Y-axis.
webnode properties
webnode properties ![Time Plot node icon](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/images/webnodeicon.png)The Web node illustrates the strength of the relationship between values of two or more symbolic (categorical) fields. The graph uses lines of various widths to indicate connection strength. You might use a Web node, for example, to explore the relationship between the purchase of a set of items at an e-commerce site. webnode properties Table 1. webnode properties webnode properties Data type Property description use_directed_web flag fields list to_field field from_fields list true_flags_only flag line_values AbsoluteOverallPctPctLargerPctSmaller strong_links_heavier flag num_links ShowMaximumShowLinksAboveShowAll max_num_links number links_above number discard_links_min flag links_min_records number discard_links_max flag links_max_records number weak_below number strong_above number link_size_continuous flag web_display CircularNetworkDirectedGrid graph_background color Standard graph colors are described at the beginning of this section. symbol_size number Specifies a symbol size. directed_line_values AbsoluteOverallPctPctToPctFrom Specify a threshold type. show_legend boolean You can specify whether the legend is displayed. For plots with a large number of fields, hiding the legend may improve the appearance of the plot. labels_as_nodes boolean You can include the label text within each node rather than displaying adjacent labels. For plots with a small number of fields, this may result in a more readable chart.
Managing the lifecycle of notebooks and scripts
Managing the lifecycle of notebooks and scripts After you have created and tested your notebooks, you can add them to pipelines, publish them to a catalog so that other catalog members can use the notebook in their projects, or share read-only copies outside of Watson Studio so that people who aren't collaborators in your Watson Studio projects can see and use them. R scripts and Shiny apps can't be published or shared using functionality in a project at this time. You can use any of these methods for notebooks: * [Add notebooks to a pipeline](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-orchestration-config.html) * [Share a URL on social media](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/share-notebooks.html) * [Publish on GitHub](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/github-integration.html) * [Publish as a gist](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/create-gist.html) * [Publish your notebook to a catalog](https://dataplatform.cloud.ibm.com/docs/content/wsj/catalog/publish-asset-project.html) Make sure that before you share or publish a notebook, you hide any sensitive code, like credentials, that you don't want others to see! See [Hide sensitive cells in a notebook](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/hide_code.html). Parent topic:[Notebooks and scripts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-and-scripts.html)
Filter node (SPSS Modeler)
Filter node You can rename or exclude fields at any point in a flow. For example, as a medical researcher, you may not be concerned about the potassium level (field-level data) of patients (record-level data); therefore, you can filter out the K (potassium) field. This can be done using a separate Filter node or using the Filter tab on an import or output node. The functionality is the same regardless of which node it's accessed from. * From import nodes, you can rename or filter fields as the data is read in. * Using a Filter node, you can rename or filter fields at any point in the flow. * You can use the Filter tab in various nodes to define or edit multiple response sets. * Finally, you can use a Filter node to map fields from one import node to another.
The Categories tab (SPSS Modeler)
The Categories tab In the Text Analytics Workbench, you can use the Categories tab to create and explore categories as well as tweak the extraction results. Extraction results can be refined by modifying the linguistic resources, which you can do directly from the Categories tab. Figure 1. Categories tab ![Categories tab](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tmwb_categoryview.png)
Viewing and setting information about types (SPSS Modeler)
Viewing and setting information about types From the Type node, you can specify field metadata and properties that are invaluable to modeling and other work. These properties include: * Specifying a usage type, such as range, set, ordered set, or flag, for each field in your data * Setting options for handling missing values and system nulls * Setting the role of a field for modeling purposes * Specifying values for a field and options used to automatically read values from your data * Specifying value labels
Creating environment templates
Creating environment templates You can create custom environment templates if you do not want to use the default environments provided by Watson Studio. Required permissions : To create an environment template, you must have the Admin or Editor role within the project. You can create environment templates for the following types of assets: * Notebooks in the Notebook editor * Notebooks in RStudio * Modeler flows in the SPSS Modeler * Data Refinery flows * Jobs that run operational assets, such as Data Refinery flows, or Notebooks in a project Note: To create an environment template: 1. On the Manage tab of your project, select the Environments page and click New template under Templates. 2. Enter a name and a description. 3. Select one of the following engine types: * Default: Select for Python, R, and RStudio runtimes for Watson Studio. * Spark: Select for Spark with Python or R runtimes for Watson Studio. * GPU: Select for more computing power to improve model training performance for Watson Studio. 4. Select the hardware configuration from the Hardware configuration drop-down menu. 5. Select the software version if you selected a runtime of "Default," "Spark," or "GPU." Where to find your custom environment template Your new environment template is listed under Templates on the Environments page in the Manage tab of your project. From this page, you can: * Check which runtimes are active * Update custom environment templates * Track the number of capacity units per hour that your runtimes have consumed so far * Stop active runtimes. Limitations The default environments provided by Watson Studio cannot be edited or modified. Notebook environments (Anaconda Python or R distributions): : - You can't add a software customization to the default Python and R environment templates included in Watson Studio. You can only add a customization to an environment template that you create. : - If you add a software customization using conda, your environment must have at least 2 GB RAM. : - You can't customize an R environment for a notebook by installing R packages directly from CRAN or GitHub. You can check if the CRAN package you want is available only from conda channels and, if the package is available, add that package name in the customization list as r-<package-name>. * After you have started a notebook in an Watson Studio environment, you can't create another conda environment from inside that notebook and use it. Watson Studio environments do not behave like a Conda environment manager. Spark environments: : - You can't customize the software configuration of a Spark environment template. Next steps * [Customize environment templates for Python or R](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/customize-envs.html) Learn more Parent topic:[Managing compute resources](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/manage-envs-new.html)