itsjhuang/watsonx-docs-document-type huggingface dataset:
Spatial functions (SPSS Modeler)
Spatial functions Spatial functions can be used with geospatial data. For example, they allow you to calculate the distances between two points, the area of a polygon, and so on. There can also be situations that require a merge of multiple geospatial data sets that are based on a spatial predicate (within, close to, and so on), which can be done through a merge condition. Notes: * These spatial functions don't apply to three-dimensional data. If you import three-dimensional data into a flow, only the first two dimensions are used by these functions. The z-axis values are ignored. * Geospatial functions aren't supported. CLEM spatial functions Table 1. CLEM spatial functions Function Result Description close_to(SHAPE,SHAPE,NUM) Boolean Tests whether 2 shapes are within a certain DISTANCE of each other. If a projected coordinate system is used, DISTANCE is in meters. If no coordinate system is used, it is an arbitrary unit. crosses(SHAPE,SHAPE) Boolean Tests whether 2 shapes cross each other. This function is suitable for 2 linestring shapes, or 1 linestring and 1 polygon. overlap(SHAPE,SHAPE) Boolean Tests whether there is an intersection between 2 polygons and that the intersection is interior to both shapes. within(SHAPE,SHAPE) Boolean Tests whether the entirety of SHAPE1 is contained within a POLYGON. area(SHAPE) Real Returns the area of the specified POLYGON. If a projected system is used, the function returns meters squared. If no coordinate system is used, it is an arbitrary unit. The shape must be a POLYGON or a MULTIPOLYGON. num_points(SHAPE,LIST) Integer Returns the number of points from a point field (MULTIPOINT) which are contained within the bounds of a POLYGON. SHAPE1 must be a POLYGON or a MULTIPOLYGON. distance(SHAPE,SHAPE) Real Returns the distance between SHAPE1 and SHAPE2. If a projected coordinate system is used, the function returns meters. If no coordinate system is used, it is an arbitrary unit. SHAPE1 and SHAPE2 can be any geo measurement type.
Adding attributes to a class instance
Adding attributes to a class instance Unlike in Java, in Python clients can add attributes to an instance of a class. Only the one instance is changed. For example, to add attributes to an instance x, set new values on that instance: x.attr1 = 1 x.attr2 = 2 . . x.attrN = n
{{ document.title.text }}
Output bias ![icon for fairness risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-fairness.svg)Risks associated with outputFairnessNew Description Generated model content might unfairly represent certain groups or individuals. For example, a large language model might unfairly stigmatize or stereotype specific persons or groups. Why is output bias a concern for foundation models? Bias can harm users of the AI models and magnify existing exclusive behaviors. Business entities can face reputational harms and other consequences. Example Biased Generated Images Lensa AI is a mobile app with generative features trained on Stable Diffusion that can generate “Magic Avatars” based on images users upload of themselves. According to the source report, some users discovered that generated avatars are sexualized and racialized. Sources: [Business Insider, January 2023](https://www.businessinsider.com/lensa-ai-raises-serious-concerns-sexualization-art-theft-data-2023-1) Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
SVM node (SPSS Modeler)
SVM node The SVM node uses a support vector machine to classify data. SVM is particularly suited for use with wide datasets, that is, those with a large number of predictor fields. You can use the default settings on the node to produce a basic model relatively quickly, or you can use the Expert settings to experiment with different types of SVM models. After the model is built, you can: * Browse the model nugget to display the relative importance of the input fields in building the model. * Append a Table node to the model nugget to view the model output. Example. A medical researcher has obtained a dataset containing characteristics of a number of human cell samples extracted from patients who were believed to be at risk of developing cancer. Analysis of the original data showed that many of the characteristics differed significantly between benign and malignant samples. The researcher wants to develop an SVM model that can use the values of similar cell characteristics in samples from other patients to give an early indication of whether their samples might be benign or malignant.
Derive node (SPSS Modeler)
Derive node One of the most powerful features in watsonx.ai is the ability to modify data values and derive new fields from existing data. During lengthy data mining projects, it is common to perform several derivations, such as extracting a customer ID from a string of Web log data or creating a customer lifetime value based on transaction and demographic data. All of these transformations can be performed, using a variety of field operations nodes. Several nodes provide the ability to derive new fields: * The Derive node modifies data values or creates new fields from one or more existing fields. It creates fields of type formula, flag, nominal, state, count, and conditional. * The Reclassify node transforms one set of categorical values to another. Reclassification is useful for collapsing categories or regrouping data for analysis. * The Binning node automatically creates new nominal (set) fields based on the values of one or more existing continuous (numeric range) fields. For example, you can transform a continuous income field into a new categorical field containing groups of income as deviations from the mean. After you create bins for the new field, you can generate a Derive node based on the cut points. * The Set to Flag node derives multiple flag fields based on the categorical values defined for one or more nominal fields. * The Restructure node converts a nominal or flag field into a group of fields that can be populated with the values of yet another field. For example, given a field named payment type, with values of credit, cash, and debit, three new fields would be created (credit, cash, debit), each of which might contain the value of the actual payment made. Tip: The Control Language for Expression Manipulation (CLEM) is a powerful tool you can use to analyze and manipulate the data used in your flows. For example, you might use CLEM in a node to derive values. For more information, see the [CLEM (legacy) language reference](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/clem_reference/clem_language_reference.html).
Trigonometric functions (SPSS Modeler)
Trigonometric functions All of the functions in this section either take an angle as an argument or return one as a result. CLEM trigonometric functions Table 1. CLEM trigonometric functions Function Result Description arccos(NUM) Real Computes the arccosine of the specified angle. arccosh(NUM) Real Computes the hyperbolic arccosine of the specified angle. arcsin(NUM) Real Computes the arcsine of the specified angle. arcsinh(NUM) Real Computes the hyperbolic arcsine of the specified angle. arctan(NUM) Real Computes the arctangent of the specified angle. arctan2(NUM_Y, NUM_X) Real Computes the arctangent of NUM_Y / NUM_X and uses the signs of the two numbers to derive quadrant information. The result is a real in the range - pi < ANGLE <= pi (radians) – 180 < ANGLE <= 180 (degrees) arctanh(NUM) Real Computes the hyperbolic arctangent of the specified angle. cos(NUM) Real Computes the cosine of the specified angle. cosh(NUM) Real Computes the hyperbolic cosine of the specified angle. pi Real This constant is the best real approximation to pi. sin(NUM) Real Computes the sine of the specified angle. sinh(NUM) Real Computes the hyperbolic sine of the specified angle. tan(NUM) Real Computes the tangent of the specified angle. tanh(NUM) Real Computes the hyperbolic tangent of the specified angle.
kmeansnode properties
kmeansnode properties ![K-Means node icon](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/images/kmeansnodeicon.png)The K-Means node clusters the data set into distinct groups (or clusters). The method defines a fixed number of clusters, iteratively assigns records to clusters, and adjusts the cluster centers until further refinement can no longer improve the model. Instead of trying to predict an outcome, k-means uses a process known as unsupervised learning to uncover patterns in the set of input fields. kmeansnode properties Table 1. kmeansnode properties kmeansnode Properties Values Property description inputs [field1 ... fieldN] K-means models perform cluster analysis on a set of input fields but do not use a target field. Weight and frequency fields are not used. See [Common modeling node properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/modelingnodeslots_common.htmlmodelingnodeslots_common) for more information. num_clusters number gen_distance flag cluster_label StringNumber label_prefix string mode SimpleExpert stop_on DefaultCustom max_iterations number tolerance number encoding_value number optimize SpeedMemory Specifies whether model building should be optimized for speed or for memory.
Sequence functions (SPSS Modeler)
Sequence functions For some operations, the sequence of events is important. The application allows you to work with the following record sequences: * Sequences and time series * Sequence functions * Record indexing * Averaging, summing, and comparing values * Monitoring change—differentiation * @SINCE * Offset values * Additional sequence facilities For many applications, each record passing through a stream can be considered as an individual case, independent of all others. In such situations, the order of records is usually unimportant. For some classes of problems, however, the record sequence is very important. These are typically time series situations, in which the sequence of records represents an ordered sequence of events or occurrences. Each record represents a snapshot at a particular instant in time; much of the richest information, however, might be contained not in instantaneous values but in the way in which such values are changing and behaving over time. Of course, the relevant parameter may be something other than time. For example, the records could represent analyses performed at distances along a line, but the same principles would apply. Sequence and special functions are immediately recognizable by the following characteristics: * They are all prefixed by @ * Their names are given in uppercase Sequence functions can refer to the record currently being processed by a node, the records that have already passed through a node, and even, in one case, records that have yet to pass through a node. Sequence functions can be mixed freely with other components of CLEM expressions, although some have restrictions on what can be used as their arguments.
Creating the flow (SPSS Modeler)
Creating the flow 1. Add a Data Asset node that points to broadband_1.csv. 2. To simplify the model, use a Filter node to filter out the Market_6 to Market_85 fields and the MONTH_ and YEAR_ fields. Figure 1. Example flow to show Time Series modeling ![Example flow to show Time Series modeling](https://dataplatform.cloud.ibm.com/docs/content/wsd/images/tut_bandwidth_forecast_build.png)
Python node properties
Python node properties Refer to this section for a list of available properties for Python nodes.
Importing a stream (SPSS Modeler)
Importing an SPSS Modeler stream You can import a stream ( .str) that was created in SPSS Modeler Subscription or SPSS Modeler client. 1. From your project's Assets tab, click . 2. Select Local file, select the .str file you want to import, and click Create. If the imported stream contains one or more source (import) or export nodes, you'll be prompted to convert the nodes. Watsonx.ai will walk you through the migration process. Watch the following video for an example of this easy process: This video provides a visual method to learn the concepts and tasks in this documentation. Video disclaimer: Some minor steps and graphical elements in this video might differ from your platform. [https://www.ustream.tv/embed/recorded/127732173](https://www.ustream.tv/embed/recorded/127732173) If the stream contains multiple import nodes that use the same data file, then you must first add that file to your project as a data asset before migrating because the conversion can't upload the same file to more than one import node. After adding the data asset to your project, reopen the flow and proceed with the migration using the new data asset. Nodes with the same name will be automatically mapped to project assets. Configure export nodes to export to your project or to a connection. The following export nodes are supported: Table 1. Export nodes that can be migrated Supported SPSS Modeler export nodes Analytic Server Database Flat File Statistics Export Data Collection Excel IBM Cognos Analytics Export TM1 Export SAS XML Export Notes: Keep the following information in mind when migrating nodes. * When migrating export nodes, you're converting node types that don't exist in watsonx.ai. The nodes are converted to Data Asset export nodes or a connection. Due to a current limitation for automatically migrating nodes, only existing project assets or connections can be selected as export targets. These assets will be overwritten during export when the flow runs. * To preserve any type or filter information, when an import node is replaced with Data Asset nodes, they're converted to a SuperNode. * After migration, you can go back later and use the Convert button if you want to migrate a node that you skipped previously. * If the stream you imported uses scripting, you may encounter an error when you run the flow even after completing a migration. This could be due to the flow script containing a reference to an unsupported import or export node. To avoid such errors, you must remove the scripting code that references the unsupported node. * If the stream you're importing contains unsupported data file types, you need to convert them to a supported type (CSV, Excel, or SPSS Statistics .sav). * In some cases, some settings from your original stream may not be restored during migration. For example, if the field delimiter in your original stream was tabs, it may be changed to commas after migration. Settings such as custom SQL also aren't migrated currently. Compare the new migrated flow to your original stream and making adjustments as needed.
cartnode properties
cartnode properties ![C&R Tree node icon](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/images/cartnodeicon.png)The Classification and Regression (C&R) Tree node generates a decision tree that allows you to predict or classify future observations. The method uses recursive partitioning to split the training records into segments by minimizing the impurity at each step, where a node in the tree is considered "pure" if 100% of cases in the node fall into a specific category of the target field. Target and input fields can be numeric ranges or categorical (nominal, ordinal, or flags); all splits are binary (only two subgroups). cartnode properties Table 1. cartnode properties cartnode Properties Values Property description target field C&R Tree models require a single target and one or more input fields. A frequency field can also be specified. See the topic [Common modeling node properties](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/scripting_guide/clementine/modelingnodeslots_common.htmlmodelingnodeslots_common) for more information. continue_training_existing_model flag objective StandardBoostingBaggingpsm psm is used for very large datasets, and requires a Server connection. model_output_type SingleInteractiveBuilder use_tree_directives flag tree_directives string Specify directives for growing the tree. Directives can be wrapped in triple quotes to avoid escaping newlines or quotes. Note that directives may be highly sensitive to minor changes in data or modeling options and may not generalize to other datasets. use_max_depth DefaultCustom max_depth integer Maximum tree depth, from 0 to 1000. Used only if use_max_depth = Custom. prune_tree flag Prune tree to avoid overfitting. use_std_err flag Use maximum difference in risk (in Standard Errors). std_err_multiplier number Maximum difference. max_surrogates number Maximum surrogates. use_percentage flag min_parent_records_pc number min_child_records_pc number min_parent_records_abs number min_child_records_abs number use_costs flag costs structured Structured property. priors DataEqualCustom custom_priors structured Structured property. adjust_priors flag trails number Number of component models for boosting or bagging. set_ensemble_method VotingHighestProbabilityHighestMeanProbability Default combining rule for categorical targets. range_ensemble_method MeanMedian Default combining rule for continuous targets. large_boost flag Apply boosting to very large data sets. min_impurity number impurity_measure GiniTwoingOrdered train_pct number Overfit prevention set. set_random_seed flag Replicate results option. seed number calculate_variable_importance flag calculate_raw_propensities flag calculate_adjusted_propensities flag adjusted_propensity_partition TestValidation
Merge node (SPSS Modeler)
Merge node The function of a Merge node is to take multiple input records and create a single output record containing all or some of the input fields. This is a useful operation when you want to merge data from different sources, such as internal customer data and purchased demographic data. You can merge data in the following ways. * Merge by Order concatenates corresponding records from all sources in the order of input until the smallest data source is exhausted. It is important if using this option that you have sorted your data using a Sort node. * Merge using a Key field, such as Customer ID, to specify how to match records from one data source with records from the other(s). Several types of joins are possible, including inner join, full outer join, partial outer join, and anti-join. * Merge by Condition means that you can specify a condition to be satisfied for the merge to take place. You can specify the condition directly in the node, or build the condition using the Expression Builder. * Merge by Ranked Condition is a left sided outer join in which you specify a condition to be satisfied for the merge to take place and a ranking expression which sorts into order from low to high. Most often used to merge geospatial data, you can specify the condition directly in the node, or build the condition using the Expression Builder.
Installing custom libraries through notebooks
Installing custom libraries through notebooks The prefered way of installing additional Python libraries to use in a notebook is to customize the software configuration of the environment runtime associated with the notebook. You can add the conda or PyPi packages through a customization template when you customize the environment template. See [Customizing environment templates](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/customize-envs.html). However, if you want to install packages from somewhere else or packages you created on your local machine, for example, you can install and import the packages through the notebook. To install packages other than conda or PyPi packages through your notebook: 1. Add the package to your project storage by clicking the Upload asset to project icon (![Shows the Upload asset to project icon](https://dataplatform.cloud.ibm.com/docs/content/wsj/console/images/find_data_icon.png)), and then browsing the package file or dragging it into your notebook sidebar. 2. Add a project token to the notebook by clicking More > Insert project token from the notebook action bar. The code that is generated by this action initializes the variable project, which is required to access the library you uploaded to object storage. Example of an inserted project token: @hidden_cell The project token is an authorization token that is used to access project resources like data sources, connections, and used by platform APIs. from project_lib import Project project = Project(project_id='7c7a9455-1916-4677-a2a9-a61a75942f58', project_access_token='p-9a4c487075063e610471d6816e286e8d0d222141') pc = project.project_context If you don't have a token, you need to create one. See [Adding a project token](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/token.html). 3. Install the library: Fetch the library file, for example the tar.gz or whatever installable distribution you created with open("xxx-0.1.tar.gz","wb") as f: f.write(project.get_file("xxx-0.1.tar.gz").read()) Install the library !pip install xxx-0.1.tar.gz 1. Now you can import the library: import xxx Parent topic:[Libraries and scripts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/libraries.html)
{{ document.title.text }}
Attribute inference attack ![icon for privacy risk](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/images/risk-privacy.svg)Risks associated with inputInferencePrivacyAmplified Description An attribute inference attack is used to detect whether certain sensitive features can be inferred about individuals who participated in training a model. These attacks occur when an adversary has some prior knowledge about the training data and uses that knowledge to infer the sensitive data. Why is attribute inference attack a concern for foundation models? With a successful attack, the attacker can gain valuable information such as sensitive personal information or intellectual property. Parent topic:[AI risk atlas](https://dataplatform.cloud.ibm.com/docs/content/wsj/ai-risk-atlas/ai-risk-atlas.html)
Uploading a text analysis package (TAP) in a Text Mining node (SPSS Modeler)
Uploading a custom asset in a Text Mining node You can add a custom text analysis package (TAP) or template directly in the Text Mining node. When your SPSS Modeler flow runs, it will use your custom asset. Procedure 1. If you want to download a TAP, save it locally. 1. Click Text analysis package while in the Text Analytics Workbench. 2. Enter details about the asset, and then click Submit. The text analysis package is saved locally as a .tap file. 2. If you want to download a template, see [Linguistic resources](https://dataplatform.cloud.ibm.com/docs/content/wsd/nodes/tmwb-linguistic-resource.htmltmwb-templates-intro__DownloadAssetsSteps). 3. Add the TAP or template file to another Text Mining node. 1. In the Text Mining node, click Select resources. 2. Click the Text analysis package or Resource template tab depending on the asset you want. 3. Click Import , and then browse to or drag-and-drop your TAP or template. 4. Enter details about the asset, and then click Add. You can now see the uploaded TAP in the list of resources. It is also saved to your project as a project asset. 5. Click Ok.