DataRobot-Research/drdocs huggingface dataset:
{% if section(page, "dr-notebooks") %}

This section outlines the management capabilities available for DataRobot Notebooks. Access the dashboard that hosts the currently available notebooks and all functionality from the top-level **Notebooks** tab.

![](images/nb-28.png)

Topic | Describes...
----- | ------
[Add notebooks](dr-create-nb) | How to create new notebooks, import existing notebooks, and export notebooks as `.ipynb` files.
[Notebook settings](dr-settings-nb) | The settings available for notebooks.
[Notebook versioning](dr-revise-nb) | How notebooks are versioned and how to load the revision history of a notebook.

{% else %}

This section outlines the management capabilities available for DataRobot Notebooks. Select and use case and then click the **Notebooks** tab to access the dashboard that displays all notebooks for that use case.

![](images/wb-nb-28.png)

Topic | Describes...
----- | ------
[Add notebooks](wb-create-nb) | How to create new notebooks, import existing notebooks, and export notebooks as `.ipynb` files.
[Notebook settings](wb-settings-nb) | The settings available for notebooks.
[Notebook versioning](wb-revise-nb) | How notebooks are versioned and how to load the revision history of a notebook.

{% endif %}
---
title: Prepare DataRobot models
description: Prepare to create deployments from DataRobot models

---

# Prepare DataRobot models for deployment

DataRobot AutoML models allow you to deploy to a DataRobot-managed prediction environment. This deployment method is the most direct route to making predictions and monitoring, managing, and governing your model in a centralized deployment hub. In addition, DataRobot AutoML models can be deployed to a containerized DataRobot prediction environment called a Portable Prediction Server (PPS). To prepare to deploy an AutoML model to a PPS, you can build models with AutoML and register the DataRobot model.

Topic | Describes
------|-----------
[Register DataRobot model packages](dr-model-reg) | How to add a DataRobot model to the Model Registry from the Leaderboard.
[Manage model packages](dr-model-manage) | How to deploy, share, or archive DataRobot models from the Model Registry.
---
title: Predict the likelihood of a loan default
description: AI models for predicting the likelihood of a loan default can be deployed within the review process to score and rank all new flagged cases.

---

# Predict the likelihood of a loan default {: #predict-the-likelihood-of-a-loan-default }

This page outlines the use case to reduce defaults and minimize risk by predicting the likelihood that a borrower will not repay their loan. This use case is captured in:

* A [Jupyter notebook](loan-default-nb.ipynb) that you can download and execute.
* A UI-based [business accelerator](loan-default).

{% include 'includes/loan-defaults-include.md' %}

### Demo {:#demo}

See the notebook [here](loan-default-nb.ipynb) or the UI-based accelerator [here](loan-default).
# Version 8.0.5 {: #version-805 }

_July 29, 2022_

The DataRobot v8.0.5 release includes some fixed issues in the DataRobot Self-Managed AI Platform platform. See the v8.0.0 release notes for:

* [Features introduced in v8.0.0](v8.0.0-aml)

In addition to the issues listed below, DataRobot addressed several outstanding security issues. This release/update is part of DataRobot's continual effort to secure the product and platform.

## Issues fixed in v8.0.5 {: #issues-fixed-in-v805 }

The following issues have been fixed since release v8.0.4:

### Enterprise {: #enterprise }

- EPS-534: Fixes issue with LDAP and SMTP password loading.


### Predictions {: #predictions }

- CODEGEN-1403: Fixes multiple Scoring Code blenders instantiation in a single class path.
- PRED-7672: Fixes an issue when making batch predictions with a JDBC intake adapter and data warehouse output adapters, like BigQuery, Snowflake, and Synapse.
- PRED-7708: Fixes an issue that caused BiqQuery write issues when feature names generated from Feature Discovery and feature names with special characters were included in passthrough columns.

### Time series {: #time-series }

- TIME-10985: Fixes an issue where time series feature derivation occasionally failed without a proper converged solution during the feature reduction process.
- TIME-11312: Fixes an issue causing supervised time series projects to suggest a different set of derived features during the feature reduction process, resulting in inconsistent derived feature sets in different project runs using the same settings.
- TIME-11371: Fixes an issue causing unsupervised time series projects to suggest a different set of derived features, resulting in inconsistent derived feature sets in different project runs using the same settings.

### Visual AI {: #visual-ai }

- VIZAI-3541: Fixes an issue that caused clustering models to experience increased run times when calculating the silhouette score for larger datatsets.

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: Prepare and leverage image data with Databricks
description: Import image files using Spark and prepare them into a data frame suitable for ingest into DataRobot.

---

# Prepare and leverage image data with Databricks {: #prepare-and-leverage-image-data-with-databricks }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/advanced-experimentation/image_data_databricks/Image%20Data%20Preparation.ipynb){ .md-button }

DataRobot's Visual AI allows you to leverage images in your models just like any other type of data. In this accelerator, you will import image files using Spark and prepare them into a data frame suitable for ingest into DataRobot. Then you will leverage DataRobot through code to rapidly train and deploy a powerful multiclass image classifier.

While there are other methods of ingesting image data into DataRobot, in this notebook you will encode the image data directly into the data frame using base64 encoding. This methodology allows you to keep all of the relevant data in a single data frame, and works well for a Databricks environment. This technique also extends widely to a wide variety of multimodal datasets.

Dive in to go from Databricks image data to a deployed classifier.
|  | Element | Description |
|---|---|---|
| ![](images/icon-1.png) | Selected word | Displays details about the selected word. (The term *word* here equates to an [*n-gram*](glossary/index#ngram), which can be a sequence of words.) <br><br>Mouse over a word to select it. Words that appear more frequently display in a larger font size in the **Word Cloud**, and those that appear less frequently display in smaller font sizes.|
| ![](images/icon-2.png) | Coefficient | Displays the [coefficient](coefficients#coefficientpreprocessing-information-with-text-variables) value specific to the word.|
| ![](images/icon-3.png) | Color spectrum | Displays a legend for the color spectrum and values for words, from blue to red, with blue indicating a negative effect and red indicating a positive effect. |
| ![](images/icon-4.png) | Appears in # rows| Specifies the number of rows the word appears in. |
| ![](images/icon-5.png) | Filter stop words | Removes stop words (commonly used terms that can be excluded from searches) from the display. |
| ![](images/icon-6.png) | Export | Allows you to [export](export-results) the **Word Cloud**. |
| ![](images/icon-7.png) | Zoom controls | Enlarges or reduces the image displayed on the canvas. Alternatively, double-click on the image. To move areas of the display into focus, click and drag. |
| ![](images/icon-8.png) | Select class | For multiclass projects, selects the class to investigate using the **Word Cloud**. |

??? info "Word Cloud availability"
    You can access **Word Cloud** from either the **Insights** page or the Leaderboard. Operationally, each version of the model behaves the same&mdash;use the Leaderboard tab to view a **Word Cloud** while investigating an individual model and the **Insights** page to access, and compare, each **Word Cloud** for a project. Additionally, they are available for multimodal datasets (i.e., datasets that mix images, text, categorical, etc.)&mdash;a **Word Cloud** is displayed for all text from the data.

The **Word Cloud** visualization is supported in the following model types and blueprints:

* Binary classification:
    * All variants of ElasticNet Classifier (linear family models) with the exception of TinyBERT ElasticNet classifier and FastText ElasticNet classifier
    * LightGBM on ElasticNet Predictions
    * Text fit on Residuals
    * Extended support for multimodal datasets (with single Auto-Tuned N-gram)

* Multiclass:
    * Stochastic Gradient Descent with at least 1 text column with the exception of TinyBERT SGD classifier and FastText SGD classifier

* Regression:
    * Ridge Regressor
    * ElasticNet Regressor
    * Lasso Regressor
    * Single Auto-Tuned Multi-Modal
    * LightGBM on ElasticNet Predictions
    * Text fit on Residuals

* Keras

!!! note
    The **Word Cloud** for a model is based on the data used to train that model, not on the entire dataset. For example, a model trained on a 32% sample size will result in a **Word Cloud** that reflects those same 32% of rows.

See [Text-based insights](analyze-insights#text-based-insights) for a description of how DataRobot handles single-character words.
---
title: Build models
description: This topic introduces elements of the basic modeling workflow as well as methods for building additional models in a project.

---

# Build models {: #build-models }

These sections describe elements of the basic modeling workflow as well as methods for building additional models in a project:

Topic | Describes...
----- | ------
[Build overview](model-data) | Set advanced modeling parameters prior to building.
[Feature lists](feature-lists) | Work with the set of features used to build models.
[Unlock Holdout](unlocking-holdout) | Release data saved for model evaluation.
[Comprehensive Autopilot](more-accuracy) | Prioritize model accuracy with more models at maximum sample size.
[Add/delete models](creating-addl-models) | Work with models after the initial model build.
[Frozen runs](frozen-run) | Freeze parameter settings to optimize resources.
[Model repository](repository) | Build from the library of modeling blueprints.
---
title: V9.0.4
description: DataRobot Release 9.0.4 release notes.

---

# V9.0.4 {: #v904 }

_July 24, 2023_

The DataRobot v9.0.4 release includes some fixed issues in the DataRobot Self-Managed AI Platform and issues found in the installation of the v9.0.3 release. For a complete list of notes since the v9.0.3 release, see the [v9.0.3 release notes](v9.0.3-aml). See the v9.0.0 release notes for:

* [Features introduced in v9.0.0](dr-9.0/indexv9.0/index
## Issues fixed in v9.0.4 {: #issues-fixed-in-v904 }

The following issues have been fixed since release v9.0.4:

### Data management {: #data-management }

DM-10790: Fixes an issue when starting Celery not running as root and using groups that do not exist.


_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: Mute deployment notifications
description: Learn how to mute notifications for a specific deployment, to stop receiving notifications for events tied to a configured notification policy.

---

# Mute deployment notifications {: #mute-deployment-notifications }

You can mute notifications for a specific deployment, allowing you to stop receiving notifications for the deployment-specific events tied to a configured [notification policy](web-notify#create-a-notification-policy). Deployments can create a lot of noise if they often change health status or encounter issues with data drift or anomalous predictions. This can result in a large number of notifications.

To mute deployment notifications, navigate to **Deployments**, select a deployment, and go to the **Settings > Notifications** tab.

![](images/webhooks-11.png)

This tab lists all of the notification policies applied to the deployment.

![](images/webhooks-12.png)

Identify the notification policy you wish to mute, and toggle on in **Mute for channel**. Once the toggle is turned on, you no longer receive notifications applied by the selected policy.

![](images/webhooks-13.png)
---
title: Enable data export
description: Enable prediction row storage for a deployment, allowing you to export the stored prediction and training data to compute and monitor custom business or performance metrics.

---

# Enable data export {: #enable-data-export }

You can enabled prediction row storage to activate the [Data Export tab](data-export), where you can export a deployment's stored training data, prediction data, and actuals to compute and monitor custom business or performance metrics on the [Custom Metrics tab](custom-metrics) or outside DataRobot.

![](images/data-export-settings.png)

| Field                         | Description                |
|-------------------------------|----------------------------|
| Enable prediction row storage | Enables prediction data storage, a setting required to store and export a deployment's prediction data for use in custom metrics. |
---
title: Build a recommendation engine
description: Explore how to use historical user purchase data in order to create a recommendation model which will attempt to guess which products out of a basket of items the customer will be likely to purchase at a given point in time.

---

# Build a recommendation engine {: #build-a-recommendation-engine}

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/game-changer/Recommendation%20Engine/Recommendation%20Engine.ipynb){ .md-button }

The accelerator provided in this notebook trains a model on historical customer purchases in order to make recommendations for future visits. The DataRobot features that will be utilized in this notebook are multi-Label modeling and feature discovery. Together the resulting model can provide rank ordered suggestions of content, product, or services that a specific customer might like.

In the notebook, you will:

* Analyze the datasets required
* Create a multilabel dataset for training
* Connect to DataRobot
* Configure a feature discovery project
* Generate features and models
* Generate recommendations for new visits
---
title: Environment variables
description: Describes the environment variables specific to operating the monitoring agent.

---

# Monitoring agent environment variables {: #monitoring-agent-environment-variables }

In addition to the environment variables used to configure the attached [spooler](spooler), you can configure the monitoring agent with the environment variables documented below.

###  General configuration {: #general-configuration }

When you run the agent using the provided `start-agent.sh` script from the `bin/` directory, the following environment configuration options are available:

| Variable |  Description  |
|----------|---------------|
| `MLOPS_AGENT_CONFIG_YAML`| The full path to a custom configuration YAML file. |
| `MLOPS_AGENT_LOG_DIR`    | The directory for writing the agent log file and stdout/error. |
| `JAVA_HOME`              | The Java Virtual Machine (JVM) to run the agent code. If you don't provide a JVM, Java should be included in the system PATH. |

###  Containerized configuration {: #container-configuration }

When you run the agent using the provided `Dockerfile` from the `tools/agent_docker/` directory, the following environment configuration options are available:

| Variable | Description     |
|----------|-----------------|
| `MLOPS_AGENT_CONFIG_YAML`  | The full path to a custom configuration YAML file. |
| `MLOPS_AGENT_LOG_DIR`      | The directory for writing the agent log file and stdout/error. |
| `MLOPS_AGENT_TMP_DIR`      | The directory for writing temporary files (a useful override if the container runs with a read-only root filesystem). |
| `MLOPS_SERVICE_URL`        | Specify the service URL to access MLOps via this environment variable instead of specifying it in the YAML configuration file. |
| `MLOPS_API_TOKEN`          | Provide your API token through this environment variable instead of specifying it in the YAML configuration file. |
| **Advanced configuration** | :~~: |
| `MLOPS_AGENT_JVM_OPT`        | Configure to override the default JVM option `-Xmx1G`.
| `MLOPS_AGENT_LOGGING_CONFIG` | Specify a full path to a completely custom Log4J2 configuration file for the MLOps monitoring agent. |
| `MLOPS_AGENT_LOGGING_FORMAT` | If using our default logging configuration, you can set the logging output format to either `plain` or `json`. |
| `MLOPS_AGENT_LOGGING_LEVEL`  | If using our default logging configuration, set the overall logging level for the agent (e.g. `trace`, `debug`, `info`, `warning`, `error`). |
| `MLOPS_AGENT_LOG_PROPERTIES` | Configure to override the default path to `mlops.log4j2.properties`. |
| `MLOPS_AGENT_SERVER_PORT`    | Set a free port number to activate the embedded HTTP server; this is useful for health and metric monitoring. |
---
title: V9.0.2
description: DataRobot Release 9.0.2 release notes.

---

# V9.0.2 {: #v902 }

_June 9, 2023_

The DataRobot v9.0.2 release includes some fixed issues in the DataRobot Self-Managed AI Platform platform. See the v9.0.0 release notes for:

* [Features introduced in v9.0.0](v9.0/index)

## Issues fixed in v9.0.2 {: #issues-fixed-in-v902 }

The DataRobot v9.0.2 release addresses issues found in the version numbering of the v9.0.1 release. For a complete list of notes since the v9.0.0 release, see the [v9.0.1 release notes](v9.0.1-aml).

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: AI Catalog
description: DataRobot's AI Catalog is a collection of registered objects that contains definitions and relationships between various object types.

---

# AI Catalog {: #import-data }

The AI Catalog is a browsable and searchable collection of registered objects that contains definitions and relationships between various object types. Items stored in the catalog include data connections, data sources, data metadata, and blueprints.

Topic | Describes...
----- | ------
**Import datasets** | :~~:
[Import data and create projects](catalog) | Import data into the AI Catalog and from there, create a DataRobot project.
**Manage catalog assets** | :~~:
[Work with catalog assets](catalog-asset) | Understand the details of catalog assets and learn how to manage them, including downloading, deleting, and sharing.
[Schedule data snapshots](snapshot)  |  Set up schedules for data snapshots in the AI Catalog to keep a dataset in sync with its data source.
**Prepare data** | :~~:
[Prepare data with SparkSQL](spark) | Enrich, transform, shape, and blend together datasets using Spark SQL queries within the AI Catalog.
# Overview

Applications are highly customizable, shareable -- therefore, there are many different ways you can modify and use apps, depending on your deployment type, user role, and use case.

This page provides a basic overview of how you might navigate through an application built from a binary classification model as an "Owner", from configuring your application to using it.

1. After [creating an application](create-app), navigate to the **Applications** tab.
2. Click **Open** to the right of the application you just created.
3. You will be prompted to sign in with DataRobot and authorize access.
4. After doing so, the application opens in Consume mode.

    Applications have two modes: **Build** mode, where you customize and configure your applications, and **Consume** mode, where you use the application to make predictions and analyze insights.

5. To modify the application, click **Build** in the upper-right corner; Build mode opens to the page you were viewing in Consume mode. In this case, that's the Home page. Note that each application is preconfigured based on the template you selected upon creation.
6. Applications are made up of widgets; some are required (default widgets) and others are optional, however, all widgets can be customized by selecting them.

   Click to select the All Rows widget (a default widget). A panel opens on the left where you can configure the widget. Configuration options vary depending on the widget.

7. On the **Data** tab, you can add or remove feature from the view. Click **Manage**, add the **Country** feature, and click **Save**. Notice that Country now appears under Selected features and as a column in the All Rows widget.
8. Click the **Properties** tab. On this tab, you can configure the widget's appearance and behavior.
9. Open the **Widgets** menu and drag-and-drop the Bar chart (an optional widget) onto the canvas.
---
title: Describe
description: Introduces the Leaderboard tabs, including Blueprint, Coefficients, Constraints, Data Quality Handling Reports, Eureqa Models, Log, Model Info, and Rating Table.

---

# Describe {: #describe }

The **Describe** tabs provide model building information and feature details:

Leaderboard tab  | Description | Source
------------------|-------------|------------
[Blueprint](blueprints)  | Provides a graphical representation of the data preprocessing and parameter settings via blueprint. | Blueprints can be DataRobot- or [user-](cml/index)generated. For DataRobot blueprints, the structure is decided once (after the partitioning stage), taking the dataset, project options, and column metadata into account. Values of `auto` hyperparameters may be decided later in the training process. Certain blueprint inputs and paths may be eliminated before training if the feature list does not have the corresponding feature types. For user-generated blueprints, the structure can be decided at any time.
 [Coefficients](coefficients)   | Provides, for select models, a visual representation of the most important variables and a coefficient export capability. | Training data
 [Constraints](monotonic)  | Forces certain XGBoost models to learn only monotonic (always increasing or always decreasing) relationships between specific features and the target. | Training, Validation data
 [Data Quality Handling Report](dq-report) (Formerly Missing Values) | Provides transformation and imputation information for blueprints. | Training data
 [Eureqa Models](eureqa) | Provides access to model blueprints for Eureqa generalized additive models (GAM), regression models, and classification models. | The Pareto front uses the Eureqa validation set, a subset of DataRobot training. The plots shown for regression and classification models use validation data.
 [Log](log)   | Lists operation status results. | N/A
 [Model Info](model-info)  | Displays model information.  | Training data
 [Rating Table](rating-table)   | Provides access to an export of the model’s complete, validated parameters.  | Training data
---
title: Explore AI insights
description: The tutorials and FAQ in this section provide quick, task-based instructions for achieving common tasks related to modeling.

---

# Explore AI insights {: #explore-ai-insights }

The tutorials in this section provide quick, task-based instructions for achieving common tasks related to modeling.

Topic | Describes how to...
----- | ------
[Interpret the Leaderboard](tut-read-leaderboard) | Interpret Leaderboard model summary information.
[Understand the Word Cloud ](tut-wordcloud) | Understand text-based insights using the Word Cloud.
[Use anomaly detection with time series](tut-ts-anomaly-detection) | Build and interpret anomaly detection models in time series projects.