DataRobot-Research/drdocs huggingface dataset:
---
title: ROC Curve tools
description: The ROC Curve tools help you explore classification, performance, and statistics related to a selected model at any point on the probability scale.
---

# ROC Curve tools {: #roc-curve-tools }

The **ROC Curve** tab provides tools for exploring classification, performance, and statistics related to a selected model at any point on the probability scale. The following topics show how to use these tools:

| Topic | Describes... |
|---|---|
| [Use the ROC Curve tools](roc-curve-tab-use) | Accessing the ROC Curve tab and understanding its components. |
| [Select data and display threshold](threshold) | Setting the data source and display threshold used for ROC Curve visualizations. |
| [Confusion matrix](confusion-matrix) | Using a confusion matrix to evaluate model accuracy by comparing actual versus predicted values. |
| [Prediction Distribution graph](pred-dist-graph) | Viewing the distribution of actual values in relation to the display threshold. |
| [ROC curve](roc-curve) | Using a ROC curve to view a plot of the true positive rate against the false positive rate for given data source. |
| [Profit curve](profit-curve) | Generating a profit curve to estimate the business impact of a selected model. |
| [Cumulative charts](cumulative-charts) | Generating charts to help assess a model's cumulative characteristics. |
| [Custom charts](custom-charts) | Generating your own charts to  explore classification, performance, and statistics for a model. |
| [Metrics](metrics) | Viewing statistics that describe model performance at the selected display threshold. |
---
title: AI Catalog impact analysis
description: View additional details that show how AI Catalog entities in the DataRobot application are related to--or dependent on--the current asset.
section_name: Data
maturity: public-preview
---

# AI Catalog impact analysis {: #ai-catalog-impact-analysis }

!!! info "Availability information"
     **AI Catalog** impact analysis is off by default. Contact your DataRobot representative or administrator for information on enabling the feature.

     <b>Feature flag:</b> Enable **AI Catalog** item impact analysis

The [**AI Catalog**](catalog) serves as a centralized collaboration hub for working with data and related assets. Additionally, it provides at-a-glance details (metadata) about these assets.

Available as a public preview feature, you can view additional details that show how other entities in the application are related to&mdash;or dependent on&mdash;the current asset. This is useful for a number of reasons, allowing you to:

* View how popular an item is based on the number of projects in which it is used.
* Understand which other entities might be affected if you were to makes changes or deletions.
* Gain understanding on how the entity is used.

All of the following associations are reported (with frequency values) as applicable:

* Projects
* Prediction datasets
* Feature Discovery configurations
* Time series calendars
* Spark SQL queries
* External model packages
* Deployment retraining

To view details, click on the asset title and tiles relevant to the selection display:

![](images/catalog-analysis.png)

Click on a tile for summary details and then click on the associated button for specific detail. For example, click on **Project** for a summary and **Open Project** for detail:

![](images/catalog-analysis-1.png)

In this example, DataRobot opens to the **Start** screen if [EDA1](eda-explained) was the last step completed or the **Data** page if EDA2 completed.

Each tile-type provides different (self-explanatory) details.

If you do not have permission to access an asset, you can view an entry that represents the asset but the entry does not disclose any additional information.

![](images/catalog-analysis-2.png)

This functionality is also available from the **Version History** tab for an asset:

![](images/catalog-analysis-3.png)
---
title: Humility tab
description: After configuring rules and making predictions with humility monitoring enabled, you can view the humility data collected over time for a deployment from the Humility tab.

---

# Humility tab {: #humility-tab }

After [configuring humility rules](humility-settings) and making predictions with humility monitoring enabled, you can view the humility data collected over time for a deployment from the **Humility > Summary** tab.

![](images/humility-9.png)

The X-axis measures the range of time that predictions have been made for the deployment. The Y-axis measures the number of times humility rules have triggered for the given period of time.

The controls&mdash;model version and data time range selectors&mdash;work the same as those available on the [Data Drift](data-drift) tab.
---
title: Manage monitoring job definitions
description: Manage monitoring job definitions
---

# Manage monitoring job definitions

To view and manage monitoring job definitions, select a deployment on the **Deployments** tab and navigate to the **Job Definitions > Monitoring Jobs** tab. 

![](images/batch-pred-job-def-list.png)

Click the action menu for a job definition and select one of the actions described below:

| Element | Description |
|---|---|
| View job history | Displays the **Deployments > Batch Jobs** tab listing all monitoring jobs generated from the job definition. |
| Run now | Runs the job definition immediately. Go to the **Deployments > Batch Jobs** tab to view progress. |
| Edit definition| Displays the job definition so that you can update and save it. | 
| Disable definition| Suspends a job definition. Any scheduled batch runs from the job definition are suspended. From the action menu of a job definition, click **Disable definition**. After you select **Disable definition**, the menu item becomes **Enable definition**. Click **Enable definition** to re-enable batch runs from this job description. |
| Clone definition | Creates a new job definition populated with the values from an existing job definition. From the action menu of the existing job definition, click **Clone definition**, update the fields as needed, and click **Save prediction job definition**. Note that the **Jobs schedule** settings are turned off by default.|
| Delete definition | Deletes the job definition. Click **Delete definition**, and in the confirmation window, click **Delete definition** again. All scheduled jobs are cancelled.  |
---
title: Compliance
description: The Compliance tabs compile model development documentation that can be used for regulatory validation.

---

# Compliance {: #compliance }

!!! info "Availability information"
     Availability of compliance documentation is dependent on your configuration. Contact your DataRobot representative for more information.

The **Compliance** tabs compile model development documentation that can be used for regulatory validation. Work with the report from the following Leaderboard tabs:

Leaderboard tab  | Description | Source
------------------|-------------|------------
[Compliance Documentation](compliance) |  Generate individualized documentation to provide comprehensive guidance on what constitutes effective model risk management. | Data sources used for insights vary across use cases.
[Template Builder](template-builder)   | Create, edit, and share custom compliance documentation templates. | Data sources used for insights vary across use cases.
---
title: Perform multi-model analysis
description: Use Python functions to aggregate DataRobot model insights into visualizations.

---

# Perform multi-model analysis {: #perform-multi-model-analysis }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/advanced-experimentation/multi_model_analysis/Multi-Model%20Analysis.ipynb){ .md-button }

DataRobot is designed to help you experiment with different modeling approaches, data preparation techniques, and problem framings. You can iterate fast with a tight feedback loop to quickly arrive at the best approach.

Sometimes you may wish to break your use case into multiple models, likely across multiple DataRobot projects. Maybe you want to build a separate model for each country or one for different periods of the year. In this case, it helps to bring all of your model performances and insights into one chart.

This accelerator shares several Python functions which can take the DataRobot insights - specifically model error, feature effects (partial dependence), and feature importance (SHAP or permutation-based) and bring them together into one chart, allowing you to understand all of your models in one place and more easily share your findings with stakeholders.
---
title: Maintenance notifications
description: Learn how you as an administrator can configure notifications that communicate service interruptions to users.

---

# Maintenance notifications {: #maintenance-notifications }

Administrators can configure notifications that communicate service interruption to users. Admins will create a banner to notify users when the system is planning maintenance or is currently impacted by an incident. The banner communicates the incident start time, scheduled end time (optional), and a link for more details (also optional). Users will see the banner if they are logged-in during the incident or during the configured notification window.

##  Create a new notification {: #create-a-new-notification }

To create a maintenance notification:

Click on your user icon and navigate to the **Maintenance Notifications** dashboard. This page is also accessible from the app administrator page.

![](images/maintenance-1.png)

Select **Add notification**. A dialog box prompts you to provide information about the new notification.

![](images/maintenance-2.png)

|Field | Description      |
|------|------------------|
| Event type | The type of event to notify users about. Select "Incident" or "Maintenance" from the dropdown.                                                             |
| Event start |  The time when the event starts. Use the calendar modal to indicate the date and time.                                                                     |
| Event end (optional)        |  The time when the event ends. Use the calendar modal to indicate the date and time.    Display notifications on                                           |
| Read more" link (optional)  |  The URL users can select to learn more information about the maintenance event. Appears as a clickable "Read more" link at the end of the notification.   |


When you have fully configured the fields, click **Add**. Note that these settings can be configured at a later time after the notification is saved. Your notification is available in the **Maintenance Notifications** dashboard.

###  Edit a notification {: #edit-a-notification }

When you have created a maintenance notifications, you can edit them from the dashboard. Select a notification to expand it and edit the fields.

![](images/maintenance-4.png)

When you have finished editing, click **Save**. If you wish to abandon the changes, click **Discard changes**.

Select **Preview** to view the notification banner that will display at the configured time.

![](images/maintenance-3.png)

###  Notification actions {: #notification-actions }

Notifications have two actions available: preview and deletion.

* Select the eye icon (![](images/icon-eye.png)) to preview the notification banner.

* Select the trash icon (![](images/icon-trash.png)) to permanently delete a notification.
# Version 7.3.6 {: #version-736 }

_May 4, 2022_

The DataRobot v7.3.6 release includes some fixed issues in the DataRobot Self-Managed AI Platform platform. See the v7.3.0 release notes for:

* [Features introduced in v7.3.0](v7.3.0-aml)

## Issues fixed in v7.3.6 {: #issues-fixed-in-v736 }

The following issues have been fixed since Enterprise release v7.3.5:

### Platform {: #platform }

* PLT-6264: Fixes error messaging when permanently deleting a user and their assets.
* PLT-6265: Fixes an issue causing permanently deleted projects to appear in search.

### Visual AI {: #visual-ai }

* VIZAI-3312: Fixes an issue affecting CSV and PNG export for external datasets confusion charts.

### MLOps {: #mlops }

* RAPTOR-7478: Fixes date column handling for Compliance Docs for Custom Inference Models. Date columns are now excluded from Feature Fit and Feature Effects.

### Predictions {: #predictions }

* CODEGEN-1333: Removes the unused junit directory from the Scoring Code JAR files.

### Time series {: #time-series }

* TIME-9875: Resolves an issue causing the feature over time calculation to fail when the feature is categorical and has a unique value.

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
# Version 7.2.3 {: #version-723 }

_October 13, 2021_

The DataRobot v7.2.3 release includes some fixed issues in the DataRobot Self-Managed AI Platform platform. See the v7.2.0 release notes for:

* [Features introduced in v7.2.0](v7.2.0-aml)

## Issues fixed in v7.2.3 {: #issues-fixed-in-v723 }

The following issues have been fixed since Enterprise release v7.2.2:

### Trust and explainability {: #trust-and-explainability }

* TREX-296: Fixes an issue with a database migration that could cause some models to lose their word cloud data during the on-premise upgrade process.
* TREX-282: Fixes an issue with a database migration that could cause some models to lose their lift chart data during the on-premise upgrade process.
* TREX-250: Fixes an issue with the compliance documentation rendering for slim run models trained to 100% of the data.

### Time Series {: #time-series }

* TIME-9425: Fixes issue that causes a blank page when accessing smart-sampled OTV projects with custom backtest settings.

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: DRUM CLI tool
description: DataRobot Model Runner (DRUM) is a tool that allows you to work with and test Python, R, and Java custom models and custom tasks.

---

{% include 'includes/drum-tool.md' %}

{% include 'includes/drum-for-ubuntu.md' %}

{% include 'includes/drum-for-mac.md' %}

{% include 'includes/drum-for-windows.md' %}
---
title: Eureqa advanced tuning
description: Eureqa models use expressions to represent mathematical relationships and transformations. DataRobot provides specialized workflows for tuning Eureqa models.

---

# Eureqa advanced tuning {: #eureqa-advanced-tuning }

Eureqa models use expressions to represent mathematical relationships and transformations. You can tune your Eureqa models by modifying building blocks, customizing the target expression, and modifying other model parameters, such as support for building blocks, error metrics, row weighting, and data splitting. To customize a Eureqa model, select the model from the Leaderboard and then click **Evaluate > Advanced Tuning**.

The following sections detail specialized workflows for tuning Eureqa models:

Topic | Describes...
----- | ------
[Tune Eureqa models](advanced-options) | Customize Eureqa models by modifying **Advanced Tuning** parameters.
[Configure building blocks](building-blocks) | Combine and configure building blocks to create a new target expression.
[Building blocks reference](eureqa-reference) | Definitions and usage of building blocks available to Eureqa models.
[Customize target expressions](custom-expressions) | Custom tune Eureqa models by modifying the target expression.
[Configure error metrics](error-metrics) | Optimize for different error metrics.
[Guidance for using error metrics](guidance) | Understand the top error metrics for different problem types.
[Configure row weighting](row-weighting) | Improve model performance with weighting.
---
title: Prediction monitoring jobs
description: To integrate more closely with external data sources, monitoring job definitions allow DataRobot to monitor deployments running and storing feature data and predictions outside of DataRobot.
---

# Prediction monitoring jobs

To integrate more closely with external data sources, monitoring job definitions allow DataRobot to monitor deployments running and storing feature data and predictions outside of DataRobot. For example, you can create a monitoring job to connect to Snowflake, fetch raw data from the relevant Snowflake tables, and send the data to DataRobot for monitoring purposes.

Method | Description
-------|-------------
[Create monitoring jobs](ui-monitoring-jobs) | Use the job definition UI to create monitoring jobs, allowing DataRobot to monitor deployments running and storing feature data and predictions outside of DataRobot.
[Monitoring jobs API](api-monitoring-jobs) | Use the Batch Monitoring API to create monitoring job definitions, allowing DataRobot to monitor deployments running and storing feature data and predictions outside of DataRobot.
[Manage monitoring job definitions](manage-monitoring-job-def) | Use a deployment's Monitoring Jobs tab to manage the monitoring job definitions you create.
[View and manage batch jobs](batch-jobs) | Use the Batch Jobs tab to view and manage monitoring and prediction jobs.
---
title: Enrich data using the Hyperscaler API
description: Call the GCP API and enrich a modeling dataset that predicts customer churn.

---

# Enrich data using the Hyperscaler API {: #enrich-data-using-the-hyperscaler-api }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/advanced-experimentation/gcp_sentiment/GCP_enrich_sentiment.ipynb){ .md-button }

Many companies are recognizing the value of unstructured data, particularly in the form of text, and are looking for ways to extract insights from it. This data includes emails, social media posts, customer feedback, call transcripts, and more. One of the most powerful tools for analyzing text data is sentiment analysis.

Sentiment analysis is the process of identifying the emotional tone of a piece of text, such as positive, negative, or neutral. It is a valuable tool to enrich the dataset for building machine learning models. For example, the sentiment expressed through a customer's recent call transcript with customer service could be predictive of the customer's likelihood to churn.

However, building sentiment analysis models is not an easy task. It requires a significant investment of time, resources, and expertise, especially in terms of accurately labeled data with large corpus to train. Most companies do not have the resources or expertise to develop their own sentiment analysis models.

Fortunately, there are now APIs available that provide sentiment analysis as a service. By using these APIs, companies can save time and money while still gaining the benefits of sentiment analysis. One of the most significant benefits of using hyperscaler APIs for sentiment analysis is their accuracy. The models behind the APIs are trained on large amounts of data, making them highly accurate at identifying emotions and sentiments in text data.

This accelerator demonstrates how easy it is to call the GCP API and enrich a modeling dataset that predicts customer churn, which shows an improvement in the model performance because of the retrieved sentiment scores based on the customers' call transcripts.
---
title: Solution accelerators
description: This section provides access to the catalog of use case-based solution accelerators, segmented by industry.
---

# Solution accelerators {: #solution-accelerators }

Solution accelerators provide a packaged solution, based on best practices and patterns, to address the most common use cases.

These sections provide a variety of industry-based solutions:

Topic | Provides solutions for...
----- | ------
[Banking](banking/index) | The banking industry.
[Healthcare](healthcare/index) | Healthcare providers, healthcare payers, and life sciences practitioners.
---
title: Predict steel plate defects
description: Help manufacturers significantly improve the efficiency and effectiveness of identifying defects of all kinds, including those for steel sheets.

---

# Predict steel plate defects {: #predict-steel-plate-defects }

Steel plates, whether used for general construction, industrial applications, or highly critical applications, must be of a structural quality to ensure safety and provide good formability and high strength. Every steel plate produced has some level of defects and faults. Normally, plates that don’t pass muster are scrapped, but not until the end of an expensive testing process. The time, money, and expertise spent on unusable product lowers the margins on each plate and adds complications to the manufacturing process.

## Business problem {: #business-problem }

Traditionally, each plate (or a representative sample) runs through a gauntlet of tests designed to reveal faults. The goal in terms of efficiency and cost is to accurately identify defective steel plates as early in the manufacturing process as possible, saving time and expense. Predictive modeling leverages data from that process, allowing you to target tests and identify faults in steel plates as early as possible. With good, accurate models, a plant can scrap bad plates earlier for higher efficiency.

The challenge, however, doesn’t end there. Plants make constant adjustments and improvements to their processes. If the predictive models aren’t updated with new data, they will quickly become outdated and suffer from a degradation in accuracy. It’s necessary to set up retraining processes that automatically identify the best model available based on new information.

Consider Michelle, who is responsible for maintaining the quality steel plates her company produces. She knows the ins and outs of the production process, like the temperature-to-pressure ranges needed to ensure pristine results. Automated machine learning offers a new way to build upon her expertise and leverage the data her team collects. Instead of worrying about hiring a data scientist to support her team, she and the other engineers can take their data and build their own models. Those models can then predict which steel plates will have the highest likelihood of faults and also identify the reasons that may be driving those defective results.

## Use case data {: #use-case-data }

This notebook uses a [dataset](http://archive.ics.uci.edu/ml/datasets/steel+plates+faults) (UCI Machine Learning Repository: Steel Plates Faults dataset) provided by Semeion, Research Center of Sciences of Communication, Via Sersale 117, 00128, Rome, Italy.

Before proceeding, review the [API quickstart guide](api-quickstart/index) to ensure that your client and API credentials have been properly configured.

* [Download training data](https://s3.amazonaws.com/datarobot-use-case-datasets/steel_plates_fault_training.csv)
* [Download prediction data](https://s3.amazonaws.com/datarobot-use-case-datasets/steel_plates_fault_testing.csv)
---
title: Amazon SageMaker
description: Integrate DataRobot with Amazon SageMaker.

---

# Amazon SageMaker {: #amazon-sagemaker }

The sections described below provide techniques for integrating Amazon SageMaker with DataRobot.

Topic | Describes...
----- | ------
[Deploy models on Sagemaker](sagemaker-deploy) | Deploying on SageMaker and monitoring with MLOps agents.
[Monitor SageMaker models in MLOps](sagemaker-monitor) | Monitoring a SageMaker model that has been deployed to AWS for real-time scoring in DataRobot MLOps.
[Use Scoring Code with AWS SageMaker](sc-sagemaker) | Making predictions using Scoring Code deployed on AWS SageMaker.