DataRobot-Research/drdocs huggingface dataset:
---
title: Analyze data
description: Interpret the findings and visualizations created by DataRobot.

---

# Analyze data {: #analyze-data }

These sections help you interpret the findings and visualizations created by DataRobot.

Topic | Describes...
----- | ------
[Data Quality Assessment](data-quality) | Understand a dataset's Data Quality Assessment results, including the logic DataRobot applies to detect, and often repair, common data quality issues.
**Post data ingest analysis (EDA1)** | :~~:
[Feature details](histogram) | Interpret histograms, frequent values charts, and transformations.
[EDA1](eda-explained#eda1) | View summary statistics based on a sample of your data.
[ESDA](lai-esda) | Interactively visualize, explore, and aggregate target, numeric, and categorical features on a map.
[Fast EDA for large datasets](fast-eda) | Understand Fast Exploratory Data Analysis (EDA) for large datasets, and how to apply early target selection.
[Over Time chart](ts-leaderboard#understand-a-features-over-time-chart). | Review time-aware visualizations of how a feature changes over time feature (time-aware only).
**Post modeling analysis (EDA2)** | :~~:
[EDA2](eda-explained#eda2) | View summary statistics based on the portion of the data used for EDA1, excluding rows that are also in the holdout data and rows where the target is `N/A`.
[Feature Associations](feature-assoc) | Interpret feature correlations.
# Amazon S3 {: #amazon-s3 }

## Supported authentication {: #supported-authentication }

- AWS Credential

## Prerequisites {: #prerequisites }

The following is required before connecting to Amazon S3 in DataRobot:

- Amazon S3 account

## Required parameters {: #required-parameters }

The table below lists the minimum required fields to establish a connection with Amazon S3:

Required field | Description |  Documentation
--------------- | ---------- | -----------
`bucketName` |     A container that stores your data in Amazon S3.        |          [AWS documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html){ target=_blank }
`bucketRegion`  |  The AWS region of your bucket.      |      [AWS documentation](https://docs.aws.amazon.com/general/latest/gr/s3.html){ target=_blank }

## Caveats {: #caveats }

- This connector does not use refresh tokens, however, you can specify it as part of your AWS Credential. Only a static access key, which does not get refreshed periodically, works with the connector.
---
title: Python v2.x use cases
description: Review Jupyter notebooks that outline common use cases for version 2.x of DataRobot's Python client.

---

# Python v2.x use cases {: #python-v2x-use-cases }

Review Jupyter notebooks that outline common use cases and machine learning workflows using DataRobot's Python client.

Topic | Describes... |
----- | ------ |
[Large scale demand forecasting](demand-forecast/index) | An end-to-end demand forecasting use case that uses DataRobot's Python package. |
[Predict loan defaults](loan-default/index) | A use case that reduces defaults and minimizes risk by predicting the likelihood that a borrower will not repay their loan. |
[Predict late shipments](predict-shipment/index) | A use case that determines whether a shipment will be late or if there will be a shortage of parts. |
[Predict steel plate defects](steel/index) | A use case that helps manufacturers significantly improve the efficiency and effectiveness of identifying defects of all kinds, including those for steel sheets.
[Reduce 30-Day readmissions rate](readmission/index) | A use case to reduce the 30-day readmission rate at a hospital. |
[Identify money laundering with anomaly detection](aml/index) | How to train anomaly detection models to detect outliers. |
[No-show appointment forecasting](no-show-appt/index) | How to build a model that identifies patients most likely to miss appointments, with correlating reasons. |
[Measure price elasticity of demand](elasticity/index) | A use case to identify relationships between price and demand, maximize revenue by properly pricing products, and monitor price elasticities for changes in price and demand. |
[Predict equipment failure](part-fail.ipynb) | A use case that that determines whether or not equipment part failure will occur. |
[Identify money laundering with anomaly detection](outlier-detect.ipynb) | How to train anomaly detection models to detect outliers. |
[Lead scoring](lead-scoring.ipynb) | A binary classification problem of whether a prospect will become a customer.  |
[Predict fraudulent medical claims](pred-fraud.ipynb) | The identification of fraudulent medical claims using the DataRobot Python package. |
[Predict CO₂ levels with out-of-time validation modeling](otv-nb.ipynb) | How to use [out-of-time validation (OTV)](otv) modeling with DataRobot's Python client to predict monthly CO₂ levels for one of Hawaii's active volcanoes, Mauna Loa. |
[Forecast sales with multiseries modeling](multiseries-nb.ipynb) | How to forecast future sales for multiple stores using multiseries modeling. |
[Predictions for fantasy baseball](fantasy.ipynb) | An estimate of a baseball player's true talent level and their likely performance for the coming season. |
[Predict customer churn](customer-churn.ipynb) | How to predict customers that are at risk to churn and when to intervene to prevent it. |
[Configure datetime partitioning](datetime-nb.ipynb) | How to use [datetime partitioning](datetime_partitioning) to guard a project against time-based target leakage. |
---
title: Experiments
description: Create experiments and iterate quickly to evaluate and select the best predictive models.
---

# Experiments {: #experiments }

The following sections help to understand Workbench experiments:

Topic   |   Describes...
---------|-----------------
[Create experiments](wb-experiment-create)  | Create experiments in Workbench.
[Evaluate models](wb-experiment-evaluate) | Filter the Leaderboard, use visualization tools to evaluate models, and manage experiments.
[Add/retrain models](wb-experiment-add) | Retrain existing models and add models from the blueprint repository
[Reference and FAQ](wb-experiment-ref)	| Questions and tips for working with experiments.
---
title: DataRobot model in a DataRobot environment
description: How to deploy a DataRobot model in a DataRobot environment.

---

# Deploy a DataRobot model in a DataRobot environment {: #deploy-a-datarobot-model-in-a-datarobot-environment }

DataRobot AutoML models allow you to deploy to a DataRobot-managed prediction environment. This deployment method is the most direct route to making predictions and monitoring, managing, and governing your model in a centralized deployment hub.

To create and deploy an AutoML model on DataRobot, follow the workflow outlined below:

``` mermaid
graph TB
  A{Deployment method?} --> |Leaderboard|B[Deploy a model from the Leaderboard];
  A --> |Model registry|C[Register a model]
  C --> D[Deploy a model from the Model Registry]
```

## Deploy a model from the Leaderboard {: #deploy-a-model-from-the-leaderboard }

DataRobot AutoML automatically generates models and displays them on the Leaderboard. The [model recommended for deployment](model-rec-process) appears at the top of the page. You can deploy this (or any other) model directly from the Leaderboard to start making and monitoring predictions. When you create a deployment from a model, DataRobot automatically creates a model package for the deployed model. You can access the model package at any time in the Model Registry.

[Deploy from the Leaderboard <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](deploy-model#deploy-from-the-leaderboard){ .md-button }

## Register a model {: #register-a-model }

If you don't want to deploy immediately from the Leaderboard, you can add a model package to the Model Registry to deploy later.

!!! note
    This method allows you to [share a model package](reg-action) or [generate compliance documentation](reg-compliance) before deploying a model.

[Register a model <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](dr-model-reg){ .md-button }

## Deploy a model from the Model Registry {: #deploy-a-model-from-the-model-registry }

After you've added a model to the Model Registry, you can deploy it at any time to start making and monitoring predictions.

[Deploy from the Model Registry <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](deploy-model#deploy-from-the-model-registry){ .md-button }
If you expect to be able to increase your worker count but cannot, the reasons may be:

* You have hit your [worker limit](#worker-limit).
* Your workers are part of a [shared pool](#pooled-workers).
* Your workers are [in use by another project](#workers-in-use).

#### Worker limit {: #worker-limit }

[Modeling worker allocations](admin-overview#modeling-worker-allocation) are set by your administrator. Each worker processes a modeling job. This job count applies across all projects in the cluster, that is, multiple browser windows building models are all a part of your personal worker count&mdash;more windows does not provide more workers.

#### Pooled workers {: #pooled-workers }

If you are in an organization, it may implement a [shared pool of workers](admin-overview#what-are-organizations). In this case, workers are allocated across all jobs for all users in the organization on a first-in, first-out basis. While you may not have to wait for jobs of other users in your organization to complete, your (and their) jobs will be seeded in the queue and processed as they were received.

#### Workers in use {: #workers-in-use }

If you believe you should be able to increase the worker count but you cannot, for example, "using X of Y workers," there are two values to consider for debugging. When Y is lower than you expect, check your worker limit and the [org limit](#worker-limit). When X is less than you expect, check whether workers are being allocated to other projects or users in your organization.

To check worker use in your projects, navigate to the [**Manage Projects** inventory](manage-projects#manage-projects-control-center) and look for queued jobs. You can identify them by:

* An icon and count in the inventory.

	![](images/queue-queued.png)

* The list that results from using [**Filter Projects**](manage-projects#filter-projects) and selecting **Running or queued**.

If you find a project with queued jobs, you can stop Worker Queue processing.

1. Click on the project in the inventory to make it the active project.
2. In the Worker Queue, click the X icon (![](images/icon-x.png)) to remove all tasks. This removes all in-progress or queued models.

	![](images/queue-remove-tasks.png)
You can also pause the project. As workers complete active jobs, they will become available to pick up jobs from other projects.
---
title: Membership assignments
description: How to see which organization and groups your system administrator has assigned you to.
---

#  Membership assignments {: #membership-assignments }

From the **Membership** page, you can see which organization and groups your system administrator has assigned you to and join other groups.

![](images/membership.png)

Organization and group assignments allow system administrators to manage users, control project sharing, apply actions across user populations, etc. You may be a member of up to ten groups or not a member of any groups. Likewise, you may be a member of a single organization or not a member of any organization. System administrators choose how they want to configure user membership assignments. Only they can create and modify these memberships.
The **Over time** chart helps you identify trends and potential gaps in your data by displaying, for both the original modeling data and the derived data, how a feature changes over the primary date/time feature. It is available for all time-aware projects (OTV, single series, and multiseries). For time series, it is available for each user-configured forecast distance.

Using the page's tools, you can focus on specific time periods. Display options for OTV and single-series projects differ from those of multiseries. Note that to view the **Over time** chart you must first compute chart data. Once computed:

1. Set the chart's granularity. The resolution options are auto-detected by DataRobot. All project types allow you to set a resolution (this option is under **Additional settings** for multiseries projects).

	![](images/fot-resolution.png)

2. Toggle the histogram display on and off to see a visualization of the bins DataRobot is using for [EDA1](eda-explained#eda1).

3. Use the date range slider below the chart to highlight a specific region of the time plot. For smaller datasets, you can drag the sliders to a selected portion. Larger data sets use block pagination.

	![](images/fot-slide.png)

4. For multiseries projects, you can set both the forecast distance and an individual series (or average across series) to plot:

	![](images/fot-resolution-multi.png)

For time series projects, the **Data** page also provides a [Feature Lineage](#feature-lineage-tab) chart to help understand the creation process for derived features.

## Partition without holdout {: #partition-without-holdout }

Sometimes, you may want to create a project without a holdout set, for example, if you have limited data points. Date/time partitioning projects have a minimum data ingest size of 140 rows. If **Add Holdout fold** is not checked, minimum ingest becomes 120 rows.

By default, DataRobot creates a holdout fold. When you toggle the switch off, the red holdout fold disappears from the representation (only the backtests and validation folds are displayed) and backtests recompute and shift to the right. Other configuration functionality remains the same&mdash;you can still modify the validation length and gap length, as well as the number of backtests. On the Leaderboard, after the project builds, you see validation and backtest scores, but no holdout score or **Unlock Holdout** option.

The following lists other differences when you do not create a holdout fold:

* Both the [**Lift Chart**](lift-chart#change-the-display) and [**ROC Curve**](pred-dist-graph#data-selection) can only be built using the validation set as their **Data Source**.
* The [**Model Info**](model-info) tab shows no holdout backtest and or warnings related to holdout.
* You can only compute predictions for **All data** and the **Validation** set from the [**Predict**](predict#why-use-training-data-for-predictions) tab.
* The [**Learning Curves**](learn-curve) graph does not plot any models trained into Validation or Holdout.
* [**Model Comparison**](model-compare) uses results only from validation and backtesting.
To configure the **Time series options**, under **Time series prediction method**, select [**Forecast point** or **Forecast range**](ts-predictions#forecast-settings).

=== "Forecast point"

    Select **Forecast point** to choose the specific date from which you want to begin making predictions, and then select a **Forecast point selection method**:

    * **Automatic**: DataRobot sets the forecast point for you based on the scoring data.

        ![](images/ts-makepred-1.png)

    * **Fixed**: You set the forecast point date.
    
        ![](images/ts-batch-pred-forecast-pt-fixed.png)

=== "Forecast range"

    Select **Forecast range** if you intend to make bulk, historical predictions (instead of forecasting future rows from the forecast point). Then, select a forecast range: 
    
    * **Use all dates from prediction source**: Predictions use all forecast distances within the selected time range. 
    
    * **Use specific date range**: Set a specific date range using the date selector.

    ![](images/ts-makepred-2.png)
---
title: Register custom models as model packages
description: Register a custom model in the Model Registry. The Model Registry is an archive of your model packages where you can also deploy and share the packages.

---

# Register custom models as model packages {: #register-custom-models-as-model-packages }

You can create a model package for a custom inference model to replace the model package in existing deployments with a new one, or to [share](reg-action#sharing) with another user who wants to deploy your custom model package.

When you have successfully created and tested a custom inference model, you have the option to add it to the Model Registry as a model package. To do so, navigate to **Model Registry** > **Custom Model Workshop** and select the custom model you wish to add.

Under the **Test and Deploy** tab, click **Add to registry**.

![](images/reg-create-1.png)

The custom model is then added to the **Model Registry**. The **Add to registry** link is replaced with **View registry package**, which if clicked, takes you to your newly created model package in the registry under the **Model Packages** tab.

Note that although a model package can be created without testing the custom model, DataRobot recommends that you confirm the model passes testing before proceeding. Untested custom models prompt a dialog box warning that the custom model is not tested.

![](images/reg-create-2.png)

For untested custom models, click **Test now** to start a model test, or **Create package without testing** to proceed with the creation of a model package.
---
title: Bias vs Accuracy
description: The Bias vs Accuracy chart compares predictive accuracy and fairness, removing the need to manually note each model's accuracy and fairness scores.

---

# Bias vs Accuracy {: #bias-vs-accuracy }

The **Bias vs Accuracy** chart shows the tradeoff between predictive accuracy and fairness, removing the need to manually note each model's accuracy score and fairness score for the protected features. Consider your use case when deciding if the model needs to be more accurate or more fair. The **Bias vs Accuracy** display is based on the validation score, using the currently selected metric.

- The Y-axis displays the validation score of each model. To change this metric, switch to the Leaderboard, change the metric via the **Metric** dropdown, and then return to **Bias vs Accuracy**.
- The X-axis displays the fairness score of each model, that is the lowest relative fairness score for a class in the protected feature.

![](images/bias-v-acc-1.png)

##  Bias vs Accuracy chart {: #bias-vs-accuracy-chart }

Consider the following when evaluating the **Bias vs Accuracy** chart:

- You must calculate [**Per-Class Bias**](per-class) for a model before it can be displayed on the chart.
- **Protected Features**, **Fairness metric**, and **Fairness threshold** were defined in [advanced options](fairness-metrics) prior to model building.
- Use the **Feature List** dropdown to compare the models trained on different feature lists.
- The left side highlights models with fairness scores below the fairness threshold, and the right side highlights models with scores above the threshold.
- Hover on any point to view scores for a specific model.
- Some models may not report scores because there is not enough data (as indicated with a tooltip). Requirements are:
	- More than 100 rows.
	- If between 100 and 1,000 rows, more than 10% of the rows must belong to the majority class (the class with the most rows of data).
---
title: Advanced options
description: Set advanced parameters before building models to set non-default characteristics of the model build.

---

# Advanced options {: #advanced-options }

After importing data and selecting a target variable, the **Data** page appears. From this page you can click the **Show advanced options** link to access advanced modeling parameters.


These parameters are summarized in the following sections:

Option   |   Description   
---------|-----------------
[Additional](additional)  | Set additional parameters and modify values that can effect model builds.   
[Bias and Fairness](fairness-metrics)  | Set conditions that help calculate fairness, as well as identify and attempt to mitigate bias in a model's predictive behavior.  
[Clustering](time-series-cluster-adv-opt.md) | Set the number of clusters that DataRobot discovers in a time series clustering project.
[External model prediction insights](external-preds) | Bring external model(s) into the DataRobot AutoML environment, view them on the Leaderboard, and run a subset of DataRobot's evaluative insights for comparison against DataRobot models.
[Feature Constraints](feature-con)  | Set monotonic constraints to control the influence between variables and  target.
[Partitioning](partitioning)  | Set how data is partitioned for training/validation/holdout and the validation type.
Partitioning: Date/time  | Set how data is partitioned for [OTV](otv#advanced-options) or [time series](ts-date-time) projects.
[Smart Downsampling](smart-ds)  | Downsample the majority class for faster model build time.  
[Time Series](ts-adv-opt)   | Set a variety or time series-specific advanced options.
[Train-time image augmentation](ttia)  | Create new training images to increase the amount of training data.


![](images/advanced-options-link.png)
---
title: DataRobot Classic
description: Get started with DataRobot's value-driven AI. Analyze data, create and deploy models, and leverage code-first accelerators and notebooks.

---

# DataRobot Classic {: #datarobot-classic }

Get started with DataRobot's classic experience. Analyze data, create and deploy models, and leverage code-first accelerators and notebooks.

Topic          | Describes...
---------------|---------------
[Fundamentals of DataRobot Classic](gs-dr-fundamentals) | Understand the types of ML modeling projects you can create in DataRobot. Learn the general process of modeling, analyzing, and selecting models for deployment.
[Get started with data](gs-data)        | Use DataRobot's integrated capabilities to import, explore, and prepare your data for modeling.
[Get started with modeling](gs-model)   | Data in, models out&mdash;click to model and immediately start investigating model insights.
[Get started with MLOps](gs-mlops)      | Deploy, monitor, and manage models in production.