DataRobot-Research/drdocs huggingface dataset:
---
title: End-to-end ML workflow with Snowflake
description: Work with Snowflake and DataRobot's Python client to import data, build and evaluate models, and deploy a model into production to make new predictions.

---

# End-to-end ML workflow with Snowflake {: #end-to-end-ml-workflow-with-snowflake }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/tree/main/end-to-end/Snowflake_End_to_End){ .md-button }

This AI accelerator walks through how to work with Snowflake (as a data source) and DataRobot's Python client to import data, build and evaluate models, and deploy a model into production to make new predictions. More broadly, the DataRobot API is a critical tool for data scientists to accelerate their machine learning projects with automation while integrating the platform's capabilities into their code-first workflows and coding environments of choice.

By using this accelerator, you will:

* Connect to DataRobot.
* Import data from Snowflake into DataRobot.
* Create a DataRobot project and run Autopilot.
* Select and evaluate the top performing model.
* Deploy the recommended model with MLOps model monitoring.
* Orchestrate scheduled batch predictions that write results back to Snowflake.
---
title: Train-time image augmentation
description: Train-time image augmentation is a processing step in the DataRobot blueprint that creates new images for training by randomly transforming existing images.
---

# Train-time image augmentation {: #train-time-image-augmentation }

**Train-time image augmentation** is a processing step in the DataRobot blueprint that creates new images for training by randomly transforming existing images, thereby increasing the size of (i.e., "augmenting") the training data. These sections describe components of augmentation, the process in which each image is transformed.

Topic | Describes...
----- | ------
[About augmented models](ttia-introduction) | Read an overview of augmention.
[Augmentation lists](ttia-lists) | Store all the parameter settings for a given augmentation strategy.
[Use case examples](ttia-examples) | See examples of leveraging domain knowledge to craft a beneficial augmentation strategy.
---
title: Reference

---

# Reference {: #reference }

The information in this section provides reference information for managing your DataRobot account.


Topic | Describes...
----- | ------------
[Role-based access control (RBAC)](rbac-ref) | View descriptions of each role in RBAC.
[Custom RBAC roles](custom-roles) | System and organization administrators can create roles and define access at a more granular level, and assign them to users and groups.
[User Activity Monitor reference](uam-ref) | View descriptions of the fields included in each activity report.
---
title: DataRobot status
description: Status page announcements provide information on service outages, scheduled maintenance, and historical uptime.

---

# Check platform status {: #check-platform-status }

DataRobot performs service maintenance regularly. Although most maintenance will occur unnoticed, some may cause a temporary impact. Status page announcements provide information on service outages, scheduled maintenance, and historical uptime. You can view and subscribe to notifications from the [DataRobot status page](https://status.datarobot.com/){ target=_blank }.

![](images/dr-status-page.png)
---
title: Deploy tab
description: How to deploy a model from the Leaderboard

---

# Deploy tab {: #deploy-tab }

You can deploy models you build with DataRobot AutoML from the Leaderboard. In most cases, before deployment, you should unlock holdout and [retrain your model](creating-addl-models#retrain-a-model) at 100% to improve predictive accuracy. DataRobot automatically runs [**Feature Impact**](feature-impact) for the model (this also calculates **Prediction Explanations**, if available).

{% include 'includes/deploy-leaderboard.md' %}
---
title: Likelihood of a loan default
description: AI models for predicting the likelihood of a loan default can be deployed within the review process to score and rank all new flagged cases.
---

# Likelihood of a loan default {: #likelihood-of-a-loan-default }

This page outlines the use case to reduce defaults and minimize risk by predicting the likelihood that a borrower will not repay their loan. It is captured below as a UI-based walkthrough. It is also available as a [Jupyter notebook](loan-default-nb.ipynb) that you can download and execute.

{% include 'includes/loan-defaults-include.md' %}

### No-Code AI Apps {: #no-code-ai-apps }

A [no-code](app-builder/index) or Streamlit app can be useful for showing aggregate results of the model (e.g., risky transactions at an entity level). Consider building a custom application where stakeholders can interact with the predictions and record the outcomes of the investigation. Once the model is deployed, predictions can be consumed for use in the [decision process](#decision-process). For example, this [No-Code AI App](app-builder/index) is an easily shareable, AI-powered application using a no-code interface:

![](images/late-ship-app-1.png)

![](images/late-ship-app-2.png)

### Notebook demo {:#notebook-demo}

See the notebook version of this accelerator [here](loan-default-nb.ipynb).
---
title: Batch prediction scripts
description: Use the Prediction API with these scripts to score large files efficiently.
---

# Batch prediction scripts {: #batch-prediction-scripts }

The Batch prediction scripts are command-line tools for Windows, macOS, and Linux.
They wrap the [Batch Prediction API](batch-prediction-api/index).

To access the scripts, you need a trained model and an active deployment. Then, navigate to the [Predictions API tab](code-py):

* Navigate to the **Deployments** tab and click to select a deployment.
* Select the **Predictions** > [**Prediction API** tab](code-py)
* Select `Batch` as `Predictions Type`

To understand more about how to interact with the code samples, see the [Prediction API Scripting Code](code-py).
---
title: Document AI
description: Learn how to use documents as a data source without manual intervention to make documents available for modeling.
section_name: AutoML
maturity: public-preview
---

# Document AI {: #document-ai }

!!! info "Availability information"
    Document AI modeling is off by default. Contact your DataRobot representative or administrator for information on enabling the feature.

    <b>Feature flag:</b> Enable Document Ingest, Enable OCR for Document Ingest


Public preview for... | Describes...
----- | ------
[Workflow overview](doc-ai-overview) | Read background information and a simplified workflow overview.
[Document ingest and modeling](doc-ai-ingest) | Learn how to prepare raw documents as an input to modeling.
[Document AI insights](doc-ai-insights) | Use the Document AI-specific visualizations to better understand text within your documents.
[Making predictions](doc-ai-predictions) | Make real-time or batch predictions on Document AI models.
# Exasol {: #exasol }

## Supported authentication {: #supported-authentication }

- Username/password

## Prerequisites {: #prerequisites }

The following is required before connecting to Exasol in DataRobot:

- Exasol account

## Required parameters {: #required-parameters }

The table below lists the minimum required fields to establish a connection with Exasol:

Required field | Description | Documentation
--------------- | ---------- | -----------
`address`       | The connection URL that supplies connection information for Exasol. **Example:** `jdbc:exa:testdb.exasol.com:5599`  |    [Exasol documentation](https://docs.exasol.com/db/latest/connect_exasol/drivers/jdbc.htm){ target=_blank }
`port`  | Specific port of network services with which Exasol databases may communicate.<br><br>Auto-populated by DataRobot.  |   [Exasol documentation](https://docs.exasol.com/db/latest/administration/on-premise/manage_network/system_network_settings.htm){ target=_blank }

Learn about additional [configuration options for Exasol](https://docs.exasol.com/db/latest/connect_exasol/drivers/jdbc.htm){ target=_blank }. See also an end-to-end overview of [connecting to Exasol with DataRobot](https://community.exasol.com/t5/tech-blog/how-to-automate-your-machine-learning-with-datarobot-and-exasol/ba-p/2333){ target=_blank } on the Exasol Community.

{% include 'includes/data-conn-trouble.md' %}
---
title: Deploy DataRobot models
description: How to create new deployments, deploy custom models, create deployments with training data, and add data post-deployment.

---

# Deploy DataRobot models {: #deploy-datarobot-models }

You can deploy models you build with DataRobot AutoML using the following methods:

* Deploy the model [from the Leaderboard](#deploy-from-the-leaderboard).
* Save the model as a model package in the Model Registry and [deploy from the registry](#deploy-from-the-model-registry).

!!! tip
    In most cases, before deployment, you should unlock holdout and [retrain your model](creating-addl-models#retrain-a-model) at 100% to improve predictive accuracy. DataRobot automatically runs [**Feature Impact**](feature-impact) for the model (this also calculates **Prediction Explanations**, if available).

## Deploy from the Leaderboard {: #deploy-from-the-leaderboard }

{% include 'includes/deploy-leaderboard.md' %}

## Deploy from the Model Registry {: #deploy-from-the-model-registry }

1. Navigate to **Model Registry** > **Model Packages**.

2. Select **Deploy** from the action menu for the model package you wish to deploy.

	![](images/deploy-from-model-registry.png)

3. Add [deployment information and create the deployment](add-deploy-info).


###  Use shared modeling workers {: #use-shared-modeling-workers }

If you don't have a dedicated prediction server instance available, you can use a node that shares workers with your model building activities. In this case, the page has a different interface.

![](images/deploy-5.png)

Click **Show Example** to generate and display a usage example:

![](images/deploy-6.png)

When using the sample code, specify your [API key](api-key-mgmt) (1). The project and model IDs (2) are available in the sample, as is the shared instance endpoint (3). The DataRobot Python client uses the API key for authentication and so no key or username is required. To execute the file, follow the instructions in the commented section of the snippet.
---
title: Version 7.0.x
description: DataRobot Release 7.0 release announcements index page.
---

# Version 7.0.x {: #version-70x }

Following are the AutoML, time series, and MLOps announcements for the features that make up DataRobot's 7.0.x releases, first released _March 15, 2021_.

* [v7.0.x AutoML](v7.0.0-aml)
* [v7.0.x time series](v7.0.0-ats)
* [v7.0.x MLOps](v7.0-mlops)
* [Maintenance release notes](7.0-maintenance/index) for DataRobot v7.0.x.
---
title: Get help
description: This help section provides basic account access troubleshooting and quick, task-based instructions for success in modeling.

---

# Get help {: #get-help }

## Troubleshooting {: #troubleshooting }

This section provides information on troubleshooting DataRobot authentication and access:

Section | Describes how to...
--------|--------------------
[Trial FAQ](trial-faq) | Questions and answers about the DataRobot Self-Service SaaS trial.
[Signing in](signin-help) | Things to try if you are having issues signing in.
[Two-factor authentication](2fa) | Common issues when working with 2FA.
[Check platform status](status-help) | View and subscribe to platform status announcements.
[Number of workers](workers-help) | Learn about increasing the worker count.
[Troubleshooting the Python client](py-help) | Review cases that can cause issues with using the Python client and known fixes

## Tutorials {: #tutorials }

DataRobot offers a variety of tutorials to assist you in using different aspects of the application, outlined below:

Section | Describes how to...
--------|--------------------
[Prepare learning data](prep-learning-data/index) | Assemble, curate, and prepare data for use with DataRobot.
[Create AI models](creating-ai-models/index) | Execute common tasks related to modeling and DataRobot projects.
[Explore AI insights](explore-ai-insights/index) | Analyze model-specific insights generated by DataRobot.
---
title: ACE score and row order
description: Does DataRobot's ACE score, a univariate measure of correlation, depend on row order?
---

# ACE score and row order {: #ace-score-and-row-order }

!!! faq "What is ACE?"
	ACE scores (Alternating Conditional Expectations) are a univariate measure of correlation between the feature and the target. ACE scores detect non-linear relationships, but as they are univariate, they do not detect interaction effects.


<span style="color:red;font-size: 1rem"> `Robot 1`</span>

**Does ACE score depend on the order of rows?**

Is there any sampling done in EDA2 when calculating ACE score? The case is two datasets which are only different in the order of rows, are run separately with same OTV setting (same date range for partitions, same number of rows in each partition), and there is a visible difference in the ACE scores. Does the ACE score depend on the order of rows? 

<span style="color:red;font-size: 1rem"> `Robot 2`</span>


EDA1 sample will vary based on order of rows for sure. EDA2 starts with EDA1 and then removes rows that are in the holdout too, so project settings can also matter.

<span style="color:red;font-size: 1rem"> `Robot 1`</span>

There are 8k rows and 70 features, fairly small datasets.

<span style="color:red;font-size: 1rem"> `Robot 2`</span>

ACE doesn’t need a large sample: It could be 1k or even 100. If the dataset is less than 500MB, then all rows may be in the sample, but the order may be different.
---
title: Data connection UI improvements
description: Set up data connections in DataRobot using a more intuitive workflow.
section_name: Data
maturity: public-preview
---

# Data connection UI improvements {: #data-connection-ui-improvements }

!!! info "Availability information"
    The new data connection page is off by default. Contact your DataRobot representative or administrator for information on enabling the feature.

    <b>Feature flag:</b> Enable New Data Connection UI

Now available for public preview, DataRobot introduces improvements to the data connection user interface that simplifies the process of adding and configuring data connections. Instead of opening multiple windows to set up a data connection, after selecting a data store, you can configure parameters and authenticate credentials in the same window. For each data connection, only the required fields are displayed, however, you can define additional parameters under **Advanced Options** at the bottom of the page.

1. To access the new data connections page, go to the **AI Catalog**.
2. Click **Add to Catalog** and select **Data Connection**.

    ![](images/connect-ui-1.png)

3. All existing connections are displayed on the left. If you select a configured connection, its configuration options are displayed in the center.

    ![](images/connect-ui-5.png)

4. To add a new data connection, click **Add new connection**.

    ![](images/connect-ui-2.png)

5. Select a connection type.

    ![](images/connect-ui-3.png)

6.  Name the data connection (1), select an authentication method (2), and fill in the required fields (see the documentation for your specific data store).

    Note that the visible configuration options are the required parameters for the selected data store, therefore, these options vary for each data store. You can add more parameters under **Show advanced options** (3). <!--is this correct? it didn't seem to be.-->

    ![](images/connect-ui-4.png)

    ??? note "Saved credentials"
        If you previously added credentials for your datastore via the [**Credentials Management**](stored-creds#credentials-management) page, you can click **Select saved credentials** and choose them from the list instead of adding them manually.

7. Click **Add from connection**; once selected, the **Schema** tab opens.
8. The **Schema** tab lists the available schemas for your database&mdash;select a schema from the list. Once selected, the **Tables** tab opens.

    ![](images/connect-ui-6.png)

    To use a SQL query to select specific elements of the named database, click the [**SQL query**](catalog#use-a-sql-query) tab.


9. Select the table(s) you want to register in the AI Catalog and click **Proceed to confirmation**. Each table will be registered as a separate catalog asset.

    ![](images/connect-ui-7.png)

9. Under **Settings**, select the appropriate policy (1) and data upload amount (2), then review and confirm your selections by clicking **Register in the AI Catalog**.

    ![](images/connect-ui-8.png)

View the existing workflow procedure in the [AI Catalog documentation](catalog#add-data-from-external-connections).
# Version 8.0.16 {: #version-8016 }

_June 26, 2023_

The DataRobot v8.0.16 release includes some fixed issues in the DataRobot Self-Managed AI Platform. See the v8.0.0 release notes for:

* [Features introduced in v8.0.0](v8.0.0-aml)

In addition to the issues listed below, DataRobot addressed several outstanding security issues. This release/update is part of DataRobot's continual effort to secure the product and platform.

## Issues fixed in v8.0.16 {: #issues-fixed-in-v8016 }

The following issues have been fixed since release v8.0.15:

### MLOps {: #mlops }

- MMM-13255: Fixes the calculation of "Accuracy" and "Balanced accuracy" metrics on the model management **Accuracy** tab for models with a threshold other than 0.5.

### Platform {: #platform }

- PY3-4137: Fixes an issue for Python 2 projects that were not shown as deprecated after the cluster upgrade to 8.0. 

### Predictions {: #predictions }

- PRED-8856: Fixes an issue when writing predictions using JDBC for time-series segmented models.

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: Use Gramian angular fields to improve datasets
description: Generate advanced features used for high frequency data use cases.
---

# Use Gramian angular fields to improve datasets {: #use-gramian-angular-fields-to-improve-datasets }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/game-changer/high_freq_data_to_images/high_frequency_data_classification_using_gramian_angular_fields.ipynb){ .md-button }

Prerequisites: [PYTS library](https://pyts.readthedocs.io/){ target=_blank }

Traditional feature engineering methods like time aware aggregation and spectrograms can have limitations. Spectrograms cannot capture correlations between each segment of the signal with other segments of the signal. If you try to do this with tabular aggregates it becomes a high dimensionality problem.

Gramian Angular Field images of signal data can solve the above problem using a matrix which can be used with computer vision models easily without the limitations of dimensionality.
{% if section(page, "dr-notebooks") %}

These pages outline the coding experience when using DataRobot notebooks. Learn about common cell actions, keyboard shortcuts, and how to run notebooks.

Topic | Describes...
----- | ------
[Environment management](dr-env-nb) | How to configure and start the notebook environment.
[Create and execute cells](dr-cell-nb) | How to create and execute code and Markdown cells in a notebook.
[Cell actions](dr-action-nb) | The actions and keyboard shortcuts available in a notebook.
[Code intelligence](dr-code-int) | The code intelligence features provided throughout the notebook coding experience.
[Notebook terminals](dr-terminal-nb) | Use integrated terminals to execute commands such as .py scripts or package installations.
[Azure OpenAI Service integration](dr-openai-nb) | Leverage Azure's OpenAI Code Assistant to generate code in DataRobot Notebooks using ChatGPT.

{% else %}

These pages outline the coding experience when using DataRobot notebooks. Learn about common cell actions, keyboard shortcuts, and how to run notebooks.

Topic | Describes...
----- | ------
[Environment management](wb-env-nb) | How to configure and start the notebook environment.
[Create and execute cells](wb-cell-nb) | How to create and execute code and Markdown cells in a notebook.
[Cell actions](wb-action-nb) | The actions and keyboard shortcuts available in a notebook.
[Code intelligence](wb-code-int) | The code intelligence features provided throughout the notebook coding experience.
[Notebook terminals](wb-terminal-nb) | Use integrated terminals to execute commands such as .py scripts or package installations.
[Azure OpenAI Service integration](wb-openai-nb) | Leverage Azure's OpenAI Code Assistant to generate code in DataRobot Notebooks using ChatGPT.

{% endif %}
---
title: Feature Discovery support in No-Code AI Apps
description: Create No-Code AI Apps from Feature Discovery projects.
section_name: Apps
maturity: public-preview
platform: cloud-only
---

# Feature Discovery support in No-Code AI Apps {: #feature-discovery-support-in-no-code-ai-apps }

!!! info "Availability information"
    Feature Discovery support in No-Code AI Apps is off by default. Contact your DataRobot representative or administrator for information on enabling the feature.

    <b>Feature flags:</b>

    - Enable Application Builder Feature Discovery Support
    - Enable Feature Cache for Feature Discovery


Now available for public preview, you can create No-Code AI Apps from Feature Discovery projects (i.e., projects built with multiple datasets) with feature cache enabled. [Feature cache](safer-ft-cache) instructs DataRobot to source data from multiple datasets and generate new features in advance, storing this information in a "cache," which is then drawn from to make predictions.

Before [creating an application](create-app), ensure the deployment has feature cache enabled in the deployment's **Settings** tab if the project contains multiple datasets.

![](images/ft-cache-2.png)

Once created, you can [use the app](use-apps/index) to build simulations and charts, run optimizations, and create what-if scenarios.

## Considerations {: #considerations }

Consider the following when enabling Feature Discovery support for No-Code AI Apps:

- Postgres DB must be installed for feature cache to work properly.
- Feature cache must be enabled to make single record predictions.
- Feature cache only caches features for the most recent period.