DataRobot-Research/drdocs huggingface dataset:
---
title: Import for Keras or TF
description: Use DataRobot custom models to import custom inference models.
---

# Import for Keras or TF {: #import-for-keras-or-tf }

<span style="color:red;font-size: 1rem"> `Robot 1`</span>

**Is there a way to import .tf or .keras models?**

<span style="color:red;font-size: 1rem"> `Robot 2`</span>

We don’t allow any form of model import (.pmml, .tf, .keras, .h5, .json etc.), except for [custom models](custom-inf-model). That said, you can use custom models to do _whatever you want_, including _importing whatever you wish_.


<span style="color:red;font-size: 1rem"> `Robot 1`</span>

I have a customer trying to do Custom Inference. Can he use this or only .h5? I don't really understand the tradeoff of one version of the model objects versus another. I've only used .h5 and JSON. I think he was also curious if we have any support for importing JSON weights.

<span style="color:red;font-size: 1rem"> `Robot 2`</span>

We do not support importing model files, except for custom models&mdash;he'll need to write a custom inference model to load the file and score data.

But yes, we support custom inference models. You can do literally whatever you want in custom inference models.


<span style="color:red;font-size: 1rem"> `Robot 1`</span>

Thanks, sorry if I'm being dense / confused&mdash;so with Custom Inference he should be able to load the .pb, .tf, .keras files?

<span style="color:red;font-size: 1rem"> `Robot 2`</span>

Yes. He will need to write his own Python code. So if he can write a Python script to load the .pd, .tf, .keras, or .whatever file and score data with it, he can make that script a custom inference model.


<span style="color:red;font-size: 1rem"> `Robot 1`</span>

Ohhh of course :) now I understand. Duh, Robot 1. Thanks!
??? faq "Drift metric support"
    <span id="drift-metric-support">While the DataRobot UI only supports the Population Stability Index (PSI) metric, the API supports Kullback-Leibler Divergence, Hellinger Distance, Kolmogorov-Smirnov, Histogram Intersection, Wasserstein Distance, and Jensen–Shannon Divergence. In addition, using the Python API client, you can [retrieve a list of supported metrics](https://datarobot-public-api-client.readthedocs-hosted.com/reference/mlops/deployment.html#data-drift){ target=_blank }.</span>
The **Histogram** chart is the default display for numeric features. It "buckets" numeric feature values into equal-sized ranges to show frequency distribution of the variable&mdash;the target observation (left Y-axis) plotted against the frequency of the value (X-axis). The height of each bar represents the number of rows with values in that range.

??? note "Histogram display variations"
    The display differs depending on whether the [data quality](data-quality#interpret-the-histogram-tab) issue "Outliers" was found.

    Without data quality issues:

    ![](images/histogram.png)

    With data quality issues:

    ![](images/histogram-outlier.png)

Initially, the display shows the bucketed data:

![](images/dq-8.png)

Select the **Show outliers** checkbox to calculate and display outliers:

![](images/dq-10.png)

The traditional box plot above the chart (shown in gold) highlights the middle quartiles for the data to help you determine whether the distribution is skewed. To determine whisker length, DataRobot uses [Ueda's algorithm](https://jsdajournal.springeropen.com/articles/10.1186/s40488-015-0031-y){ target=_blank } to identify the outlier points&mdash;the whiskers depict the full range for the lowest and highest data points in the dataset excluding those outliers. This is useful for helping to determine whether a distribution is skewed and/or whether the dataset contains a problematic number of outliers.

Note the change in the X-axis scale and compression of the box plot to allow for outlier display. Because there tend to be fewer rows recording an outlier value (it's what makes them outliers), the blue bar may not display. Hover on that column to display a tooltip with the actual row count.

After EDA2 completes, the histogram also displays an [average target value](histogram#average-target-values) overlay.
## DRUM on Mac {: #drum-on-mac }

The following instructions describe installing DRUM with `conda` (although you can use other tools if you prefer) and then using DRUM to test a task locally. Before you begin, DRUM requires:

* An installation of [`conda`](https://docs.conda.io/en/latest/miniconda.html){ target=_blank }.

* A Python environment (also required for R) of 3.7+.

### Install DRUM on Mac {: #install-drum-mac }

1. Create and activate a virtual environment with Python 3.7+. In the terminal for 3.8, run:

	``` sh
	conda create -n DR-custom-tasks python=3.8 -y
	conda activate DR-custom-tasks
	```

2. Install DRUM:

	``` sh
	conda install -c conda-forge uwsgi -y
	pip install datarobot-drum
	```

3. To set up the environment, install [Docker Desktop](https://www.docker.com/products/docker-desktop){ target=_blank } and download from GitHub the DataRobot [drop-in environments](https://github.com/datarobot/datarobot-user-models/tree/master/public_dropin_environments){ target=_blank } where your tasks will run. This recommended procedure ensures that your tasks run in the same environment both locally and inside DataRobot.

	Alternatively, if you plan to run your tasks in a local `python` environment, install packages used by your custom task into the same environment as DRUM.
---
title: Generate Java models in an existing project
description: Retrain legacy models for which you want to download Scoring Code.

---

# Generate Java models in an existing project {: #generate-java-models-in-an-existing-project }

If you have projects that were created before the Scoring Code feature was enabled for your organization, you must retrain the models for which you want to download code. You do not need to recreate the entire project.

To retrain a model:

1. Click the checkbox at the left of the model to select it. Note the blueprint number (BPxx) as you will need this information later.

2. From the dropdown menu, select **Delete**.

    ![](images/scorecode-2.png)

3. Open the [**Repository**](repository) and search for a model with same blueprint number. Check the box to the left of the model to select it.

4. Set the [values](repository#create-a-new-model) and click the **Run task(s)** button.

5. When the model has finished training, return to the Leaderboard and enter the blueprint number in the search field.

6. Expand the model version with the SCORING CODE ![](images/icon-scorecode.png) tag, navigate to **Predict > Downloads**, select a download option, and click **Download** to access the scoring code.

!!! note
    A retrained model may have slightly different predictions than the original model due to the nature of the parameter initialization process used by machine learning algorithms.
# Microsoft SQL Server {: #microsoft-sql-server }

## Supported authentication {: #supported-authentication }

- Username/password

## Prerequisites {: #prerequisites }

The following is required before connecting to Microsoft SQL Server in DataRobot:

- Microsoft SQL account

## Required parameters {: #required-parameters }

The table below lists the minimum required fields to establish a connection with Microsoft SQL Server:

Required field | Description | Documentation
--------------- | ---------- |  -----------
`address`       | The service endpoint used to connect to SQL Server.<br><br>**Example:**<br> `jdbc-cert-ms-sql-server-2016.cqz9ezetwbf4.us-east-1.rds.amazonaws.com:1489`  |    [Microsoft documentation](https://docs.microsoft.com/en-us/sql/connect/jdbc/using-the-jdbc-driver){ target=_blank }

{% include 'includes/data-conn-trouble.md' %}
!!! note "Time of Prediction" 
    The <span id="time-of-prediction">Time of Prediction</span> value differs between the [Data Drift](data-drift) and [Accuracy](deploy-accuracy) tabs and the [Service Health](service-health) tab:
    
    * On the Service Health tab, the "time of prediction request" is _always_ the time the prediction server _received_ the prediction request. This method of prediction request tracking accurately represents the prediction service's health for diagnostic purposes. 
    
    * On the Data Drift and Accuracy tabs, the "time of prediction request" is, _by default_, the time you _submitted_ the prediction request, which you can override with the prediction timestamp in the [Prediction History](add-deploy-info#prediction-history-and-service-health) settings.
---
title: Predict steel plate defects
description: Help manufacturers significantly improve the efficiency and effectiveness of identifying defects of all kinds, including those for steel sheets.

---

# Predict steel plate defects {: #predict-steel-plate-defects }

Steel plates, whether used for general construction, industrial applications, or highly critical applications, must be of a structural quality to ensure safety and provide good formability and high strength. Every steel plate produced has some level of defects and faults. Normally, plates that don’t pass muster are scrapped, but not until the end of an expensive testing process. The time, money, and expertise spent on unusable product lowers the margins on each plate and adds complications to the manufacturing process.

## Business problem {: #business-problem }

Traditionally, each plate (or a representative sample) runs through a gauntlet of tests designed to reveal faults. The goal in terms of efficiency and cost is to accurately identify defective steel plates as early in the manufacturing process as possible, saving time and expense. Predictive modeling leverages data from that process, allowing you to target tests and identify faults in steel plates as early as possible. With good, accurate models, a plant can scrap bad plates earlier for higher efficiency.

The challenge, however, doesn’t end there. Plants make constant adjustments and improvements to their processes. If the predictive models aren’t updated with new data, they will quickly become outdated and suffer from a degradation in accuracy. It’s necessary to set up retraining processes that automatically identify the best model available based on new information.

Consider Michelle, who is responsible for maintaining the quality steel plates her company produces. She knows the ins and outs of the production process, like the temperature-to-pressure ranges needed to ensure pristine results. Automated machine learning offers a new way to build upon her expertise and leverage the data her team collects. Instead of worrying about hiring a data scientist to support her team, she and the other engineers can take their data and build their own models. Those models can then predict which steel plates will have the highest likelihood of faults and also identify the reasons that may be driving those defective results.

## Use case data {: #use-case-data }

This notebook uses a [dataset](http://archive.ics.uci.edu/ml/datasets/steel+plates+faults) (UCI Machine Learning Repository: Steel Plates Faults dataset) provided by Semeion, Research Center of Sciences of Communication, Via Sersale 117, 00128, Rome, Italy.

Before proceeding, review the [API quickstart guide](https://docs.datarobot.com/en/docs/api/api-quickstart/index.html) to ensure that your client and API credentials have been properly configured.

* [Download training data](https://s3.amazonaws.com/datarobot-use-case-datasets/steel_plates_fault_training.csv)
* [Download prediction data](https://s3.amazonaws.com/datarobot-use-case-datasets/steel_plates_fault_testing.csv)
# Version 7.2.6 {: #version-726 }

_November 23, 2021_

The DataRobot v7.2.6 release does not include any customer-impacting issues. See the v7.2.0 release notes for:

* [Features introduced in v7.2.0](v7.2.0-aml)

## Updated language localization {: #updated-language-localization }

Localization of the documentation has been updated to include Japanese content for all new 7.2 features.

To change languages, open your [user settings](getting-help) and select a language for your session. The selection remains until you reset it.

![](images/languages-set-pt-jp.png)

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: Time series modeling data
description: This topic describes the creation and management of the modeling dataset that is a result of the feature derivation process.

---

# Time series modeling data {: #time-series-modeling-data }

This topic describes the creation and management of the modeling dataset that is a result of the feature derivation process.

Topic | Describes...
----- | ------
[Create the modeling dataset](ts-create-data) | How DataRobot's feature derivation process creates a new modeling dataset for time series projects.
[Data prep for time series](ts-data-prep) | Using the time series data prep tool to correct data quality time step issues. 
[Restore features removed by reduction](restore-features) | Adding features back into your available derived modeling data after running EDA2.
---
title: AI Platform releases
description: A monthly record of the new Public Preview and GA features announced for DataRobot's managed AI Platform.

---

# AI Platform releases {: #ai-platform-releases }

A monthly record of the new Public Preview and GA features announced for DataRobot's managed AI Platform. Deprecation announcements are also included and link to deprecation guides, as appropriate.

* [May 2023 release announcements](may2023-announce)
* [April 2023 release announcements](april2023-announce)
* [March 2023 release announcements](march2023-announce)
* [February 2023 release announcements](february2023-announce)
* [January 2023 release announcements](january2023-announce)
* [November 2022 release announcements](november2022-announce)
* [October 2022 release announcements](october2022-announce)
* [September 2022 release announcements](sept2022-announce)
* [August 2022 release announcements](august2022-announce)
* [July 2022 release announcements](july2022-announce)
* [June 2022 release announcements](june2022-announce)
* [May 2022 release announcements](may2022-announce)

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
# SAP HANA {: #sap-hana }

## Supported authentication {: #supported-authentication }

- Username/password

## Prerequisites {: #prerequisites }

The following is required before connecting to SAP HANA in DataRobot:

- SAP HANA account

## Required parameters {: #required-parameters }

The table below lists the minimum required fields to establish a connection with SAP HANA:

Required field | Description |  Documentation
--------------- | ---------- | -----------
`address`       | The server's address used to connect to SAP HANA.  | [SAP documentation](https://help.sap.com/docs/SAP_HANA_PLATFORM/0eec0d68141541d1b07893a39944924e/b250e7fef8614ea0a0973d58eb73bda8.html?version=2.0.03)

For more information, see the [SAP HANA connector documentation](https://help.sap.com/docs/SAP_HANA_CLIENT/f1b440ded6144a54ada97ff95dac7adf/109397c2206a4ab2a5386d494f4cf75e.html).

{% include 'includes/data-conn-trouble.md' %}
---
title: Custom RBAC roles
description: Custom role-based access control (RBAC) roles are a solution for organizations with use cases that are not addressed by default roles in DataRobot.
section_name: Administrator
platform: cloud-only

---

# Custom RBAC roles {: #custom-rbac-roles }

Custom role-based access control (RBAC) roles are a solution for organizations with use cases that are not addressed by [default roles](rbac-ref) in DataRobot. System and organization administrators can create roles and define access at a more granular level, and assign them to users and groups. You can access custom RBAC roles from **User Settings** > **User Roles**.

The **User Roles** page lists each available role an admin can assign to a user in their organization, including DataRobot default roles. Before creating a custom role, review the [considerations](#feature-considerations).

![](images/custom-rbac-1.png)

## Create custom roles {: #create-custom-roles }

To create a new role, click **+ Add a user role**. Enter a name for the role and assign permissions.

DataRobot features are listed on the left, and role access options&mdash;[read, write, admin](roles-permissions#role-definitions), and entity-specific permissions&mdash;are listed on the right. Select an object, then select an access level for the object under _Permissions_.

![](images/custom-rbac-2.png)

## Manage roles {: #manage-roles }

Unlike default DataRobot roles, you can edit and delete any custom role created for an organization from the **User Roles** page. To determine if a role can be modified, refer to the **Custom Role** column; default roles display _Yes_ and custom roles display _No_.

![](images/custom-rbac-3.png)

From this page, you can:

| &nbsp; | Element       | Description |
| ---------- | ----------- | ---------- |
| ![](images/icon-1.png) | Edit      | Select a custom user role to modify its permissions.        |
| ![](images/icon-2.png) | Delete      | Click the **Menu** icon and select **Delete**. If the role is currently assigned to a user, a warning message appears before deleting the role.        |
| ![](images/icon-3.png) | Duplicate      | Click the duplicate icon and name the new role, then select the role to modify permissions.        |

## Feature considerations {: #feature-considerations }

* The RBAC feature flag must be enabled.
* When system administrators create a custom role, the role is enabled system-wide across all organizations. When organization administrators create a custom role, the role is only enabled for that organization.
* The feature flag for custom roles must be enabled system- or org-wide.
* Feature flags cannot be configured through roles.
* Multiple roles cannot be assigned to a user/group/organization at this time.
---
title:  Model Info
description: To view the Model Info tab, click a model on the Leaderboard, then click Model Info. The tab’s tiles report general model and performance information.

---

# Model Info {: #model-info }

To display model information, click a model on the Leaderboard list then click **Model Info**. The tab provides tiles that report general model and performance information.

![](images/model-info.png)

For time series projects, the output also includes backtesting information, including execution time for each backtest.

![](images/model-info-1.png)

The backtest summaries show partitioning against the full date range:

* Date ranges the model is trained on (blue)
* Validation (green) and Holdout (red) partitions
* Any configured gaps (yellow)

Training periods can be changed by clicking the plus sign next to a model on the [Leaderboard](otv#change-the-training-period).

If a model uses specified start and end dates note that:

* Insights on validation and holdout sets are only available if the model was not trained into those sets (if data is out-of-sample).
* If _any_ partitions remain out-of-sample for the model, insights are provided for that partition.
* If any partition is wholly or partially in the training period for the model, insights are _not_ provided for that partition.

##  Model File Size {: #model-file-size }

Model File Size reports the sum total of the cache files DataRobot uses to store the model data. It's generated from an internal storage mechanism and indicates your system footprint, which can be especially useful for Self-Managed AI Platform deployments.

##  Prediction Time {: #prediction-time }

Displays the estimated time, in seconds, to score 1000 rows of the dataset.

##  Sample Size {: #sample-size }

Sample Size reports the number of observations used to train and validate the model (and also for each cross-validation repetition, if applicable). When [smart downsampling](smart-ds) is in play or DataRobot has downsampled the project, Sample Size reports the number of rows in the minority class rather than the total number of rows used to train the model.
---
title: Version 9.0.x
description: DataRobot Release 9.0 release announcements index page.

---

# Version 9.0.x {: #version-90x }

_March 29, 2023_

View feature announcements for data, AutoML and time series (AutoTS) modeling, MLOps, and platform changes and enhancements for version 9.0.x of DataRobot's Self-Managed AI Platform deployment option.

* [Data](90-data)
* [Time series](90-autots)
* [MLOps](90-mlops)
* [Platform](90-platform)
* [Maintenance release notes](9.0-maintenance/index) for DataRobot v9.0.x.


### In the spotlight {: #in-the-spotlight }

This release brings DataRobot’s new user interface&mdash;Workbench. Workbench is an intuitive, guided, machine learning workflow focused on experimentation and iteration. It lets you easily move from raw data to prepared, partitioned data that’s ready for modeling. Then, as with DataRobot Classic, you can build many models at once and generate value quickly through key insights and predictions.

![](images/wb-uc-2.png)

The Workbench user interface lets you group, organize, and share your modeling assets to better leverage DataRobot for enhanced experimentation. These assets&mdash;datasets, experiments, notebooks, and no-code apps&mdash;are housed within folder-like containers known as Use Cases.

![](images/wb-conceptual.png)

Workbench is a public preview feature, but is on by default and enabled for all users. The notebooks component, also public preview, is not enabled by default (contact your DataRobot representative or administrator for information on enabling them).
---
title: End-to-end ML workflow with AWS
description: Work with AWS and DataRobot's Python client to import data, build and evaluate models, and deploy a model into production to make new predictions.

---

# End-to-end ML workflow with AWS {: #end-to-end-ml-workflow-with-aws}

Being one of the largest cloud providers in the world, AWS has multiple ways of storing data within its cloud.

You can use either of two AI accelerators that allow you to source data from S3 or Athena, build and evaluate a model using DataRobot, and send predictions from that model back to S3.

[Access the AI accelerator for S3 on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/end-to-end/AWS_End_to_End/Amazon_S3_End_to_End.ipynb){ .md-button }

[Access the AI accelerator for AWS Athena on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/end-to-end/AWS_End_to_End/AWS_Athena_End_to_End.ipynb){ .md-button }

Each AI accelerator will perform the following steps to help you integrate DataRobot with your data in AWS:

* **Import data for training**:

    * In the S3 AI accelerator, you will be able to take data in the parquet file format, assemble it, and upload it to DataRobot's AI Catalog.

    * In the Athena AI Accelerator, you will create a JDBC data source within DataRobot to connect to Athena and then pull data in via a SQL query.

* Using the DataRobot Python API, you will have DataRobot build up to 50 different machine learning models while also evaluating how those models perform on this dataset.
---
title: AWS Lambda
description: Integrate DataRobot with AWS Lambda.

---

# AWS Lambda {: #aws-lambda }

The sections described below provide techniques for integrating AWS Lambda with DataRobot.

Topic | Describes...
----- | ------
[Serverless MLOps agents](monitor-serverless-mlops-agents) | Monitoring external models with serverless MLOps agents running on AWS Lambda.
[AWS Lambda reporting to MLOps](aws-lambda-reporting-to-mlops) | AWS Lambda serverless reporting of actuals to DataRobot MLOps.
[Use DataRobot Prime models with AWS Lambda](prime-lambda) | Using DataRobot Prime models with AWS Lambda.
[Use Scoring Code with AWS Lambda](sc-lambda) | Making predictions using Scoring Code deployed on AWS Lambda.