DataRobot-Research/drdocs huggingface dataset:
---
title: N-grams and prediction confidence
description: Which DataRobot tools help understand n-gram predictions?
---

# N-grams and prediction confidence {: #n-grams-and-prediction-confidence }

<span style="color:red;font-size: 1rem"> `Robot 1`</span>

**Customer question: how do we know which words/n-grams increase confidence in the predictions?**

<span style="color:red;font-size: 1rem"> `Robot 2`</span>

The [Word Cloud](word-cloud) is useful for this!


<span style="color:red;font-size: 1rem"> `Robot 3`</span>

You can also look at the [coefficients](coefficients) directly for any linear model with n-grams.


<span style="color:red;font-size: 1rem"> `Robot 2`</span>

Finally we’ve got [Prediction Explanations](predex-text) for text features.

  
<span style="color:red;font-size: 1rem"> `Robot 1`</span>

Thanks for the response.  I love these and they make sense. I recommended word cloud but she indicated that that was intensity not confidence (to me they are highly related).


<span style="color:red;font-size: 1rem"> `Robot 2`</span>

Our linear models are regularized so low confidence words will be dropped.
---
title: Work with data (Classic)
description: An overview of the tools DataRobot Classic provides for importing, preparing, and managing data for machine learning.
---

# Work with data (Classic) {: #work-with-data-classic }

DataRobot knows that high-quality data is integral to the ML workflow—from importing and cleaning data to transforming and engineering features, from scoring with prediction datasets to deploying on a prediction server—data is critical. DataRobot provides tools to help you seamlessly and securely interact with your data. 

**Import data from various sources, including from external data sources** to minimize data movement and control data governance across your cloud data warehouses and lakes.

![](images/gs-data-1.png)

**Explore patterns and insights** in your data; automate the discovery, testing, and creation of hundreds of valuable new features.

![](images/gs-data-2.png)

## 1: Import data {: #import-data }

Import data into the DataRobot platform from the [AI Catalog](catalog), directly from a [connected data source](data-conn), or as a [local file](import-to-dr).

![](images/overview-data.png)

??? tip "Learn more"
    To learn more about the topics discussed in this section, see:

    - [File size requirements](file-types)
    - [Import data documentation](import-data/index)

## 2: Explore data {: #explore-data }

After importing your data, DataRobot performs [exploratory data analysis](eda-explained), a process that analyzes the datasets, summarizes their main characteristics, and [automatically creates feature transformations](auto-transform) &mdash;the results of which are displayed on the [Data page](histogram) of your project.

Once EDA1 completes, you can use the [Data Quality Assessment](data-quality) to find and address quality issues surfaced in your dataset.

![](images/dq-3.png)

??? tip "Learn more"
    To learn more about the topics discussed in this section, see:

    - [EDA Explained](eda-explained)
    - [View the results of EDA1](histogram)
    - [Data Quality Assessment](data-quality)

## 3: Prepare data {: #prepare-data }

Now that you've explored your dataset and identified areas for improvement, you can: 

Perform [manual feature transformations](feature-transforms). 

![](images/create-derived-feature6.png)

[Prepare your data using Spark SQL](spark).

![](images/catalog-2.png)

[Add secondary datasets and then define those relationships to the primary in Feature Discovery projects](fd-overview).

![](images/safer-1.png)
    
??? tip "Learn more"
    To learn more about the topics discussed in this section, see:

    - [Manual feature transformations](feature-transforms)
    - [Prepare your data with Spark SQL](spark)
    - [Configure a Feature Discovery project](fd-overview)

## Next steps {: #next-steps }

Now that your data is where it needs to be, you're ready to start [modeling](gs-model).
---
title: Ping health check
description: Health check to determine if the service is "alive".

---

# Ping health check {: #ping-health-check }

**Endpoint:** `/ping`

Health check to determine if the service is "alive".

**Request Method:** `GET`

**Request URL:** deployed URL, example: `https://your-company.orm.datarobot.com/predApi/v1.0`

###  Request parameters {: #request-parameters }

None required.

###  Response 200 {: #response-200 }

|   Data   |  Type  | Example(s) |
|----------|--------|------------|
| response | string | pong       |
---
title: Prediction Explanations
description: Index page for SHAP, XEMP, and Text Prediction Explanations.

---

# Prediction Explanations {: #prediction-explanations }

The following sections provide an overview and describe the alternate methodologies for working with Prediction Explanations:


Topic | Describes...
----- | ------
[Prediction Explanations overview](predex-overview) | Describes the SHAP and XEMP methodologies, including benefits and tradeoffs.
[SHAP Prediction Explanations](shap-pe) | Describes how to work with SHAP-based Prediction Explanations.
[XEMP Prediction Explanations](xemp-pe) | Describes how to work with XEMP-based Prediction Explanations.
[Text Prediction Explanations](predex-text) | Helps to interpret the output of text-based explanations.
---
title: Scoring Code JAR integrations
description: How to import DataRobot Scoring Code JARs into external platforms.

---

# Scoring Code JAR integrations {: #scoring-code-jar-integrations }

!!! info "Availability information"
     Contact your DataRobot representative for information on enabling the Scoring Code feature.
     
Although DataRobot provides its own scalable prediction servers that are fully integrated with other platforms, there are several reasons you may decide to deploy Scoring Code on another platform:

* Company policy or governance decision.
* Custom functionality on top of the DataRobot model.
* Low-latency scoring without the API call overhead.
* The ability to integrate models into systems that cannot communicate with the DataRobot API.

To use the Scoring Code, download the JAR from the [Leaderboard](sc-download-leaderboard) or from a [deployment](sc-download-deployment) and import it to the platform of your choice, as described in the following topics.

Topic | Describes...
------|-------------
[Use Scoring Code with Amazon SageMaker](sc-sagemaker) | Importing DataRobot Scoring Code models to SageMaker.
[Use Scoring Code with AWS Lambda](sc-lambda) | Making predictions using Scoring Code deployed on AWS Lambda.
[Use Scoring Code with Azure ML](sc-azureml) | Importing DataRobot Scoring Code models to Azure ML.
[Android Scoring Code integration](android) | Using DataRobot Scoring Code on Android.
[Apache Spark API for Scoring Code](sc-apache-spark) | Using the Spark API to integrate DataRobot Scoring Code JARs into Spark clusters.
[Generate Snowflake UDF Scoring Code](snowflake-sc) | Using the DataRobot Scoring Code JAR as a user-defined function (UDF) on Snowflake.
---
title: Predict factory order quantities for new products
description: Build a model to improve decisions about initial order quantities using future product details and product sketches.

---

# Predict factory order quantities for new products {: #predict-factory-order-quantities-for-new-products }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/advanced-experimentation/Retail_Industry_Predicting_Factory_Orders_New_Products/Retail%20Industry%20-%20Predicting%20Factory%20Order%20Quantities%20for%20New%20Products.ipynb){ .md-button }

Retailers face many decisions when launching new products. One key decision is the amount of product to order from the manufacturer.

Ordering too much wastes working capital and can lead to products being heavily discounted. Ordering too little squanders an opportunity for revenue and may cause customers to purchase other brands.

Getting initial orders quantities right is particularly difficult for luxury products where first year demand for a new purse, a new belt or a new shoe can vary by several orders of magnitude based on factors unrelated to the product specifications.

This notebook illustrates how to build a model to improve decisions about initial order quantities using future product details and product sketches.
---
title: Visual AI
description: This topic introduces the workflow and reference materials for including images as part of your DataRobot project.

---

# Visual AI {: #visual-ai }

These sections describe the workflow and reference materials for including images as part of your DataRobot project.

Topic | Describes...
----- | ------
[Workflow overview](vai-overview) | View a simplified workflow overview to aid understanding.
[Build models](vai-model) | Prepare the dataset, build models, and make predictions.
[Train-time image augmentation](tti-augment/index) | Transform images to create new images to enlarge the dataset.
[Model insights](vai-insights) | Visually assess, understand, and evaluate model performance.
[Tune models](vai-tuning) | Apply advanced tuning to calibrate models.
[Predictions](vai-predictions) | Explore options for making image predictions.
[Visual AI reference](vai-reference/index) | Learn about [technological components](vai-ref) of Visual AI and see a [tuning walkthrough](vai-tuning-guide).

See considerations for working with [Visual AI](vai-model#feature-considerations).
---
title: Azure
description: Integrate DataRobot with Azure cloud services.

---

# Azure {: #azure }

The sections within describe techniques for integrating Azure cloud services with DataRobot:

Topic | Describes...
----- | ------
[Run Batch Prediction jobs from Azure Blob Storage](azure-blob-storage-batch-pred) | Running Batch Prediction jobs from Azure Blob Storage with the Batch Prediction API.
[Deploy and monitor DataRobot models in Azure Kubernetes Service](aks-deploy-and-monitor) | Deploying and monitoring DataRobot models on Azure Kubernetes Services (AKS) to create production scoring pipelines with the Portable Prediction Server (PPS).
[Deploy and monitor Spark models](spark-deploy-and-monitor) | Deploying and monitoring Spark models in DataRobot MLOps with the monitoring agent.
[Deploy and monitor ML.NET models](mlnet-deploy-and-monitor) | Deploying and monitoring ML.NET models in DataRobot MLOps with monitoring agent.
[Use Scoring Code with Azure ML](sc-azureml) | Importing Scoring Code models to Azure ML to make prediction requests with Azure.
---
title: Reduce 30-day readmissions rate
description: Build ML models with the DataRobit UI to reduce the 30-Day readmissions rate by predicting at-risk patients. 
---

# Reduce 30-day readmissions rate {: #reduce-30day-readmissions-rate }

This page outlines a use case to reduce the 30-day hospital readmission rate by predicting at-risk patients. It is captured below as a UI-based walkthrough. It is also available as a [Jupyter notebook](readmission.ipynb) that you can download and execute.

{% include 'includes/hospital-readmit-include.md' %}

### No-Code AI Apps {: #no-code-ai-apps }

Consider building a custom application where stakeholders can interact with the predictions and record the outcomes of the investigation. Once the model is deployed, predictions can be consumed for use in the [decision process](#decision-process). For example, this [No-Code AI App](app-builder/index) is an easily shareable, AI-powered application using a no-code interface:

![](images/biz-readmit-1.png)

![](images/biz-readmit-2.png)

### Notebook demo {:#notebook-demo}

See the notebook version of this accelerator [here](loan-default-nb.ipynb).
---
title: Force delete deployments
description: Delete a deployment without waiting for the resolution of the deployment deletion request sent to the management agent.

---

# Force delete management agent deployments

If the management agent is not running or has errored, you can delete a deployment without waiting for the resolution of the deployment deletion request sent to the management agent.

!!! warning
    This will remove the deployment from the deployments area for all users. This action cannot be undone.

To force the deletion of a management agent deployment without waiting for the resolution of the deletion request sent to the agent, take the following steps:

1. On the [**Deployments**](deploy-inventory) page or any tab within a deployment, next to the name of the deployment you want to delete, click the [Actions menu](actions-menu) and click **Delete**.

    ![](images/mgmt-agent-delete.png)

2. In the **Delete deployment** dialog box, click **Ignore Management Agent**, and then click **Delete deployment**.

    ![](images/mgmt-agent-delete-confirm.png)
??? faq "How does DataRobot track drift?"
    For data drift, DataRobot tracks:

    * **Target drift**: DataRobot stores statistics about predictions to monitor how the distribution and values of the target change over time. As a baseline for comparing target distributions, DataRobot uses the distribution of predictions on the holdout.

    * **Feature drift**: DataRobot stores statistics about predictions to monitor how distributions and values of features change over time. As a baseline for comparing distributions of features:

        * For training datasets larger than 500 MB, DataRobot uses the distribution of a random sample of the training data.

        * For training datasets smaller than 500 MB, DataRobot uses the distribution of 100% of the training data.
---
title: Manage custom models
description: How to use the Actions menu, which lets you share and delete custom models and environments.

---

# Manage custom models {: #manage-custom-models }

There are several **Actions** available from the menu on the **Model Registry** > **Custom Model Workshop** page, such as [sharing](#share) and [deleting](#delete-a-deployment) custom models or environments.

##  Share {: #share }

The sharing capability allows [appropriate user roles](roles-permissions#custom-model-and-environment-roles) to grant permissions on a custom model or environment. This is useful, for example, for allowing others to use your models and environments without requiring them to have the expertise to create them.

When you have created a custom model or environment and are ready to share it with others, open the action menu to the right of the **Created** header and select **Share** (![](images/icon-share.png)).

![](images/custom-share-1.png)

This takes you to the sharing modal, which lists each associated user and their role. To remove a user, click the X button to the right of their role.

![](images/custom-share-2.png)

To re-assign a user's role, click on the assigned role and assign a new one from the dropdown.

![](images/custom-share-3.png)

To add a new user, enter their username in the **Share With** field and choose their role from the dropdown. Then click **Share**.

![](images/custom-share-6.png)

This action initiates an email notification.

##  Delete a model or environment {: #delete-a-model-or-environment }

If you have the appropriate [permissions](roles-permissions#custom-model-and-environment-roles), you can delete a custom model or environment from the Model Registry by clicking the trash can icon ![](images/icon-delete.png). This action initiates an email notification to all users with sharing privileges for the model or environment.

![](images/custom-share-5.png)
---
title: Feature Reduction with FIRE
description: Learn about the benefits of Feature Importance Rank Ensembling (FIRE)&mdash;a method of advanced feature selection that uses a median rank aggregation of feature impacts across several models created during a run of Autopilot.
---

# Feature Reduction with FIRE {: #feature-reduction-with-fire}

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/advanced-experimentation/feature_reduction_with_fire/feature_reduction_with_fire.ipynb){ .md-button }

You can significantly reduce the number of features in your dataset by leveraging DataRobot's ability to train hundreds of high-quality models in a matter of minutes.

Feature Importance Rank Ensembling (FIRE) aggregates the rankings of individual features using Feature Impact from several blueprints on the leaderboard. This approach can provide greater accuracy and robustness over other feature reduction methods.

This accelerator shows how to apply FIRE to your dataset and dramatically reduce the number of features without impacting the performance of the final model.
---
title: Deprecations and migrations
description: Documentation related to deprecated functionality and guides on how to migrate your workflows.

---

# Deprecations and migrations {: #deprecation-migration-guides }

Deprecation _notices_ are included within the Cloud announcement pages. The pages within _this_ section provide more detailed guidance related to deprecated features. When applicable, a guide that explains the process for migrating your workflows to support the new functionality is provided.


* [Python 2](python2) deprecation/migration guide
* Deprecated: [Batch scoring script](python-batch-scoring)
* Deprecated: [Open source and USER models](oss-user-models)
---
title: No-show appointment forecasting
description: How to build a model that identifies patients most likely to miss appointments, with correlating reasons.

---

# No-show appointment forecasting {: #no-show-appointment-forecasting }

[Access this AI accelerator on GitHub <span style="vertical-align: sub">:material-arrow-right-circle:{.lg }</span>](https://github.com/datarobot-community/ai-accelerators/blob/main/advanced-experimentation/appointment_forecasting/no_show.ipynb){ .md-button }

Many people are guilty of having canceled a doctor’s appointment. However, although canceling an appointment does not seem too disastrous from the patient’s point of view, no-shows cost outpatient health centers a staggering 14% of anticipated daily revenue (JAOA). Missed appointments trickle into lower utilization rates for not only doctors and nurses but also the overhead costs required to run outpatient centers. In addition, patients missing their appointments risk facing poorer health outcomes as they are unable to access timely care.

While outpatient centers employ solutions such as calling patients ahead of time, these high touch resources investments are often not prioritized for patients with the highest risk of no-shows. Low touch solutions such as automated texts are effective tools for mass reminders but do not offer necessary personalization for patients at the highest risk of no-shows. This accelerator shows how to identify clients who are likely to miss appointments ("no-shows") and take action to prevent that from happening.
# Version 8.0.3 {: #version-803 }

_May 19, 2022_

The DataRobot v8.0.3 release includes some fixed issues in the DataRobot Self-Managed AI Platform platform. See the v8.0.0 release notes for:

* [Features introduced in v8.0.0](v8.0.0-aml)

## Issues fixed in v8.0.3 {: #issues-fixed-in-v803 }

The following issues have been fixed since release v8.0.2:

### Platform {: #platform }

* EPS-412: The Enterprise Installer example configuration file (`multi-node-ha.yaml`) now includes the `internalnginx` service alongside the `appsinternalapi` service.

* PLT-6734: SAML assertions for role and group membership can now be provided in a delimiter-separated list in a single attribute value.

* PY3-3953: The project lifecycle transition can now mark Python 2 projects as **Deprecated** during a background process that runs after the upgrade.

* PY3-3957: The **Python Version** field is now added to User Activity Monitor reports for App Usage and Predictions Usage in the UI, API, and exported CSV data.

* UIUX-9414: The deprecated project banner can now be dismissed. When you click the **Dismiss** button, the related project ID persists in local storage. The banner won't appear again until you clear your browser's local storage.

### Predictions {: #predictions }

* PRED-2620: Fixes a prediction performance issue that could increase latency for multiprocessing batch predictions.

* PRED-7418: Fixes an issue that blocked JDBC writing due to duplicate columns in a schema if pass-through columns included a feature with the same name as the deployment's association ID.

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
---
title: Time series public preview features
description: Read preliminary documentation for time series features currently in the DataRobot public preview pipeline.
section_name: Time Series
maturity: public-preview
---


# Time series public preview features {: #time-series-public-preview-features }

{% include 'includes/pub-preview-notice-include.md' %}

Public preview for... | Describes...
----- | ------
[Period Accuracy](ts-period-accuracy) | View model performance over periods within the training dataset.
[Time series model package prediction intervals](pp-ts-pred-intervals-mlpkg) | Export time series models with prediction intervals in model package (.mlpkg) format.
[Batch predictions for TTS and LSTM models](pp-ts-tts-lstm-batch-pred) | Make batch predictions for Traditional Time Series (TTS) and Long Short-Term Memory (LSTM) models.
---
title: Local files
description: Add locally-stored datasets to your Use Case in Workbench.
---

# Upload local files {: #upload-local-files }

By uploading a local file via the Add data modal, you are both adding the dataset to your Use Case and [registering it in the Data Registry](wb-data-registry). This method of adding data is a good approach if your dataset is already ready for modeling.

Before uploading a file, review DataRobot's [dataset requirements](file-types){ target=_blank } for accepted file formats and size guidelines. See the associated [considerations](wb-data-ref/index#add-data) for important additional information.

To upload a local file:

1. In the **Add data** modal, click **Upload**.

    ![](images/wb-local-1.png)

2. Locate and select your dataset in the file explorer. Then, click **Open**.

    !!! note "Supported file types"
        Workbench supports the following file types for upload: .csv, .tsv, .dsv, .xls, .xlsx, .sas7bdat, .geojson, .gz, .bz2, .tar, .tgz, .zip.

3. DataRobot opens the **Datasets** tab, where you can monitor the progress of the dataset as it's registering in the Data Registry.

    === "Data registering"

        ![](images/wb-local-2.png)

    === "Data registered"

        ![](images/wb-local-3.png)

    When registration is complete, DataRobot displays the source, row count, feature count, and size.

## Next steps {: next-steps }

From here, you can:

- [Add more data.](wb-add-data/index)
- [View Exploratory Data Insights for the dataset.](wb-data-tab)
- [Use the dataset to set up an experiment and start modeling.](wb-experiment/index)

## Read more {: #read-more}

To learn more about the topics discussed on this page, see:

- [DataRobot's dataset requirements.](file-types){ target=_blank }
- [Upload local files in DataRobot Classic.](import-to-dr#drag-and-drop){ target=_blank }