# Supported foundation models available with watsonx\.ai #

A collection of open source and IBM foundation models are deployed in IBM watsonx\.ai\.

The following models are available in watsonx\.ai:

<!-- <ul> -->

 *  flan\-t5\-xl\-3b
 *  flan\-t5\-xxl\-11b
 *  flan\-ul2\-20b
 *  gpt\-neox\-20b
 *  granite\-13b\-chat\-v2
 *  granite\-13b\-chat\-v1
 *  granite\-13b\-instruct\-v2
 *  granite\-13b\-instruct\-v1
 *  llama\-2\-13b\-chat
 *  llama\-2\-70b\-chat
 *  mpt\-7b\-instruct2
 *  mt0\-xxl\-13b
 *  starcoder\-15\.5b

<!-- </ul> -->

You can prompt these models in the Prompt Lab or programmatically by using the Python library\.

## Summary of models ##

To understand how the model provider, instruction tuning, token limits, and other factors can affect which model you choose, see [Choosing a model](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-model-choose.html)\.

The following table lists the supported foundation models that IBM provides\.

<!-- <table> -->

Table 1\. IBM foundation models in watsonx\.ai

| Model name                                               | Provider | Instruction\-tuned | Billing class | Maximum tokens  <br>Context (input \+ output) | More information                                                          |
| -------------------------------------------------------- | -------- | ------------------ | ------------- | --------------------------------------------- | ------------------------------------------------------------------------- |
| [granite\-13b\-chat\-v2](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#granite-13b-chat)         | IBM      | Yes                | Class 2       | 8192                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-chat-v2?context=wx)  <br>• [Website](https://www.ibm.com/blog/watsonx-tailored-generative-ai/)  <br>• [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM)             |
| [granite\-13b\-chat\-v1](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#granite-13b-chat-v1)         | IBM      | Yes                | Class 2       | 8192                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-chat-v1?context=wx)  <br>• [Website](https://www.ibm.com/blog/watsonx-tailored-generative-ai/)  <br>• [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM) |
| [granite\-13b\-instruct\-v2](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#granite-13b-instruct) | IBM      | Yes                | Class 2       | 8192                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-instruct-v2?context=wx)  <br>• [Website](https://www.ibm.com/blog/watsonx-tailored-generative-ai/)  <br>• [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM) |
| [granite\-13b\-instruct\-v1](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#granite-13b-instruct-v1) | IBM      | Yes                | Class 2       | 8192                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-instruct-v1?context=wx)  <br>• [Website](https://www.ibm.com/blog/watsonx-tailored-generative-ai/)  <br>• [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM) |

<!-- </table ""> -->

The following table lists the supported foundation models that third parties provide through Hugging Face\.

<!-- <table> -->

Table 2\. Supported third party foundation models in watsonx\.ai

| Model name                                 | Provider   | Instruction\-tuned | Billing class | Maximum tokens  <br>Context (input \+ output) | More information                                                                                          |
| ------------------------------------------ | ---------- | ------------------ | ------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| [flan\-t5\-xl\-3b](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#flan-t5-xl-3b)       | Google     | Yes                | Class 1       | 4096                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/google/flan-t5-xl?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2210.11416)                                    |
| [flan\-t5\-xxl\-11b](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#flan-t5-xxl-11b)   | Google     | Yes                | Class 2       | 4096                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/google/flan-t5-xxl?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2210.11416)                                    |
| [flan\-ul2\-20b](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#flan-ul2-20b)           | Google     | Yes                | Class 3       | 4096                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/google/flan-ul2?context=wx)  <br>• [UL2 research paper](https://arxiv.org/abs/2205.05131v1)  <br>• [Flan research paper](https://arxiv.org/abs/2210.11416) |
| [gpt\-neox\-20b](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#gpt-neox-20b)           | EleutherAI | No                 | Class 3       | 8192                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/eleutherai/gpt-neox-20b?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2204.06745)                                    |
| [llama\-2\-13b\-chat](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#llama-2) | Meta       | Yes                | Class 1       | 4096                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/meta-llama/llama-2-13b-chat?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2307.09288)                                    |
| [llama\-2\-70b\-chat](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#llama-2) | Meta       | Yes                | Class 2       | 4096                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/meta-llama/llama-2-70b-chat?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2307.09288)                                    |
| [mpt\-7b\-instruct2](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#mpt-7b-instruct2)   | Mosaic ML  | Yes                | Class 1       | 2048                                          | • [Model card](https://huggingface.co/ibm/mpt-7b-instruct2)  <br>• [Website](https://www.mosaicml.com/blog/mpt-7b)                                                 |
| [mt0\-xxl\-13b](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#mt0-xxl-13b)             | BigScience | Yes                | Class 2       | 4096                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/bigscience/mt0-xxl?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2211.01786)                                   |
| [starcoder\-15\.5b](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=cdpaas&locale=en#starcoder-15.5b)     | BigCode    | No                 | Class 2       | 8192                                          | • [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/bigcode/starcoder?context=wx)  <br>• [Research paper](https://arxiv.org/abs/2305.06161)                                   |

<!-- </table ""> -->

<!-- <ul> -->

 *  For a list of which models are provided in each regional data center, see [Regional availability of foundation model](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/regional-datactr.html#data-centers)\.
 *  For information about the billing classes and rate limiting, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html#ru-metering)\.

<!-- </ul> -->

## Foundation model details ##

The available foundation models support a range of use cases for both natural languages and programming languages\. To see the types of tasks that these models can do, review and try the [sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html)\.

### flan\-t5\-xl\-3b ###

The flan\-t5\-xl\-3b model is provided by Google on Hugging Face\. This model is based on the pretrained text\-to\-text transfer transformer (T5) model and uses instruction fine\-tuning methods to achieve better zero\- and few\-shot performance\. The model is also fine\-tuned with chain\-of\-thought data to improve its ability to perform reasoning tasks\.

**Note**: This foundation model can be tuned by using the Tuning Studio\.

**Usage** : General use with zero\- or few\-shot prompts\.

**Cost** : Class 1\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html)

**Size** : 3 billion parameters

**Token limits** : Context window length (input \+ output): 4096

: **Note**: Lite plan output is limited to 700

**Supported natural languages** : English, German, French

**Instruction tuning information** : The model was fine\-tuned on tasks that involve multiple\-step reasoning from chain\-of\-thought data in addition to traditional natural language processing tasks\.

    Details about the training data sets used are published.

**Model architecture** : Encoder\-decoder

**License** : [Apache 2\.0 license](https://www.apache.org/licenses/LICENSE-2.0.txt)

**Learn more** : [Research paper](https://arxiv.org/abs/2210.11416) : [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/google/flan-t5-xl?context=wx) : [Sample notebook: Tune a model to classify CFPB documents in watsonx](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/bf57e8896f3e50c638b5a378780f7502)

### flan\-t5\-xxl\-11b ###

The flan\-t5\-xxl\-11b model is provided by Google on Hugging Face\. This model is based on the pretrained text\-to\-text transfer transformer (T5) model and uses instruction fine\-tuning methods to achieve better zero\- and few\-shot performance\. The model is also fine\-tuned with chain\-of\-thought data to improve its ability to perform reasoning tasks\.

**Usage** : General use with zero\- or few\-shot prompts\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html) : [Sample notebook: Use watsonx and Google flan\-t5\-xxl to generate advertising copy](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/73243d67b49a6e05f4cdf351b4b35e21?context=wx) : [Sample notebook: Use watsonx and LangChain to make a series of calls to a language model](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/c3dbf23a-9a56-4c4b-8ce5-5707828fc981?context=wx)

**Size** : 11 billion parameters

**Token limits** : Context window length (input \+ output): 4096

: **Note**: Lite plan output is limited to 700

**Supported natural languages** : English, German, French

**Instruction tuning information** : The model was fine\-tuned on tasks that involve multiple\-step reasoning from chain\-of\-thought data in addition to traditional natural language processing tasks\. Details about the training data sets used are published\.

**Model architecture** : Encoder\-decoder

**License** : [Apache 2\.0 license](https://www.apache.org/licenses/LICENSE-2.0.txt)

**Learn more** : [Research paper](https://arxiv.org/abs/2210.11416) : [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/google/flan-t5-xxl?context=wx)

### flan\-ul2\-20b ###

The flan\-ul2\-20b model is provided by Google on Hugging Face\. This model was trained by using the Unifying Language Learning Paradigms (UL2)\. The model is optimized for language generation, language understanding, text classification, question answering, common sense reasoning, long text reasoning, structured\-knowledge grounding, and information retrieval, in\-context learning, zero\-shot prompting, and one\-shot prompting\.

**Usage** : General use with zero\- or few\-shot prompts\.

**Cost** : Class 3\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html) : [Sample notebook: Use watsonx to summarize cybersecurity documents](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/1cb62d6a5847b8ed5cdb6531a08e9104?context=wx) : [Sample notebook: Use watsonx and LangChain to answer questions by using retrieval\-augmented generation (RAG)](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/d3a5f957-a93b-46cd-82c1-c8d37d4f62c6?context=wx&audience=wdp) : [Sample notebook: Use watsonx, Elasticsearch, and LangChain to answer questions (RAG)](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/ebeb9fc0-9844-4838-aff8-1fa1997d0c13?context=wx&audience=wdp) : [Sample notebook: Use watsonx, and Elasticsearch Python SDK to answer questions (RAG)](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/bdbc8ad4-9c1f-460f-99ee-5c3a1f374fa7?context=wx&audience=wdp)

**Size** : 20 billion parameters

**Token limits** : Context window length (input \+ output): 4096

: **Note**: Lite plan output is limited to 700

**Supported natural languages** : English

**Instruction tuning information** : The flan\-ul2\-20b model is pretrained on the colossal, cleaned version of Common Crawl's web crawl corpus\. The model is fine\-tuned with multiple pretraining objectives to optimize it for various natural language processing tasks\. Details about the training data sets used are published\.

**Model architecture** : Encoder\-decoder

**License** : [Apache 2\.0 license](https://www.apache.org/licenses/LICENSE-2.0.txt)

**Learn more** : [Unifying Language Learning (UL2) research paper](https://arxiv.org/abs/2205.05131v1) : [Fine\-tuned Language Model (Flan) research paper](https://arxiv.org/abs/2210.11416)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/google/flan-ul2?context=wx)

### gpt\-neox\-20b ###

The gpt\-neox\-20b model is provided by EleutherAI on Hugging Face\. This model is an autoregressive language model that is trained on diverse English\-language texts to support general\-purpose use cases\. GPT\-NeoX\-20B has not been fine\-tuned for downstream tasks\.

**Usage** : Works best with few\-shot prompts\. Accepts special characters, which can be used for generating structured output\. : The data set used for training contains profanity and offensive text\. Be sure to curate any output from the model before using it in an application\.

**Cost** : Class 3\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html)

**Size** : 20 billion parameters

**Token limits** : Context window length (input \+ output): 8192

: **Note**: Lite plan output is limited to 700

**Supported natural languages** : English

**Data used during training** : The gpt\-neox\-20b model was trained on the Pile\. For more information about the Pile, see [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://arxiv.org/abs/2101.00027)\. The Pile was not deduplicated before being used for training\.

**Model architecture** : Decoder

**License** : [Apache 2\.0 license](https://www.apache.org/licenses/LICENSE-2.0.txt)

**Learn more** : [Research paper](https://arxiv.org/abs/2204.06745)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/eleutherai/gpt-neox-20b?context=wx)

### granite\-13b\-chat\-v2 ###

The granite\-13b\-chat\-v2 model is provided by IBM\. This model is optimized for dialogue use cases and works well with virtual agent and chat applications\.

**Usage** : Generates dialogue output like a chatbot\. Uses a model\-specific prompt format\. Includes a keyword in its output that can be used as a stop sequence to produce succinct answers\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompt](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample7a)

**Size** : 13 billion parameters

**Token limits** : Context window length (input \+ output): 8192

**Supported natural languages** : English

**Instruction tuning information** : The Granite family of models is trained on enterprise\-relevant data sets from five domains: internet, academic, code, legal, and finance\. Data used to train the models first undergoes IBM data governance reviews and is filtered of text that is flagged for hate, abuse, or profanity by the IBM\-developed HAP filter\. IBM shares information about the training methods and data sets used\.

**Model architecture** : Decoder

**License** : [Terms of use](https://www.ibm.com/support/customer/csol/terms/?id=i126-6883) : For more information about contractual protections related to IBM watsonx\.ai, see the [IBM watsonx\.ai service description](https://www.ibm.com/support/customer/csol/terms/?id=i126-7747)\.

**Learn more** : [Model information](https://www.ibm.com/blog/watsonx-tailored-generative-ai/) : [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-chat-v2?context=wx)

### granite\-13b\-chat\-v1 ###

The granite\-13b\-chat\-v1 model is provided by IBM\. This model is optimized for dialogue use cases and works well with virtual agent and chat applications\.

**Usage** : Generates dialogue output like a chatbot\. Uses a model\-specific prompt format\. Includes a keyword in its output that can be used as a stop sequence to produce succinct answers\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompt](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample7a)

**Size** : 13 billion parameters

**Token limits** : Context window length (input \+ output): 8192

**Supported natural languages** : English

**Instruction tuning information** : The Granite family of models is trained on enterprise\-relevant data sets from five domains: internet, academic, code, legal, and finance\. Data used to train the models first undergoes IBM data governance reviews and is filtered of text that is flagged for hate, abuse, or profanity by the IBM\-developed HAP filter\. IBM shares information about the training methods and data sets used\.

**Model architecture** : Decoder

**License** : [Terms of use](https://www.ibm.com/support/customer/csol/terms/?id=i126-6883) : For more information about contractual protections related to IBM watsonx\.ai, see the [IBM watsonx\.ai service description](https://www.ibm.com/support/customer/csol/terms/?id=i126-7747)\.

**Learn more** : [Model information](https://www.ibm.com/blog/watsonx-tailored-generative-ai/) : [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-chat-v1?context=wx)

### granite\-13b\-instruct\-v2 ###

The granite\-13b\-instruct\-v2 model is provided by IBM\. This model was trained with high\-quality finance data, and is a top\-performing model on finance tasks\. Financial tasks evaluated include: providing sentiment scores for stock and earnings call transcripts, classifying news headlines, extracting credit risk assessments, summarizing financial long\-form text, and answering financial or insurance\-related questions\.

**Usage** : Supports extraction, summarization, and classification tasks\. Generates useful output for finance\-related tasks\. Uses a model\-specific prompt format\. Accepts special characters, which can be used for generating structured output\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample 3b: Generate a numbered list on a particular theme](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample3b) : [Sample 4c: Answer a question based on a document](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample4c) : [Sample 4d: Answer general knowledge questions](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample4d)

: [Sample notebook: Use watsonx and ibm/granite\-13b\-instruct to analyze car rental customer satisfaction from text](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/61c1e967-8d10-44bb-a846-cc1f27e9e69a?context=wx)

**Size** : 13 billion parameters

**Token limits** : Context window length (input \+ output): 8192

**Supported natural languages** : English

**Instruction tuning information** : The Granite family of models is trained on enterprise\-relevant data sets from five domains: internet, academic, code, legal, and finance\. Data used to train the models first undergoes IBM data governance reviews and is filtered of text that is flagged for hate, abuse, or profanity by the IBM\-developed HAP filter\. IBM shares information about the training methods and data sets used\.

**Model architecture** : Decoder

**License** : [Terms of use](https://www.ibm.com/support/customer/csol/terms/?id=i126-6883) : For more information about contractual protections related to IBM watsonx\.ai, see the [IBM watsonx\.ai service description](https://www.ibm.com/support/customer/csol/terms/?id=i126-7747)\.

**Learn more** : [Model information](https://www.ibm.com/blog/watsonx-tailored-generative-ai/) : [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-instruct-v2?context=wx)

### granite\-13b\-instruct\-v1 ###

The granite\-13b\-instruct\-v1 model is provided by IBM\. This model was trained with high\-quality finance data, and is a top\-performing model on finance tasks\. Financial tasks evaluated include: providing sentiment scores for stock and earnings call transcripts, classifying news headlines, extracting credit risk assessments, summarizing financial long\-form text, and answering financial or insurance\-related questions\.

**Usage** : Supports extraction, summarization, and classification tasks\. Generates useful output for finance\-related tasks\. Uses a model\-specific prompt format\. Accepts special characters, which can be used for generating structured output\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample 3b: Generate a numbered list on a particular theme](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample3b) : [Sample 4d: Answer general knowledge questions](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample4d)

: [Sample notebook: Use watsonx and ibm/granite\-13b\-instruct to analyze car rental customer satisfaction from text](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/61c1e967-8d10-44bb-a846-cc1f27e9e69a?context=wx)

**Size** : 13 billion parameters

**Token limits** : Context window length (input \+ output): 8192

**Supported natural languages** : English

**Instruction tuning information** : The Granite family of models is trained on enterprise\-relevant data sets from five domains: internet, academic, code, legal, and finance\. Data used to train the models first undergoes IBM data governance reviews and is filtered of text that is flagged for hate, abuse, or profanity by the IBM\-developed HAP filter\. IBM shares information about the training methods and data sets used\.

**Model architecture** : Decoder

**License** : [Terms of use](https://www.ibm.com/support/customer/csol/terms/?id=i126-6883) : For more information about contractual protections related to IBM watsonx\.ai, see the [IBM watsonx\.ai service description](https://www.ibm.com/support/customer/csol/terms/?id=i126-7747)\.

**Learn more** : [Model information](https://www.ibm.com/blog/watsonx-tailored-generative-ai/) : [Research paper](https://www.ibm.com/downloads/cas/X9W4O6BM)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/ibm/granite-13b-instruct-v1?context=wx)

### Llama\-2 Chat ###

The Llama\-2 Chat model is provided by Meta on Hugging Face\. The fine\-tuned model is useful for chat generation\. The model is pretrained with publicly available online data and fine\-tuned using reinforcement learning from human feedback\.

You can choose to use the 13 billion parameter or 70 billion parameter version of the model\.

**Usage** : Generates dialogue output like a chatbot\. Uses a model\-specific prompt format\.

**Cost** : 13b: Class 1 : 70b: Class 2 : For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompt](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#sample7b) : [Sample notebook: Use watsonx and Meta llama\-2\-70b\-chat to answer questions about an article](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/b59922d8-678f-44e4-b5ef-18138890b444?context=wx) : [Sample notebook: Use watsonx and Meta llama\-2\-70b\-chat to answer questions about an article](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/b59922d8-678f-44e4-b5ef-18138890b444?context=wx)

**Available sizes** : 13 billion parameters : 70 billion parameters

**Token limits** : Context window length (input \+ output): 4096

: Lite plan output is limited as follows: : \- 70b version: 900 : \- 13b version: 2048

**Supported natural languages** : English

**Instruction tuning information** : Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources\. The fine\-tuning data includes publicly available instruction data sets and more than one million new examples that were annotated by humans\.

**Model architecture** : Llama 2 is an auto\-regressive decoder\-only language model that uses an optimized transformer architecture\. The tuned versions use supervised fine\-tuning and reinforcement learning with human feedback\.

**License** : [License](https://ai.meta.com/llama/license/)

**Learn more** : [Research paper](https://arxiv.org/abs/2307.09288)

: [13b Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/meta-llama/llama-2-13b-chat?context=wx) : [70b Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/meta-llama/llama-2-70b-chat?context=wx)

### mpt\-7b\-instruct2 ###

The mpt\-7b\-instruct2 model is provided by MosaicML on Hugging Face\. This model is a fine\-tuned version of the base MosaicML Pretrained Transformer (MPT) model that was trained to handle long inputs\. This version of the model was optimized by IBM for following short\-form instructions\.

**Usage** : General use with zero\- or few\-shot prompts\.

**Cost** : Class 1\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html)

**Size** : 7 billion parameters

**Token limits** : Context window length (input \+ output): 2048

: **Note**: Lite plan output is limited to 500

**Supported natural languages** : English

**Instruction tuning information** : The dataset that was used to train this model is a combination of the Dolly dataset from Databrick and a filtered subset of the Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback training data from Anthropic\.

    During filtering, parts of dialog exchanges that contain instruction-following steps were extracted to be used as samples.

**Model architecture** : Encoder\-decoder

**License** : [Apache 2\.0 license](https://www.apache.org/licenses/LICENSE-2.0.txt)

**Learn more** : [Model card](https://huggingface.co/ibm/mpt-7b-instruct2) : [Blog](https://www.mosaicml.com/blog/mpt-7b)

### mt0\-xxl\-13b ###

The mt0\-xxl\-13b model is provided by BigScience on Hugging Face\. The model is optimized to support language generation and translation tasks with English, languages other than English, and multilingual prompts\.

**Usage** : General use with zero\- or few\-shot prompts\. For translation tasks, include a period to indicate the end of the text you want translated or the model might continue the sentence rather than translate it\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html)

: [Sample notebook: Simple introduction to retrieval\-augmented generation with watsonx\.ai](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/fed7cf6b-1c48-4d71-8c04-0fce0e000d43?context=wx)

**Size** : 13 billion parameters

**Token limits** : Context window length (input \+ output): 4096

: **Note**: Lite plan output is limited to 700

**Supported natural languages** : The model is pretrained on multilingual data in 108 languages and fine\-tuned with multilingual data in 46 languages to perform multilingual tasks\.

**Instruction tuning information** : BigScience publishes details about its code and data sets\.

**Model architecture** : Encoder\-decoder

**License** : [Apache 2\.0 license](https://www.apache.org/licenses/LICENSE-2.0.txt)

**Learn more** : [Research paper](https://arxiv.org/abs/2211.01786)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/bigscience/mt0-xxl?context=wx)

### starcoder\-15\.5b ###

The starcoder\-15\.5b model is provided by BigCode on Hugging Face\. This model can generate code and convert code from one programming language to another\. The model is meant to be used by developers to boost their productivity\.

**Usage** : Code generation and code conversion : Note: The model output might include code that is taken directly from its training data, which can be licensed code that requires attribution\.

**Cost** : Class 2\. For pricing details, see [Watson Machine Learning plans](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/wml-plans.html)\.

**Try it out** : [Sample prompts](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-prompt-samples.html#code) : [Sample notebook: Use watsonx and BigCode starcoder\-15\.5b to generate code based on instruction](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/b5792ad4-555b-4b68-8b6f-ce368093fac6?context=wx)

**Size** : 15\.5 billion parameters

**Token limits** : Context window length (input \+ output): 8192

**Supported programming languages** : Over 80 programming languages, with an emphasis on Python\.

**Data used during training** : This model was trained on over 80 programming languages from GitHub\. A filter was applied to exclude from the training data any licensed code or code that is marked with opt\-out requests\. Nevertheless, the model's output might include code from its training data that requires attribution\. The model was not instruction\-tuned\. Submitting input with only an instruction and no examples might result in poor model output\.

**Model architecture** : Decoder

**License** : [License](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement)

**Learn more** : [Research paper](https://arxiv.org/abs/2305.06161)

: [Model card](https://dataplatform.cloud.ibm.com/wx/samples/models/bigcode/starcoder?context=wx)

**Parent topic:**[Foundation models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-overview.html)

<!-- </article "role="article" "> -->