# ERNIE

## Overview

ERNIE is a series of powerful models proposed by baidu, especially in Chinese tasks,

including [ERNIE1.0](https://arxiv.org/abs/1904.09223), [ERNIE2.0](https://ojs.aaai.org/index.php/AAAI/article/view/6428),

[ERNIE3.0](https://arxiv.org/abs/2107.02137), [ERNIE-Gram](https://arxiv.org/abs/2010.12148), [ERNIE-health](https://arxiv.org/abs/2110.07244), etc.

These models are contributed by [nghuyong](https://huggingface.co/nghuyong) and the official code can be found in [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) (in PaddlePaddle).

### Usage example

Take `ernie-1.0-base-zh` as an example:

```Python

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")

model = AutoModel.from_pretrained("nghuyong/ernie-1.0-base-zh")

### Model checkpoints

|     Model Name      | Language |           Description           |

|:-------------------:|:--------:|:-------------------------------:|

|  ernie-1.0-base-zh  | Chinese  | Layer:12, Heads:12, Hidden:768  |

|  ernie-2.0-base-en  | English  | Layer:12, Heads:12, Hidden:768  |

| ernie-2.0-large-en  | English  | Layer:24, Heads:16, Hidden:1024 |

|  ernie-3.0-base-zh  | Chinese  | Layer:12, Heads:12, Hidden:768  |

| ernie-3.0-medium-zh | Chinese  |  Layer:6, Heads:12, Hidden:768  |

|  ernie-3.0-mini-zh  | Chinese  |  Layer:6, Heads:12, Hidden:384  |

| ernie-3.0-micro-zh  | Chinese  |  Layer:4, Heads:12, Hidden:384  |

|  ernie-3.0-nano-zh  | Chinese  |  Layer:4, Heads:12, Hidden:312  |

|   ernie-health-zh   | Chinese  | Layer:12, Heads:12, Hidden:768  |

|    ernie-gram-zh    | Chinese  | Layer:12, Heads:12, Hidden:768  |

You can find all the supported models from huggingface's model hub: [huggingface.co/nghuyong](https://huggingface.co/nghuyong), and model details from paddle's official

repo: [PaddleNLP](https://paddlenlp.readthedocs.io/zh/latest/model_zoo/transformers/ERNIE/contents.html)

and [ERNIE](https://github.com/PaddlePaddle/ERNIE/blob/repro).

## Resources

- [Text classification task guide](../tasks/sequence_classification)

- [Token classification task guide](../tasks/token_classification)

- [Question answering task guide](../tasks/question_answering)

- [Causal language modeling task guide](../tasks/language_modeling)

- [Masked language modeling task guide](../tasks/masked_language_modeling)

- [Multiple choice task guide](../tasks/multiple_choice)

## ErnieConfig

[[autodoc]] ErnieConfig

    - all

## Ernie specific outputs

[[autodoc]] models.ernie.modeling_ernie.ErnieForPreTrainingOutput

## ErnieModel

[[autodoc]] ErnieModel

    - forward

## ErnieForPreTraining

[[autodoc]] ErnieForPreTraining

    - forward

## ErnieForCausalLM

[[autodoc]] ErnieForCausalLM

    - forward

## ErnieForMaskedLM

[[autodoc]] ErnieForMaskedLM

    - forward

## ErnieForNextSentencePrediction

[[autodoc]] ErnieForNextSentencePrediction

    - forward

## ErnieForSequenceClassification

[[autodoc]] ErnieForSequenceClassification

    - forward

## ErnieForMultipleChoice

[[autodoc]] ErnieForMultipleChoice

    - forward

## ErnieForTokenClassification

[[autodoc]] ErnieForTokenClassification

    - forward

## ErnieForQuestionAnswering

[[autodoc]] ErnieForQuestionAnswering

    - forward