|
--- |
|
license: cc-by-4.0 |
|
tags: |
|
- vision |
|
- nougat |
|
pipeline_tag: image-to-text |
|
--- |
|
|
|
# Nougat model, small-sized version |
|
|
|
Nougat model trained on PDF-to-markdown. It was introduced in the paper [Nougat: Neural Optical Understanding for Academic Documents](https://arxiv.org/abs/2308.13418) by Blecher et al. and first released in [this repository](https://github.com/facebookresearch/nougat/tree/main). |
|
|
|
Disclaimer: The team releasing Nougat did not write a model card for this model so this model card has been written by the Hugging Face team. |
|
|
|
Note: this model corresponds to the "0.1.0-small" version of the original repository. |
|
|
|
## Model description |
|
|
|
Nougat is a [Donut](https://huggingface.co/docs/transformers/model_doc/donut) model trained to transcribe scientific PDFs into an easy-to-use markdown format. The model consists of a Swin Transformer as vision encoder, and an mBART model as text decoder. |
|
|
|
The model is trained to autoregressively predict the markdown given only the pixels of the PDF image as input. |
|
|
|
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/nougat_architecture.jpg" |
|
alt="drawing" width="600"/> |
|
|
|
<small> Nougat high-level overview. Taken from the <a href="https://arxiv.org/abs/2308.13418">original paper</a>. </small> |
|
|
|
## Intended uses & limitations |
|
|
|
You can use the raw model for transcribing a PDF into Markdown. See the [model hub](https://huggingface.co/models?search=nougat) to look for other |
|
fine-tuned versions that may interest you. |
|
|
|
### How to use |
|
|
|
We refer to the [docs](https://huggingface.co/docs/transformers/main/en/model_doc/nougat). |
|
|
|
### BibTeX entry and citation info |
|
|
|
```bibtex |
|
@misc{blecher2023nougat, |
|
title={Nougat: Neural Optical Understanding for Academic Documents}, |
|
author={Lukas Blecher and Guillem Cucurull and Thomas Scialom and Robert Stojnic}, |
|
year={2023}, |
|
eprint={2308.13418}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.LG} |
|
} |
|
``` |
