TrOCR Backslash problem
Author: win5923
Pull request: no

hello,I use https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Fine_tune_TrOCR_on_IAM_Handwriting_Database_using_Seq2SeqTrainer.ipynb this jupyter to fine-tune my task .It about  generate the math expression LATEX sequence according to the handwritten math expression image.
but when I go predicting the image,the model predicts an extra backslash like this
![image](https://user-images.githubusercontent.com/56353753/157398183-05c0c796-b8f5-4170-9c19-11639e8aea3c.png)
Would like to ask if this can be solved?

processor = TrOCRProcessor.from_pretrained("microsoft/trocr-base-handwritten")
model = VisionEncoderDecoderModel.from_pretrained("microsoft/trocr-small-stage1") 


labels before training is no problem.Do I need a custom tokenizer?
![image](https://user-images.githubusercontent.com/56353753/157399841-abb335ce-5347-449e-aeeb-ffb324d9b22a.png)


Comments:
I think that I get the problem's point.
Why is there no difference between Beam Search and greedy?
Do I need to customize tokenizers?
![image](https://user-images.githubusercontent.com/56353753/157649252-d1ae592f-7ff7-47bf-8ace-75bc4cffbfaa.png)
![image](https://user-images.githubusercontent.com/56353753/157649350-d0fa0e8a-5a7d-433f-8ad7-28a018090da3.png)



@win5923 the double backslash quirk is a Python "feature" :D See this StackOverflow issue, which explains what's going on: https://stackoverflow.com/questions/24085680/why-do-backslashes-appear-twice

As for beam search / greedy generate, they can get the same results (beam search is an elaborated greedy generation). If you were looking for multiple outputs with beam search, try using the `num_return_sequences` argument. If you want to add more entropy to the mix, try with `sample=True` and explore other arguments.

Finally, the application you're trying to build is super interesting! We'd be super happy if you would share your fine-tuned model with us, and/or create a Space showcasing it <3

thank you so much! I will.

Let us know if this solves your problem (so we can close the issue), or if there are further bugs :)  

I'm not sure how to upload the model from checkpoint, I upload the code to my repo.
https://github.com/win5923/TrOCR-Handwritten-Mathematical-Expression-Recognition
Not sure if there is something wrong with my code.The current model does not predict very well.
Consider adjusting the parameters of beam search to see if it can improve performance or accuracy.
It's ok now.Thanks a lot.


Hi,

You can easily upload a model to the hub using the `push_to_hub`[method](https://huggingface.co/docs/transformers/main_classes/model#transformers.file_utils.PushToHubMixin.push_to_hub). For example:

```
model.push_to_hub(
        repo_path_or_name="trocr-handwritten-math",
        organization="nielsr",
        commit_message="Add model",
        use_temp_dir=True,
)
```

This way, you'll be able to just do:
```
model = VisionEncoderDecoderModel.from_pretrained("nielsr/trocr-handwritten-math")
```

Closing this issue as I believe we've answered your questions. Feel free to re-open if you have further questions.

@NielsRogge @gante can you please explain how to annotate the below files for custom handwritten mathematical equation training. More importantly s^2 
![FCA(1)-1-0tensor(0 89785)720](https://user-images.githubusercontent.com/51528367/198884771-5b7bf056-ae81-43f7-8ce9-a187cf371602.png)
![FCA(1)-4-3tensor(0 89384)888](https://user-images.githubusercontent.com/51528367/198884777-95d35008-c665-460d-b996-d0835e7f5a0e.png)
![FCA6-1-0tensor(0 83108)1827](https://user-images.githubusercontent.com/51528367/198884814-8cbfb1e8-66e0-4956-bb44-f9c912bc35f3.png)
![FCA6-3-1tensor(0 88415)2013](https://user-images.githubusercontent.com/51528367/198884835-19619bf3-2a8c-4113-b8fb-e28614cd44d2.png)

![FCA6-4-1tensor(0 82750)1928](https://user-images.githubusercontent.com/51528367/198884849-267d3228-2253-4eb1-a005-23626760c6a5.png)