Huggingface mt5 does not reach the performance of original mt5 on paws-x 
Author: dorost1234
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.2
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?): 1.7 
- Tensorflow version (GPU?): - 
- Using GPU in script?: yes 
- Using distributed or parallel set-up in script?: yes 

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:



Library:



Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

 t5: @patrickvonplaten, @patil-suraj

## Information

Hi
I ran the model of mT5-small on paws-x in zero-shot cross-lingual setup where we tune on english and evaluate on all languages in paws-x dataset and obtain only 80.2 while the reported original mt5-small on this datasets is 82.4. (see table 2 in mt5 paper) I used the setup in mt5 paper, is there any missing details from original mt5 work in the huggingface implementation? thanks 
 

## Expected behavior

reaching the performance of original model.

Comments:
Hey @dorost1234, 

could you post your question on the [forum](https://discuss.huggingface.co/) and see whether you can get help from the community there? We try to keep GitHub issues for bug reports mostly. 

It would also be very important that you attach a notebook or something that allows people to understand what you have done and what you might have missed...

Hi  Patrick
thanks, sure, I posted here because I mainly wanted to ask if you have in
the past compared the performance of original mt5 with the HuggingFace
model side by side on one setting?
this is unfortunately a lot of codes for me to share, since I build on top
of my codebase, and this is not very easy to make a small example showing
these differences.
Overall, knowing if in the past such comparison is done is great.
thanks.

On Mon, Feb 22, 2021 at 3:02 PM Patrick von Platen <notifications@github.com>
wrote:

> Hey @dorost1234 <https://github.com/dorost1234>,
>
> could you post your question on the forum
> <https://discuss.huggingface.co/> and see whether you can get help from
> the community there? We try to keep GitHub issues for bug reports mostly.
>
> It would also be very important that you attach a notebook or something
> that allows people to understand what you have done and what you might have
> missed...
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/10315#issuecomment-783396814>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AS37NMXVTAFKQLNWCBFUG43TAJPXBANCNFSM4X65FR7Q>
> .
>


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.