Ecco package - integration with HuggingFace
Author: JoaoLages
Pull request: no

Hi there! 

I'm writing this issue to reach the HuggingFace team and to ask if it would make sense to merge the [Ecco package](https://github.com/jalammar/ecco) with huggingface or, for example, if it makes sense to integrate it in the transformers' API.

Note: I do not speak behalf of the Ecco package, I'm just a code contributor of the repository and heavy user of it + transformers! 

Comments:
Hello! We appreciate @jalammar's work on `ecco`, and we're happy to work with Jay and other members of ecco's community on making sure that `transformers` remains compatible with it - but I wonder if merging the two packages would improve users' workflows or if keeping the two as they are isn't the best for ecco's development.

In general we're happy to work with other package maintainers to provide the best compatibility we can, and we're also happy to integrate tools in transformers but only if that's a wish from the package maintainers and if it comes with a significantly improved user experience.

Agreed! Would it make sense to make `ecco` a `transformers` extra dependency for an explainability API? 
`ecco` can be extended to deal with [other models](https://huggingface.co/transformers/model_doc/auto.html?highlight=auto). Now we are only dealing with CausalLM, MaskedLM and Seq2SeqLM. Any kind of model classification would be a fairly easy integration, for example :)  

Thank you @JoaoLages for the initiative and @LysandreJik for your support!

`transformers` is the main dependency for Ecco and insuring compatibility is indeed key.

Certainly happy to brainstorm on how to improve the user experience. And would certainly love Ecco to be considered for interpretability/explainability initiatives around `transformers`. This can potentially fall under [Bertology ](https://huggingface.co/transformers/master/bertology.html) research.  Another possibility is one or multiple language model interpretability notebooks as a part of community notebooks. @JoaoLages, the Integrated Gradients feature is likely a great candidate for something like this. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

> Thank you @JoaoLages for the initiative and @LysandreJik for your support!
> 
> `transformers` is the main dependency for Ecco and insuring compatibility is indeed key.
> 
> Certainly happy to brainstorm on how to improve the user experience. And would certainly love Ecco to be considered for interpretability/explainability initiatives around `transformers`. This can potentially fall under [Bertology ](https://huggingface.co/transformers/master/bertology.html) research. Another possibility is one or multiple language model interpretability notebooks as a part of community notebooks. @JoaoLages, the Integrated Gradients feature is likely a great candidate for something like this.

@LysandreJik how do you see Ecco being integrated into the `transformers` package? Any of the possibilities pointed here?
 
It would be easily generalizable to most NLP tasks but it is a component that is not present in this package, from what I see - that's why I believe it would be an awesome integration😄. An idea is to have a new transformers' explainability API with Ecco dependencies or simply import the Ecco code into the package.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.