Inference widgets for self-hosted models?
Author: cceyda
Pull request: no

# 🚀 Feature request
I'm loving the new [huggingface](https://huggingface.co/bert-base-uncased?) dataset browsing & hosted model interfaces 🤯  
So firstly a huge thank you to everyone <3

This is a question/feature request ~
- Can we use inference widgets for self-hosted models? 

I see that there is a [serving.py](https://github.com/huggingface/transformers/blob/master/src/transformers/commands/serving.py) ( transformer-cli ) but nothing about widgets as far as I can see. If this is possible I would love an example on how; if not, will it be in the future?

## Motivation
Inference widgets would be nice to have during model testing & demos

## Your contribution
I was planning on doing something similar using `huggingface -> spacy(displacy) -> streamlit`  (for NER)



Comments:
do you mind sending an email to clement [at] huggingface [dot] co explaining your usecase?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I have created a simple package that demos how to self-host Hugging Face NER models with an UI akin to the Inference api~

Self-Hosted inference for NER:
![image](https://user-images.githubusercontent.com/15624271/93560907-264e7580-f9be-11ea-8c57-8f878e95cb3a.png)

available @ https://github.com/cceyda/lit-NER
❤️ using HuggingFace+Torchserve+Displacy+Streamlit  


looks good @cceyda! Do you have a publicly hosted demo somewhere?

@julien-c  my main objective was to provide an easy way to have an interface API&AI for self-hosted models vs public~ but why not do both 😄 so I put up a demo [here](https://share.streamlit.io/cceyda/lit-ner/public/lit_ner.py) (I recently got access to streamlit sharing beta 🥳 )
You can enter the model name of any NER model hosted at [huggingface ❤️ ](https://huggingface.co/models?filter=token-classification&search=ner) like so:
![image](https://user-images.githubusercontent.com/15624271/95163741-7f524200-07e3-11eb-8c41-7714b8ed3ac8.png)


**_OR_** even your custom self hosted model by using the example torchserve recipe I made [lit_ner/serve.py](https://github.com/cceyda/lit-NER/blob/master/examples/serve.ipynb) (currently there is no security setup) 

BTW, There are some problems with the current NER pipeline (which I provide a fix PR for [here](https://github.com/huggingface/transformers/pull/5970))
Example error:
![image](https://user-images.githubusercontent.com/15624271/95163217-5aa99a80-07e2-11eb-8a85-4144f7deb636.png)
At my local with changes from the PR:
![image](https://user-images.githubusercontent.com/15624271/95163363-ab20f800-07e2-11eb-9184-8c300ea7c46b.png)

PS: this can be easily expanded to other pipelines and also highly customizable  😉 will polish it more as soon as I have some time

That's neat! Do you mind if I tweet it? (what's your Twitter handle :-)

@julien-c of course I would like it very much.  I have also written a [blog post](https://cceyda.github.io/blog/huggingface/torchserve/streamlit/ner/2020/10/09/huggingface_streamlit_serve.html) about it,  my first blog post! 🥳  My [twitter](https://twitter.com/ceyda_cinarel) is so unused it is embarrassing 😆 until now I used it just for following the news, but in the future I will be using it for sharing my blog post notifications. hoping that someone will be reading it 🤞 😄 


[Tweet is up](https://twitter.com/julien_c), I'll close this issue now, thanks again for sharing