Strange Behaviour using Transformer pipline with FLASK
Author: nimahm2020
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

I am building a web-frame  with Flask to run Transformer pipeline ( sentiment - analysis ) over Excel columns text. 
However ,  In first try  running Flask app , gives following error as :   

ModuleNotFoundError: No module named 'tensorflow_core._api' 

after few days, without any success, As mentioned in other pages , issue will be resolved or better saying ignored by  change debug Flask run flag from TRUE to FALSE as below :

  app.run(host='0.0.0.0', debug=False)

It runs ! Webpage opens, Excel sheet  uploaded to server and then after calling pipline Transformer 

!!  Now i am facing very strange!!! behaviour ! 

It seems that pipline works !! but results are different than stand alone run!! !

In light be better clarification  
For instance : 

Downloading: 100%|█████████████████████████████████████████████████████████████| 230/230 [00:00<00:00, 255kB/s]
amazing job
----------------
[{'label': 'POSITIVE', 'score': 0.99987316}]
sounds misleading
----------------
[{'label': 'NEGATIVE', 'score': 0.9996892}]
amazing job
----------------
[{'label': 'POSITIVE', 'score': 0.99987316}]
sounds cheap
----------------
[{'label': 'NEGATIVE', 'score': 0.99980205}]
Uninteresting.
----------------
[{'label': 'NEGATIVE', 'score': 0.9997152}]
This is an Interesting item.
----------------
[{'label': 'POSITIVE', 'score': 0.99984246}]
very good
----------------
[{'label': 'POSITIVE', 'score': 0.999852}]
this is inappropriate for you
----------------
[{'label': 'NEGATIVE', 'score': 0.99970394}]
this is appropriate for you
----------------
[{'label': 'POSITIVE', 'score': 0.99979025}]
very bad
----------------
[{'label': 'NEGATIVE', 'score': 0.99978846}]

will be like as below after calling in FLASK app

amazing job | 0.793442070484161 | NEGATIVE
sounds misleading | 0.940393328666687 | POSITIVE
amazing job | 0.793442070484161 | NEGATIVE
sounds cheap | 0.934843122959137 | NEGATIVE
Uninteresting. | 0.995108544826508 | NEGATIVE
This is an Interesting item. | 0.946804642677307 | NEGATIVE
very good | 0.77636182308197 | POSITIVE
this is inappropriate for you | 0.945333242416382 | NEGATIVE
this is appropriate for you | 0.942687690258026 | NEGATIVE
very bad | 0.623483657836914 | NEGATIVE



Your help and comments are greatly appreciated. 



## Details


<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
Hi @nimahm2020, 

Thanks for reporting. 
It seems the tokenizer used for the default Sentiment Analysis pipeline was not aligned with the actual model. More precisely, the model is using an **uncased** vocab while the tokenizer is **cased**.

I've opened a PR(#4264) to solve the issue, if you can/want to give it a try and let us know if it solves the issue :).

> Hi @nimahm2020,
> 
> Thanks for reporting.
> It seems the tokenizer used for the default Sentiment Analysis pipeline was not aligned with the actual model. More precisely, the model is using an **uncased** vocab while the tokenizer is **cased**.
> 
> I've opened a PR(#4264) to solve the issue, if you can/want to give it a try and let us know if it solves the issue :).

Hi @mfuntowicz 
In term of accuracy , accidentally, I found that Transformer 2.8 provide the same  accuracy performance  as it was before . It seems that nlp=pipline (sentiment- analysis ) in Transformer 2.9 is a bit under-estimated . 
As a brief ; Transformer connects to Flask with following modifications that i have no clue why it should be : 
1- run Flask app with debug=False 
2- install Transfomer 2.8 ( NOT last version is 2.9) 

more deep explanation and comments are always appreciated . 


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.