Repository: evidentlyai/evidently
Issue: ProbClassificationPerformanceTab - Report not getting displayed
Root cause: Resource Mishandling

Hey,

I was following the documentation to try out Evidently. I save the reports as html and open them up to analyze. I recently tried out the "ProbClassificationPerformanceTab" as it proved to be useful for my problem. The code runs fine but when I open the output HTML file, I am just able to see "Loading..." and not any Visualizations. But I have used the drift report and cat target drift report and both of them worked fine.

Can you please help with this? 

Hi @KN1507  ! 

Thanks for opening up the issue!

We are trying to reproduce the bug. 

Can you share a bit more data 
- version of evidently package
- your operating system
- how you generate the reports - via ipython, Jupyter, Jupyter Lab 
- classification problem type - binary, multiclass
- approximate size of the file you have?

Hi @emeli-dral ,

Yes. I can give you the details.

Version of evidently :- 0.1.15.dev0
Evaluated models :- RandomForestClassifier, CatBoostClassifier
OS:- Win10
System config:- 32GB ram, i7 processor
Report generation :- I saved the file using Jupyter notebook.
Classification type: Binary
Size of the output html file:- 859888 KB if full data & 647220 only for train data.

Eventually, I checked for only the validation datasets between the two models as separate reports which came at 230000 kb. The reports populated for this.

An additional question - On the Drift tab, I have 24 features and it says all 24 drifted but the distribution only shows a strong change for 3 features out of the 24. I couldn't evaluate it using the p-value as it showed me 0.0000 for all the features. Could you help on this?

On a lighter note, Kudos to the package and for the informative blogs. Really helped a lot. 

We are trying to reproduce the issue, but we haven’t succeeded yet. Our hypothesis is that the problem arose because of the file size.
For now we suggest you to try taking 20% of the dataset and compare the models on them, I wonder if the report will be visualized correctly?

Additionally, could you please show us the errors from the browser (they can be viewed in the JavaScript console)? 
It seems like the problem occurred at the rendering stage.

Hi @kn1507  , i forgot to tag you in the previous reply. Let me know if you had the chance to have a look!

Hi @emeli-dral , 

I did a workaround. I tried  a smaller validation set and compared it  with the test set . It worked fine.

I also attribute it to the file size but I don't know why it doesn't load even after hours of keeping the file open.

Not working report image console:-
![image](https://user-images.githubusercontent.com/41093453/118978336-7cbdea80-b977-11eb-91a4-8542322502b9.png)

Working report image console:-
![image](https://user-images.githubusercontent.com/41093453/118978240-6021b280-b977-11eb-878b-57353db144d7.png)



@kn1507 Thanks for an update! We figured out that it is a UI problem when the dataset is too large. Working on a fix now!

Hi @kn1507,

It seems that UI should work fine as of now.  If not - please reopen this issue or join our Discord https://discord.com/invite/xZjKRaNp8b , our dev team will help to fix it.

Also, we made UI compatible with Google Collab. It might be convenient to create some Dashboards there. 
Here are some  examples of Probabilistic Classification Performance Dashboard: [Iris](https://colab.research.google.com/drive/1uFN0ZWNLk3ExylBywq_M8oGkmac0lDrq), 
[Breast Cancer](https://colab.research.google.com/drive/1vwEBgGnDVk5BJG2PXT1w41VfZ-_bq_Sw)