Repository: evidentlyai/evidently
Issue: [BUG] Grafana example throws 500 Internal Server Error
Root cause: External Dependency Breakage

Currently grafana example is not working. Throwing:

> Send a data item for bike_gradient_boosting
Cannot reach a metrics application, error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')), data: {'instant': 121, 'dteday': '2011-05-01', 'season': 2, 'yr': 0, 'mnth': 5, 'holiday': 0, 'weekday': 0, 'workingday': 0, 'weathersit': 2, 'temp': 0.451667, 'atemp': 0.448204, 'hum': 0.762083, 'windspeed': 0.106354, 'casual': 1138, 'registered': 2213, 'cnt': 3351, 'prediction': 3348.349282595368}
Send a data item for bike_random_forest
Cannot reach a metrics application, error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')), data: {'instant': 121, 'dteday': '2011-05-01', 'season': 2, 'yr': 0, 'mnth': 5, 'holiday': 0, 'weekday': 0, 'workingday': 0, 'weathersit': 2, 'temp': 0.451667, 'atemp': 0.448204, 'hum': 0.762083, 'windspeed': 0.106354, 'casual': 1138, 'registered': 2213, 'cnt': 3351, 'prediction': 3087.98}
Send a data item for kdd_k_neighbors_classifier
Cannot reach a metrics application, error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')), data: {'duration': 1, 'protocol_type': "b'tcp'", 'service': "b'smtp'", 'flag': "b'SF'", 'src_bytes': 1022, 'dst_bytes': 389, 'land': 0, 'wrong_fragment': 0, 'urgent': 0, 'hot': 0, 'num_failed_logins': 0, 'logged_in': 1, 'num_compromised': 0, 'root_shell': 0, 'su_attempted': 0, 'num_root': 0, 'num_file_creations': 0, 'num_shells': 0, 'num_access_files': 0, 'num_outbound_cmds': 0, 'is_host_login': 0, 'is_guest_login': 0, 'count': 1, 'srv_count': 1, 'serror_rate': 0.0, 'srv_serror_rate': 0.0, 'rerror_rate': 0.0, 'srv_rerror_rate': 0.0, 'same_srv_rate': 1.0, 'diff_srv_rate': 0.0, 'srv_diff_host_rate': 0.0, 'dst_host_count': 83, 'dst_host_srv_count': 175, 'dst_host_same_srv_rate': 0.64, 'dst_host_diff_srv_rate': 0.02, 'dst_host_same_src_port_rate': 0.01, 'dst_host_srv_diff_host_rate': 0.02, 'dst_host_serror_rate': 0.0, 'dst_host_srv_serror_rate': 0.0, 'dst_host_rerror_rate': 0.0, 'dst_host_srv_rerror_rate': 0.0, 'labels': 'normal.', 'prediction': 'normal.'}
Wait 2 seconds till the next try.
Send a data item for bike_gradient_boosting
Success.
Send a data item for bike_random_forest
Success.
Send a data item for kdd_k_neighbors_classifier
Success.
Wait 2 seconds till the next try.
Send a data item for bike_gradient_boosting
Got an error code 500 for the data chunk. Reason: INTERNAL SERVER ERROR, error text: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
500 Internal Server Error
Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

and so on every 2 seconds. Adding this line in Dockerfile before APP command 
`ENV FLASK_DEBUG=1` 

Results in catching the exact error via debugger:


> Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.8/site-packages/werkzeug/middleware/dispatcher.py", line 78, in __call__
    return app(environ, start_response)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2076, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/app/app.py", line 232, in iterate
    SERVICE.iterate(dataset_name=dataset, new_rows=pd.DataFrame.from_dict(item))
  File "/app/app.py", line 117, in iterate
    current_data = self.current[dataset_name].append(new_rows, ignore_index=True)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/generic.py", line 5989, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'append'


So basically append was removed in Pandas > 2.0 https://github.com/pandas-dev/pandas/issues/35407 . Even when setting correct version in requirements.txt, the correct version is overwritten via

`RUN pip3 install evidently==0.2.8` in Dockerfile

Precisely in setup.py is no max version of pandas, so 2.0.1 is installed by default

```
setup_args = dict(
    cmdclass=cmdclass,
    author_email="emeli.dral@gmail.com",
    include_package_data=True,
    install_requires=[
        "plotly>=5.5.0",
        "statsmodels>=0.12.2",
        "scikit-learn>=0.24.0",
        **"pandas>=1.3.5",**
        "numpy>=1.19.5",
        "nltk>=3.6.7",
        "scipy>=1.5.4",
        "requests>=2.19.0",
        "PyYAML>=5.1",
        "pydantic<2",
    ],
```

Currently the only way to override it without modifying setup.py is adding additional line to Dockerfile after installing evidently (tested working)

`RUN pip3 install "pandas>=1.1.5,<2.0"`


I can make PR for that - to decide is whether:
- Modify setup.py of evidently 0.28
- Modify only Dockerfile 
- Leave flask debug (in my opinion it is valuable for such testing)


i mentioned the grafana example being broken over in #573. i also noticed the pandas version issue later, and fixed it by pinning the versions in the various `requirements.txt`. they merged PR #575 this morning, but it only changed the evidently version.

i'd just pin the library versions and be done with it. i think pinned pandas 1.4 and evidently 0.2.8

Hi,
Thanks for reporting the problem.

It was unexpected mismatch in versions of pandas which led to installing pandas 2.0 instead of 1.3.5.

We fixed this issue in main branch: changed the version in container to 1.3.5 and changed append call (replaced by pd.concat) so it should work with new pandas as well.

appreciate you.

Indeed it works now out of the box. Thanks! I admire really fast response and commit

```python
$ docker exec -it grafana_monitoring_service-evidently_service-1 sh
# python
Python 3.8.16 (default, Apr 12 2023, 07:05:33)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> pd.__version__
'1.3.5'
>>>
```