Repository: Giskard-AI/giskard-oss
Issue: [GSK-885] Group dependencies of Python client
Root cause: Architectural Constraint

Currently, we require all dependencies to be satisfied when installing the `giskard` package via `pip`. We should group them based on the use case so that we have less problems of incompatibility with the user’s dependencies.

For example, if the user needs to test sklearn models, they do not need to install `transformers`, `catboost`, etc. They should be able to install Giskard by

```
pip install "giskard[sklearn]"
```

<sub>From [SyncLinear.com](https://synclinear.com) | [GSK-885](https://linear.app/giskard/issue/GSK-885/group-dependencies-of-python-client)</sub>

@andreybavt @rabah-khalek-khalek 

<!-- From Matteo Dora on Linear -->

**Problems**

* Scikit learn version (AI shepherd)
* A lot of warnings in google colab
* A lot of errors in workbench
* Torch issues because of Lime
* big query cloud issue (webedia)

**Possible solutions**

* Give up 3.7
* optional dependencies groups for specific functionalities (e.g. `pip install giskard[sklearn]`)
* less strict version requirements:
  * remove shap or lime
* dynamic imports to load only what we need (less dependencies = less problems)
* make the Python client more independent of the Java codebase & build process

<!-- From jmjm on Linear -->

A comment on grouping: if we do pip install giskard\[tabular\], we would not need to install lime (since there are no text features). But, since we are talking about removing lime (because it does not work well), I'm not sure this is the right solution. 

<!-- From jmjm on Linear -->

pip install giskard\[ui\] can be a solution, since we won't need to install shap. But, we need to make sure that we won't need shap for the scan feature

<!-- From jmjm on Linear -->

Yeah, I think the complete scan feature will require special things to work with textual data. I think giskard should install by default with tabular support and basic scan, and then allow users to refine the use case by `pip install giskard[nlp]` or equivalent. During the scan or on dataset creation we can print warning if we detect text columns, e.g. “We noticed that your dataset includes textual features, to take full advantage of the Giskard Autoscan, install the optional `giskard[nlp]`.”

<!-- From Matteo Dora on Linear -->

Here's our review of libraries that are required by core giskard and the UI:

```
[tool.poetry.dependencies]
python = ">=3.8,<3.11"
# serialization
cloudpickle = ">=2.0.0" #UI
zstandard = ">=0.18.0" #UI
mlflow-skinny = ">=2" #UI
#ML
numpy = ">=1.21.6,<1.22.0"
scikit-learn = ">=1.0"
scipy = ">=1.7.2,<1.8"
shap = ">=0.41.0" #UI
eli5 = ">=0.13.0" #UI
#Communication
mixpanel = ">=4.10.0"
requests-toolbelt = ">=0.10.1" #UI
grpcio = ">=1.46.3,<=1.51.1" #UI
grpcio-status = ">=1.46.3,<=1.51.1" #UI
requests = ">=2.28.2" #UI
protobuf = "<3.20" #UI
#Utils
pydantic = ">=1.10.7"
tenacity = ">=8.2.2" #UI
python-daemon = ">=2.3.2" #UI
click = ">=8.1.3"
lockfile = ">=0.12.2" #UI
psutil = ">=5.9.4" #UI
tqdm = ">=4.65.0"  #UI
setuptools = ">=65.4.1,<68.0.0" # to check if we need it
pycryptodome = ">=3.17" # to check if we need it
```

<!-- From Andrey Avtomonov on Linear -->

* To test on AI Shepherds notebook

<!-- From Andrey Avtomonov on Linear -->