File size: 5,382 Bytes
b00385f 5350122 0914710 b00385f 0914710 b00385f 0914710 0c9e62a 0914710 fcb8c81 0914710 fcb8c81 0914710 fcb8c81 0914710 0c9e62a 0914710 0c9e62a 0914710 7d75d30 0c9e62a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 |
---
title: SamGIS - LISA on CUDA
emoji: 🗺️
colorFrom: red
colorTo: blue
sdk: docker
pinned: false
license: mit
---
## Segment Anything models
It's possible to prepare the model files using https://github.com/vietanhdev/samexporter/ or using the ones
from https://huggingface.co/aletrn/sam-quantized (copy them within the folder `/machine_learning_models`).
## SamGIS - HuggingFace version
The SamGIS HuggingSpace url is https://huggingface.co/spaces/aletrn/samgis-lisa-on-cuda.
Build the docker image this way:
```bash
# clean any old active containers
docker stop $(docker ps -a -q); docker rm $(docker ps -a -q)
# build the base docker image from the repository root folder using ARGs:
# - DEPENDENCY_GROUP=fastapi used by poetry
# VITE__MAP_DESCRIPTION, VITE__SAMGIS_SPACE used by 'docker build'
(
set -o allexport && source <(cat ./static/.env|grep VITE__) && set +o allexport;
env|grep VITE__;
docker build . -f dockerfiles/dockerfile-samgis-base --progress=plain \
--build-arg DEPENDENCY_GROUP=fastapi \
--build-arg VITE__MAP_DESCRIPTION=${VITE__MAP_DESCRIPTION} \
--build-arg VITE__SAMGIS_SPACE=${VITE__SAMGIS_SPACE} \
--tag registry.gitlab.com/aletrn/gis-prediction
)
# build the image, use the tag "samgis-huggingface"
docker build . --tag example-docker-namespace/samgis-huggingface --progress=plain
```
Run the container (keep it on background) and show logs
```bash
docker run -d --name samgis-huggingface -p 7860:7860 example-docker-namespace/samgis-huggingface; docker logs -f samgis-huggingface
```
Test it with curl using a json payload:
```bash
URL=http://localhost:7860/infer_samgis
curl -d@./events/payload_point_eolie.json -H 'accept: application/json' ${URL}
```
or better visiting the swagger page on http://localhost:7860/docs
### Dependencies installation and local tests
The docker build process needs only the base dependency group plus the `aws_lambda` or `fastapi` optional one.
Install also the `test` and/or `docs` groups if needed.
### Tests
Tests are defined in the `tests` folder in this project. Use PIP to install the test dependencies and run tests.
```bash
python -m pytest --cov=samgis_lisa_on_cuda --cov-report=term-missing && coverage html
```
### How to update the static documentation with sphinx
This project documentation uses sphinx-apidoc: it's a tool for automatic generation of Sphinx sources that, using the autodoc
extension, document a whole package in the style of other automatic API documentation tools. See the
[documentation page](https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html) for details.
Run the command from the project root:
```bash
# missing docs folder (run from project root) initialize this way
cd docs && sphinx-quickstart -p SamGIS -r 1.0.0 -l python --master index
# update docs folder (from project root)
sphinx-apidoc -f -o docs samgis
```
Then it's possible to generate the HTML pages
```bash
cd docs && make html && ../
# to clean old files
cd docs && make clean html && cd ../
```
The static documentation it's now ready at the path `docs/_build/html/index.html`.
To create a work in progress openapi json or yaml file use
- `extract-openapi-fastapi.py`
- `extract-openapi-lambda.py` (useful to export the json schema request and response from lambda app api)
### Handle dynamic folder creation
it's possible to dynamically create a new support folder adding it to a json string env FOLDERS_MAP, e.g.:
```json
{
"WORKDIR": "/var/task",
"XDG_CACHE_HOME": "/data",
"PROJECT_ROOT_FOLDER": "/data",
"MPLCONFIGDIR": "/data/.cache/matplotlib",
"TRANSFORMERS_CACHE": "/data/.cache/transformers",
"PYTORCH_KERNEL_CACHE_PATH": "/data/.cache/torch/kernels",
"FASTAPI_STATIC": "/var/task/static",
"VIS_OUTPUT": "/data/vis_output"
}
```
The python script create_folders_and_variables_if_not_exists.py will read this env variable, removing any files that exists with these pathnames and assert the correct creation of all the folders. Also these folders must exist as env variables, so the script assert that an env variable exists with its path.
It's possible to use the project in a bare metal installation (not within a docker container). To do this
- download this project
- prepare a virtualenv and install the python dependencies
- install nodejs LTS
- create a .env_source file (in this case `HOME=/home/jovyan`)
```bash
export FOLDER_MAP='{"WORKDIR":"/home/jovyan/workspace/samgis-lisa-on-cuda","XDG_CACHE_HOME":"/home/jovyan/.cache","PROJECT_ROOT_FOLDER":"/home/jovyan/","MPLCONFIGDIR":"/home/jovyan/.cache/matplotlib","TRANSFORMERS_CACHE":"/home/jovyan/.cache/transformers","PYTORCH_KERNEL_CACHE_PATH":"/home/jovyan/.cache/torch/kernels","FASTAPI_STATIC":"/home/jovyan/workspace/samgis-lisa-on-cuda/static","VIS_OUTPUT":"/home/jovyan/workspace/samgis-lisa-on-cuda/vis_output"}'
export WORKDIR="$HOME/workspace/samgis-lisa-on-cuda"
export XDG_CACHE_HOME="$HOME/.cache"
export PROJECT_ROOT_FOLDER="$HOME/"
export MPLCONFIGDIR="$HOME/.cache/matplotlib"
export TRANSFORMERS_CACHE="$HOME/.cache/transformers"
export PYTORCH_KERNEL_CACHE_PATH="$HOME/.cache/torch/kernels"
export FASTAPI_STATIC="$HOME/workspace/samgis-lisa-on-cuda/static"
export VIS_OUTPUT="$HOME/workspace/samgis-lisa-on-cuda/vis_output"
```
- execute the script `baremetal_entrypoint.sh` instead than `docker_entrypoint.sh`.
|