Package: validata-table
Validata TableValidata Tableis python package used as a tabular data validation service.It includes four subpackages, where you can find their respective documentations :validata_corevalidata_uivalidata_apitestsused for testing the project (in development environment or in continuous integration)It offers also a command line toolvalidataused to validate tabular data.
Seevalidata_core/README.mdfor more details.Usingvalidata-tablepackageYou can use locally this packagevalidata-table, doing:pip install validata-tableThis allows you to usevalidatacommand line tool to validate tabular data:validata --helpDevelopmentThis project is based onDockerto use a local environement developement.This project includes a Makefile, which allows you to run predefined actions
by running specific commands.Dependency management tool used :Poetry version 1.6.1RequirementsFirst installDocker Engineanddocker-compose >= version 2on your machine if not already done.Then you may clone source code in your deployment directory:git clone https://gitlab.com/validata-table/validata-table.git
cd validata-tableRun on development local environmentConfiguration is done by editing environment variables in.env,
(see.env.examplefile to set.envfile)Warning: Docker env files do not support using quotes around variable values!Launch the development local environment, thanks to the makefile command:# in validata-table/makeserve_devThis launches two docker containers:validata-table-api-devvalidata-table-ui-devValidata Table API (usingvalidata-table-api-devdocker container)To access to the API of Validata Table click onhttp://localhost:5000/Try Validata Table APIValidata Table UI (usingvalidata-table-ui-devdocker container)To access to the API of Validata Table click onhttp://localhost:5001/Validata Table command line tool (usingvalidata-table-api-devdocker container)To use validata command line tool in the docker development environment, run:docker exec -it validata-table-api-dev bash
validata --helpTestTo launch tests in the development environment, run:make testLintingSome code linting tools are configured for this project:black: to format code, runmake blackisort: to format import, runmake isortflake8: to enforce style coding, runmake flake8pyright: to check static types.pyrightwill be executed in local virtual environment withpoetry:First you need to create a virtual environment .venv at the root of the project
and configure it:# At /validata-table/ root 
python3.9 -m venv .venv  # install virtual environement locally
poetry config virtualenvs.in-project true
poetry config --list # Check if correctly configured
>>>
...
virtualenvs.in-project = true
...

poetry install  # install project dependenciesThen execute locallypyrightwithpoetry:poetry run pyright .Continuous IntegrationThe continuous integration is configured in.gitlab-ci.ymlfileRelease a new versionOn master branch :Update version inpyproject.tomlandCHANGELOG.mdfilesUpdate version Docker images used indocker-compose.ymlfile:registry.gitlab.com/validata-table/validata-table/validata-table-api:vX.X.Xregistry.gitlab.com/validata-table/validata-table/validata-table-ui:vX.X.XCommit changes usingReleaseas commit messageCreate git tag (starting with "v" for the release)git tag -aGit push:git push && git push --tagsCheck that pypi package is created and container images for validata_ui and validata_api are well-built
(validata-table pipelines)Creating and pushing a new release will trigger the pipeline in order to automatically:publish a new version ofvalidata-tablepackage onPyPIbuild a new tag of the Docker imagevalidata-table-ui, based on the new version just created of the packagevalidata-table, and publish it on the gitlab container
registryvalidata-table-ui,
used to run user interface Validatabuild a new tag of the Docker imagevalidata-table-api, based on the new version just created of the packagevalidata-tableand publish it on the gitlab container
registryvalidata-table-api,
used to run the API of ValidataThis pipeline runs when a new tag under the format 'vX.X.X' is pushed.Deploy to productionYou can deploy this project on your own production server by using Docker.Production environment is based on Docker imagesvalidata-table-uiandvalidata-table-apihosted on gitlab container registriesvalidata-table-uiandvalidata-table-apiTo deploy in production, you can follow these steps described below.First you may clone source code in your deployment directory:git clone https://gitlab.com/validata-table/validata-table.git
cd validata-tableConfiguration is done by editing environment variables in.env,
(see.env.examplefile to set.envfile).Warning: Docker env files do not support using quotes around variable values !Launch the docker production environment with makefile:make serve_prodOR launch the docker production environment withdocker composecommand:docker compose -f docker-compose.yml up --build -dThen:To access to the API of Validata Table click onhttp://localhost:<PORT_NUMBER_API>/
(replacing PORT_NUMBER_API with the value you choose)To access to the UI of Validata Table click onhttp://localhost:<PORT_NUMBER_UI>/
(replacing PORT_NUMBER_UI with the value you choose)To access to thevalidatacommand lines tool:docker exec -it validata-table-api bash
validata --helpHistoryTo keep track of the project's history,Validata Tablecomes from the merge of four gitlab repositories :Validata coreValidata UIValidata APIValidata Docker