Package: geowrangler
GeowranglerTools for wrangling with geospatial dataOverviewGeowrangleris a python package for geodata wrangling. It helps you build data transformation workflows with no out-of-the-box solutions from other geospatial libraries.We have surveyed our past geospatial projects to extract these solutions for our work and hope it will be useful for others as well.Our audience are researchers, analysts and engineers delivering geospatial projects.Wewelcome your comments, suggestions, bug reports and code contributionsto makeGeowranglerbetter.ModulesGrid Tile GenerationGeometry ValidationVector Zonal StatsRaster Zonal StatsArea Zonal StatsDistance Zonal StatsDemographic and Health Survey (DHS) Processing UtilsGeofabrik (OSM) Data DownloadOokla Data DownloadCheckthis page for more details about our RoadmapInstallationpip install geowranglerDocumentationThe documentation forthe package is available hereDevelopmentDevelopment SetupIf you want to learn more aboutGeowranglerand explore its inner workings, you can setup a local development environment. You can run geowrangler's jupyter notebooks to see how the different modules are built and how they work.Pre-requisitesOS: Linux, MacOS, Windows Subsystem for Linux (WSL) on WindowsRequirements:python 3.7 or highervirtualenv, venv or conda for python environment virtualizationpoetry for dependency managementGithub Repo ForkIf you plan to make contributions to geowrangler, we encourage you tocreate your forkof the Geowrangler repo.This will then allow you to push commits to your forked repo and
then create a Pull Request (PR) from your repo to the main geowrangler
repo for approval by geowrangler's maintainers.Development InstallationWe recommend creating a virtual python environment viavirtualenvorcondafor your development environment. Please see the relevant documentation for more details on installing these python environment managers. Afterward, continue the geowrangler setup instructions below.Set-up with virtualenvFirst, installlibgeos( version >=3.8 required for building pygeos/shapely) through the ff. commands.Seelibgeos documentationfor installation details on other systems.sudo apt update # updates package info
sudo apt install build-essential # installs GCC
sudo apt install libgeos-devNext, set-up your Python env withvirtualenvand install pre-commits and the necessary python libs by running the following commands.Remember to replace<your-github-id>in the github url below with your GitHub ID to clone from your fork.git clone https://github.com/<your-github-id>/geowrangler.git
cd geowrangler
virtualenv -p /usr/bin/python3.9 .venv
source .venv/bin/activate
pip install pre-commit "poetry>=1.2.0"
pre-commit install
poetry installYou're all set!Run the teststo make sure everything was installed properly.Whenever you open a new terminal, you cancd <your-local-geowrangler-folder>and  runpoetry shellto activate the geowrangler environment.Set-up with condaRun the following commands to set-up a conda env and install geos.conda create -y -n geowrangler-env python=3.9 # replace geowrangler-env if you prefer a different env name
conda deactivate # important to ensure libs from other envs aren't used
conda activate geowrangler-env
conda install -y geosThen run the following to install the pre-commits and python libs.cd geowrangler # cd into your geowrangler local directory
pip install pre-commit poetry>=1.2.0
pre-commit install
poetry installYou're all set!Run the teststo make sure everything was installed properly.Whenever you open a new terminal, runconda deactivate && conda activate geowrangler-envto deactivate any conda env, and then activate the geowrangler environment.Jupyter Notebook DevelopmentThe code for thegeowranglerpython package resides in Jupyter notebooks located in thenotebooksfolder.Usingnbdev, we generate the python modules residing in thegeowranglerfolder from code cells in jupyter notebooks marked with an#exportcomment. A#default_exp <module_name>comment at the first code cell of each notebook directsnbdevto put the code in a module named<module_name>in thegeowranglerfolder.See thenbdev clidocumentation for more details on the commands to generate the package as well as the documentation.Running notebooksRun the following to view the jupyter notebooks in thenotebooksfolderpoetry run jupyter labGenerating and viewing the documentation siteTo generate and view the documentation site on your local machine, the quickest way is to setupDocker. The following assumes that you have setup docker on your system.poetry run nbdev_build_docs --mk_readme False --force_all True
docker-compose up jekyllAs an alternative if you don't want to useDockeryou caninstall jekyllto view the documentation site locally.nbdevconverts notebooks within thenotebooks/folder into a jekyll site.From this jekyll site, you can then create a static site.To generate the docs, run the followingpoetry run nbdev_build_docs -mk_readme False --force_all True
cd docs && bundle i && cd ..To run the jekyll site, run the followingcd docs
bundle exec jekyll serveRunning testsWe are usingpytestas our test framework. To run all tests and generate a generate a coverage report, run the following.poetry run pytest --cov --cov-config=.coveragerc -n autoTo run a single test or test file# for a single test functionpoetryrunpytesttests/test_grids.py::test_create_grids# for a single test filepoetryrunpytesttests/test_grids.pyContributingPlease readCONTRIBUTING.mdandCODE_OF_CONDUCT.mdbefore anything.Development NotesFor more details regarding our development standards and processes, please seeour wiki.