Package: threedi-raster-edits
threedi-raster-editsIntroductionThreedi Raster Edits is used internally at NENS for raster conversion, edits and checks.
Currently it includes the functions as alignment, replace_nodata, alignment checks, property checks and raster conversions to friction, interception and others.UsageWe can download rasters:>>> import threedi_raster_edits as tre
>>> landuse = tre.UUID.THREEDI_LANDUSE
>>> vector = tre.Vector(vector_path)
>>> feature = vector[0]
>>> rextract = tre.RasterExtraction(lizard_api_key)
>>> rextract.run("ouput/raster.tif", landuse_uuid, feature.geometry, cellsize=0.5, threads=1)Raster conversion:>>> from threedi_raster_edits import ThreediRasterGroup
>>> group = ThreediRasterGroup(dem_path, landuse_path=landuse_path, soil_path=soil_path)
>>> group.load_landuse_conversion_table(csv_landuse_path)
>>> group.load_soil_conversion_table(csv_soil_path)
>>> group.generate_friction()
>>> group.generate_permeability()
>>> group.generate_interception()
>>> group.generate_crop_type()
>>> group.generate_max_infiltration()
>>> group.generate_infiltration()
>>> group.generate_hydraulic_conductivity()
>>> group.friction.write("friction.tif")Raster checks:>>> group = ThreediRasterGroup(dem_path, friction_path="friction.tif")
>>> align_checks = group.check_alignemnt()
>>> property_checks = group.check_properties()Building interception can also be generated:>>> from threedi_raster_edits import Vector
>>> buildings = Vector("buildings.shp")
>>> group = ThreediRasterGroup(dem_path, buildings=buildings)
>>> group.generate_building_interception(10)
>>> group.interception.write("interception.tif")One can also use the internal raster class for easy raster processing as reproject, clipping, resampling, aligning, reading geometries and creating copies:>>> from threedi_raster_edits import Raster
>>> import numpy as np
>>> raster = Raster("dem.tif")
>>> raster.reproject(28992)
>>> raster.resample(1,1)
>>> raster.clip(Vector("clip.shp"))
>>> raster.align(Raster("template.tif"))
>>> copy = raster.copy(shell=True)
>>> clip = Vector("clip.shp")
>>> geometry = clip[0].geometry # geometry of first feature
>>> clip_values = np.nansum(raster.read(clip))For more advanced/larger processing you can also use raster loops:>>> output = raster.copy(shell=True)
>>> for tile in raster:
>>>     array = tile.array
>>>     array[array==1] = 20
>>>     output.array = array, tile.location
>>> output.write("tiles_adding.tif")Or just do something with the array:>>> output = raster.copy(shell=True)
>>> array = raster.array
>>> array = array *100
>>> output.array = array
>>> output.write("output.tif")InstallationWe can be installed with:$ pip install threedi-raster-edits(TODO: after the first release has been made)Development installation of this project itselfGDAL is not automatically installed, hence not available in the requirement,so please install gdal 3.2.0 yourself or use anaconda:$ conda create -n threedi_raster_edits python=3 gdal=3.2.0 rtree black pytestWe use python’s build-in “virtualenv” to get a nice isolated directory. You
only need to run this once:$ python3 -m venv --system-site-packages .A virtualenv puts its commands in thebindirectory. Sobin/pip,bin/pytest, etc. Set up the dependencies like this:$ bin/pip install -r requirements.txtThere will be a script you can run like this:$ bin/run-threedi-raster-editsIt runs themain()function inthreedi-raster-edits/scripts.py,
adjust that if necessary. The script is configured insetup.py(seeentry_points).In order to get nicely formatted python files without having to spend manual
work on it, run the following command periodically:$ bin/black threedi_raster_editsRun the tests regularly. This also checks with pyflakes, black and it reports
coverage. Pure luxury:$ bin/pytestThe tests are also run automaticallyon “github actions”for
“master” and for pull requests. So don’t just make a branch, but turn it into
a pull request right away:Prepend the title with “[WIP]”, work in progress. That way you make clear it
isn’t ready yet to be merged.Important: it is easy to give feedback on pull requests. Little comments
on the individual lines, for instance. So use it to get early feedback, if
you think that’s useful.On your pull request page, you also automatically get the feedback from the
automated tests.There’s alsocoverage reportingon coveralls.io (once it has been set up).If you need a new dependency (likerequests), add it insetup.pyininstall_requires. Local development tools, like “black”, can be added to therequirements.txtdirectoy. In both cases, run install again to actuall
install your dependency:$ bin/pip install -r requirements.txtReleasing the projectTo make a release, first install zest.releaser (globally):$ pip install zest.releaserMake sure you’re on master and that you’ve updated the changelog. Then release
it with “fullrelease”. You can press “enter” at every question:$ fullreleaseThe github action ought to upload the fresh package to pypi.Steps to do after generating with cookiecutterAdd a new project onhttps://github.com/nens/with the same name. Set
visibility to “public” and do not generate a license or readme.Note: “public” means “don’t put customer data or sample data with real
persons’ addresses on github”!Follow the steps you then see (from “git init” to “git push origin master”)
and your code will be online.Go tohttps://github.com/nens/threedi-raster-edits/settings/collaborationand add the teams with write access (you might have to ask someone with
admin rights to do it).Update this readme. Use.rstas the format.Ask Reinout to configure travis and coveralls.Remove this section as you’ve done it all :-)Changelog of threedi-raster-edits0.27 (2022-08-29)Bugfix: For raster tiles everything is not done in memory anymore.Cleanup for lizard including dependency installation.Updated for postgres upload. Works again.Updated blocksize estimation.Differences between stacked and adjacent rasters.No automatic console logging.0.26 (2022-07-11)Geometry field changes of connection nodes.0.25 (2022-07-08)Hotfix: removed aggregation_in_space in the v2_aggregation_settings0.24 (2022-07-08)Hotfix: requirements.txt0.23 (2022-07-07)Added two extra dependencies.0.22 (2022-07-07)Removed 3Di results.Removed 3Di scenarios.Klondike support.0.21 (2022-03-22)Scenarios > 3.8.00.20 (2022-03-22)QGIS changes for raster conversions.Include global cache, for storage in a pickle.Added easy threedi scenario downloads.0.19 (2022-02-28)Increased speed of vector loopsAdded examples in scriptsAdded intuitive progress bars inclusing for gdal0.18 (2021-11-29)Changed input for rextract.0.17 (2021-11-29)Added threading to lizard rextractAlso qgis compatible way of showing progressSimulations now in queue0.16 (2021-10-28)Fixed release error0.15 (2021-10-28)More efficient loading of threedirastergroup0.14 (2021-10-28)ThreediAPI and ThreediResults are now optional.0.13 (2021-10-19)Fixed vector reprojectionsAdded inflated rasterloops for rastersAdded rasterloops for rastergroupsAdded inflated rasterloops for rastergroupsAdded bbox clipping for rasters0.12 (2021-09-13)Improved difference algorithmRemove geometry fixed at every call, now call once with veotor.fix()fid= -1 will result in a fid which is the count0.11 (2021-09-06)Token release0.10 (2021-09-06)Added sqlite-model supportAdded api support0.9 (2021-05-06)Changed black format.0.8 (2021-05-06)Added clips for rastersAdded custom line string lengthsAdded vector interpolationAdded (partly) fix threedi rasters0.7 (2021-03-26)Fixed release process (same for 0.6/0.6).0.4 (2021-03-26)Fixed release process.Fixed tests.Added logging.Better memory usage of rasters.Small changes in vector, geometries.0.3 (2021-03-25)Automated pypi release.0.2 (2021-03-12)Changed the syntax of raster classChanged the imports to the main script: E.g., from threedi_raster_edits import raster, rastergroup etc.Changed the readme.Rewritten the geometry structure.0.1 (2021-03-11)Initial project structure created with cookiecutter andhttps://github.com/nens/cookiecutter-python-template