Package: threedigrid
Threedigrid: The 3Di grid admin frameworkThe Python package for the threedigrid administration.Free software: BSD licenseDocumentation:https://threedigrid.readthedocs.io.OverviewFeaturesaccess to the threedicore administration by a single instance of theGridH5Adminobjectquery the model data by pre-defined subsets and django style filtersexport model data to gis formats like shapefile, geopackageserialize model data as geojsonQuick startThe standard threedigrid distribution is pretty lightweight, installing as little dependencies
as possible. If you want to make use of all capabilities threedigrid has to ofter (e.g. spatial
operations and command line tools) install like this:$ pip install threedigrid[geo,results]Console scriptsUsing the 3digrid_explore shortcut, simply run:$ 3digrid_explore --grid-file=<path to grid file> --ipyThis will invoke an ipython session with aGridH5Admininstance already loaded.To get a quick overview of the threedimodels meta data omit the--ipyoption or
explicitly run:$ 3digrid_explore --grid-file=<the to grid file> --no-ipyThis will give you output like this:Overview of model specifics:

model slug:              v2_bergermeer-v2_bergermeer_bres_maalstop-58-b1f8179f1f3c2333adb08c9e6933fa7b9a8cd163
threedicore version:     0-20180315-3578e9b-1
threedi version:         1.63.dev0
has 1d:                  True
has 2d:                  True
has groundwater:         True
has levees:              True
has breaches:            True
has pumpstations:        True(I)Python shellGet a grid admin instance:from threedigrid.admin.gridadmin import GridH5Admin

f = 'gridadmin.h5'
ga = GridH5Admin(f)The grid admin directly holds some model specific attributes like whether the model has a 1D or 2D
or groundwater section:In [4]: ga.has_groundwater
Out[4]: False

In [5]: ga.has_1d
Out[5]: TrueThere are different types of filters but a filter, generally speaking, acts on field. That means you can
filter by value. If you have a line model instance you can filter the data by the kcu field:ga.lines.filter(kcu__in=[100,102])or by the lik value:ga.lines.filter(lik__eq=4)The filtering is lazy, that is, to retrieve data you have to call data explicitly:ga.lines.filter(lik__eq=4).data  # will return an ordered dictThe structure control actions netcdf can also be analyzed and exported using threedigrid:from threedigrid.admin.gridresultadmin import GridH5StructureControl
from threedigrid.admin.structure_controls.exporters import structure_control_actions_to_csv

gst = GridH5StructureControl("gridadmin.h5", "structure_control_actions_3di.nc")
gst.table_control
structure_control_actions_to_csv(gst, "test.csv")Remote procedure callsCurrently only the client-side is included. The server-side might be added in a later stage.
Note: this is an advanced feature used inside the 3Di stack, probably you don’t need this.
Note2: you need Python 3.7 or higher for this to work.Installation:$ pip install threedigrid[rpc]Basic usage:ga = GridH5ResultAdmin('rpc://REDIS_HOST/SIMULATION_ID', 'rpc://REDIS_HOST/SIMULATION_ID')
# Replace REDIS_HOST and SIMULATION_ID with actual values.
future_result = ga.nodes.filter(lik__eq=4).data
data = await future_result.resolve()Subscription usage:subscription = await future_result.subscribe()

async for item in subscription.enumerate():
      # do something with itemCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History2.2.4 (2024-02-06)Substance count regex fixSet substance name on Nodes2.2.3 (2024-01-12)Fix 1D/2D data swap for water quality results2.2.3b (2024-01-02)Added build/release Github action.2.2.2 (2023-12-21)Bugfix:onlyin case ofboundaries: correct sorting of timeseries in allTimeSeriesSubsetArrayField. This
affected: Nodes [infiltration_rate_simple, ucx, ucy, leak, intercepted_volume, q_sss]
Lines: [qp, up1, breach_depth, breach_width]Note: Thedepthandwidthfield are broken forbreachesobject.
Please usebreach_depthandbreach_widthon thelinesobject instead.2.2.1 (2023-12-05)Fix Lines lookup index bug for single cell models2.2.0 (2023-11-07)Add water quality results NetCDF interface.Drop Python 3.7 support2.1.2 (2023-11-03)Retry PyPi release.2.1.1 (2023-11-03)Add missing init file to structure control folder2.1.0 (2023-10-16)Added structure control actions NetCDF interface and CSV exporter.2.0.6 (2023-07-07)Addedsewerageandsewerage_typefields to Line model.2.0.5 (2023-03-27)Switch to using ‘line_geometries’ for Breaches model.2.0.4 (2023-03-24)Make ‘line_coords’ the geometry on breaches geojson output.2.0.3 (2023-03-22)Bugfix: Dataset instance check is not generic.2.0.2 (2023-03-21)Added ‘line_coords’ to Breaches model and default export fields.2.0.1 (2023-02-22)Addedmetatable to Geokpackage export with meta information about 3Di model.Added 2D groundwater boundary line types 600, 700, 800, 900.
Eextended the ‘2D_GROUNDWATER’ subset.2.0.0 (2023-01-11)Removed gridadmin.lines.breaches.Removed MappedSubsetArrayField.Added ‘code’ and ‘display_name’ to breaches.Added missing Geopackage fieldsbottom_levelanddrain_levelfor nodes. Renamedconnection_node_storage_aretoconnection_node_storage_area.Dropped python 3.6 support.Fixed compatibility with shapely 2.*.1.2.5 (2022-10-17)Optimized geojson export memory usage.Removed majority of properties from the flowlines (1D2D) export.1.2.4 (2022-10-03)Allow exporting flowlines (1D2D) to geojson or shapefile.1.2.3 (2022-06-01)Divers Geopackage exports improvements.1.2.2 (2022-05-31)New release since 1.2.1 was already present at PyPi.1.2.1 (2022-05-31)Added new OGR based exporter for Geopackage exports.
Currently only has default export settings for lines, cells, nodes and pumps.Fixed error in crs attribute.Removed deprecation warning on gridadmin.breaches and added it to
gridadmin.lines.breaches.Fixed export_breaches for new gridadmins.1.2.0 (2022-03-09)Added discharge_coefficient_positive and discharge_coefficient_negative
to breaches.Removed threedicore version check on GridH5ResultAdmin initialization.Added crs attribute to GridH5Admin. Pyproj >=2.2 is required.Drop python 2.7 support.1.1.14 (2022-02-16)Fix geometry selection filtering, gridadmin can now contain nan values.Added dimp attribute to nodes.1.1.13 (2021-12-09)Added flod and flou attributes to lines for possible reading obstacle heights.1.1.12 (2021-11-18)Added ‘has_dem_averaged’ attribute to cells.1.1.11 (2021-11-02)Exporters now export NaN and -9999.0 float values as NULL.Fixed exporting string dtype fields (e.g. cont_type) in OGR exporter. For instance,
the string “b’something’” now is written as “something”.Skip the dummy element (with id=0) in all exporters.Set the FID (feature ID) in the OGR (shapefile/geopackage/some geojson) exporters.Deprecate specific serializers.py under threedigrid.admin.breaches, .lines, .nodes and
.pumps.1.1.10 (2021-11-01)Fix GeoJSON levees coordinate order.1.1.9 (2021-10-25)Renamed the ‘s_1d’ field under lines to ‘ds1d_half’.Added ‘initial_waterlevel’ to nodes.1.1.8 (2021-10-25)Added ‘nodes_embedded’, available under gridadmin Class.1.1.7 (2021-10-18)Fixed timeseries filtering with h5py>=3.1.x1.1.6 (2021-08-31)Added CrossSection model to ORM.Added following fields to lines:dpumax cross1 cross2 ds1d s1d cross_weight invert_level_start_point invert_level_end_pointAdded following fields to nodes:calculation_type drain_level storage_area dmaxCreated new subset 1D for Nodes.1.1.5 (2021-08-10)Release on pypi (repo has no Github actions)1.1.4 (2021-08-10)Replace nan with null in geojson output.Remove requirements files, only keep one for development in docker.1.1.3 (2021-06-01)Bugfix: geojson levees export also crashed
due to 3.8.10 and numpy 1.19.11.1.2 (2021-05-28)Fixed only filter for aggregate result admin. (#121)Added cross_pix_coords field to lines.Bugfix: geojson line_geometries export crashes with
Python 3.8.10 and numpy 1.19.1. Needed explicit astype conversion1.1.1 (2021-03-30)Reduced the source distribution filesize by removing the tests.1.1.0 (2021-03-29)Bumped asyncio-rpc to 0.1.10Fixed GeoJSON export with pyproj <= 1.9.6.Fixed compatibility with h5py 3.*.Added gridadmin.grid.transform.Fixed gridadmin.grid.n2dtot and .dx propagation.Added gridadmin.cells.iter_by_tile() and .get_extent_pixels().1.0.27 (2021-02-22)Bumped asyncio-rpc to 0.1.91.0.26 (2021-02-05)Fixed rpc gridadmin properties1.0.25 (2020-09-15)Bugfix: crest_level is also inverted by Inpy. Use
the raw value in the prepare step to include the
correct (non inverted) value.1.0.24 (2020-09-02)Removed numba as dependency, since it did not really give any
performance gain on Linux.1.0.23 (2020-09-02)Creating fresh release after upload failed.1.0.22 (2020-09-02)Added extra field ‘discharge_coefficient’ to channels and pipes. These
fields default to 1.0.Bugfix: don’t use the z-coordinate when making line_geometries during the prepare step1.0.21 (2020-07-17)Invert_level_start_point and end point where inverted
by Inpy. After this change the values will be correct again,
however present gridadmin files will still have the incorrect value.AddedBreachesmodel under lines with specific breach fields
mapped from the ‘breaches’ h5py datagroup.AddedMappedSubsetArrayFieldallowing to map arrays from other
h5py datagroups to a model on another datagroup subset. Breaches
uses this to map the array’s under ‘breaches’ to ‘lines’1.0.20.12 (2020-07-14)Fixed problem with previous release1.0.20.11 (2020-07-14)Nodgrid generation bugfixes1.0.20.10 (2020-07-07)Added missing numba requirement in setup.py1.0.20.9 (2020-07-07)Fixed RPC breaches/pumps bugAdded fast nod_grid generation on cells1.0.20.8 (2020-05-22)Add groundwater_cells to exporter for frontend.1.0.20.7 (2020-05-18)Addedcontent_pkto the export_constants of all structures which have acontent_pk.1.0.20.6 (2020-04-15)An empy array [] is returned now instead of None if there is no
dataset.Line geojson items need to use line_geometry valuesBugfix forModel._get_subset_idxnot instantiating new subsets with their parent’s
mixins.1.0.20.5 (2020-04-01)Use ‘ga.xxx.id.size’ to check if certain submodels (like channels/weirs/manholes)
are available for geojson exports.1.0.20.4 (2020-03-31)Allow older pyrpoj versions. (pre 2.2.0)Bugfix for timeseries start_time=0 selection and allow indexes=slice(x,x,x)
in combination with sample() method.Added try-except surrounding all imports of the packagegeojson. This package is
only available when threedigrid is installed with the extra [geo] extension.AddORIFICES_EXPORT_FIELDSto export_constants.1.0.20.3 (2020-03-18)Add extra fieldpixel_widthto cellsBugfix for GeometryIntersectionFilter: filter was only checking on
intersecting bounding boxes1.0.20.2 (2020-03-06)Thesamplemethod needs to skip the last timestamp for SWMR
to work correctly. (time dataset can have one item more
than datasets with timeseries)1.0.20.1 (2020-02-26)Bugfix: reprojection with no coordinates (empty array’s)1.0.20 (2020-02-19)AddedGeometryIntersectionFilter.Added general GeoJsonSerializer which allows you to specify the field names
you want to serialize and extract to geojson. The GeoJsonSerializer allows
you to specify nested fields.Added a set of standard export fields for each model.Automatically pick the correct serializer based on file extention.json/.geojson –> to_geojson.gpgk –> to_geopackage.shp –> to_shape1.0.19.1 (2020-02-04)Minor bugfix, need to check if h5py filepath is a str or bytes string
during initialization1.0.19 (2020-01-31)First release with RPC integration.1.0.19rc3 (2020-01-14)Bumped version of asyncio-rpc1.0.19rc2 (2020-01-14)Fixed incorrect version number1.0.19rc1 (2020-01-14)Added RPC datasource which enables to use the majority of
threedigrid in a RPC setting. Uses asyncio-rpc for
sending/handling RPC calls.RPC datasource allows both one time executing (resolve()) and pub/sub
(subscribe()) functionialty.Refactored to allow using RPC datasource1.0.18 (2019-11-28)Only use pyproj Transformer if it is present
else revert to old transform method1.0.17 (2019-11-28)Addedcontent_pkto the pumps model.Bumped package versionsReduced reprojection overhead of line_geometries.1.0.16 (2019-07-08)Removed max capacity from Orifice model/serializer.1.0.15 (2019-07-05)Fixed group update for default null values.1.0.14 (2019-06-19)Do not use0has a default when converting database objects to numpy
arrays in the prepare phase.1.0.13 (2019-05-01)Fixed_field_model_dictbeing a class variable.1.0.12 (2019-04-18)Added sumax to nodes1.0.11 (2019-02-01)Bug fix inh5py_filemethod mapping.1.0.10 (2019-01-31)Added sources and sinks (q_sss) to threedigrid.1.0.9 (2019-01-31)Manholes preparation fixed mapping inconnection_node_pk.Addedto_structured_arraymethod for retrieving (filtered) results
as Numpy structured array instead of an OrderedDict1.0.8 (2019-01-03)Set fixed type to the fieldscode,display_nameandshape. These fields
now have a fixed lenght of 32, 64 and 4 characters respectively.1.0.7 (2018-11-21)Bug fix: dict.values() and dict.keys() in python 3 are causing some
unintended behaviour.1.0.6 (2018-11-14)New release due to failing uploads.1.0.5 (2018-11-14)Add aggregation option ‘current’ to volume and intercepted_volume.Using a non-tuple sequence for multidimensional indexing is deprecated; usearr[tuple(seq)]instead ofarr[seq].Properties should be strings so we can use string methods on them.Do not prepare levees if there aren’t any.Split requirements file to allow for finer grained builds (for instance to
generate the documentation).Add ‘intercepted_volume’ to NodesAggregateResultsMixin.Split requirements file to allow for finer grained builds (for instance to
generate the documentation).1.0.4 (2018-10-17)Added BooleanArrayField for boolean values and use it foris_manholefilter.
NO_DATA_VALUE is interpreted as False.1.0.3 (2018-09-17)Do not throw exception on cftimeImportError1.0.2 (2018-09-17)Add boolean filter for manholes.1.0.1 (2018-09-11)Patch for converting numpy strings/bytes to float for both python2/3.Dropped NetCDF library and replaced opening NetCDF files with h5pyBumped h5py to 2.8.01.0 (2018-09-04)Made threedigrid >= Python 3.5 compatible.0.2.8 (2018-07-23)Bug fix for issue #44: use the methodget_filtered_field_value()instead
ofget_field_value()for the count property.Properly closes netcdf-file inGridH5ResultAdmin.0.2.7 (2018-05-24)Add export functions for 2D to theexport_all()collection.0.2.6 (2018-05-17)Do not usepkg_resourcesto determine the current version but use
zest_releaser to update the version string in threedigrid/init.py0.2.5 (2018-05-16)Use the customNumpyEncoderto convert specific numpy types to native
python types when calling(geo-)json.dumps().0.2.4 (2018-05-15)Introducing subset fields that can be used to query results that are collected
only for subsets of the model, like the 2D section.0.2.3 (2018-05-14)Fix lookup_index functionality for composite fields.Make model name property optional. That is, ‘unknown’ will be returned if the
name cannot be derived.Changed Depth/width fields on breach-timeseries to breach_depth and breach_width.0.2.2 (2018-04-30)_get_composite_meta()does not raise an AssertionError anymore if
composite field attributes differ. Instead a warning is issued.0.2.1 (2018-04-26)Bug fix:threedicore_result_versionmust be a property.0.2 (2018-04-26)Added additional exporters for2D_GROUNDWATER2D_OPEN_WATER2D_VERTICAL_INFILTRATIONAdded methodget_model_instance_by_field_nameto theGridH5ResultAdminclass. This makes it possible to do reverse lookups
in situations where you have a field name but do not know which model it
belongs to. N.B the field must be unique otherwise anIndexErrorwill
be raised.Added propertydt_timestampsto the timeseries_mixin module.The version number is added to the__init__file dynamically using thepkg_resourcesAPI.Timestamps of all timeseries fields are shown for aggregation results.Timestamps in the aggregation results are filtered when retrieving subsets of timeseries.Introducing the ModelMeta class. Its main purpose at this moment is to compute all
possible combinations of composite_fields and aggregation variables.Fixed return statement of method slice (in class Model) which now takes**new_class_kwargs.Empty or missing datasets are now displayed asnp.array(None)instead of
raising an error.0.1.6 (2018-04-18)New release using twine 1.11.0.1.5 (2018-04-18)Added support for composite fields which can be used to fetch data from
multiple source variables as a single field. Like this
result_3di netcdfs can be queried the same way as gridadmin files.0.1.4 (2018-04-08)Changed ResultMixin to dynamically add attributes based on the netcdf
variables.Added basic result proccesing for line/node data.The filter mask is computed only for array’s affected and
before applying it to all array’sThe ‘only’ filter works much faster because the filter mask
is only applied on fields that are affected.The filter mask is cached on the line/node instance after getting
the first value. You can thus do something like:queryset = gridadmin.lines.filter(kcu=2)
ids = queryset.id
line_coords = queryset.line_coordsand the filter mask will only be computed once.Add click console scripts3digrid_exploreand3digrid_exportfor
quick overviews and data exports.Make ogr/gdal imports optional to avoid breaking parts of the documentation.Added documentation and setup forsphinxdocumentation pipeline.Use linear referencing for embedded channels to keep the original geometry
intact when preparing line geometries for visualisation.Define extra’s to make the standard threedigrid distribution as
lightweight as possible.Convert strings inattrstonumpy.string_to fix crashes under
Windows.0.1.3 (2018-03-16)Remove propertyhas_groundwaterfromGridH5Admin.
Should always be provided by the threedicore itself. Gives a warning for
backwards compatibility.0.1.2 (2018-03-12)Get model extent now always returns a bbox (minX, minY, maxX, maxY)0.1.1 (2018-03-06)All imports are absolute.Added install info using pip.0.1.0 (2018-03-05)First release with fullrelease.