DOI: 10.7910/dvn/yy2xmg
title: WaterNet Outputs and Code

description: &lt;h1&gt;Introduction&lt;/h1&gt;
&lt;p&gt;
This data repository contains global raster and vector outputs for the WaterNet model at 20 meters, and it contains the code for three python modules related to the generation of this data. If using this data please cite the original WaterNet paper, where you can also find details of the modelling and data production process: "Pierson, Matthew., and Mehrabi, Zia. 2024. Mapping waterways worldwide with deep learning. arXiv. https://doi.org/10.48550/arXiv.2412.00050". 
&lt;/p&gt;
description: &lt;br&gt;
&lt;hr&gt;
&lt;h1&gt;Projection&lt;/h1&gt;
All files use EPSG: 4326.
&lt;hr&gt;
description: &lt;h2&gt;Raster Data&lt;/h2&gt;
&lt;h3&gt;Naming Convention&lt;/h3&gt;
The raster outputs have the following naming convention {xtile}_{ytile}.tif and correspond to a zoom level 6
&lt;a href="https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Resolution_and_Scale"&gt;xyz map tile.&lt;/a&gt;
We have included geoparquet file named
&lt;a href="https://dataverse.harvard.edu/file.xhtml?fileId=10651101"&gt; WaterNet_20m_raster_files_bbox.parquet &lt;/a&gt;
that has the bounding boxes for each raster file.

&lt;h3&gt;Datatypes&lt;/h3&gt;

The raster files have an unsigned 8-bit integer datatype, with 0 indicating a low probability of a waterway, and 255 indicating a high probability of waterway. During the vectorization process, we consider two sharp cut offs, with a value less than 255*0.1 to not a waterway, and a value greater than 255*0.5 to be a waterway if it is connected to a stream in TDX-Hydro (i.e if its connected component intersects a stream in TDX-Hydro). See the paper for more information.

&lt;h3&gt;Known Issues&lt;/h3&gt;

There are known issues with the raster product. These include lower accuracy of capturing swampy areas and deserts (although alternative feature weightings of the model can allow for better capture of swamps, and our vectorization process aims to resolve the issues in deserts). We also note higher noise in areas where it is difficult to create cloud free composites (coastal areas, near the equator); and future integration of SAR data may help alleviate these particularly on a near-real time deploy. There is also artifact in Greenland (missing cells) that we expect is due to Sentinel-2 feature inputs, for which further investigation and back fill is likely required.

&lt;hr&gt;
description: &lt;h2&gt;Vector Data&lt;/h2&gt;
&lt;h3&gt;Naming Convention&lt;/h3&gt;
The vector outputs have the following naming convention {hydrobasins level 2 id}_{part}.parquet. &lt;a href="https://www.hydrosheds.org/products/hydrobasins"&gt;The level 2 HydroBasins can be obtained for their website.&lt;/a&gt; As the naming convention suggests, each file corresponds to part of a hydrobasin level 2 basin. Since many of the files were larger than 2.5 GB, they were split into parts to satisfy the Harvard Dataverse individual file size limit.
&lt;h3&gt;Dataframe Column Descriptions&lt;/h3&gt;
&lt;dl&gt;
 &lt;dt&gt;&lt;b&gt;stream_id&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data type: int &lt;br&gt;
 &lt;/dd&gt;
 &lt;dd&gt;
 Description: A unique id for the stream segment.
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;target_stream_id&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: int &lt;br&gt;
 Description: The stream_id of the target. IE the stream_id of the next adjacent downstream stream.
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;source_stream_ids&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: list(int) &lt;br&gt;
 Description: A list of the source stream_id. IE a list containing the ids of all adjacent streams flowing into this stream. Can contain more than 2 ids.
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;stream_order&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: int &lt;br&gt;
 Description: &lt;a href="https://en.wikipedia.org/wiki/Strahler_number"&gt; The Strahler stream order of this stream segment. &lt;/a&gt;
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;from_tdx&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: bool &lt;br&gt;
 Description: True if this stream segment appears in &lt;a href="https://earth-info.nga.mil/index.php?dir=geosci&action=geosciences"&gt;TDX-Hydro&lt;/a&gt;, otherwise False. Note, this may only be a segment of a stream in TDX-Hydro.
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;tdx_stream_id&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: int &lt;br&gt;
 Description: Each stream in this dataset falls in a drainage basin in the TDX-Hydro dataset. This value corresponds to the streamID in the TDX-Hydo drainage basins dataset that this stream is in (or LINKNO in the TDX-Hydo stream network dataset).
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;intersects_lake&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: bool &lt;br&gt;
 Description: True if the geometry intersects a lake in the &lt;a href="True if the geometry intersects a lake in the HydroLakes v 1.0 Dataset"&gt; HydroLakes v 1.0 Dataset. &lt;/a&gt;
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;length_m&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: &lt;br&gt;
 Description: The length of the geometry computed using &lt;a href="https://pyproj4.github.io/pyproj/stable/api/geod.html#pyproj.Geod.geometry_length"&gt;pyproj.Geod.geometry_length&lt;/a&gt;
 &lt;/dd&gt;
 &lt;dt&gt;&lt;b&gt;geometry&lt;/b&gt;&lt;/dt&gt;
 &lt;dd&gt;
 Data Type: LineString (polyline) &lt;br&gt;
 Description: The Geometry of this segment.
 &lt;/dd&gt;
&lt;/dl&gt;
&lt;h3&gt;Known Issues&lt;/h3&gt;
Vectorization of any large body of water (lakes, swamps, wide rivers, etc) can result in multiple geometries that should all be a single geometry. One method for identifying such artifacts are to search for streams of order 1 whose
target streams are high in order. Since these artifacts often occur in lakes, we have included the column "intersects_lake" in the output,
which indicates if the geometry intersects a lake in HydroLakes.
&lt;hr&gt;
description: &lt;hr&gt;
&lt;h1&gt;WaterNet Code&lt;/h1&gt;
The code needed to make and train WaterNet and vectorize its outputs has been supplied in WaterNet_code directly in this repository.The code for inference of WaterNet globally has not been included as the large amount of data required makes such an effort difficult to generalize across compute hardware. All of the code will also be made public on GitHub at the links below in the near future.
&lt;ul&gt;
 &lt;li&gt;&lt;a href="https://github.com/Better-Planet-Laboratory/WaterNet_vectorize"&gt;WaterNet&lt;/a&gt;&lt;/li&gt;
 &lt;li&gt;&lt;a href="https://github.com/Better-Planet-Laboratory/WaterNet_training_and_evaluation"&gt;WaterNet Training and Evaluation.&lt;/a&gt;&lt;/li&gt;
 &lt;li&gt;&lt;a href="https://github.com/Better-Planet-Laboratory/WaterNet_vectorize"&gt;WaterNet Vectorization&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
description: &lt;h2&gt;
 Acknowledgments
&lt;/h2&gt;
The authors would like to thank Bridges to Prosperity for enabling the project. This project was in part funded by Bridges to Prosperity under the grant “Remote Impact Assessment of Rural Infrastructure Development” to the Better Planet Laboratory (https://betterplanetlab.com/).