Package: raster-loader
raster-loaderPython library for loading GIS raster data to standard cloud-based data warehouses that
don't natively support raster data.Raster Loader is currently tested on Python 3.8, 3.9, 3.10, and 3.11.DocumentationThe Raster Loader documentation is available atraster-loader.readthedocs.io.Installpipinstallraster-loaderInstalling from sourcegitclonehttps://github.com/cartodb/raster-loadercdraster-loader
pipinstall.UsageThere are two ways you can use Raster Loader:Using the CLI by runningcartoin your terminalUsing Raster Loader as a Python library (import raster_loader)CLIAfter installing Raster Loader, you can run the CLI by typingcartoin your terminal.Currently, Raster Loader supports uploading raster data toBigQuery.
Accessing BigQuery with Raster Loader requires theGOOGLE_APPLICATION_CREDENTIALSenvironment variable to be set to the path of a JSON
file containing your BigQuery credentials. See theGCP documentationfor more information.Two commands are available:Uploading to BigQuerycarto bigquery uploadloads raster data from a local file to a BigQuery table.
At a minimum, thecarto bigquery uploadcommand requires afile_pathto a local
raster file that can beread by GDALand processed withrasterio. It also requires
theproject(theGCP project name)
anddataset(theBigQuery dataset name)
parameters. There are also additional parameters, such astable(BigQuery table
name) andoverwrite(to
overwrite existing data).For example:cartobigqueryupload\--file_path/path/to/my/raster/file.tif\--projectmy-gcp-project\--datasetmy-bigquery-dataset\--tablemy-bigquery-table\--overwriteThis command uploads the TIFF file from/path/to/my/raster/file.tifto a BigQuery
project namedmy-gcp-project, a dataset namedmy-bigquery-dataset, and a table
namedmy-bigquery-table. If the table already contains data, this data will be
overwritten because the--overwriteflag is set.Inspecting a raster file on BigQueryUse thecarto bigquery describecommand to retrieve information about a raster file
stored in a BigQuery table.At a minimum, this command requires aGCP project name,
aBigQuery dataset name, and aBigQuery table name.For example:cartobigquerydescribe\--projectmy-gcp-project\--datasetmy-bigquery-dataset\--tablemy-bigquery-tableUsing Raster Loader as a Python libraryAfter installing Raster Loader, you can import the package into your Python project. For
example:fromraster_loaderimportrasterio_to_bigquery,bigquery_to_recordsCurrently, Raster Loader supports uploading raster data toBigQuery. Accessing BigQuery with Raster Loader requires theGOOGLE_APPLICATION_CREDENTIALSenvironment variable to be set to the path of a JSON
file containing your BigQuery credentials. See theGCP documentationfor more information.You can use Raster Loader to upload a local raster file to an existing
BigQuery table using therasterio_to_bigquery()function:rasterio_to_bigquery(file_path='path/to/raster.tif',project_id='my-project',dataset_id='my_dataset',table_id='my_table',)This function returnsTrueif the upload was successful.You can also access and inspect a raster file from a BigQuery table using thebigquery_to_records()function:records_df=bigquery_to_records(project_id='my-project',dataset_id='my_dataset',table_id='my_table',)This function returns a DataFrame with some samples from the raster table on BigQuery
(10 rows by default).DevelopmentSeeCONTRIBUTING.mdfor information on how to contribute to this
project.ROADMAP.mdcontains a list of features and improvements planned for future
versions of Raster Loader.