Package: torchgeo
TorchGeo is aPyTorchdomain library, similar totorchvision, providing datasets, samplers, transforms, and pre-trained models specific to geospatial data.The goal of this library is to make it simple:for machine learning experts to work with geospatial data, andfor remote sensing experts to explore machine learning solutions.Community:Packaging:Testing:InstallationThe recommended way to install TorchGeo is withpip:$pipinstalltorchgeoForcondaandspackinstallation instructions, see thedocumentation.DocumentationYou can find the documentation for TorchGeo onReadTheDocs. This includes API documentation, contributing instructions, and severaltutorials. For more details, check out ourpaperandblog.Example UsageThe following sections give basic examples of what you can do with TorchGeo.First we'll import various classes and functions used in the following sections:fromlightning.pytorchimportTrainerfromtorch.utils.dataimportDataLoaderfromtorchgeo.datamodulesimportInriaAerialImageLabelingDataModulefromtorchgeo.datasetsimportCDL,Landsat7,Landsat8,VHR10,stack_samplesfromtorchgeo.samplersimportRandomGeoSamplerfromtorchgeo.trainersimportSemanticSegmentationTaskGeospatial datasets and samplersMany remote sensing applications involve working withgeospatial datasets—datasets with geographic metadata. These datasets can be challenging to work with due to the sheer variety of data. Geospatial imagery is often multispectral with a different number of spectral bands and spatial resolution for every satellite. In addition, each file may be in a different coordinate reference system (CRS), requiring the data to be reprojected into a matching CRS.In this example, we show how easy it is to work with geospatial data and to sample small image patches from a combination ofLandsatandCropland Data Layer (CDL)data using TorchGeo. First, we assume that the user has Landsat 7 and 8 imagery downloaded. Since Landsat 8 has more spectral bands than Landsat 7, we'll only use the bands that both satellites have in common. We'll create a single dataset including all images from both Landsat 7 and 8 data by taking the union between these two datasets.landsat7=Landsat7(root="...",bands=["B1",...,"B7"])landsat8=Landsat8(root="...",bands=["B2",...,"B8"])landsat=landsat7|landsat8Next, we take the intersection between this dataset and the CDL dataset. We want to take the intersection instead of the union to ensure that we only sample from regions that have both Landsat and CDL data. Note that we can automatically download and checksum CDL data. Also note that each of these datasets may contain files in different coordinate reference systems (CRS) or resolutions, but TorchGeo automatically ensures that a matching CRS and resolution is used.cdl=CDL(root="...",download=True,checksum=True)dataset=landsat&cdlThis dataset can now be used with a PyTorch data loader. Unlike benchmark datasets, geospatial datasets often include very large images. For example, the CDL dataset consists of a single image covering the entire continental United States. In order to sample from these datasets using geospatial coordinates, TorchGeo defines a number ofsamplers. In this example, we'll use a random sampler that returns 256 x 256 pixel images and 10,000 samples per epoch. We also use a custom collation function to combine each sample dictionary into a mini-batch of samples.sampler=RandomGeoSampler(dataset,size=256,length=10000)dataloader=DataLoader(dataset,batch_size=128,sampler=sampler,collate_fn=stack_samples)This data loader can now be used in your normal training/evaluation pipeline.forbatchindataloader:image=batch["image"]mask=batch["mask"]# train a model, or make predictions using a pre-trained modelMany applications involve intelligently composing datasets based on geospatial metadata like this. For example, users may want to:Combine datasets for multiple image sources and treat them as equivalent (e.g., Landsat 7 and 8)Combine datasets for disparate geospatial locations (e.g., Chesapeake NY and PA)These combinations require that all queries are present in at least one dataset, and can be created using aUnionDataset. Similarly, users may want to:Combine image and target labels and sample from both simultaneously (e.g., Landsat and CDL)Combine datasets for multiple image sources for multimodal learning or data fusion (e.g., Landsat and Sentinel)These combinations require that all queries are present in both datasets, and can be created using anIntersectionDataset. TorchGeo automatically composes these datasets for you when you use the intersection (&) and union (|) operators.Benchmark datasetsTorchGeo includes a number ofbenchmark datasets—datasets that include both input images and target labels. This includes datasets for tasks like image classification, regression, semantic segmentation, object detection, instance segmentation, change detection, and more.If you've usedtorchvisionbefore, these datasets should seem very familiar. In this example, we'll create a dataset for the Northwestern Polytechnical University (NWPU) very-high-resolution ten-class (VHR-10) geospatial object detection dataset. This dataset can be automatically downloaded, checksummed, and extracted, just like with torchvision.dataset=VHR10(root="...",download=True,checksum=True)dataloader=DataLoader(dataset,batch_size=128,shuffle=True,num_workers=4)forbatchindataloader:image=batch["image"]label=batch["label"]# train a model, or make predictions using a pre-trained modelAll TorchGeo datasets are compatible with PyTorch data loaders, making them easy to integrate into existing training workflows. The only difference between a benchmark dataset in TorchGeo and a similar dataset in torchvision is that each dataset returns a dictionary with keys for each PyTorchTensor.Pre-trained WeightsPre-trained weights have proven to be tremendously beneficial for transfer learning tasks in computer vision. Practitioners usually utilize models pre-trained on the ImageNet dataset, containing RGB images. However, remote sensing data often goes beyond RGB with additional multispectral channels that can vary across sensors. TorchGeo is the first library to support models pre-trained on different multispectral sensors, and adopts torchvision'smulti-weight API. A summary of currently available weights can be seen in thedocs. To create atimmResnet-18 model with weights that have been pretrained on Sentinel-2 imagery, you can do the following:importtimmfromtorchgeo.modelsimportResNet18_Weightsweights=ResNet18_Weights.SENTINEL2_ALL_MOCOmodel=timm.create_model("resnet18",in_chans=weights.meta["in_chans"],num_classes=10)model=model.load_state_dict(weights.get_state_dict(progress=True),strict=False)These weights can also directly be used in TorchGeo Lightning modules that are shown in the following section via theweightsargument. For a notebook example, see thistutorial.Reproducibility with LightningIn order to facilitate direct comparisons between results published in the literature and further reduce the boilerplate code needed to run experiments with datasets in TorchGeo, we have created Lightningdatamoduleswith well-defined train-val-test splits andtrainersfor various tasks like classification, regression, and semantic segmentation. These datamodules show how to incorporate augmentations from the kornia library, include preprocessing transforms (with pre-calculated channel statistics), and let users easily experiment with hyperparameters related to the data itself (as opposed to the modeling process). Training a semantic segmentation model on theInria Aerial Image Labelingdataset is as easy as a few imports and four lines of code.datamodule=InriaAerialImageLabelingDataModule(root="...",batch_size=64,num_workers=6)task=SemanticSegmentationTask(model="unet",backbone="resnet50",weights=True,in_channels=3,num_classes=2,loss="ce",ignore_index=None,lr=0.1,patience=6,)trainer=Trainer(default_root_dir="...")trainer.fit(model=task,datamodule=datamodule)TorchGeo also supports command-line interface training usingLightningCLI. It can be invoked in two ways:#Iftorchgeohasbeeninstalledtorchgeo#Iftorchgeohasbeeninstalled,orifithasbeenclonedtothecurrentdirectorypython3 -m torchgeoIt supports command-line configuration or YAML/JSON config files. Valid options can be found from the help messages:#Seevalidstagestorchgeo --help#Seevalidtraineroptionstorchgeo fit --help#Seevalidmodeloptionstorchgeo fit --model.help ClassificationTask#Seevaliddataoptionstorchgeo fit --data.help EuroSAT100DataModuleUsing the following config file:trainer:max_epochs:20model:class_path:ClassificationTaskinit_args:model:"resnet18"in_channels:13num_classes:10data:class_path:EuroSAT100DataModuleinit_args:batch_size:8dict_kwargs:download:truewe can see the script in action:#Trainandvalidateamodeltorchgeo fit --config config.yaml#Validate-onlytorchgeo validate --config config.yaml#Calculateandreporttestaccuracytorchgeo test --config config.yaml --trainer.ckpt_path=...It can also be imported and used in a Python script if you need to extend it to add new features:fromtorchgeo.mainimportmainmain(["fit","--config","config.yaml"])See theLightning documentationfor more details.CitationIf you use this software in your work, please cite ourpaper:@inproceedings{Stewart_TorchGeo_Deep_Learning_2022,
    address = {Seattle, Washington},
    author = {Stewart, Adam J. and Robinson, Caleb and Corley, Isaac A. and Ortiz, Anthony and Lavista Ferres, Juan M. and Banerjee, Arindam},
    booktitle = {Proceedings of the 30th International Conference on Advances in Geographic Information Systems},
    doi = {10.1145/3557915.3560953},
    month = {11},
    pages = {1--12},
    publisher = {Association for Computing Machinery},
    series = {SIGSPATIAL '22},
    title = {{TorchGeo}: Deep Learning With Geospatial Data},
    url = {https://dl.acm.org/doi/10.1145/3557915.3560953},
    year = {2022}
}ContributingThis project welcomes contributions and suggestions. If you would like to submit a pull request, see ourContribution Guidefor more information.This project has adopted theMicrosoft Open Source Code of Conduct. For more information see theCode of Conduct FAQor contactopencode@microsoft.comwith any additional questions or comments.