Package: cropharvest
CropHarvestCropHarvest is an open source remote sensing dataset for agriculture with benchmarks. It collects data from a variety of agricultural land use datasets and remote sensing products.The dataset consists of95,186datapoints, of which33,205(35%) have multiclass labels. All other datapoints only have binary crop / non-crop labels.70,213(74%) of these labels are paired with remote sensing and climatology data, specificallySentinel-2,Sentinel-1, theSRTM Digital Elevation ModelandERA 5 climatology data.21 datasets are aggregated into CropHarvest - these are documentedhere.More details about CropHarvest and the benchmarks are available inthis paper.PipelineThe code in this repositorycombines the constituent datasets into a single geoJSON file,exports the associated satellite data from Earth Engine,combines both datasets to create(X,y)training tuples andexposes those tuples via aDatasetobject.The pipeline through which this happens is shown below:All blue boxes are associated with code in this repository. Anything green is data accessible viaZenodo. By default, the Zenodo data will get downloaded to thedata folder- the data folder'sReadmehas more information about the exact structure of the data.There are unique cases where you may need to use theEarthEngineExporterdirectly, these use cases are demonstrated in thedemo_exporting_data.ipynbnotebook.InstallationLinux and MacOS users can install the latest version of CropHarvest with the following command:pipinstallcropharvestWindows users must install the CropHarvest within acondaenvironment to ensure all dependencies are installed correctly:condainstall'fiona>=1.5''rasterio>=1.2.6'pipinstallcropharvestGetting startedSee thedemo.ipynbnotebook for an example on how to download the data fromZenodoand train a random forest against this data.For more examples of models trained against this dataset, see thebenchmarks.ContributingIf you would like to contribute a dataset, please see thecontributing readme.FAQQuestions asked at least onceHow do I use CropHarvest for a specific geography?All the data is accessible through thecropharvest.datasets.CropHarvestobject. The main parameters which you might be interested in manipulating are controllable through acropharvest.datasets.Task, which takes as input the following parameters:A bounding box, which defines the spatial boundaries of the labels retrievesA target label, which defines the class of the positive labels (if this is left toNone, then the positive class will be crops and the negative class will be non-crops)A boolean defining whether or not to balance the crops and non-crops in the negative classA test_identifier string, which tells the dataset whether or not to retrieve a file from thetest_featuresfolder and return it as the test data.So if I wanted to use this to train a model to identify crop vs. non crop in France, I might do it like this:fromsklearn.ensembleimportRandomForestClassifierfromcropharvest.datasetsimportTask,CropHarvestfromcropharvest.countriesimportget_country_bbox# folder into which the data will be downloaded / already existsdata_dir="data"# get_country_bbox returns a list of bounding boxes.# the one representing Metropolitan France is the 2nd boxmetropolitan_france_bbox=get_country_bbox("France")[1]task=Task(bounding_box=metropolitan_france_bbox,normalize=True)my_dataset=CropHarvest(data_dir,task)X,y=my_dataset.as_array(flatten_x=True)model=RandomForestClassifier(random_state=0)model.fit(X,y)How do I load a specific pixel timeseries?Thespecific use casehere is to retrieve NDVI values for a specific row in thelabels.geojson. Here is how you might go about doing that:Firstly, I will load the geosjon. I'll do it through aCropHarvestLabelsobject, which is just a wrapper around the geojson but provides some nice utility functions.>>>fromcropharvest.datasetsimportCropHarvestLabels>>>>>>labels=CropHarvestLabels("cropharvest/data")>>>labels_geojson=labels.as_geojson()>>>labels_geojson.head()harvest_dateplanting_date...is_testgeometry0NoneNone...FalsePOLYGON((37.0825210.71274,37.0834810.71291...1NoneNone...FalsePOLYGON((37.0872110.72398,37.0871410.72429...2NoneNone...FalsePOLYGON((37.0849810.71371,37.0848110.71393...3NoneNone...FalsePOLYGON((37.0902110.71320,37.0901410.71341...4NoneNone...FalsePOLYGON((37.0830710.72160,37.0828110.72197...[5rowsx13columns]Now, I can use thelabelsobject to retrieve the filepath of the processed satellite data for each row in the labels geojson:>>>path_to_file=labels._path_from_row(labels_geojson.iloc[0])This processed satellite data is stored ash5pyfiles, so I can load it up as follows:>>>importh5py>>>h5py_file=h5py.File(path_to_file,"r")>>>x=h5py_file.get("array")[:]>>>x.shape(12,18)The shape ofxrepresents 12 timesteps and 18 bands. To retrieve the band I am interested in:>>>fromcropharvest.bandsimportBANDS>>>x[:,BANDS.index("NDVI")]array([0.28992072,0.28838343,0.26833579,0.22577633,0.27138986,0.06584114,0.498998,0.50147203,0.50437743,0.44326343,0.33735849,0.28375967])These are 12 NDVI values, corresponding to the 12 months captured in this timeseries. To find out exactly which month each timestep represents, I can do>>>labels_geojson.iloc[0].export_end_date'2021-02-01T00:00:00'Wich tells me that the last timestep represents January 2021. I can work backwards from there.What is the data format?The structure of the different data files is now described in depth in the data folder's [Readme](https://github.com/nasaharvest/cropharvest/blob/main/data/README.md)LicenseCropHarvest has aCreative Commons Attribution-ShareAlike 4.0 Internationallicense.CitationIf you use CropHarvest in your research, please use the following citation:@inproceedings{
    tseng2021cropharvest,
    title={CropHarvest: A global dataset for crop-type classification},
    author={Gabriel Tseng and Ivan Zvonkov and Catherine Lilian Nakalembe and Hannah Kerner},
    booktitle={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)},
    year={2021},
    url={https://openreview.net/forum?id=JtjzUXPEaCu}
}