Package: copernicusmarine
Copernicus Marine Service toolbox (CLI & Python)FeaturesThecopernicusmarineoffers capabilities through bothCommand Line Interface (CLI)andPython API:Metadata Information: List and retrieve metadata information on all variables, datasets, products, and their associated documentation.Subset Datasets: Subset datasets to extract only the parts of interest, in preferred format, such as Analysis-Ready Cloud-Optimized (ARCO) Zarr or NetCDF file format.Advanced Filters: Apply simple or advanced filters to get multiple files, in original formats like NetCDF/GeoTIFF, via direct Marine Data Store connections.No Quotas: Enjoy no quotas, neither on volume size nor bandwidth.InstallationFor installation, multiple options are available depending on your setup:Conda|MambaThough no conda package has been created yet, these steps cover the installation in a new isolated environment (safer). Replace conda by mamba if necessary.Create the filecopernicusmarine-env.ymlthat contains:name:cmcchannels:-conda-forgedependencies:-python>=3.9,<3.13-pip-pip:-copernicusmarineUse the terminal or a [Conda|Mamba] Prompt to create thecmcenvironment from theymlfile:condaenvcreate--filecopernicusmarine-env.ymlOpen the newcmcenvironment by running:condaactivatecmcVerify that the newcmcenvironment was installed correctly:condaenvlistPipOtherwise, if you already have an environment (safer to clone it), the package can be installed using thepipcommand:python-mpipinstallcopernicusmarineAnd toupgrade the packageto the newest available version, run:python-mpipinstallcopernicusmarine--upgradeUser GuideFor more comprehensive details on how to use thecopernicusmarine, please refer to ourHelp Center. It ensures a smooth migration for existing users of legacy services such as MOTU, OPeNDAP, and FTP.General configurationCache UsageCachier library is used for caching part of the requests (as describe result or login). By default, the cache will be located in the home folder. If you need to change the location of the cache, you can set the environment variableCOPERNICUSMARINE_CACHE_DIRECTORYto point to the desired directory.Disable SSLA global SSL context is used when making HTTP calls using thecopernicusmarinetoolbox. For some reason, it can lead to unexpected behavior depending on your network configuration. You can set theCOPERNICUSMARINE_DISABLE_SSL_CONTEXTenvironmnent variable to globally disable the usage of SSL in the client.Command Line Interface (CLI)The--helpoptionTo discover commands and their available options, consider appending--helpon any command line.Example:copernicusmarine--helpReturns:Usage:copernicusmarine[OPTIONS]COMMAND[ARGS]...

Options:-V,--versionShowtheversionandexit.--helpShowthismessageandexit.

Commands:describePrintCopernicusMarinecatalogasJSONloginThiscommandcheckthecopernicusmarinecredentialsprovided...getDownloadoriginallyproduceddatafilessubsetDownloadssubsetsofdatasetsasNetCDFfilesorZarrstoresCommanddescribeRetrieve metadata information about all products/datasets and display as JSON output:copernicusmarinedescribe--include-datasetsThe JSON output can also be saved like follows:copernicusmarinedescribe--include-datasets>all_datasets_copernicusmarine.jsonCommandloginCreate a single configuration file.copernicusmarine-credentialsallowing to access all Marine Data Store data services. By default, it saves file in user's home directory.Example:>copernicusmarinelogin
username:johndoe
password:
INFO-Configurationfilesstoredin/Users/foo/.copernicusmarineIf.copernicusmarine-credentialsalready exists, the user is asked for confirmation to overwrite (--overwrite/--overwrite-configuration-file).Access points migration and evolutionIf you already have a configuration for current services (e.g.~/motuclient/motuclient-python.ini,~/.netrcor~/_netrc) in your home directory, it will automatically be taken into account with commandsgetandsubsetwithout the need for running thelogincommand.
If the configuration files are already available in another directory, when running commandssubsetorget, you can use the--credentials-fileoption to point to the file.CommandsubsetRemotely subset a dataset, based on variable names, geographical and temporal parameters.Example:copernicusmarinesubset--dataset-idcmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m--variablethetao--variableso--start-datetime2021-01-01--end-datetime2021-01-03--minimum-longitude0.0--maximum-longitude0.1--minimum-latitude28.0--maximum-latitude28.1Returns:INFO-DownloadthroughS3
<xarray.Dataset>
Dimensions:(depth:50,latitude:2,longitude:1,time:1)Coordinates:*depth(depth)float320.50581.5562.668...5.292e+035.698e+03*latitude(latitude)float3228.028.08*longitude(longitude)float320.08333*time(time)datetime64[ns]2021-01-01
Datavariables:thetao(time,depth,latitude,longitude)float32dask.array<chunksize=(1,50,2,1),meta=np.ndarray>so(time,depth,latitude,longitude)float32dask.array<chunksize=(1,50,2,1),meta=np.ndarray>
Attributes:(12/19)Conventions:CF-1.0bulletin_date:2022-11-01......title:CMEMSIBIREANALYSIS:YEARLYPHYSICALPRODUCTS
Doyouwanttoproceedwithdownload?[Y/n]:By default, after the display of the summary of the dataset subset, a download confirmation is asked. To skip this user's action, call option--force-download.Note about longitude rangeOptions--minimum-longitudeand--maximum-longitudework as follows:If the result of the substraction (--maximum-longitudeminus--minimum-longitude) is superior or equal to 360, then return the full dataset.If the requested longitude range:does not crossthe antemeridian, then return the dataset between range -180° and 180°.does crossthe antemeridian, then return the dataset between range 0° and 360°.Note that you can request any longitudes you want. A modulus is applied to bring the result between -180° and 360°. For example, if you request [530°, 560°], the result dataset will be in [170°, 200°].Access point migration and evolutionThe copernicus marine toolbox will download the data in the most efficient way according to your request:if the target datasetis availablein ARCO version, then files are downloaded in a fresh new folder in the current working directory.if the target datasetis not yet availablein ARCO version, then a file is downloaded in the current working directory.NOTE:The filename will be with the following formatdataset_id-longitude_range-latitude_range-depth_range-date_range.[nc|zarr]However, the output directory and filename can be specified using-o/--output-directoryand-f/--output-filenamerespectively. If the later ends with.nc, it will be written as a NetCDF file.You can force the use of a specific data access service with option--service.Note about netcdf-compression-enabled and --netcdf-compression-enabled optionsWhen subseting data, if you decide to write your data as a NetCDF file (which is the default behavior), then you can provide the extra option "--netcdf-compression-enabled". The downloaded file will be lighter but it will take you more time to write it (because of the compression task). If you don't provide it, the task will be faster, but the file heavier.
Finally, if you decide to write your data in ZARR format (.zarr extension), then the original compression that is used in the Marine Data Store will be apply, which mean that the download task will be fast AND the file compressed. In that case, you cannot use the "netcdf-compression-enabled" as it has no sense.Here is the default options added to xarray in the background when using the option: {'zlib': True, 'complevel': 1, 'contiguous': False, 'shuffle': True}In addition to this option, you can also provide the--netcdf-compression-enabledoption and customize the NetCDF compression level between 0 (no compression) and 9.CommandgetDownload the dataset file(s) as originally produced, based on the datasetID or the path to files.Example:copernicusmarineget--dataset-urlftp://my.cmems-du.eu/Core/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m--serviceoriginal-filesReturns:INFO-Youforcedselectionofservice:original-files
INFO-Downloadingusingserviceoriginal-files...
INFO-Yourequestedthedownloadofthefollowingfiles:
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_19930101_19931231_R20221101_RE01.nc-8.83MB[...truncatedforbrevity..]s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20120101_20121231_R20221101_RE01.nc-8.62MB
Printed20outof29files

Totalsizeofthedownload:252.94MB

Doyouwanttoproceedwithdownload?[Y/n]You can force the use of a specific data access service with option--service.By default:after the header displays a summary of the request, a download confirmation is asked. To skip this user's action, add option--force-download.files are downloaded to the current directory applying the original folder structure. To avoid this behavior, add--no-directoriesand specify a destination with the--output-directoryoption.Option--show-outputnamesdisplays the full paths of the output files, if required.Option--download-file-listonly creates a filefiles_to_download.txtcontaining the names of the targeted files instead of downloading them. If specified, no other action will be performed.Note about sync optionOption--syncallows to download original files only if not exist and not up to date. The toolbox checks the destination folder against the source folder. It can be combined with filters. Note that if set with--overwrite-output-data, the latter will be ignored.
The logic is largely inspired froms5mp package sync commandOption--sync-deletewill work as--syncwith the added fonctionnality that it deletes any local file that has not been found on the remote server. Note that the files found on the server are also filtered. Hence, a file present locally might be deleted even if it is on the server because,  for example, the executedgetcommand contains a filter that excludes this specific file.
Limitations:is not compatible with--no-directories.Version needs to be set when using sync (with--force-dataset-versionflag)As for now, the sync functionality is not available for datasets with several parts (like INSITU or static datasets for example).Note about filtering optionsOption--filterallows to specify a Unix shell-style wildcard pattern (seefnmatch — Unix filename pattern matching) and select specific files:copernicusmarineget--dataset-idcmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m--filter"*01yav_200[0-2]*"Returns:INFO-Downloadingusingservicefiles...
INFO-Yourequestedthedownloadofthefollowingfiles:
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20000101_20001231_R20221101_RE01.nc-8.93MB
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20010101_20011231_R20221101_RE01.nc-8.91MB
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20020101_20021231_R20221101_RE01.nc-8.75MB

Totalsizeofthedownload:26.59MB
Doyouwanttoproceedwithdownload?[Y/n]:Option--regexallows to specify a regular expression for more advanced files selection:copernicusmarineget-icmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m--regex".*01yav_20(00|01|02).*.nc"Returns:INFO-Downloadingusingservicefiles...
INFO-Yourequestedthedownloadofthefollowingfiles:
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20000101_20001231_R20221101_RE01.nc-8.93MB
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20010101_20011231_R20221101_RE01.nc-8.91MB
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20020101_20021231_R20221101_RE01.nc-8.75MB

Totalsizeofthedownload:26.59MB
Doyouwanttoproceedwithdownload?[Y/n]:Option--file-listallows to specify a list of files for more advanced files selection:An examplefile_list.txtwould look like this:CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20000101_20001231_R20221101_RE01.nc
CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20010101_20011231_R20221101_RE01.nc
CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20020101_20021231_R20221101_RE01.ncNOTE:This option is compatible with the file generated by the--download-file-listoption.Then the following command:copernicusmarineget-icmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m--file-listfile_list.txtReturns:INFO-Downloadingusingservicefiles...
INFO-Yourequestedthedownloadofthefollowingfiles:
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20000101_20001231_R20221101_RE01.nc-8.93MB
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20010101_20011231_R20221101_RE01.nc-8.91MB
s3://mdl-native/native/IBI_MULTIYEAR_PHY_005_002/cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m_202211/CMEMS_v5r1_IBI_PHY_MY_NL_01yav_20020101_20021231_R20221101_RE01.nc-8.75MB

Totalsizeofthedownload:26.59MB
Doyouwanttoproceedwithdownload?[Y/n]:Shared optionsBothsubsetandgetcommands provide these options:Option--overwrite-output-dataWhen specified, the existing files will be overwritten.
Otherwise, if the files already exist on destination, new ones with a unique index will be created once the download has been accepted (or once--force-downloadis provided).Option--request-fileThis option allows to specify CLI options but in a provided JSON file, useful for batch processing.Template forsubsetdata request:{"dataset_id":"cmems_mod_glo_phy-thetao_anfc_0.083deg_P1D-m","start_datetime":"2022-04-11","end_datetime":"2023-08-11","minimum_longitude":-182.79,"maximum_longitude":-179.69,"minimum_latitude":-40,"maximum_latitude":-36,"minimum_depth":0,"maximum_depth":0,"variables":["thetao"],"output_directory":"./data/","output_filename":"temperature_small_pacific_2022208-202308.zarr","force_download":false}Example:copernicusmarinesubset--request-filetemplate_subset_data_request.jsonTemplate forgetdata request:{"dataset_id":"cmems_mod_ibi_phy_my_0.083deg-3D_P1Y-m","filter":"*01yav_200[0-2]*","force_download":false,"service":"files","log_level":"INFO","no_directories":false,"no_metadata_cache":false,"output_directory":"./data/","overwrite_output_data":false,"overwrite_metadata_cache":false,"show_outputnames":true}Example:copernicusmarineget--request-filetemplate_get_data_request.jsonOption--credentials-fileYou can use the--credentials-fileoption to point to a credentials file. The file can be either.copernicusmarine-credentials,motuclient-python.ini,.netrcor_netrc.Option--dataset-versionYou can use the--dataset-versionoption to fetch a specific dataset version.Option--dataset-partYou can use the--dataset-partoption to fecth a specific part for the chosen dataset version.Python package (API)Thecopernicusmarineexposes a Python interface to allow you tocall commands as functions.DocumentationSee theHelp Center. A detailed standalone API documentation is under construction and will come at a later stage.ContributionWe welcome contributions from the community to enhance this package. If you find any issues or have suggestions for improvements, please check out ourReport Template.Future improvements & RoadmapMake available the currently not managed Analysis-Ready Cloud-Optimized (ARCO) versions of Ocean Monitoring Indicator (OMI), in situ and static datasets.Allow to specify the compression level when downloading your subset as NetCDF file.Allow to subset variables using theirstandard_name(s)and not only theirname(s).To keep up to date with the most recent and planned advancements, including revisions, corrections, and feature requests generated from users' feedback, please refer to ourRoadmap.Join the communityGet in touch!Create yourCopernicus Marine AccountLog inand chat with us (bottom right corner ofCopernicus Marine Service)Join ourtraining workshopsNetwork y/ourCopernicus StoriesWatchour videosLicenceLicensed under the (EUPL)[https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12]