Package: ecmwf-opendata
ecmwf-opendataecmwf-opendatais a package to simplify the download of ECMWFopen data. It implements a request-based interface to the dataset using ECMWF's MARS language to select meteorological fields, similar to the existingecmwf-api-clientPython package.A collection of Jupyter Notebooks that make use of that package is availablehere.The example below will download the latest available 10-day forecast for themean sea level pressure(msl) into a local file calleddata.grib2:fromecmwf.opendataimportClientclient=Client()client.retrieve(step=240,type="fc",param="msl",target="data.grib2",)❗NOTE:This package is designed for users that want to download a subset of the whole dataset. If you plan to download a large percentage of each data file, it may be more efficient to download whole files and filter out the data you want locally. See the documentation on thefile naming conventionfor more information. Alternatively, you can use this tool to download whole files by only specifyingdate,time,step,streamandtype. Please be aware that all data for a full day is in the order of 726 GiB.OptionsThe constructor of the client object takes the following options:client=Client(source="ecmwf",beta=True,preserve_request_order=False,infer_stream_keyword=True,)where:sourceis either the name of server to contact or a fully qualified URL. Possible values areecmwfto access ECMWF's servers, orazureto access data hosted on Microsoft's Azure. Default isecmwf.betais a boolean that indicates whether to access the beta or the production version of the dataset. Current onlybeta=Trueis supported.preserve_request_order. If this flag is set toTrue, the library will attempt to write the retrieved data into the target file in the order specified by the request. For example, if the request specifiesparam=[2t,msl]the library will ensure that the field2tis first in the target file, while withparam=[msl,2t], the fieldmslwill be first. This also works across different keywords:...,levelist=[500,100],param=[z,t],...will produce different output to...,param=[z,t],levelist=[500,100],...If the flag is set toFalse, the library will sort the request to minimise the number of HTTP requests made to the server, leading to faster download speeds. Default isFalse.infer_stream_keyword. Thestreamkeyword represents the ECMWF forecasting system that creates the data. Setting it properly requires knowledge of how ECMWF runs its operations. If this boolean is set toTrue, the library will try to infer the correct value for thestreamkeyword based on the rest of the request. Default isTrue.⚠️NOTE:It is  recommendednotto set thepreserve_request_orderflag toTruewhen downloading a large number of fields as this will add extra load on the servers.MethodsClient.retrieve()TheClient.retrieve()method takes request as input and will retrieve the corresponding data from the server and write them in the user's target file.A request is a list of keyword/value pairs used to select the desired data. It is possible to specify a list of values for a given keyword.The request can either be specified as a dictionary:fromecmwf.opendataimportClientclient=Client(source="ecmwf")request={"time":0,"type":"fc","step":24,"param":["2t","msl"],}client.retrieve(request,"data.grib2")# or:client.retrieve(request=request,target="data.grib2",)or directly as arguments to theretrieve()method:fromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,type="fc",step=24,param=["2t","msl"],target="data.grib2",)Thedateandtimekeyword are used to select the date and time of the forecast run (seeDate and timebelow). Ifdateor bothdateandtimeare not specified, the library will query the server for the most recent matching data. Thedateandtimeof the downloaded forecast is returned by theretrieve()method.fromecmwf.opendataimportClientclient=Client(source="ecmwf")result=client.retrieve(type="fc",step=24,param=["2t","msl"],target="data.grib2",)print(result.datetime)may print2022-01-23 00:00:00.Client.download()TheClient.download()method takes the same parameters as theClient.retrieve()method, but will download the whole data files from the server, ignoring keywords likeparam,levelistornumber.The example below will download all field from the latest time step 24, ignoring the keywordparam:fromecmwf.opendataimportClientclient=Client(source="ecmwf")client.download(param="msl",type="fc",step=24,target="data.grib2",)Client.latest()TheClient.latest()method takes the same parameters as theClient.retrieve()method, and returns the date of the most recent matching forecast without downloading the data:fromecmwf.opendataimportClientclient=Client(source="ecmwf")print(client.latest(type="fc",step=24,param=["2t","msl"],target="data.grib2",))may print2022-01-23 00:00:00.⏰NOTE: The data is available between 7 and 9 hours after the forecast starting date and time, depending on the forecasting system and the time step specified.Request keywordsThe supported keywords are:type: the type of data (compulsory, defaults tofc).stream: the forecast system (optional if unambiguous, compulsory otherwise). See theinfer_stream_keywordabove.date: the date at which the forecast starts.time: the time at which the forecast starts.step: the forecast time step in hours, orfcmonth, the time step in months for the seasonal forecast (compulsory, default to0and1, respectively).and (all optional, with no defaults):param: the meteorological parameters, such as wind, pressure or humidity.levtype: select between single level parameters and parameters on pressure levels.levelist: the list of pressure levels when relevant.number: the list of ensemble member numbers when relevant.The keywords in the first list are used to identify which file to access, while the second list is used to identify which parts of the files need to be actually downloaded. Some HTTP servers are able to return multiple parts of a file, while other can only return a single part from a file. In the latter case, the library may perform many HTTP requests to the server. If you wish to download whole files, only provide keywords from the first list.Date and timeThe date and time parameters refer to the starting time of the forecast. All date and time are expressed in UTC.There are several ways to specify the date and time in a request.Date can be specified using strings, numbers and Pythondatetime.datetimeordatetime.dateobjects:...date='20220125',time=12,...date='2022-01-25',time=12,...date='2022-01-25 12:00:00',...date=20220125,time=12,...date=datetime.datetime(2022,1,25,12,0,0),...date=datetime.date(2022,1,25),time=12,...Dates can also be given as a number less than or equal to zero. In this case, it is equivalent to the current UTC date minus the given number of days:...date=0,# todaydate=-1,# yesterdaydate=-2,# the day before yesterday...The keywordtimecan be given as a string or an integer, or a Pythondatetime.timeobject. All values of time below are equivalent:...time=12,...time=1200,...time='12',...time='1200',...time=datetime.time(12),...List of valid values for time0, 6, 12 and 18Iftimeis not specified, the time is extracted from the date....date='2022-01-25 12:00:00',...is equivalent to:...date='2022-01-25',time=12,...If thetimekeyword is specified, it overrides any time given in the request....date='2022-01-25 12:00:00',time=18,...is equivalent to:...date='2022-01-25',time=18,...As stated before, ifdateor bothdateandtimeare not specified, the library will query the server for the most recent matching data. Thedateandtimeof the downloaded forecast is returned by theretrieve()method:Example without thedatekeyword:fromecmwf.opendataimportClientclient=Client(source="ecmwf")result=client.retrieve(time=12,type="fc",param="2t",step="24",target="data.grib2",)print(result.datetime)will print2022-01-22 12:00:00if run in the morning of 2022-01-23.Example without thedateandtimekeywords:fromecmwf.opendataimportClientclient=Client(source="ecmwf")result=client.retrieve(type="fc",param="2t",step="24",target="data.grib2",)print(result.datetime)will print2022-01-23 00:00:00if run in the morning of 2022-01-23.Stream and typeECMWF runs several forecasting systems:HRES: High Resolution Forecast.ENS: Ensemble Forecasts.SEAS: Long-Range (Seasonal) Forecast.Each of these forecasts also produces several types of products, that are referred to using the keywordsstreamandtype.Valid values fortypeare:HRES:fc: Forecast.ENS:cf: Control forecast.pf: Perturbed forecast.em: Ensemble mean.es: Ensemble standard deviation.ep: Probabilities.Valid values forstreamare:oper: Atmospheric fields from HRES - 00 UTC and 12 UTC.wave: Ocean wave fields from HRES - 00 UTC and 12 UTC.enfo: Atmospheric fields from ENS.waef: Ocean wave fields from ENS.scda: Atmospheric fields from HRES - 06 UTC and 18 UTC.scwv: Ocean wave fields from HRES - 06 UTC and 18 UTC.📌NOTE: if the client's flaginfer_stream_keywordis set toTrue, the library will infer the stream from thetypeandtime. In that case, you just need to specifystream=waveto access ocean wave products, and don't provide a value forstreamin other cases.Time stepsTo select a time step, use thestepkeyword:...step=24,...step=[24,48],...Forecasting systemTimeList of time stepsHRES00 and 120 to 144 by 3, 144 to 240 by 6ENS00 and 120 to 144 by 3, 144 to 360 by 6HRES06 and 180 to 90 by 3ENS06 and 180 to 144 by 3Probabilities - Instantaneous weather events00 and 120 to 360 by 12Probabilities - Daily weather events00 and 120-24 to 336-360 by 12📌NOTE: Not specifyingstepwill return all available time steps.Parameters and levelsTo select a parameter, use theparamkeyword:...param="msl",...param=["2t","msl"]...for pressure level parameters, use thelevelistkeyword:...param="t",levelist=850,...param=["u","v"],levelist=[1000,850,500],...📌NOTE: Not specifyinglevelistwill return all available levels, and not specifyingparamwill return all available parameters.List of pressure levels (hPa)1000, 925, 850, 700, 500, 300, 250, 200 and 50Below is the list of all parameters:Atmospheric fields on pressure levelsParameterDescriptionUnitsdDivergences-1ghGeopotential heightgpmqSpecific humiditykg kg-1rRelative humidity%tTemperatureKuU component of windm s-1vV component of windm s-1voVorticity (relative)s-1Atmospheric fields on a single levelParameterDescriptionUnits10u10 metre U wind componentm s-110v10 metre V wind componentm s-12t2 metre temperatureKmslMean sea level pressureParoRunoffmsktSkin temperatureKspSurface pressurePastSoil TemperatureKstl1Soil temperature level 1KtcwvTotal column vertically-integrated water vapourkg m-2tpTotal PrecipitationmOcean waves fieldsParameterDescriptionUnitsmp2Mean zero-crossing wave periodsmwdMean wave directionDegree truemwpMean wave periodspp1dPeak wave periodsswhSignificant height of combined wind waves and swellmEnsemble mean and standard deviation - pressure levelsParameterDescriptionUnitsghGeopotential heightgpmtTemperatureKwsWind speedm s-1Ensemble mean and standard deviation - single levelParameterDescriptionUnitsmslMean sea level pressurePaInstantaneous weather events - atmospheric fields - 850 hPaParameterDescriptionUnitsptsa_gt_1p5stdevProbability of temperature standardized anomaly greater than 1.5 standard deviation%ptsa_gt_1stdevProbability of temperature standardized anomaly greater than 1 standard deviation%ptsa_gt_2stdevProbability of temperature standardized anomaly greater than 2 standard deviation%ptsa_lt_1p5stdevProbability of temperature standardized anomaly less than -1.5 standard deviation%ptsa_lt_1stdevProbability of temperature standardized anomaly less than -1 standard deviation%ptsa_lt_2stdevProbability of temperature standardized anomaly less than -2 standard deviation%Daily weather events - atmospheric fields - single levelParameterDescriptionUnits10fgg1010 metre wind gust of at least 10 m/s%10fgg1510 metre wind gust of at least 15 m/s%10fgg2510 metre wind gust of at least 25 m/s%tpg1Total precipitation of at least 1 mm%tpg10Total precipitation of at least 10 mm%tpg100Total precipitation of at least 100 mm%tpg20Total precipitation of at least 20 mm%tpg25Total precipitation of at least 25 mm%tpg5Total precipitation of at least 5 mm%tpg50Total precipitation of at least 50 mm%Instantaneous weather events - ocean waves fieldsParameterDescriptionUnitsswhg2Significant wave height of at least 2 m%swhg4Significant wave height of at least 4 m%swhg6Significant wave height of at least 6 m%swhg8Significant wave height of at least 8 m%Ensemble numbersYou can select individual members of the ensemble forecast use the keywordnumber....stream="enfo",step=24,param="msl",number=1,...stream="enfo",step=24,param="msl",number=[1,10,20],...List of ensemble numbers1 to 50📌NOTE: Not specifyingnumberwill return all ensemble forecast members.ExamplesDownload a single surface parameter at a single forecast step from ECMWF's 00UTC HRES forecastfromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="oper",type="fc",step=24,param="2t",target="data.grib2",)Download the tropical cyclone tracks from ECMWF's 00UTC HRES forecastfromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="oper",type="tf",step=240,target="data.bufr",)The downloaded data are encoded in BUFR edition 4For the HRES Tropical Cyclone tracks at time=06 and time=18 use:...step=90,...❗NOTE:Tropical cyclone tracks products are only available when there are tropical cyclones observed or forecast.Download a single surface parameter at a single forecast step for all ensemble members from ECMWF's 12UTC 00UTC ENS forecastfromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="enfo",type="pf",param="msl",target="data.grib2",)To download a single ensemble member, use thenumberkeyword:number=1.All of the odd numbered ensemble members usenumber=[num for num in range(1,51,2)].To download the control member, usetype="cf".Download the Tropical Cyclone tracks from ECMWF's 00UTC ENS forecastThe Tropical Cyclone tracks are identified by the keywordtype="tf".fromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="enfo",type="tf",step=240,target="data.bufr",)The downloaded data are encoded in BUFR edition 4For the ENS Tropical Cyclone tracks at time=06 and time=18 replacestep=240withstep=144.Download the ensemble mean and standard deviation for all parameters at a single forecast step from ECMWF's 00UTC ENS forecastThe ensemble mean and standard deviation are identified by the keywordstype="em":fromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="enfo",type="em",step=24,target="data.grib2",)andtype="es", respectively:fromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="enfo",type="es",step=24,target="data.grib2",)Download the ensemble probability productsThe ensemble probability products are identified by the keywordtype="ep".  The probability products are available only fortime=00andtime=12.Two different products are available.Probabilities - Instantaneous weather events - Pressure levelsThe probability of temperature standardized anomalies at a constant
pressure level of 850hPa are available at 12 hourly forecast steps.fromecmwf.opendataimportClientclient=Client(source="ecmwf")client.retrieve(time=0,stream="enfo",type="ep",step=[iforiinrange(12,361,12)],levelist=850,param=["ptsa_gt_1stdev","ptsa_gt_1p5stdev","ptsa_gt_2stdev","ptsa_lt_1stdev","ptsa_lt_1p5stdev","ptsa_lt_2stdev",],target="data.grib2",)Probabilities - Daily weather events - Single levelThe probabilities of total precipitation and wind gusts exceeding specified thresholds in a 24 hour period are available for step ranges 0-24 to 336-360 by 12​​.  These are specified in the retrieval request using, e.g.:step=["0-24", "12-36", "24-48"].fromecmwf.opendataimportClientclient=Client(source="ecmwf")steps=[f"{12*i}-{12*i+24}"foriinrange(29)]client.retrieve(time=0,stream="enfo",type="ep",step=steps,param=["tpg1","tpg5","10fgg10"],target="data.grib2",)ECMWF open data licenseBy downloading data from the ECMWF open data dataset, you agree to the their terms: Attribution 4.0 International (CC BY 4.0). If you do not agree with such terms, do not download the data. Visitthis pagefor more information.LicenseApache License 2.0In applying this licence, ECMWF does not waive the privileges and immunities
granted to it by virtue of its status as an intergovernmental organisation nor does it submit to any jurisdiction.