Package: imfp
imfpimfp, by Christopher C. Smith, is a Python package for downloading data from theInternational Monetary
Fund'sRESTful JSON
API.InstallationTo install the stable version of imfp from PyPi, use pip.pipinstall-q--upgradeimfpTo load the library, useimport:importimfpUsageSuggested packagesimfpoutputs data in apandasdata frame, so you will want to use thepandaspackage for its functions for viewing and manipulating this object type. I also recommendmatplotliborseabornfor making plots, andnumpyfor computation. These packages can be installed usingpipand loaded usingimport:importseabornFetching an Index of Databases with theimf_databasesFunctionTheimfppackage introduces four core functions:imfp.imf_databases,imfp.imf_parameters,imfp.imf_parameter_defs, andimfp.imf_dataset. The function for downloading datasets isimfp.imf_dataset, but you will need the other functions to determine what arguments to supply toimfp.imf_dataset. For instance, all calls toimfp.imf_datasetrequire adatabase_id. This is because the IMF serves many different databases through its API, and the API needs to know which of these many databases you're requesting data from. To obtain a list of databases, useimfp.imf_databases, like so:#Fetch the list of databases available through the IMF APIdatabases=imfp.imf_databases()databases.head()database_iddescription0BOP_2017M06Balance of Payments (BOP), 2017 M061BOP_2020M3Balance of Payments (BOP), 2020 M032BOP_2017M11Balance of Payments (BOP), 2017 M113DOT_2020Q1Direction of Trade Statistics (DOTS), 2020 Q14GFSMAB2016Government Finance Statistics Yearbook (GFSY 2...This function returns the IMF’s listing of 259 databases available through the API. (In reality, 7 of the listed databases are defunct and not actually available: FAS_2015, GFS01, FM202010, APDREO202010, AFRREO202010, WHDREO202010, BOPAGG_2020.)To view and explore the database list, it’s possible to explore subsets of the data frame by row number withdatabases.loc:# View a subset consisting of rows 5 through 9databases.loc[5:9]database_iddescription5BOP_2019M12Balance of Payments (BOP), 2019 M126GFSYFALCS2014Government Finance Statistics Yearbook (GFSY 2...7GFSE2016Government Finance Statistics Yearbook (GFSY 2...8FM201510Fiscal Monitor (FM) October 20159GFSIBS2016Government Finance Statistics Yearbook (GFSY 2...Or, if you already know which database you want, you can fetch the corresponding code by searching for a string match usingstr.containsand subsetting the data frame for matching rows. For instance, here’s how to search for the Primary Commodity Price System:databases[databases['description'].str.contains("Commodity")]database_iddescription237PCTOTCommodity Terms of Trade239PCPSPrimary Commodity Price System (PCPS)Fetching a List of Parameters and Input Codes withimf_parametersandimf_parameter_defsOnce you have adatabase_id, it’s possible to make a call toimfp.imf_datasetto fetch the entire database:imfp.imf_dataset(database_id). However, while this will succeed for a few small databases, it will fail for all of the larger ones. And even in the rare case when it succeeds, fetching an entire database can take a long time. You’re much better off supplying additional filter parameters to reduce the size of your request.Requests to databases available through the IMF API are complicated by the fact that each database uses a different set of parameters when making a request. (At last count, there were 43 unique parameters used in making API requests from the various databases!) You also have to have the list of valid input codes for each parameter. Theimfp.imf_parametersfunction solves this problem. Use the function to obtain the full list of parameters and valid input codes for a given database:# Fetch list of valid parameters and input codes for commodity price databaseparams=imfp.imf_parameters("PCPS")Theimfp.imf_parametersfunction returns a dictionary of data frames. Each dictionary key name corresponds to a parameter used in making requests from the database:# Get key names from the params objectparams.keys()dict_keys(['freq', 'ref_area', 'commodity', 'unit_measure'])In the event that a parameter name is not self-explanatory, theimfp.imf_parameter_defsfunction can be used to fetch short text descriptions of each parameter:# Fetch and display parameter text descriptions for the commodity price databaseimfp.imf_parameter_defs("PCPS")parameterdescription0freqFrequency1ref_areaGeographical Areas2commodityIndicator3unit_measureUnitEach named list item is a data frame containing a vector of valid input codes that can be used with the named parameter, and a vector of text descriptions of what each code represents.To access the data frame containing valid values for each parameter, subset theparamsdict by the parameter name:# View the data frame of valid input codes for the frequency parameterparams['freq']input_codedescription0AAnnual1MMonthly2QQuarterlyViewing Data FramesNote thatpandasdata frames in Python can be a little difficult to work with, because Python doesn't have a built-in variable explorer. If you're doing data science, I recommend using an IDE like RStudio or Spyder that has a built-in variable explorer. However, if you don't have a variable explorer, you can prevent Python from truncating data frames using theoptionsinpandas. For instance, to increase the maximum allowed column width to 100 characters, we can usepandas.options.display.max_colwidth = 100.Alternatively, it's possible to open the data frame in a new window to view it in full:importimfpimporttempfileimportwebbrowser# Define a simple function to view data frame in a browser windowdefView(df):html=df.to_html()withtempfile.NamedTemporaryFile('w',delete=False,suffix='.html')asf:url='file://'+f.namef.write(html)webbrowser.open(url)# Open data frame in a new browser window using the functiondf=imfp.imf_databases()View(df)Supplying Parameter Arguments toimf_dataset: A Tale of Two WorkflowsThere are two ways to supply parameters toimfp.imf_dataset: by supplying list arguments or by supplying a modified parameters dict. The list arguments workflow will be more intuitive for most users, but the dict argument workflow requires a little less code.The List Arguments WorkflowTo supply list arguments, just find the codes you want and supply them toimfp.imf_datasetusing the parameter name as the argument name. The example below shows how to request 2000–2015 annual coal prices from the Primary Commodity Price System database:# Fetch the 'freq' input code for annual frequencyselected_freq=list(params['freq']['input_code'][params['freq']['description'].str.contains("Annual")])# Fetch the 'commodity' input code for coalselected_commodity=list(params['commodity']['input_code'][params['commodity']['description'].str.contains("Coal")])# Fetch the 'unit_measure' input code for indexselected_unit_measure=list(params['unit_measure']['input_code'][params['unit_measure']['description'].str.contains("Index")])# Request data from the APIdf=imfp.imf_dataset(database_id="PCPS",freq=selected_freq,commodity=selected_commodity,unit_measure=selected_unit_measure,start_year=2000,end_year=2015)# Display the first few entries in the retrieved data framedf.head()freqref_areacommodityunit_measureunit_multtime_formattime_periodobs_value0AW00PCOALIX0P1Y200039.35102302932021AW00PCOALIX0P1Y200149.33785872840392AW00PCOALIX0P1Y200239.49490916480063AW00PCOALIX0P1Y200343.28788769507884AW00PCOALIX0P1Y200482.9185858052862The Parameters Argument WorkflowTo supply a list object, modify each data frame in theparamslist object to retain only the rows you want, and then supply the modified list object toimfp.imf_datasetas its parameters argument. Here is how to make the same request for annual coal price data using a parameters list:# Fetch the 'freq' input code for annual frequencyparams['freq']=params['freq'][params['freq']['description'].str.contains("Annual")]# Fetch the 'commodity' input code(s) for coalparams['commodity']=params['commodity'][params['commodity']['description'].str.contains("Coal")]# Fetch the 'unit_measure' input code for indexparams['unit_measure']=params['unit_measure'][params['unit_measure']['description'].str.contains("Index")]# Request data from the APIdf=imfp.imf_dataset(database_id="PCPS",parameters=params,start_year=2000,end_year=2015)# Display the first few entries in the retrieved data framedf.head()freqref_areacommodityunit_measureunit_multtime_formattime_periodobs_value0AW00PCOALIX0P1Y200039.35102302932021AW00PCOALIX0P1Y200149.33785872840392AW00PCOALIX0P1Y200239.49490916480063AW00PCOALIX0P1Y200343.28788769507884AW00PCOALIX0P1Y200482.9185858052862Working with the Returned Data FrameNote that all columns in the returned data frame are character vectors, and that to plot the series we will need to convert to valid numeric or date formats. Usingseabornwithhue, we can plot different indicators in different colors:# Convert obs_value to numeric and time_period to integer yeardf=df.astype({"time_period":int,"obs_value":float})# Plot prices of different commodities in different colors with seabornseaborn.lineplot(data=df,x='time_period',y='obs_value',hue='commodity');Also note that the returned data frame has mysterious-looking codes as values in some columns.Codes in thetime_formatcolumn are ISO 8601 duration codes. In this case, “P1Y” means “periods of 1 year.” Theunit_multcolumn represents the number of zeroes you should add to the value column. For instance, if value is in millions, then the unit multiplier will be 6. If in billions, then the unit multiplier will be 9.The meanings of the other codes are stored in ourparamsobject and can be fetched with a join. For instance to fetch the meaning of theref_areacode “W00”, we can perform a left join with theparams['ref_area']data frame and use select to replaceref_areawith the parameter description:# Join df with params['ref_area'] to fetch code descriptiondf=df.merge(params['ref_area'],left_on='ref_area',right_on='input_code',how='left')# Drop redundant columns and rename description columndf=df.drop(columns=['ref_area','input_code']).rename(columns={"description":"ref_area"})# View first few columns in the modified data framedf.head()freqcommodityunit_measureunit_multtime_formattime_periodobs_valueref_area0APCOALIX0P1Y200039.351023All Countries, excluding the IO1APCOALIX0P1Y200149.337859All Countries, excluding the IO2APCOALIX0P1Y200239.494909All Countries, excluding the IO3APCOALIX0P1Y200343.287888All Countries, excluding the IO4APCOALIX0P1Y200482.918586All Countries, excluding the IORate and Bandwidth Limit ManagementSetting a Unique Application Name withset_imf_app_nameimfp.set_imf_app_name()allows users to set a custom application name to be used when making API calls to the IMF API. The IMF API has an application-based rate limit of 50 requests per second, with the application identified by the "user_agent" variable in the request header.This could prove problematic if theimfplibrary became too popular and too many users tried to make simultaneous API requests using the default app name. By setting a custom application name, users can avoid hitting rate limits and being blocked by the API.imfp.set_imf_app_name()sets the application name by changing theIMF_APP_NAMEvariable in the environment. If this variable doesn't exist,imfp.set_imf_app_name()will create it.To set a custom application name, simply call theimfp.set_imf_app_name()function with your desired application name as an argument:# Set custom app name as an environment variableimfp.set_imf_app_name("my_custom_app_name")The function will throw an error if the provided name is missing, NULL, NA, not a string, or longer than 255 characters. If the provided name is "imfr" (the default) or an empty string, the function will issue a warning recommending the use of a unique app name to avoid hitting rate limits.Changing the enforced wait time between API calls withset_imf_wait_timeBy default,imfpenforces a mandatory 1.5-second wait time between API calls to prevent repeated or recursive calls from exceeding the API's bandwidth/rate limit. This wait time should be sufficient for most applications. However, if you are running parallel processes usingimfp(e.g. during cross-platform testing), this wait time may be insufficient to prevent you from running up against the API's rate and bandwidth limits. You can change this wait time by calling theset_imf_wait_timefunction with a numeric value, in seconds. For instance, to enforce a five-second wait time between API calls, useset_imf_wait_time(10).Also note that by default,imfpfunctions will retry any API call rejected for bandwidth or rate limit reasons. The number of timesimfpwill attempt the call is set by thetimesargument, with a default value of 3. (With this value, requests will be retried twice after an initial failure.) Note thatimfpenforces an exponentially increasing wait time between function calls, with a base wait time of 5 seconds on the first retry, so it is not recommended to set a high value fortimes.ContributingI would love to have your help in improvingimfr. If you encounter a bug while using the library, please open an issue. Alternatively, fix the bug and open a pull request. Thanks in advance for your help!