Package: bblocks
The bblocks packagebblocksis a python package with tools to download and analyse
development data. These tools are meant to be thebuilding blocksof
further analysis.We have builtbblocksto support our work at ONE, but we hope that
it will be useful to others working with development data. We welcome feedback,
feature requests, and collaboration.bblocksis organised around the following main features:Import toolsto help with import data from:The World Bank (building on the wbgapi package)The IMF World Economic outlook (building on the weo package)The IMF data on Special Drawing RightsThe World Food Programme (WFP) data on food security and inflationThe FAO (notably the price index)The UNDP Human Development Report dataUNAIDSThe WHO Government Health Expenditure dataCleaning toolsto help with:Cleaning numbers/numeric seriesTransforming country identifiers (ISO2, ISO3, WB, UN, etc., building on the country_converter package)Transforming text to datetime objects, and datetime objects to textFormatting numbers as text (percentages, millions, billions, etc.)Analysis toolsto help with:Calculating period averagesCalculating the change from one period to anotherDataFrame toolsto help with:Adding a population column to a DataFrameAdding a "share of population" / "per capita" column to a DataFrameAdding a population density column to a DataFrameAdding a GDP column to a DataFrameAdding a "share of GDP" column to a DataFrameAdding a poverty ratio column to a DataFrameAdding a government expenditure column to a DataFrameAdding a "share of government expenditure" column to a DataFrameAdding a "World Bank income level" column to a DataFrameAdding a column with short country names to a DataFrameAdding a column with ISO3 codes to a DataFrameAdding the median observation of a groupAdding a column with geojson geometries to a DataFrameOther toolslike:Dictionaries mapping ISO3 codes (and vice-versa) toOECD DAC codesWB income groupsgeojson geometriesG7, EU27, G20 countriesIncome levelsLife expectancyPopulationMore information is available:Documentation:https://bblocks.readthedocs.io/GitHub:https://github.com/ONECampaign/bblocksPyPI:https://pypi.org/project/bblocks/Installationbblocks can be installed from using pippipinstallbblocks--upgradeThe package is compatible with Python 3.10 and above.Basic usageTo get started, import the package. It is strongly recommended that you specify
the path to the folder where you want to store the data.You only have to do this once per file/notebook.frombblocksimportset_bblocks_data_path# Set to the folder you wantset_bblocks_data_path("path/to/data/folder")All the examples below assume that you have done this.Importing data from the World BankfrombblocksimportWorldBankData# create a WorldBankData object. This object will allow you# to download indicators from the World Bank and get them as DataFrameswb=WorldBankData()# For example to get "primary completion rate" (SE.PRM.CMPT.ZS) from 2010 to 2020.# If the data is not already in your data folder, it will be downloadedwb.load_data(indicator="SE.PRM.CMPT.ZS",start_year=2010,end_year=2020)# Get the data as a DataFramedf=wb.get_data()# Print a sample of 10 rowsprint(df.sample(10))The above would return a DataFrame like this:dateiso_codeindicator_codevalue2010-01-01LMCSE.PRM.CMPT.ZS87.7531892012-01-01SWZSE.PRM.CMPT.ZS84.6974722013-01-01NAMSE.PRM.CMPT.ZS93.0200422012-01-01PAKSE.PRM.CMPT.ZS63.4862102015-01-01LICSE.PRM.CMPT.ZS63.4634702016-01-01BGDSE.PRM.CMPT.ZSNaN2019-01-01SYRSE.PRM.CMPT.ZSNaN2013-01-01NACSE.PRM.CMPT.ZS99.0257032011-01-01ANDSE.PRM.CMPT.ZSNaN2013-01-01GRLSE.PRM.CMPT.ZSNaNYou can also get the latest data (most recent non-empty observation) for one or more indicators:frombblocksimportWorldBankData# create a WorldBankData object.wb_data=WorldBankData()# Load the indicators. If they are not downloaded, they will bewb_data.load_data(indicator=["SH.XPD.CHEX.PC.CD","SH.XPD.CHEX.GD.ZS"],most_recent_only=True)# Get the data as a DataFramedf=wb_data.get_data(indicators="all")# Print a sample of the dataprint(df.sample(10))This would return a DataFrame like this:dateiso_codeindicator_codevalue2019-01-01HRVSH.XPD.CHEX.PC.CD1040.0856932019-01-01ERISH.XPD.CHEX.GD.ZS4.4587672019-01-01JAMSH.XPD.CHEX.PC.CD327.4035342019-01-01MYSSH.XPD.CHEX.PC.CD436.6120302019-01-01BHSSH.XPD.CHEX.GD.ZS5.7497752015-01-01YEMSH.XPD.CHEX.PC.CD73.1767432019-01-01PERSH.XPD.CHEX.PC.CD370.1099552019-01-01IDASH.XPD.CHEX.PC.CD52.0762852019-01-01ERISH.XPD.CHEX.PC.CD25.2679352019-01-01WLDSH.XPD.CHEX.PC.CD1115.008730In all cases, if you had already downloaded the data and you want to update it
you can call.update_data()after loading the data in order to refresh it.wb_data.update_data(reload_data=True)Importing data from UNAIDSfrombblocksimportAids# create an Aids object. This object will allow you# to download indicators from UNAIDS and get them as DataFramesaids=Aids()# To view all the indicators that can be downloaded using this tool# you can use the `.available_indicators` propertyaids.available_indicatorsHer are the first 10 indicators, but over 50 are available:indicatorcategory0Trend of new HIV infectionsEpidemic transition metrics1Trend of AIDS-related deathsEpidemic transition metrics2Incidence:prevalence ratioEpidemic transition metrics3Incidence:mortality ratioEpidemic transition metrics4People living with HIV - All agesPeople living with HIV5People living with HIV - Children (0-14)People living with HIV6People living with HIV - Adolescents (10-19)People living with HIV7People living with HIV - Young people (15-24)People living with HIV8People living with HIV - Adults (15+)People living with HIV9People living with HIV - Adults (15-49)People living with HIV# to load/download indicators, you can use the `.load_data` method# you can also specify whether to download "country", "region", or "all"aids.load_data(indicator="Trend of AIDS-related deaths",area_grouping="region")# get the data as a DataFramedf=aids.get_data()# print a sample of 10 rowsprint(df.sample(10))area_namearea_idyearindicatordimensionvalueGlobal03M49WLD2013Trend of AIDS-related deathsAll ages estimate1.061395e+06Latin AmericaUNALA2021Trend of AIDS-related deathsAll ages estimate2.916500e+04Middle East and North AfricaUNAMENA2018Trend of AIDS-related deathsAll ages lower estimate4.089657e+03Western & Central Europe and North AmericaUNAWCENA2019Trend of AIDS-related deathsAll ages estimate1.305140e+04CaribbeanUNACAR2021Trend of AIDS-related deathsAll ages lower estimate4.213485e+03Middle East and North AfricaUNAMENA2021Trend of AIDS-related deathsAll ages upper estimate6.867407e+03Western & Central Europe and North AmericaUNAWCENA2016Trend of AIDS-related deathsAll ages upper estimate1.771698e+04Western & Central Europe and North AmericaUNAWCENA2020Trend of AIDS-related deathsAll ages upper estimate1.632782e+04Eastern Europe and Central AsiaUNAEECA2017Trend of AIDS-related deathsAll ages upper estimate4.553729e+04Latin AmericaUNALA2020Trend of AIDS-related deathsAll ages upper estimate4.577862e+04As with other bblocks tools, you can also get multiple indicators at once
(see the WorldBank example).In all cases, if you had already downloaded the data and you want to update it
you can call.update_data()after loading the data in order to refresh it.aids.update_data(reload_data=True)Importing SDR data from the IMF# Import the SDR object from the sdr module of "import_tools"frombblocks.import_tools.sdrimportSDR# Create an SDR objectsdr=SDR()# To view the latest date for which data is available,# call the `.latest_date()` methodsdr.latest_date()# To download the latest datasdr.load_data(date="latest")# To get the data as a DataFrame. You can specify getting a# specific indicator by using 'indicator'. In this case,# we'll get holdings (allocations are also available)df=sdr.get_data(indicator="holdings")# Print a sample of 10 rowsprint(df.sample(10))entityindicatorvaluedateSamoaholdings1.584296e+072023-01-31Iraqholdings3.301367e+072023-01-31Lao People\'s Democratic Republicholdings5.870183e+072023-01-31Haitiholdings9.169516e+072023-01-31Bahamas, Theholdings1.245326e+082023-01-31Totalholdings6.606989e+112023-01-31Libyaholdings3.187335e+092023-01-31Namibiaholdings1.783556e+082023-01-31Tajikistan, Republic ofholdings1.891507e+082023-01-31Maltaholdings2.499760e+082023-01-31In all cases, if you had already downloaded the data and you want to update it
you can call.update_data()after loading the data in order to refresh it.sdr.update_data(reload_data=True)Adding World Bank income levels to a DataFrameFor this example, we will continue using the SDR data as above.frombblocksimportadd_income_level_column# We can add the column by passing the dataframe to the functiondf=add_income_level_column(df=df,id_column="entity",id_type="regex",# so the text can be matched to the right country)Which adds the income level column:entityindicatorvaluedateincome_levelMontenegro, Republic ofholdings7.404593e+072023-01-31Upper middle incomeGambia, Theholdings5.857020e+072023-01-31Low incomeSurinameholdings1.211070e+082023-01-31Upper middle incomeSyrian Arab Republicholdings5.636629e+082023-01-31Low incomeIran, Islamic Republic ofholdings4.976198e+092023-01-31Lower middle incomeUruguayholdings6.330507e+082023-01-31High incomeSouth Africaholdings4.424154e+092023-01-31Upper middle incomeNigeriaholdings3.755370e+092023-01-31Lower middle incomeDominican Republicholdings4.498683e+082023-01-31Upper middle incomeTrinidad and Tobagoholdings7.722810e+082023-01-31High incomeAn optional argument can be passed to the function to redownload the income classification
data from the World Bank.df=add_income_level_column(df=df,id_column="entity",id_type="regex",update_data=True,)Adding a GDP share column to a DataFrameFor this example, we will continue working with data on military expenditure downloaded using
the World Bank tool.# First import the function from the `add` module of `dataframe_tools`frombblocks.dataframe_tools.addimportadd_gdp_share_columnfrombblocksimportWorldBankData# this data is in local currency unitsdf=WorldBankData().load_data(indicator="MS.MIL.XPND.CN",most_recent_only=True).get_data()dateiso_codeindicator_codevalue2021-01-01BDIMS.MIL.XPND.CN1.351000e+112014-01-01YEMMS.MIL.XPND.CN3.685000e+112021-01-01AFGMS.MIL.XPND.CN2.304000e+102021-01-01PERMS.MIL.XPND.CN9.086000e+092021-01-01AUSMS.MIL.XPND.CN4.229595e+10# Then call the function, passing the DataFrame and the column namedf=add_gdp_share_column(df=df,id_column="iso_code",id_type="ISO3",date_column="date",# to match the gdp values with the year of the datavalue_column="value",decimals=1,usd=False,# since the data is in local currency unitsinclude_estimates=True,# to include official data and IMF estimates for GDP)print(df.sample(10))Which returns a dataframe with an extra column "gdp_share".dateiso_codeindicator_codevaluegdp_share2021-01-01GINMS.MIL.XPND.CN2.406750e+121.52014-01-01AREMS.MIL.XPND.CN8.356800e+105.62021-01-01NGAMS.MIL.XPND.CN1.783120e+121.02021-01-01GNQMS.MIL.XPND.CN9.439700e+101.42021-01-01ISLMS.MIL.XPND.CN0.000000e+000.02021-01-01ESPMS.MIL.XPND.CN1.652680e+101.42021-01-01BHRMS.MIL.XPND.CN5.194000e+083.62021-01-01GEOMS.MIL.XPND.CN9.723000e+081.62021-01-01MDAMS.MIL.XPND.CN9.144000e+080.42013-01-01LAOMS.MIL.XPND.CN1.782500e+110.2Cleaning a numeric series which contains numbers with textSometimes dataframes contain columns which don't have clean text. For example,
something likeiso_codevalue0USA10%1GBR+12%2FRA13.4%3DEU%14.34ITA15.3  %5ESP16%6CAN17%7JPN18%8AUS19%9CHN20%bblocks can help clean that data.frombblocksimportclean_numeric_seriesdf['value']=clean_numeric_series(data=df['value'],to=float# or if dealing with integers, use to=int)Returns a clean version of the dataiso_codevalue0USA10.01GBR12.02FRA13.43DEU14.34ITA15.35ESP16.06CAN17.07JPN18.08AUS19.09CHN20.0ContributingInterested in contributing to the package? Please reach out.