Package: weightipy
WeightipyWeightipy is a cut down version ofQuantipy3for weighting people data using the RIM (iterative raking) algorithm.Changes from QuantipyRemoved all quantipy overhead. Weightipy supports the latest versions of Pandas and Numpy and is tested for Python 3.7, 3.8, 3.9, 3.10 and 3.11.Weightipy runs up to 6 times faster than Quantipy, depending on the dataset.Rim class will not generate reports like Quantipy did, unless the parameter verbose is set to True on the Rim constructor.Installationpip install weightipyorpython3 -m pip install weightipyCreate a virtual envirionmentIf you want to create a virtual environment when using Weightipy:condacondacreate-nenvwppython=3with venvpython-mvenv[your_env_name]5-minutes to WeightipyGet startedAssuming we have the variablesgenderandagecatwe can weight the dataset like this:importweightipyaswptargets={"agecat":{"18-24":5.0,"25-34":30.0,"35-49":26.0,"50-64":19.0,"65+":20.0},"gender":{"Male":49,"Female":51}}scheme=wp.scheme_from_dict(targets)df_weighted=wp.weight_dataframe(df=my_df,scheme=scheme,weight_column="weights")efficiency=wp.weighting_efficiency(df_weighted["weights"])In case we are working with census data, which also includes a region variable and we would
like to weight the data by age and gender in each region, we can use thescheme_from_dffunction:importweightipyaswpimportpandasaspddf_data=pd.read_csv("data_to_weight.csv")df_census=pd.read_csv("census_data.csv")scheme=wp.scheme_from_df(df=df_census,cols_weighting=["agecat","gender"],col_filter="region",col_freq="freq")df_weighted=wp.weight_dataframe(df=d,scheme=scheme,weight_column="weights")efficiency=wp.weighting_efficiency(df_weighted["weights"])Or by using the underlying functions that will give more access to the weighting process, we
can use the Rim and WeightEngine classes directly:importweightipyaswp# in this example, agecat and gender are int dtypeage_targets={'agecat':{1:5.0,2:30.0,3:26.0,4:19.0,5:20.0}}gender_targets={'gender':{0:49,1:51}}scheme=wp.Rim('gender_and_age')scheme.set_targets(targets=[age_targets,gender_targets])my_df["identity"]=range(len(my_df))engine=wp.WeightEngine(data=df)engine.add_scheme(scheme=scheme,key="identity",verbose=False)engine.run()df_weighted=engine.dataframe()col_weights=f"weights_{scheme.name}"efficiency=wp.weighting_efficiency(df_weighted[col_weights])print(engine.get_report())Weightvariableweights_gender_and_ageWeightgroup_default_name_WeightfilterNoneTotal:unweighted582.000000Total:weighted582.000000Weightingefficiency60.009826Iterationsrequired14.000000Meanweightfactor1.000000Minimumweightfactor0.465818Maximumweightfactor6.187700Weightfactorratio13.283522For more references on the underlying classes, refer to the QuantipydocumentationOverview of functions to get started:FunctionDescriptionweight_dataframeWeights data by scheme, returns modified dataframe with new weight column.weighting_efficiencyTakes weights and returns efficiency of weighting. See:https://quantipy.readthedocs.io/en/staging-develop/sites/lib_doc/weights/03_diags.html#the-weighting-efficiencyscheme_from_dictTurns a dict of dicts into a Rim scheme. Keys of the dict are column names and the values are distributions. These are normalized.scheme_from_dfCreates a Rim scheme from a dataframe from specified weighting columns and frequency column. Useful when working with census data.Rim classUseful for creation of more complex weighting schemas. For example when weighting subregions or groups, which require filters. See:https://quantipy.readthedocs.io/en/staging-develop/sites/lib_doc/weights/02_rim.html#using-the-rim-classWeightEngine classUseful for more specialised manipulation of the weighting processPlanned featuresMore utility functions to simplify the weighting processMore performance improvements, in order to better support batch weighting of many datasetsSupport for multithreaded weighting (possibly using Polars)Rewrite of the API to be less oriented towards how Quantipy worked and more in line with simple weighting needsFar future: Support for more weighting algorithmsContributingThe test suite for Weightipy can be run with the commandpython3 -m pytest testsBut when developing a specific aspect of Weightipy, it might be quicker to run (e.g. for the Rim class)python3 -m unittest tests.test_rimWe welcome volunteers and supporters. Please include a test case with any pull request, especially those that run calculations.QuantipyOriginsQuantipy was concieved of and instigated by Gary Nelson:http://www.datasmoothie.comContributors on QuantipyAlexander Buchhammer, Alasdair Eaglestone, James Griffiths, Kerstin Müller :https://yougov.co.ukDatasmoothie’s Birgir Hrafn Sigurðsson andGeir Freysson:http://www.datasmoothie.com