Package: zrp
Zest Race PredictorZest Race Predictor (ZRP) is an open-source machine learning algorithm
that estimates the race/ethnicity of an individual using only their full
name and home address as inputs. ZRP improves upon the most widely used
racial and ethnic data estimation method, Bayesian Improved Surname
Geocoding (BISG), developed by RAND Corporation in 2009.ZRP was built using ML techniques such as gradient boosting and trained
on voter data from the southeastern U.S. It was then validated on a
national sample using adjusted tract-level American Community Survey
(ACS) data. (Model training procedures are provided.)Compared to BISG, ZRP correctly identified:25% more African-Americans as African-American35% fewer African-Americans as non-African American60% fewer Whites as non-WhiteZRP can be used to analyze racial equity and outcomes in critical
spheres such as health care, financial services, criminal justice, or
anywhere there's a need to impute the race or ethnicity of a population
dataset. (Usage examples are included.) The financial services industry,
for example, has struggled for years to achieve more equitable outcomes
amid charges of discrimination in lending practices.Zest AI began developing ZRP in 2020 to improve the accuracy of our
clients' fair lending analyses by using more data and better math. We
believe ZRP can greatly improve our understanding of the disparate
impact and disparate treatment of protected-status borrowers. Armed with
a better understanding of the disparities that exist in our financial
system, we can highlight inequities and create a roadmap to improve
equity in access to finance.NotesThis is the preliminary version and implementation of the ZRP tool.
We're dedicated to continue improving both the algorithm and
documentation and hope that government agencies, lenders, citizen data
scientists and other interested parties will help us improve the model.
Details of the model development process can be found in themodel
development documentationInstallInstall requires an internet connection. The package has been tested on python 3.7.4, but should likely work with 3.7.X.Note: Due to the size and number of lookup tables necesary for the zrp
package, total installation requires 3 GB of available space.Setting up your virtual environmentWe recommend installing zrp
inside apython virtual
environment.Run the following to build your virtual envrionment:python3 -m venv /path/to/new/virtual/environmentActivate your virtual environment:source /path/to/new/virtual/environment/bin/activateEx.:python -m venv /Users/joejones/Documents/ZestAI/zrpvenv
source /Users/joejones/Documents/ZestAI/zrpvenv/bin/activateUnix-like systemspip install zrpAfter installing via pip, you need to download the lookup tables and
pipelines using the following command: :python -m zrp downloadWindowspip install pipwin
pipwin install gdal
pipwin install fiona

pip install zrpAfter installing via pip, you need to download the lookup tables and
pipelines using the following command: :python -m zrp downloadIf you're experiencing issues with installation, please consult ourCommon Issuespage.DataTraining DataThe models available in this package were trained on voter registration
data from the states of Florida , Georgia, and North Carolina. Summary
statistics on these datasets and additional datasets used as validation
can be foundhere.Consult the following to download state voter registration data:North CarolinaFloridaAlabamaSouth CarolinaGeorgiaLouisianaAmerican Community Survey (ACS) Data:The US Census Bureau details that, "the American Community Survey (ACS)
is an ongoing survey that provides data every year -- giving
communities the current information they need to plan investments and
services. The ACS covers a broad range of topics about social, economic,
demographic, and housing characteristics of the U.S. population. The
5-year estimates from the ACS are "period" estimates that represent
data collected over a period of time. The primary advantage of using
multiyear estimates is the increased statistical reliability of the data
for less populated areas and small population subgroups. The 5-year
estimates are available for all geographies down to the block group
level." ( Bureau, US Census. "American Community Survey 5-Year Data
(2009-2019)." Census.gov, 8 Dec. 2021,https://www.census.gov/data/developers/data-sets/acs-5year.html. )ACS data is available in 1 or 5 year spans. The 5yr ACS data is the most
comprehensive & is available at more granular levels than 1yr data. It
is thus used in this work.Model Development and Feature DocumentationDetails of the model development process can be found in themodel
development documentation. Details of the human
readable feature definitions as well as feature importances can be foundhere.Usage and ExamplesTo get started using the ZRP, first ensure the download is complete (as
described above) and xgboost == 1.0.2Check out the guides in theexamplesfolder.
Clone the repo in order to obtain the example notebooks and data; this
is not provided in the pip installable package. If you're experiencing
issues, first consult ourcommon issues
guide.Here, we additionally
provide an interactive virtual environment, via Binder, with ZRP
installed. Once you open this link and are taken to the JupyterLab
environment, open up a terminal and run the following: :python -m zrp downloadNext, we present the primary ways you'll use ZRP.ZRP PredictionsSummary of commands::>>> from zrp import ZRP
>>> zest_race_predictor = ZRP()
>>> zest_race_predictor.fit()
>>> zrp_output = zest_race_predictor.transform(input_dataframe)Breaking down key commands:>>> zest_race_predictor = ZRP()ZRP(pipe_path=None, support_files_path="data/processed",
key="ZEST_KEY", first_name="first_name",
middle_name="middle_name", last_name="last_name",
house_number="house_number",
street_address="street_address", city="city", state="state",
zip_code="zip_code", race='race', proxy="probs",
census_tract=None, street_address_2=None, name_prefix=None,
name_suffix=None, na_values=None, file_path=None, geocode=True,
bisg=True, readout=True, n_jobs=49, year="2019", span="5",
runname="test")What it does:Prepares data to generate race & ethnicity proxiesYou can find parameter descriptions in theZRP
classand it'sparent
class.>>> zrp_output = zest_race_predictor.transform(input_dataframe)zest_race_predictor.transform(df)What it does:Processes input data and generates ZRP proxy predictions.Attempts to predict on block group, then census tract, then
zip code based on which level ACS data is found for. If Geo
level data is unattainable, the BISG proxy is computed. No
prediction returned if BISG cannot be computed either.Parametersdf : {DataFrame} Pandas dataframe containing input data
           (see below for necessary columns)Input data,df, into the prediction/modeling pipelineMUSTcontain the following columns: first name, middle name, last name, house
number, street address (street name), city, state, zip code, and zest
key. Consult ourcommon issues
guideto
ensure your input data is the correct format.Output: A dataframe with the following columns: AAPI AIAN BLACK
HISPANIC WHITE source_block_group source_zip_code source_bisg :>>> zrp_output

=========== =========== =========== =========== =========== =========== ===================== ====================== ==================  
            AAPI        AIAN        BLACK       HISPANIC    WHITE       source_block_group    source_census_tract    source_zip_code      
=========== =========== =========== =========== =========== =========== ===================== ====================== ==================  
ZEST_KEY                                                                                                                                        
10          0.021916    0.021960    0.004889    0.012153    0.939082    1.0                   0.0                    0.0                    
100         0.009462    0.013033    0.003875    0.008469    0.965162    1.0                   0.0                    0.0                    
103         0.107332    0.000674    0.000584    0.021980    0.869429    1.0                   0.0                    0.0                    
106         0.177411    0.015208    0.003767    0.041668    0.761946    1.0                   0.0                    0.0                    
109         0.000541    0.000416    0.000376    0.000932    0.997736    1.0                   0.0                    0.0                    
...         ...         ...         ...         ...         ...         ...                   ...                    ...                    
556         NaN         NaN         NaN         NaN         NaN         0.0                   0.0                    0.0                    
557         NaN         NaN         NaN         NaN         NaN         0.0                   0.0                    0.0                    
=========== =========== =========== =========== =========== =========== ===================== ====================== ==================One of the parameters to theparent
classthat
ZRP() inherits from isfile_path. This parameter allows you to specify
where theartifacts/folder is outputted during the run of the ZRP.
Once the run is complete, theartifacts/folder will contain the
outputted race/ethnicity proxies and additional logs documenting the
validity of input data.file_pathneed notbe specified. If it is
not defined, theartifacts/folder will be placed in the same
directory of the script running zrp. Subsequent runs will, however,
overwrite the files inartifacts/; providing a unique directory path
forfile_pathwill avoid this.ZRP BuildSummary of commands:>>> from zrp.modeling import ZRP_Build
>>> zest_race_predictor_builder = ZRP_Build('/path/to/desired/output/directory')
>>> zest_race_predictor_builder.fit()
>>> zrp_build_output = zest_race_predictor_builder.transform(input_training_data)Breaking down key commands:>>> zest_race_predictor_builder = ZRP_Build('/path/to/desired/output/directory')ZRP_Build(file_path, zrp_model_name = 'zrp_0',
zrp_model_source ='ct')What it does:Prepares the class that builds the new custom ZRP model.Parametersfile_path : {str} The path where pipeline, model, and
           supporting data are saved.

           zrp_model_name : {str} Name of zrp_model.

           zrp_model_source : {str} Indicates the source of
           zrp_modeling data to use.You can find more detailed parameter descriptions in theZRP_Build
class.
ZRP_Build() also inherits initlizing parameters from itsparent
class.>>> zrp_build_output = zest_race_predictor_builder.transform(input_training_data)zest_race_predictor_builder.transform(df)What it does:Builds a new custom ZRP model trained off of user input data
when supplied with standard ZRP requirements including name,
address, and raceProduces a custom model-pipeline. The pipeline, model, and
supporting data are saved automatically to
"~/data/experiments/model_source/data/" in the support
files path defined.The class assumes data is not broken into train and test
sets, performs this split itself, and outputs predictions on
the test set.Parametersdf : {DataFrame} Pandas dataframe containing input data
           (see below for necessary columns)Input data,df, into this pipelineMUSTcontain the following
columns: first name, middle name, last name, house number, street
address (street name), city, state, zip code, zest key, and race.
Consult ourcommon issues
guideto
ensure your input data is the correct format.Output: A dictionary of race & ethnicity probablities and labels.As mentioned in the ZRP Predict section above, once the run is complete,
theartifacts/folder will contain the outputted race/ethnicity
proxies and additional logs documenting the validity of input data.
Similarly, definingfile_pathneed notbe specified, but providing
a unique directory path forfile_pathwill avoid overwriting the
[artifacts/]{.title-ref} folder. When running ZRP Build, however,artifacts/also contains the processed test and train data, trained
model, and pipeline.Additional Runs of Your Custom ModelAfter having run ZRP_Build() you can re-use your custom model just like
you run the packaged model. All you must do is specify the path to the
generated model and pipelines (this path is the same path as
'/path/to/desired/output/directory' that you defined previously when
running ZRP_Build() in the example above; we call this 'pipe_path').
Thus, you would run: :>>> from zrp import ZRP
>>> zest_race_predictor = ZRP('pipe_path')
>>> zest_race_predictor.fit()
>>> zrp_output = zest_race_predictor.transform(input_dataframe)ValidationThe models included in this package were trained on publicly-available
voter registration data and validated multiple times: on hold out sets
of voter registration data and on a national sample of PPP loan
forgiveness data. The results were consistent across tests: 20-30% more
African Americans correctily identified as African American, and 60%
fewer whites identified as people of color as compared with the status
quo BISG method.To see our validation analysis with Alabama voter registration data,
please check outthis
notebook.Performance on the national PPP loan forgiveness dataset was as follows
(comparing ZRP softmax with the BISG method):African AmericanStatisticBISGZRPPct. DiffTrue Positive Rate0.5710.700+23% (F)True Negative Rate0.9540.961+01% (F)False Positive Rate0.0460.039-15% (F)False Negative Rate0.4290.300-30% (F)Asian American and Pacific IslanderStatisticBISGZRPPct. DiffTrue Positive Rate0.6830.777+14% (F)True Negative Rate0.9820.977-01% (U)False Positive Rate0.0180.023-28% (F)False Negative Rate0.3170.223-30% (F)Non-White HispanicStatisticBISGZRPPct. DiffTrue Positive Rate0.5990.711+19% (F)True Negative Rate0.9790.973-01% (U)False Positive Rate0.0210.027-29% (F)False Negative Rate0.4010.289-28% (F)White, Non-HispanicStatisticBISGZRPPct. DiffTrue Positive Rate0.7580.906+19% (F)True Negative Rate0.7580.741-02% (U)False Positive Rate0.2420.259+07% (U)False Negative Rate0.2410.094-61% (F)AuthorsKasey
Matthews(Zest AI Lead)Piotr Zak(Algomine)Austin Li(Harvard T4SG)Christien
Williams(Schmidt
Futures)Sean Kamkar(Zest AI)Jay Budzik(Zest AI)ContributingContributions are encouraged! For small bug fixes and minor
improvements, feel free to just open a PR. For larger changes, please
open an issue first so that other contributors can discuss your plan,
avoid duplicated work, and ensure it aligns with the goals of the
project. Be sure to also follow theCode of
Conduct.
Thanks!MaintainersMaintainers should additionally consult our documentation onreleasing.
Follow the steps there to push new releases to Pypi and Github releases.
With respect to Github releases, we provide new releases to ensure
relevant pipelines and look up tables requisite for package download and
use are consistently up to date.WishlistSupport for the following capabilities is planned:add multiracial classification output supportnational validation datasets and validation partnerspointers to additional training dataadd support for gender and other protected basesLicenseThe package is released under theApache-2.0
License.Results and FeedbackGenerate interesting results with the tool and want to share it or other
interesting feedback? Get in touch viaabetterway@zest.ai.