Package: iggyenrich
iggy-enrich-pythonIggymakes it easy for data scientists and machine learning engineers to include location-specific features in their models and analyses.This package helps Iggy data users to enrich the points (i.e. latitude/longitude pairs), census boundaries, and zipcodes in their data with Iggy features using Python.Getting startedGet Iggy data.Please request accesshereand we'll immediately send you a link to some sample data to play around with.What you'll receive is a sample data file in.tar.gzformat. Un-compress it in a location of your choosing, for example:tar-xzvfiggy-package-wkt-<iggy_version_id>_<iggy_prefix>.tar.gzInstall this library and dependencies.Install via pip:pipinstalliggyenrichEnrich some data!This repo contains a (very small) sample csv file with the locations of twenty four 7-11 stores in Pinellas County, FL. It haslatitudeandlongitudecolumns specifying the location of each store and a few additional attributes. The easiest way to enrich a file like this (withallthe available Iggy features) is by running:python-miggyenrich.iggy_enrich-f./sample_data/pinellas_711s.csv--iggy_base_loc<iggy_base_loc>--iggy_version_id<iggy_version_id>--latitude_collatitude--longitude_collongitude...where<iggy_base_loc>is the local directory or S3 bucket in which you have your un-compressed Iggy data, and<iggy_version_id>is the version of Iggy data you have (something like"20211110214810").After a few seconds you'll find an "enriched" version of the file insample_data/enriched_pinellas_711s.csvcontaining its original 24 data rows, but the number of columns has exploded from the original 7 to 2,808. These extra ~2,800 columns contain Iggy features.ExamplesIf you want to useIggyEnrichwithin your own code, here are a few things you can do:Create a Local Iggy Data PackageThis repo assumes that you have your Iggy data saved locally or on s3, and want to load it into memory to do your enrichment.To do this, you'll first want to create an instance of aLocalIggyDataPackageobject, which loads the data from disk or s3:fromiggyenrich.iggy_data_packageimportLocalIggyDataPackagepkg_spec={"iggy_version_id":"{your iggy version id}","crosswalk_prefix":"{your iggy crosswalk prefix}","base_loc":"{your iggy data base location on disk or s3 bucket}","iggy_prefix":"{your data's iggy prefix}"}pkg=LocalIggyDataPackage(**pkg_spec)You'll notice that thepkg_specincludes a couple parameters you need to specify. These depend on the Iggy data package you've downloaded or put in an s3 bucket. If you look at one of these packages, you'll see that it has a parent directory like:/<base_loc>/iggy-package-wkt-<iggy_version_id>_<iggy_prefix>/e.g./Users/iggy/data/iggy-package-wkt-20211110214810_fl_pinellas_quadkeys, in which casebase_loc="/Users/iggy/data"andiggy_version_id="20211110214810"andiggy_prefix="fl_pinellas_quadkeys", ors3://iggy-bucket/data/iggy-package-wkt-20211110214810_fl_pinellas_quadkeys, in which casebase_loc="s3://iggy-bucket/data"andiggy_version_id="20211110214810"andiggy_prefix="fl_pinellas_quadkeys", orWithin that parent directory will be one or more crosswalk files with a name like:/<crosswalk_prefix>_<iggy_version_id>/000000000000.snappy.parquetYou can specifyiggy_version_id,crosswalk_prefix,base_loc, andiggy_prefixbased on these naming conventions.Now, once your package is set up, you can bundle it withIggyEnrichand load the data:fromiggyenrich.iggy_enrichimportIggyEnrichiggy=IggyEnrich(iggy_package=pkg)iggy.load()Choose specific boundaries+featuresWhat if I don't want to enrich my data withallof Iggy's features, but rather want to select a few specific boundaries or features? (See ourData READMEandData Dictionary) for what's available.)You can narrow things down by specifing what you want when callingload():selected_features=["area_sqkm_qk_isochrone_walk_10m","population_qk_isochrone_walk_10m","poi_count_per_capita_qk_isochrone_walk_10m","poi_count_qk_isochrone_walk_10m",]selected_boundaries=["cbg"]iggy.load(boundaries=selected_boundaries,features=selected_features)This will loadallfeatures corresponding to thecbgboundary, plus the four selected features corresponding to theisochrone_walk_10mboundary.Enrich a DataFrame with columns for lat/lngLet's assume you have a pandasDataFramecontaining columns with latitude and longitude. You can enrich it with your chosen Iggy features:importpandasaspddf=pd.read_csv('sample_data/pinellas_711s.csv')enriched_df=iggy.enrich_df(df,latitude_col="latitude",longitude_col="longitude")Enrich a GeoDataFrameIf you prefer working in GeoPandas, theenrich_dffunction can take aGeoDataFrametoo:importgeopandasasgpdgdf=gpd.GeoDataFrame(df,geometry=gpd.points_from_xy(df.longitude,df.latitude),crs="WGS84")enriched_gdf=iggy.enrich_df(gdf)Enrich a DataFrame with census boundaries or zipcodesWhat if you don't have point coordinates in your data, but you do have zip codes, census block groups, census tracts, counties, or census-based statistical areas?You can enrich based on these columns as follows (using zip codes as an example):importpandasaspddf=pd.read_csv('sample_data/pinellas_711s.csv')enriched_df=iggy.enrich_df(df,zipcode_col="zip")These are the specific boundary types and formats supported byIggyEnrich:boundary typeenrich_dfparameterformatexampleCensus Block Groupcensus_block_group_col12-digit GEOID121030269131Census Tractcensus_tract_col11-digit GEOID12103026913Zip Codezipcode_col5-digit zip33763Countycounty_col5-digit county FIPS code12103Census-Based Statistical Areametro_col5-digit CBSA GEOID45300More resourcesYou can find our Data READMEhereand our Data Dictionaryhere.DevRunning testsTests are located intestsdirectory and can be run as:pipenvrunpythontests/test_iggy_enrich.py
pipenvrunpythontests/test_local_iggy_data_package.pyContact usFor questions or issues with using this code, pleaseadd a New Issueand we'll respond as quickly as possible.To get access to Iggy sample data please contact ushere!If you cannot find an answer to a question in here or at any of those links, please do not hesitate to reach out to Iggy Support (support@askiggy.com).