Package: elex-model
elex-live-modelThe Washington Post uses this live election model to generate estimates of the number of outstanding votes on an election night based on the current results of the race. It is agnostic to the quantity that it is estimating. For general elections, we also generate estimates of the partisan split of outstanding votes, and for primaries, we split estimates by candidate.Generally, the model works by comparing the current results to a historical baseline and regressing on the difference using demographic features. We usequantile regressionas the underlying model andconformal predictionto produce our uncertainty estimates.The first iteration of this model is written in R inthis repo.How to Run the ModelFor dev work, skip this section and go on to theDevelopmentsection below.InstallationWe recommend that you set up a virtualenv and activate it (IEmkvirtualenv elex-modelviahttp://virtualenvwrapper.readthedocs.io/en/latest/).Runpip install elex-modelIf you get stuck with an error aboutqdldl(e.g.RuntimeError: CMake must be installed to build qdldl), you are missingcmake.  You can install it with Homebrew withbrew install cmakeorcompile it from source.UsageWe can run the model with a CLI or with Python.We can use the model to generate current estimates or for a historical evaluation. Historical evaluation means running the "current reporting" subunits with data from a previous election, and then calculating the error that the current set of reporting subunits would have given us. This allows to test how representative the currently reporting subunits are.See more information on how to pass data to the model in thedata README.CLIThe CLI is for local development and testing purposes only. We cannot run a live election through the CLI because it pulls vote counts from data files located either in S3 or locally. It does not retrieve current data from the Dynamo database of election results.The CLI takes an election ID, estimands, office ID, and a geographic unit type. If you're running the model with local data files, they should be located atelex-live-model/data/{election_id}/{office_id}/data_{geographic_unit_type}.csv.  Otherwise, the model will attempt to find the data files in S3.Pass in a command like this:elexmodel 2017-11-07_VA_G --estimands=dem --office_id=G --geographic_unit_type=countyYou can also pass in multiple estimands:elexmodel 2017-11-07_VA_G --estimands=dem --estimands=turnout --office_id=G --geographic_unit_type=county --percent_reporting 40If you want to run a test with some nonreporting subunits, you can use the--percent_reportingcli parameter:elexmodel 2017-11-07_VA_G --estimands=dem --office_id=G --geographic_unit_type=county --percent_reporting 40Historical electionIf you want to run a historical election, you can use the--historicalflag. For this to succeed, the election must have historical data already prepared.elexmodel 2021-11-02_VA_G --estimands=dem --office_id=G --geographic_unit_type=county --percent_reporting 60 --historicalParametersParameters for the CLI tool:NameTypeAcceptable valueselection_idstringYYYY-MM-DD_{geography}_{election_type}geography is the state orUSAand election type isGfor general or'P'for primaryestimandslistparty name (i.e.dem,gop) or turnout in a general;{candidate_last_name}_{polID}in a primaryoffice_idstringPresidential (P), Senate (S), House (H), Governor (G), state Senate (Z), state House (Y)geographic_unit_typestringcounty,precinct,county-district, orprecinct-districtpercent_reportingnumeric0-100historicalflagfeatureslistfeatures to include in the modelfixed_effectsdictspecified as:{"<fixed_effect_column>: [which fixed effects to include]}; to include all effects for a column, use["all"]; possible values for the fixed effect variable include:postal_code,county_classificationorcounty_fips, but really any prepared categorical variableaggregateslistlist of geographies for which to calculate predictions beyond the originalpostal_code,county_fips,district,county_classificationpi_methodstringmethod for constructing prediction intervals (nonparametricorgaussian)model_parametersdictdictionary of model specific parameters e.g.--model_parameters='{"robust":True}'save_outputlistresults,data,configunexpected_unitsintnumber of unexpected units to simulate; only used for testing and does not work with historical runNote: When running the model with multiple fixed effects, make sure they are not linearly dependent. For example,county_fipsandcounty_classificationare linearly dependent when run together. That's because every county is in one county class, so all the fixed effect columns of the counties in the county class sum up to the fixed effect column of that county class.Custom EstimandsIt's possible to create a custom estimand based on other data elements.  Here's how to create a new estimand called "my_estimand":Insrc/elexmodel/handlers/data/Estimandizer.py, create a function with the signaturedef my_estimand(data_df).Inmy_estimand(), use the columns indata_dfto create a new column, eitherbaseline_my_estimandorresults_my_estimandas necessary.  See theparty_vote_share_demfunction for an example.Specifymy_estimandas one of your estimands.  For example, via the command line:elexmodel 2017-11-07_VA_G --estimands my_estimand --office_id=G --geographic_unit_type=county --percent_reporting 50Your output should have columns includingbaseline_my_estimand,results_my_estimand, and related columns for the prediction intervals, if using them.Here's an example showing multiple estimands, includingmy_estimand:elexmodel 2017-11-07_VA_G --estimands=turnout --estimands my_estimand --estimands party_vote_share_dem --office_id=G --geographic_unit_type=county --percent_reporting 50PythonModel ParametersSome model types have specific model parameters that can be included.NameTypeAcceptable valuesmodellambdanumericregularization constantallturnout_factor_lowernumericdrop units with < turnout factorallturnout_factor_uppernumericdrop units with < turnout factorallrobustbooleanlarger prediction intervalsnonparametricbetanumericvariance inflationgaussianwinsorizebooleanwinsorize std estimategaussianBnumericbootstrap samplesbootstrapTnumerictemperature for aggregatebootstrapstratalistgroups to stratify bootstrap bybootstrapagg_model_hard_thresholdboolhard threshold aggregate modelbootstrapy_LBnumericlower bound norm. margin distbootstrapy_UBnumericupper bound norm. margin distbootstrapz_LBnumericlower bound turnout fact distbootstrapz_UBnumericlower bound turnout fact distbootstrapy_unobserved_lower_boundnumericlower bound for norm. marginbootstrapy_unobserved_upper_boundnumericupper bound for norm. marginbootstrappercent_expected_vote_error_boundnumericerror tolerance on expected votebootstrapz_unobserved_lower_boundnumericlower bound for turnout factorbootstrapz_unobserved_upper_boundnumericupper bound for turnout factorbootstrapThis is the class and function that invokes the general function to generate estimates. You can installelex-modelas a Python package and use this code snippet in other projects.from elexmodel.client import ModelClient

model_client = ModelClient()
model_response = model_client.get_estimates(
  current_results,
  election_id,
  office,
  estimand, 
  prediction_intervals,
  percent_reporting_threshold,
  geographic_unit_type,
)Historical electionThis is the class and function that invokes a historical evaluation. You can installelex-modelas a Python package and use this code snippet in other projects.from elexmodel.client import HistoricalModelClient

historical_model_client = HistoricalModelClient()
model_evaluation = historical_model_client.get_historical_evaluation(
  current_data,
  election_id,
  office,
  estimand,
  prediction_intervals,
  percent_reporting_threshold,
  geographic_unit_type
)DevelopmentWe welcome contributions to this repo. Please open a Github issue for any issues or comments you have.InstallationClone the repository and install the requirements:pip install -r requirements.txt
  pip install -r requirements-dev.txtCreate a .env file in the top directory and add the below variables. Assuming your S3 bucket and path roots are namedelex-models, set these as your variables:APP_ENV=local
  DATA_ENV=dev
  MODEL_S3_BUCKET=elex-models
  MODEL_S3_PATH_ROOT=elex-modelsTestingpip install -r requirements-dev.txttoxWe also have arequirements-test.txtfile which is used for running unit tests only. It is installed automatically as part of installingrequirements-dev.txt.PrecommitTo run precommit hooks for linting, run:pre-commit run --all-filesReleaseTo release a new version manually:Decide what the next version will be per semantic versioning:X.X.XMake a new branch from develop calledrelease/X.X.XUpdate the version insetup.pyUpdate the changelog with all the chnages that will be included in the releaseCommit your updates and open a PR against mainOnce the PR is merged, tag main (or develop for a beta release) with the version's release number (git tag X.X.X) and push that tag to github (git push --tags)Merge main into developThen, we need to release this version to PyPi. This repository has a Github Action workflow that automatically builds and releases the latest version to TestPyPi and PyPi on pushes tomain. However, to release to PyPi manually:Generate a distribution archive:Make surerequirements-dev.txtis installedRunpython3 -m pip install --upgrade buildto installbuildRunpython3 -m build. This should generate two files in thedist/directory.Check to make sure the correct version is installed in thedist/folder that should now exist at the base of the repo folder. If you've previously run these commands locally for an earlier version, you may need to delete the older files indist/order to upload them correctly in the next step. You can just delete the entiredist/folder and run the above command again.Upload the distribution archive:`Runpython3 -m pip install --upgrade twineUpload to TestPyPi withpython3 -m twine upload --repository testpypi dist/*Upload to PyPipython3 -m twine upload dist/*Further ReadingWe have published multiple resources to share our progress.October 2020:"How The Washington Post Estimates Outstanding Votes for the 2020 Presidential Election"November 1, 2020:"What the Post’s election results will look like"November 2020:Github repository for the original election night model used in the 2020 electionsDecember 2020:"An Update To The Washington Post Election Night Model"2020 General election night model open sourced repositoryFebruary 21, 2021:"How The Washington Post Estimates Outstanding Votes for the 2020 Presidential Election"November 2, 2021:"How The Washington Post will model possible outcomes in the Virginia governor’s race"May 17, 2022:"How the Washington Post’s election night model works"September 14, 2022:"We're open sourcing our live election night model"