Package: united-states-congress
## unitedstates/congressPublic domain code that collects data about the bills, amendments, roll call votes, and other core data about the U.S. Congress.Includes:A data importing script for the [official bulk bill status data](https://github.com/usgpo/bill-status) from Congress, the official source of information on the life and times of legislation.Scrapers for House and Senate roll call votes.A scraper for GPO FDSys, the official repository for most legislative documents.A defunct THOMAS scraper for presidential nominations in Congress.Read about the contents and schema in the [documentation](https://github.com/unitedstates/congress/wiki) in the github project wiki.For background on how this repository came to be, see [Eric’s blog post](http://sunlightfoundation.com/blog/2013/08/20/a-modern-approach-to-open-data/).### Setting UpThis project is tested using Python 2.7.System dependenciesOn Ubuntu, you’ll needwget,pip, and some support packages:`bash sudoapt-getinstall gitpython-devlibxml2-devlibxslt1-devlibz-devpython-pip`On OS X, you’ll need developer tools installed ([XCode](https://developer.apple.com/xcode/)), andwget.`bash brew install wget `Python dependenciesIt’s recommended you use avirtualenv(virtual environment) for development. The easiest way is installvirtualenvandvirtualenvwrapper, usingsudoif necessary:`bash sudo pip install virtualenv sudo pip install virtualenvwrapper `Create a virtualenv for this project:`bash mkvirtualenv congress `And activate it before any development session using:`bash workon congress `Finally, with your virtual environment activated, install Python packages:`bash pip install-rrequirements.txt `### Collecting the dataThe general form to start the scraping process is:./run <data-type> [–force] [other options]where data-type is one of:bills(see [Bills](https://github.com/unitedstates/congress/wiki/bills)) and [Amendments](https://github.com/unitedstates/congress/wiki/amendments))votes(see [Votes](https://github.com/unitedstates/congress/wiki/votes))nominations(see [Nominations](https://github.com/unitedstates/congress/wiki/nominations))committee_meetings(see [Committee Meetings](https://github.com/unitedstates/congress/wiki/committee-meetings))fdsys(see [Bill Text](https://github.com/unitedstates/congress/wiki/bill-text))deepbills(see [Bill Text](https://github.com/unitedstates/congress/wiki/bill-text))statutes(see [Bills](https://github.com/unitedstates/congress/wiki/bills) and [Bill Text](https://github.com/unitedstates/congress/wiki/bill-text))To scrape bills, resolutions, and amendments from THOMAS, run:`bash ./run fdsys--collections=BILLSTATUS./run bills `The bills script will output bulk data into a top-leveldatadirectory, then organized by Congress number, bill type, and bill number. Two data output files will be generated for each bill: a JSON version (data.json) and an XML version (data.xml).### Common optionsDebugging messages are hidden by default. To include them, run with –log=info or –debug. To hide even warnings, run with –log=error.To get emailed with errors, copy config.yml.example to config.yml and fill in the SMTP options. The script will automatically use the details when a parsing or execution error occurs.The –force flag applies to all data types and supresses use of a cache for network-retreived resources.### Data OutputThe script will cache downloaded pages in a top-levelcachedirectory, and output bulk data in a top-leveldatadirectory.Two bulk data output files will be generated for each object: a JSON version (data.json) and an XML version (data.xml). The XML version attempts to maintain backwards compatibility with the XML bulk data that [GovTrack.us](https://www.govtrack.us) has provided for years. Add the –govtrack flag to get fully backward-compatible output using GovTrack IDs (otherwise the source IDs used for legislators is used).See the [project wiki](https://github.com/unitedstates/congress/wiki) for documentation on the output format.### ContributingPull requests with patches are awesome. Unit tests are strongly encouraged ([example tests](https://github.com/unitedstates/congress/blob/master/test/test_bill_actions.py)).The best way to file a bug is to [open a ticket](https://github.com/unitedstates/congress/issues).### Running testsTo run this project’s unit tests:`bash ./test/run `### Who’s Using This DataThe [Sunlight Foundation](http://sunlightfoundation.com) and [GovTrack.us](https://www.govtrack.us) are the two principal maintainers of this project.Both Sunlight and GovTrack operate APIs where you can get much of this data delivered over HTTP:[GovTrack.us API](https://www.govtrack.us/developers/api)[Sunlight Congress API](http://sunlightlabs.github.io/congress/)## Public domainThis project is [dedicated to the public domain](LICENSE). As spelled out in [CONTRIBUTING](CONTRIBUTING.md):> The project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the [CC0 1.0 Universal public domain dedication](http://creativecommons.org/publicdomain/zero/1.0/).> All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.[![Build Status](https://travis-ci.org/unitedstates/congress.svg?branch=master)](https://travis-ci.org/unitedstates/congress)