Package: tx_highered
The Texas Higher Education Data Project---------------------------------------[![Build Status](https://travis-ci.org/texastribune/the-dp.svg)](https://travis-ci.org/texastribune/the-dp)## A very rough guide to starting development### Example `.env` file for environment variables:```DJANGO_SETTINGS_MODULE=exampleproject.settings.devDATABASE_URL=postgis:///tx_highered```Complete guide to getting started (remove steps to suit you):```bash# install postgresql libpq-devgit clone $REPOSITORY && cd $PATHmkvirtualenv tx_higher_edsetvirtualenvprojectadd2virtualenv .pip install -r requirements.txt# if you need to create a database:# `postdoc` greatly simplifies connecting to Docker databasespip install postdocphd createdb --encoding=UTF8 -T template0echo "CREATE EXTENSION postgis;" | phd psqlecho "CREATE EXTENSION postgis_topology;" | phd psql# or if you need to reset your database:make resetdb# syncdb and load fixturesmake syncdb######################################################################## You can stop at this point if you're just playing with the project. ######################################################################### if using 2012 data, bump it up to 2014 standardspython tx_highered/scripts/2014_update.py# get ipeds data, requires https://github.com/texastribune/ipeds_reporter../ipeds_reporter/csv_downloader/csv_downloader.py \--uid data/ipeds/ipeds_institutions.uid --mvl data/ipedsmv ~/Downloads/Data_*.csv data/ipeds# get thecb datacd data && make all# load data#   timing: 10m25.069smake load# post-process the datapython exampleproject/manage.py tx_highered_process##################################### placeholder for post-2014 update ###################################### the 2012->2014 specific stuff can go out and the above importing# instructions can get updated```### DatabaseThis project currently requires a PostGIS database (hopefully not for long):```bash$ phd createdb$ phd psqlCREATE EXTENSION postgis;CREATE EXTENSION postgis_topology;```#### Moving data between databasesYou can do a sql dump to move data from one postgres database to another(excluding geo info):```bash$ phd SOURCE_DATABASE_URL pg_dump --no-owner --no-acl --table=tx_highered* --clean > tx_highered.sql$ phd DEST_DATABASE_URL psql -f tx_highered.sql```#### After deploy1. Freeze the current data in a fixture1. Edit the tx_highered_YYYY.json.gz make task2. Run the task to save the data2. Adjust the loading scripts to reference the new fixture3. Deprecate (or delete) any one-time data migration scripts, e.g.2014_update.py won't be necessary after 2015Getting Data from the IPEDS Data Center-----------------When it asks you for an Institution, enter a list of UnitIDs generated by:list(Institution.objects.filter(ipeds_id__isnull=False).values_list('ipeds_id', flat=True))Getting Data from the Texas Higher Education Coordinating Board------------------If you want to regrab data from THECB's web site, first find the data file that you want to re-grab.It will be named something like "top_10_percent.html". There will also be a file called "top_10_percent.POST". From that file you can recreate the report with the command:curl -X POST -d @top_10_percent.POST http://www.txhighereddata.org/interactive/accountability/InteractiveGenerate.cfm -s -v > blahblahblah.htmlIf you need to modify the report, you can reverse engineer it from the POST data and the form markup.(c) 2012 The Texas Tribune