Initial commit

Initial commit of py and sql scripts used to explore SDI data.
Remove files ending in ~

Remove files with filenames ending in ~ which were added before the .gitignore file was created
Calculate odds ratios for peak week +/- 6 wks

OR_6wk.sql: pull incidence data by season and incidence and popstat data by zip3 by week
ORgenerator_v070813.py: new script, update from v060713 to import data into dictionaries instead of lists; completed import_d and ORgen_seas functions
odds_c_a_6wk_v7-8-13.py: new script, draw OR by season charts with the peak week +/- 6 wks definition; completed simple OR by season chart
peak season OR charts by season and by week, zip-fips

Plot peak OR by week charts
Began cleaning OR_swk6_zip3.csv in preparation for choropleth by season charts

:wq
choropleth- OR by season - zip3 to fips conversion
1) converted zip3 spatial denomination to fips code denomination by choosing the most populous fips code associated with the zip3 as the "representative" fips code
2) since some zip3s had the same representative fips code, the ILI and popstat data for those zip3s needed to be aggregated when converting the spatial denomination
3) a Py dict was created where dict(seasonnum, fips) = OR for children to adults
zip3-fips conversion and OR calculations were done in python
4) csvfile output: fips_OR_season%d.csv
5) added previously exported files from R for SDI data and urban-rural data, mysql import
lat/long maps: OR and incidence by season

zip3_ll.txt: sourcefile = Coord3digits.csv, zip3-latlong crosswalk
zip3_OR_season.txt: OR by season map, sourcefile = R_export/zip3_ILI_season.csv, zip3-OR values by season, includes the 545 zip3s that had data for all 7 age groups (5-59 yo) for all 10 seasons.Or represents OR of attack rate of C:A during peak week of incidence +/- 6 weeks for each season. Different weeks are represented each season.
zip3_incid_season.txt: Incidence by season map, sourcefile = SQL_export/choropleth_seasonincid_v7-1-13.csv, zip3-attack rate per 1000, includes 843 zip3s that had non-zero popstat data for all 10 seasons. Seasonal incidence represents the period from Oct to Mar each year
map-zipcodes_ECLedit.R: edit of R code from Shweta to draw US lat/long maps, includes code for all maps. 1) OR for peakweeks by season, 2) incidence from Oct to Mar by season, 3) incidence for all weeks and seasons

The maps have some known flaws:
1) There may be a problem with the order of the labels (weekly incidence maps are missing the multitude of 0 attack rates but have a large number of very high attack rates it seems)
2) Legend colors are not consistent at each time step. It depends on what data is present in that week/season's time step. This means that the same color may represent different ORs or incidences in different time steps, which makes comparison difficult.
Proxy for seas severity: ILI in acute care in pwks

Explore SDI data in mysql to see what set of data would serve as a good seasonal proxy for a flu season's severity. Examined the following datasets:
1) ILI attack rate per 10,000 over entire population during peak weeks
2) ILI attack rate per 10,000 over entire population during Oct-May flu season
3) ILI attack rate per 100,000 in acute care or inpatient facilities during peak weeks
4) ILI attack rate per 100,000 in ER during peak weeks
5) ILI attack rate per 100,000 in acute care or inpatient facilities during Oct-May flu season

All attack rates were compared with qualitative descriptions provided on CDC's website about different flu seasons. It was found that:
1) 2006-07: mild compared to the previous three flu seasons, activity increased in late Dec, peaked in mid-February and decreased through May
2) 2007-08: severe relative to the previous three seasons, similar in severity to the 2004-05 season, activity increased in January and peaked in mid-Feb and decreased through May
3) 2009-10: relatively severe for children and adults, peaked at end of October and decreased through January
4) 2002-03: relatively mild depending on the region, activity peaked in the first and second weeks of February.
5) 2003-04: moderately severe, began earlier than most seasons and peaks were in early to mid December.
6) 2004-05: moderate activity, activity increase in January and peaked in mid-February.
7) 2005:06: peak activity was less intense but activity was elevated for a longer period of time than compared to the previous three season. Activity increased during Feb and peaked in early Mar.

ILI attack rate per 100,000 in acute care or inpatient facilities during peak weeks best matched this information (option 4). It was exported as seasonseverity.csv.
fix legend on lat/long maps: OR, incid by seas&wk

for all three maps:
changed legend colors to "bin" variable because the "legend" variable was misrepresenting the OR/attack rate bin factors
removed decreasing=TRUE in label sorting because it caused the labels to applied incorrectly
reversed the factor levels so blue would refer to lower OR and incidence values and red would be for high values
removed the "legend" variable from the dataframes because it was no longer needed for plotting the color scale
fixed labels dataset to "mergethree" because the other dataset would allow missing values to appear
show all OR/incidence factors, will not drop unused factors, which will keep the legends the same regardless of the presence of a certain factor on the map

To do:
take log(OR)

Notes:
The incidence attack rates seem unusually low when the week and season ILI & popstat values are summed and percentages are calculated. Look at the original mysql data to see if attack rates fall below 1% as they do here. This needs to be fixed
ID severity proxy and create fxn for OR_sev chart

added import_gen_d to ORgenerator to import seasonseverity.csv into OR_seasonseverity.py
untrack all csv and txt data files
remove pyc and R history files from tracking
map formatting, log(OR) maps, incidbywk greyed bg

changed map labels for OR by season and incidence by season
created log(OR) by season maps
changed bins for incidence by week maps
changed color scheme for incidence by week maps - all 0 values are light grey in order to maintain the shape of the US map but allow zip3s with flu activity to stand out
four completed map outputs: OR by season, incidence by season, log(OR) by season, incidence by week
saved all four sets of map outputs to /mapping_code/cleanedmapdata/mapoutputs/
add gitignore file to repo
added *.Rhistory to gitignore file
plot OR vs attack rate in acute care facilities

attack rate in acute care and inpatient facilities is a proxy for season severity
hypothesis: as severity increases, OR decreases
added *.csv and *.zip to gitignore
first commit of age dynamics paper

paper is just an outline at this point
the simulations sections are not well developed since the simulations have not yet been performed
open in LyX
corrected prominent subtype in subtype table

Discrepancies were found between prominent subtypes reported in CDC Flu Summaries and a report discussing the estimated mortality associated with seasonal flu from 1976-2007. The updated subtype and subtype columns represent the prominent flu subtypes as based on number of isolates reported at each end of flu season summary. Prominent subtypes are defined as at least 20% of typed isolates in a season.
corrected prominent subtype in db and charts

One CDC source that listed prominent subtype did not match the prominent subtypes identified by calculating the percentage of isolates in each season from the end of season CDC flu summaries. Both methods defined prominent subtype as 20% or greater isolates in a flu season. We changed all of the charts and SDI db tables to reflect the flu season summary calculations.
plot total incidence by week for each season

Incidence per 100,000 for total population
Denominator for incidence is US population for 2nd calendar year in flu season (ie. 2001-02 uses US population size in 2002)