OHI-Science/ohiprep — pull request #7 by bbest (2014-03-08 01:19:11+00:00)
Title: adding Whence files and examples

Hi guys,

Happy Friday!

I have made some progress on the [whence gapfilling business](https://github.com/OHI-Science/ohiprep/wiki/Whence) gapfilling business (using dplyr!) and want to see what you think. They are on my jules32/ohiprep repository; let's take for example the World Bank data.

As a reminder, spatial gapfilling is using the UN georegional groupings, with are r2 and r1 (we never used r0=world average; if it ever comes to that I think we would want to find a better solution). When we gapfill with georegional averages, the process is:
1. by r2 and r1 grouping calculate averages using rgn_ids that do have data (this is done by year),
1. identify which rgn_ids are missing (= need to be gapfilled),
1. insert georegional averages, by year, into necessary rgn_ids to be gapfilled, first using r2; if not possible, then r1

**Datafiles with new field: whencev01.** --let's look at [this file](https://github.com/jules32/ohiprep/blob/master/Global/WorldBank-Statistics_v2012/data/rgn_wb_gdp_2014a.csv) as an example.
* This is a start. The two labels in these files are either SG=spatial gapfilling and OG=original data. I'm happy to change OG to something better, but couldn't think of something better for now.
* I'm not sure how to elegantly add more info (eg all of these are SG, but I'd like to add info about their Southern Island status too).

**New datafiles: .*_whencev01.csv** --let's look at [this file](https://github.com/jules32/ohiprep/blob/master/Global/WorldBank-Statistics_v2012/data/rgn_wb_gdp_2014a_whencev01.csv) as an example.

The .*_whencev01.csv files are tricky, and I'm trying to wrap my head around what will be the most useful data, and how to present it. It would be awesome to hear your feedback. What would be the most useful (maybe several files?)?
* list of countries, years, and data values that were used in gapfilling. This file would have columns: rgn_id, rgn_nam, year, value, r2. This is what I have currently: it is a beast, but all the information would be there, and we could have a simple script on hand to search for whatever r2 and year to see which countries were involved, and what those values were. This still doesn't link at all to the actual rgn_ids that were gapfilled using these data--could add that as another column? But there would be way too much redundancy. See # 2.
* short list of countries that have data and were used in gapfilling. So this file would have columns: rgn_id, rgn_nam, r2. I think the value here is that you could have a list of the rgn_ids that were gapfilled using these data without it being too unmanageably huge. However, rgn_ids were only used in calculating the georegional average if they had data that year, so this could be misleading.
Also, about the naming of these files: it is hard to read now because of the intermingling between the \_whencev01 and the non ones. See what you think: Would it be better to instead use a prefix instead of a suffix, for example whencev01_rgn_wb_gdp_2014a.csv?

I hope this makes any sense at all in light of what we talked about--I will get back into this next week, maybe it's worth another discussion before I move along, but at least this gives us something to talk about!

Thanks guys,
Julie