add bare bones utility to plot ROC curves from CSV data

Summary: add bare bones utility to plot ROC curves from CSV data

Test Plan: none

Reviewers: jascha

Reviewed By: jascha

CC: eliana

Differential Revision: http://phabricator.khanacademy.org/D2357
Query to compute total number of students and teachers   using heuristic to define teacher.

Summary:
Add student_coach_count to jobflow config

Add student_coach_count to hive init

Rename coach to teacher where appropriate

coach -> teachers rename

General idea is that we need to generate partial sums.
They way it is usually done is through cross product with selection. Though, in
hive case it would be better to write UDAF (only in java) for that.

Would be open for some suggestions about simplifying and optimizing it, however, I
think that's good for a first attempt. Especially due to the fact that it's independent
of the time range.

Test Plan:
Execute the queries with ka_hive_init.q

Check that student_coach_count.q is executed daily

check that student_coach_count is available after hive startup

None, rename

Reviewers: jace

Reviewed By: jace

Differential Revision: http://phabricator.khanacademy.org/D2309
add the abilitiy to output features to mirt_train_EM.py

Summary:
This change allows the MIRT training to emit a CSV dataset
and the end of training that contains a row for each question
answered containing the correctness, the MIRT model predicted
likelihood of correctness, and the abilities vector.

It's getting to be a swiss army knife of options, I know. This
was done quick-n-easy because I hope soon to to have
standardized, abstract interfaces for our data inputs, training,
and model validation.

Test Plan:
in analytics:/ebs/kadata/accmodel/mirt, I ran
  ./mirt_train_EM.py -a 1 -n 0 -f prod_responses -o mirt_emit -e -r mirt_new_epoch=74.npz

Reviewers: eliana

Reviewed By: eliana

CC: jascha

Differential Revision: http://phabricator.khanacademy.org/D2358
fix an apparently failed merge
Rewriting company_metrics.q for performance (and accuracy?)

Test Plan:
1. Started my own hive cluster, copy+pasted company_metrics.q into hive to execute
2. `select * company_metrics_new;` produces:

2012-04 200455  234261  110996
2012-05 157416  232637  104437
2012-06 153805  169153  83004
2012-07 126050  145249  77356
2012-08 242957  148085  81290
2012-09 519804  215446  162041
2012-10 418853  326151  193460
2012-11 247161  348870  156883
2012-12 199803  329898  132050
2013-01 299278  376685  165444
2013-02 239556  356060  138591
2013-03 240054  369366  140862
2013-04 372975  406869  153169
2013-05 167038  243144  47856

Which feels right in the scheme of our webiste, but are drastically different numbers than what is currently in `select * from company_metrics;`

I have not run the old company_metrics.q script since Jace said it would take forever to complete, so my theories on the differences in output are 1) the company_metrics table is out-of-date, 2) my query is wrong, or 3) the existing query is wrong (due to the `count(distinct user)`?)

So, I need to figure out why there is a discrepancy, but wanted to get a few more eyeballs on this in case I am doing something dumb.

Next Steps:
1. Change the dashboard code to understand the new schema (are there any other callers that would need to be changed?)
2. Run the query over all-time to populate company_metrics

Reviewers: jace

CC: stephanie

Differential Revision: http://phabricator.khanacademy.org/D2329
Fix syntax error introduced by D2329 fix report_importer syntax errors

Summary: wrong else if statement and missing parenthesis

Test Plan: report_importer.py should not have syntax errors

Reviewers: mattfaus, jace

Reviewed By: mattfaus

Differential Revision: http://phabricator.khanacademy.org/D2367
Add some additional detail as to the contents of the graph.

Test Plan:
Looked at it in Chrome.
Merge branch 'master' of github.com:Khan/analytics
HTML template for exercise summary

Summary: CSS styles for exercise summary - mostly graph colours

Test Plan:
Open dashboards.khanacademy.org and check exercise-summary dashboard

On exercise-summary console should not show any less related errors.
Style should be consistent with the screenshot on the main page.

Reviewers: jace, chris

Reviewed By: chris

Differential Revision: http://phabricator.khanacademy.org/D2230
Make the lower-bound on the graphs zero

Summary:
The daily exercises and goals dashboard charts are never going to be
negative, but highcharts automatically scales so it shows negative portions of
the graph, which is unhelpful. This makes the lower bound zero, so it looks
better.

Test Plan:
Visit /exercises and /goals, see that the graphs look right but
don't have a negative bit on the y-axis

Reviewers: alpert, mattfaus, joshnetterfield

Reviewed By: mattfaus

Differential Revision: http://phabricator.khanacademy.org/D2375
Fix naive way of taking maximum, Hive max is an aggregate function. Use IFs instead.

Summary: Fix for the failed daily emr report

Test Plan: student_teacher_count.q should not fail

Reviewers: jace

Reviewed By: jace

Differential Revision: http://phabricator.khanacademy.org/D2368
update copy on dashboard index.html, remove reference to *Q3* company goals

Summary: WYSIWYG

Test Plan: none

Reviewers: mattfaus

Reviewed By: mattfaus

Differential Revision: http://phabricator.khanacademy.org/D2384
Copying metric goals from https://docs.google.com/a/khanacademy.org/document/d/12sxBJ8huMCezMRFtuLPDb8J24-G9L3taoZkJ65DVJk4/edit# on to the page

Test Plan:
Started the debug server, looked at the page in Chrome
Merge branch 'master' of github.com:Khan/analytics
Added a script to find the log lines and app logs of an issue

Test Plan:
Tried it out with a few different issues.

Auditors: jace,chris
add bingo_id to userdata_info
code written for response time learning, haven't run/debugged it yet.
move parameters class to util
runs without crashing.  parameters diverge.
Fixed gradient, and matrix shape bug for sign flipping for num abilities > 1