Speed up the sampling profiler

Summary:
The sampling profiler is designed to grab 250 samples per second, but it wasn’t
going nearly that fast, and the sampling times weren’t continuous (it would
sometimes grab many samples in succession, then wait hundreds of milliseconds
before the next sample). This slowness was due to time spent in
traceback.extract_stack, which I believe had highly variable performance because
of its use of the linecache. The fix (inspired by
https://github.com/bdarnell/plop ) was to instead grab the stack information
directly. This means that we no longer have the text of the lines at each stack
frame, but that's probably ok (it only showed up in the tooltip anyway, and it's
not provided by the instrumented profiler).

With this change, the number of samples that we get on the homepage in
dev_appserver increased from about 70 to about 300. With the “time.sleep” line
removed, the new sampler is about 100x faster rather than 4x.

Test Plan:
Load the logged-in homepage in dev_appserver with sampling enabled. Make sure
the render_learning_dashboard frames add up to something close to 100%.

Reviewers: ben, chris

Reviewed By: ben

CC: alpert

Differential Revision: http://phabricator.khanacademy.org/D6646
Improve sleeping strategy in the sampling profiler

Summary:
Previously, we were trying to achieve 250 samples per second by sleeping for 4ms
between samples, which ended up getting only about 100 samples per second, and
gave unfair weight to times where we were able to sample faster. We now figure
out the time that the next sample should theoretically take place and sleep
until then, which ensures that we get the appropriate number of samples. They're
still not distributed perfectly evenly, with occasional pauses as high as
50-100ms in extreme cases, but this should at least improve things.

Test Plan:
Run dev_appserver and load the logged in homepage with sampling enabled. Verify
that the number of samples is approximately 250 times the number of seconds
taken in the request, and that the results seem to represent the actual time
spent in the different sections of the logged in homepage.

Reviewers: ben, chris

Reviewed By: ben

Differential Revision: http://phabricator.khanacademy.org/D6647
Change the sampling profiler to display full stack traces

Summary:
The previous visualization for the sampling profiler (all frames in all
collected stack traces, sorted by frequency), wasn't very useful, except
possibly in some cases where the user knows the code very well and can spot an
unusual frame frequency.

We now send over the full stack trace for each sample and provide a way of
visualizing this information. This is useful, for example, to see what was going
on at a time when Appstats shows no RPCs active. Currently, the user picks the
sample number to view by typing it into a text box, but the plan is to change
that to a slider soon.

To avoid sending over a giant response, we compress the stack traces by keeping
a table of frame description strings and by representing a stack trace as a list
of indexes into that table.

Test Plan:
-Load the logged in home page with sampling and Appstats enabled.
-Try various valid sample numbers and make sure the changes are snappy and the
stack traces themselves look reasonable.
-Cross reference with Appstats by typing in the sample number corresponding to
an RPCless time (assuming 250 samples per second) and making sure that it's
doing what I thought it was doing.
-Try out-of-range integers, non-integers and make sure they clear the table
without any errors showing up in the JS console.

Reviewers: ben, chris

Reviewed By: ben

Differential Revision: http://phabricator.khanacademy.org/D6648
Add timestamps to the sampling profiler

Summary:
Each sample now has a timestamp associated with it, which we display along with
the stack trace. Ideally, these would be spaced exactly 4ms apart, but there's
some variance, so I provide tenth-of-millisecond precision. This should make it
easier to cross-reference sampling results with Appstats.

Test Plan:
-Load the logged in homepage with sampling on.
-Make sure the timestamp shows up when the stack trace does and that the number
is reasonable.
-Type in some invalid results and make sure the timestamp disappears whenever
the stack trace does.

Reviewers: ben, chris

Reviewed By: ben

Differential Revision: http://phabricator.khanacademy.org/D6649
Don't lose sort order in the sampling profile viewer

Summary:
Previously, the tableSorter wouldn't re-sort itself when the contents were
updated, and I ended up needing to do some hacks to make it work. The intention
here was to make it easy to watch the stack trace change over time as a way of
making it visually obvious how much time is spent in each function, but
unfortunately, that doesn't quite work yet, since there's enough framework code
at the top of the stack that the important frames are out of view.

I also split up updateSampleNumber into two functions, since it got too long.

Test Plan:
Go to the logged-in homepage in dev_appserver and make sure that the sampling
profiler results look reasonable. Sort the depth in reverse order and make sure
that the results remain sorted in reverse order as I change the sample number,
and make sure that the table doesn't flicker too much.

Reviewers: ben, chris

Reviewed By: ben

Differential Revision: http://phabricator.khanacademy.org/D6650
Change the sampling profile view to use a slider

Summary:
Instead of having to type in a sample number, you can now drag the slider to the
right place in time.

Also change the stack trace table to not change in size when dragging the slider around.

Test Plan:
Load the logged-in homepage and try out the slider. Make sure the results are
pretty much the same as with the text input and that the rendering is
responsive. Load the logged-out homepage and make sure that the slider shows up
there too and that it gives reasonable results.

Reviewers: chris

Reviewed By: chris

CC: alpert

Differential Revision: http://phabricator.khanacademy.org/D6651
Allow filtering of frames and add help message to the sampling profiler

Summary:
Add a new "Hide frames" slider that hides some number of top-level frames,
which is useful when viewing the frames in ascending order of depth to watch
the stack change over time (since without this feature, the view would be
filled with stack frames that are the same for the entire request). I
considered trying to automatically filter out these frames, but the obvious
approach would only filter out half the frames that really should be filtered
out, so I think it makes more sense to make it user-specifiable.

Also add a "how to use" message that gives some intended use cases.

Test Plan:
On both the logged in and logged out homepage:
-Load the page and make sure the profile works like it did before.
-Drag the "hide frames" slider to various values and make sure that no errors
appear and that the stack trace changes in a smooth way when viewed in
ascending order of depth.

Reviewers: ben, chris

Reviewed By: chris

CC: marcos

Differential Revision: http://phabricator.khanacademy.org/D6654