Compute optimal L.

Modified filter_data.py to compute the optimal history length L based
on a tuning set. It then outputs the performance on the tuning set, as
well as the causal state model so-generated.
Consolidated filtering code.

I wrapped the code that performs filtering on a timeseries, given a
CSM, into a single function. Since this collection of code gets called
a lot, and in a lot of different places, this just made sense.
Fair comparison between Ls.

Before, I had the different CSMs reconstructed using different Ls
running on the same timeseries, but different substrings of it (the
longer the L, the shorter the substring). This gave the larger L CSMs
an artificial advantage, since they were predicting on shorter
substrings and thus had *less* chances to make mistakes. Now, all of
the reconstructed CSMs predict on the *same* length during tuning.
Thus, we're making sure we compare apples to apples with the tuning
performance.
Added additional performance metrics.

I've added precision, recall, and F metrics. I'm not 100% convinced
I've done this correctly. But it's a start.

Also modified it to extract the userids for the top K users, so that
it's easy to run CSMs on these instead of doing it by user.
Get top K tweeters.

Modified code to make it simple to train/tune/test on the top K most
frequent tweeters.
Jerry-rigged filtering code.

After realizing that CSSR doesn't *all* possible histories that might
belong to a state, I jerry-rigged the code so that it

(1) synchronizes by using a lookup of histories and then

(2) continues ahead with the filtering by using the topology of the
reconstructed epsilon machine, until a disallowed transition occurs, at
which point we go back to (1).

This isn't the best possible solution, but it will have to do for now.

The completed fix is in filter_data_fixing1.py, which, on all future
commits (not including this one), will go back to being called
filter_data.py.
Recover name of filter_data.py

filter_data.py is now the file with the jerry-rigged filtering solution.
Incorporate Zero-order CSM.

This is a baseline test. Basically, we predict as if the process is a
biased coin. So if the person mostly tweets, we always tweet, and vice
versa. This is like using the sample mean of an output as the predictor
(since, barring any other information, this *is* the optimal predictor).
Started playing around with Tweepy.

The goal is to pull down friends of a particular user, so that I can do
*network*-based prediction.
Unknown changes.

I'm not really sure what I changed here.
Move all timeseries files to the folder timeseries.

Moved the timeseries files and their associated dot files into the
folder timeseries.
Fix don't-predict-on-'' bug in filtering code.

'Fixed' this by forcing the filter to predict the most often emitted
symbol. That is, if we see a symbol sequence we haven't seen before,
we'll predict the symbol that appeared most frequently in the training
set.
Save filtering results.

Now save the results of filtering (things like the optimal number of
states, etc.) into a tab-delimited file called filtering_results.tsv.
Modify plotting options in extract_timeseries_by_day.py.

Modified to prevent automatic plotting when extracting timeseries,
which really clogged up computer resources.
Moved methods from extract_timeseries_by_day.py to extract_timeseries_methods.py

This seems more maintainable, I think.
Created script to coarsen timeseries. Modified traintunetest.py to be just a methods file.

coarsen_timeseries.py takes in an uncoarsened (1s resolution)
timeseries and outputs a timeseries with any resolution less than 1s.

I gutted traintunetest.py so that it only contains the method for
splitting the data files. I then incorporated this method into
filter_data_batch.py so that I do the train-tune-test partition *as I
filter the data*.
Now include days even when a user *didn't* Tweet.

Modified the extraction script to include those days when a user didn't
Tweet. Before, I would skip over those days, but that doesn't seem fair.
Fixed appending.

The appending of [None] to the ts_by_day list wasn't working propertly.
Changed script to sort timepoints for each user before extracting by day.

Some of the Tweets in the database were apparently *not* sorted by
date, and this was causing an infinite while loop since we jumped
*back* in time with the timepoint.
Write script to open files based on the type of improvement the filtering provided.

open_files.py opens the raster plot and the epsilon-machine for a given
user. It ranks the users by the amount of improvement, disimprovemnt,
or no-change (these aren't ranked).