Restoring files and scripts from Project-HashtagAnalysis backup folder. Excluded twitter_proof due to its history with tcp.txt, the large file in git history that blocked git push due to size constraints.

In new branch hashtags, readded files from the backup directory I made earlier. Overwrote files with the same name from master branch. Should have a clean history now.
Did lose all git history for HashtagAnalysis, the previous name of this branch. Git history will re-start with this initial commit.
Replicating LaurentLuce tutorial using hashtags. Works as far as extract_features.

In extract_features:
Passed in the result of getting all hashtags from timeline as HASHTAGS, used a test list of hashtags (["tcot", "p2", "Maddow"]) as hashtags.
Function returned a dictionary of hashtags from HASHTAGS list paired with True or False. "Maddow" was True.
WIP. Marking point at which I will change Chris' suggestion that extract_hashtags add to a set called HASHTAGS.

extract_hashtags adds to a set called HASHTAGS defined in main(). It works - creates a set (all unique items) of all hashtags from a given timeline.
Ignore storage() function; training_set is not working.
However, I need to change this; making it a set too early precludes using FreqDist to get the frequency of different hashtags.
Create data structure needed for classifying data: a list of tuples in the form of ([data], label)

Create a list of liberal hashtags
Create a list of conservative hashtags
Label each one separate from the extraction process
Extend an overarching list called HASHTAGS with both conservative and liberal tuples.
HASHTAGS is in the correct data structure.
WIP.
Program can extract features from a new document (ie, list of hashtags) and return a dictionary of hashtags that are present (value = True) and that are not present (value = False).

In main(), extract hashtags from lib tweets to a list of liberal hashtags. Do the same for conservative tweets. A variable called 'classifier' adds a label to the list and creates a tuple. HASHTAGS is a list of these tuples.

By feeding the output of each function into the next function:
get_unique_hashtags -> get_hashtags_as_features -> extract_features returns a dictionary of 'hits' or matches in a given document.
Clean up code, delete large sections of commented-out previous iterations.

Inserted link to NLTK tutorial at top of file.
Rewrote TODO to reflect remaning steps to build classifier.
Copy flask_server and templates/index.html from twitter_proof into main project folder for frontend development.

Beginning to add in front-end components.
Updated flask_server and index.html for current filenames and routes.

Flask_server creates a user object from friends.User class to initiate api connection.
index.html submits form to /display.
Flask_server returns tweepy cursor object of friends ids (as test).
Included bootstrap css from static directory.
Adding requirements file for pip installations
Add packages to requirements.txt - dependencies for scikit-learn

Numpy
Scipy
Scikit-learn
Import libraries for scikit-learn into score.py
Add notes and small modifications to hashtag extraction and json functions at the top of score.py

Json function will have to change to get data from database, rather than json files
Added parameters and updated docstring in extract_hashtags to reflect a separate list of labels that will be created at the same time that hashtags list is populated.
Added vectorize and classifier functions to score.py, to instantiate and train classifier. Untested at present.

Created functions and added docstrings.
Use CountVectorizer from sklearn to count occurrences within features list (hashtags)
Instantiate a Bernoulli Naive Bayes classifer.
Create cross-validator with Stratified Kfold method to split data in K parts.
Print precision and recall averages.
Return classifier.
Modified extract_hashtags function to label every hashtag, even those in the same tweet. In main(), init main HASHTAGS and LABELS lists.

Indented label_list.append within extract_hashtags to create a label for every hashtag.
Deleted code that referenced LIB_HASHTAGS and CONS_HASHTAGS, simplified to a single hashtags list with accompanying labels list.
ipython notebook files
Debugging in ipy notebook - WIP
Moved ipynb files into tests directory. Finished debugging classifier in ipy notebook. Updated score.py

TODO: make sure I understand metrics, recall, precision. I was getting an error with precision.append(p[1]) originally, maybe due to wrong indentation.

Add X as parameter to init_and_train_classifier.
Added note about parameter X to docstring of init_and_train_classifier
Slight modification to precision/recall to test my understanding (print statements)
Added dummy score module for insertion into friends.py as method on User class.

Update name of route function in flask_server.py
Change import statement to from friends import User, allowing me to refer to User directly.