Function ngramsFinder() is now external and can be access from outside this code. It takes a text and finds the best bigrams and trigrams which are written into a file.

TODO: implement ngramsText() which takes a text and returns the text where the bigrams and trigrams have been converted to a single token
Added a new function in collocations which takes a text and process it joining the bigrams and trigrams stored in a external file.
Defined all related function and ngrams.py
TODO: remove the functions from collocations and import the ngrams.py file
Cleaning up and writing comments
Merge pull request #6 from elenagr/master

ngrams
Moved the stop words test as it was in the wrong place.
Merge pull request #7 from emsiepopples/master

Moved the stop words test as it was in the wrong place.
More stop words changes.
Merge pull request #8 from emsiepopples/master

More stop words changes.
- New stopwords based on results in ngrams
- ngrams: Included a new function to read the dictionary and replace the words found in text. Writes a file with the matches
- ngrams: ngramsText gives an output with the replaced words and the text id.
 - collocations: clean up
cleaning up and changing name
cleaning up and changing name
Update
Dictionary for abbreviations in finance
First version of bigrams
First version of trigrams
Merge pull request #9 from elenagr/master

ngrams and abbreviations dictionary
cleaning up and changing name
Summary of the tokenisation/stemming test.  Some useful quantities saved in the pickle
Merge pull request #10 from emsiepopples/master

Summary of the tokenisation/stemming test
Make a nice bar graph with the results from the tokenisation/stemming test