first push of docProcessor.py
push of ldaProcessor supposed to handle the lda processing, the lda part works. the json dumping part still needs to be implemented
json dump in stdout implemented, see comments in code for a pretty print of the json format
ldaProcessor.py takes bow doc as input
started tokenizer
bug fixing
skeleton of pdfProcessor.py responsible for processing the pdf file to text and then parses it in a nice format
added word to int map and tokenization
Merge branch 'LDA' of https://github.com/aubry74/matomeru into LDA
function to run pdfxstream from python
refactoring docProcessor
added generate bag of words representation
Merge branch 'LDA' of https://github.com/aubry74/matomeru into LDA
Merge branch 'LDA' of https://github.com/aubry74/matomeru into LDA
piping the modules together
added id2token
Merge branch 'LDA' of https://github.com/aubry74/matomeru into LDA
everything has been piped together and is working git add ldaProcessor.py ldaTokenizer.py pdfProcessor.py!
stopWordsList.txt added
docProcessor can handle pdfgit add docProcessor.py !