CeON/CoAnSys — pull request #40 by kawaa (2012-09-09 23:38:22+00:00)
Title: Document similarity subproject functionality

The first version of document similarity functionality.

It includes Pig scripts and Java UDFs to:
1) calculate TFIDFs (based on terms from title, abstract, keywords list)

The TFIDF is used next used to:
2a) calcuate similarity between documents that have at least one common term (simple average function is used)
2b) calculate similarity between all pairs of documents (cosine function is uded)