Abandoned the multiple parallel similarity types approach and simplified the basic similarity engine.
For benchmarking purposes, I created a similarity engine that uses the Gensim package to perform Latent Semantic Indexing. It works a lot better than my basic tf-idf approach.