johnmyleswhite/SimilarityMetrics.jl — pull request #4 by toddleo (2014-11-13 09:41:51+00:00)
Title: Modify: Added efficient cosine similarity function for sparse vectors

I am working on a project which heavily utilizes cosine similarity calculation, specifically, on very **sparse vectors**. Initially I adopted the R idiom and implemented cosine similarity in a vectorized fashion with an one-liner, and it was painfully slow. Later on I found the devectorized version of cosine similarity in this repo, but it gave me no boost of speed on my sparse dataset. Hence **I wrote several sparse-friendly routine** to calculate the norm and the dot product . As a result, **the sparse version outperformed the original one by 8 times**, and my test case is listed as following:

```julia
function cosine_test(v,w)
for i in 1:10000
cosine(v,w)
end
end

function cosine_sparse_test(v,w)
for i in 1:10000
cosine_sparse(v,w)
end
end

spmat = sprand(2, 10000, 0.1)
v = spmat[1,:]
w = spmat[2,:]
# warmup
cosine_test(v,w)
cosine_sparse_test(v,w)

gc()
@time cosine_test(v,w)
gc()
@time cosine_sparse_test(v,w)

# elapsed time: 15.056221871 seconds (80 bytes allocated)
# elapsed time: 1.870123261 seconds (172000080 bytes allocated, 6.11% gc time)
```

I thought someone might also need to run similar calculations on sparse vectors, thus this PR is created and I hope it can help.

Cheers,
Todd Leo