zenogantner/MyMediaLite — Issue #185: correlation matrix I/O: decide on sparse vs. dense representation after analysing the sparsity
Author: zenogantner
Created: 2011-12-23 11:46:24+00:00
State: open
Labels: enhancement

Currently, we represent correlation matrix as dense matrices in memory, but write them out in a "sparse" manner, wasting a lot memory for the row and column IDs.

For example, a user-user similarity matrix for the ML-1M data set uses about 333 MB of memory.

A good write method would check the sparsity of the correlation matrix, and then decide on whether to store the upper triangle in a dense or sparse way.