shogun-toolbox/shogun — pull request #2374 by khalednasr (2014-07-08 13:39:03+00:00)
Title: refactored linalg's matrix sum methods

- Refactored linalg's matrix sum methods, similar to the previous PR
- Removed the need to have 2 wrapper methods (one for global backend, another for template specified backend) for each struct by using a default template argument

However, there're some issues that I ran into while refactoring that should be discussed before this PR is merged:

- Returning matrices/vectors is probably not a good idea. For example, the colwise sum function currently looks something like: ```SGVector colwise_sum(Matrix m, bool no_diag=false) ```. This forces the return type for all backends and matrix types to be SGVector. This makes the function pretty much useless for the GPU backend, since it can only return CPU vectors. If we make the return type a template parameter, automatic deduction won't work for it, and it'll have to be specified explicitly. Also, returning a new vector each function call can be inefficient. Therefore, I suggest making this function (and similar functions) take a pre-allocated result vector as an argument.

- Supporting Eigen3 matrices seems wasteful. They behave differently from SGMatrix/GPUMatrix (i.e they do deep copies on assignment, and shogun matrices do shallow copies) and this can lead to problems. Also, if someone decided to use Eigen3 matrices in some algorithm, they're probably better off using native Eigen3 methods instead of the linalg library.

- Currently, the GPU backend is practically useless, unless the developer explicitly uses CGPUMatrix for all the heavy matrices in their algorithm. If they used SGMatrix instead, it'll result in a lot of CPU<-->GPU copies, which would severely limit the performance. And if they do decide to use CGPUMatrix, the code will only be usable with GPU backend. This problem can be solved by introducing a backend-dependent matrix/vector class - something like [this](https://gist.github.com/khalednasr/92acd8aa23d7ae26196d).

@lambday, @lisitsyn, @karlnapf Please take a look.