shogun-toolbox/shogun — pull request #585 by puffin444 (2012-06-14 16:55:16+00:00)
Title: Gaussian Process Regression Framework.

This is the implementation of the design for the GP Framework discussed last month.
I have thoroughly tested it for accuracy against the same inputs in the GPML toolbox.
I also believe I have eliminated all memory leaks with valgrind. Right now there is a proof of
concept gradient search method in the ExactInferenceMethod Class. Soon I will finish incorporating
gradient search into the ModelSelection framework, where I will use a more rigorous algorithm from NLOPT.

I current have a few questions about the code:

1. Right now I am using get_name() to obtain the identity of classes. Would adding enumerated
types for specific classes be better for this situation?

2. This will be unique to ExactInferenceMethod: It can only use a GaussianLikelihood model. It thus
checks its model if its really the GaussianLikelhood class, and then downcasts the model. Is it appropriate
to use downcasting in this way?

3. In update_alpha_and_chol, I use lapack functions to calculate the cholesky decomposition of a matrix.
Strangely, it returns the correct upper triangle of the matrix, but does not fill the lower triangle with zeros (it
leaves junk data in those entries). Is lapack supposed to do this? I go ahead and fill the lower triangle with zeros anyway.

4. The GP uses the cholesky decomposition matrix L and alpha vector m_alpha to perform inference in a general way. Thus both GaussianProcessRegression and InferenceMethod (its subclasses) will call get_cholesky and get_alpha. Each time these are called the matrix and vector respectively are updated in update_alpha_and_chol. I fear this might be inefficient. Do you think its better and safe to simply to call update_alpha_and_chol only when InferenceMethod is constructed or the features are changed in set_features?

5. Right now the apply_regression in GaussianProcessRegression throws an SG_ERROR if the data sent is NULL. Is this appropriate? Should it try to handle this situation more gracefully?

I have tried to comment everything/make sure everything is formatted correctly. If you have any concerns about formatting/coding conventions/commenting, let me know.

puffin444