shogun-toolbox/shogun — pull request #603 by puffin444 (2012-06-26 13:12:10+00:00)
Title: Gradient Search Framework.

Hi,

Here is finally the Gradient Search Framework applied to learning hyperparameters for Gaussian Processes.
I updated the regression_gaussian_process example to include this learning.

I do have some comments/questions.

1. With the new framework, MachineEvaluation returns a general EvaluationResult which then must be downcast. Heiko suggested that I add some enum types to do this safely. What do you all think?

2. Heiko, I added two functions to your ParameterCombinations class and ModelSelectionParameter class. Should I add my name to the copyright?

3. What do you think of the rand() function I use in ModelSelectionParameters::get_random_combination?
Is it appropriate? Should I generate a random number using some other method?

4. Does it make sense to overload the set_parameter function for int and other types, or should there
be one function that uses one input as a switch to determine the type?

5. It is easy in NLOPT to do maximization instead of minimization. I should probably use get_evaluation_direction from an evaluation criterion to determine direction. Currently GradientModelSelection does not use an evaluation criterion. Does it make sense to add an evaluation criterion class solely for determining gradient search direction?

6. GCC complains if I do not add get_name to abstract classes (like machineevaluation). Is this supposed to happen? If not, what am I doing wrong?

7. Right now I have automatically set lower bounds for parameters. Would it make sense to take the lowest
value from the array built by build_values to determine lower bounds for individual parameters?

8. Valgrind complains about uninitialized values in the CMap structure. I am not exactly sure why it complains, but I suspect its because an internal array is allocated, but not all the values are used. Should I worry about this?

Again, please let me know if you have any concerns, or if you find an issue with my code/coding style.

puffin444