Module Expr Str Import alias identifier ImportFrom alias identifier ImportFrom alias Import alias identifier FunctionDef arguments args NameParam defaults Num body Expr Str For NameStore Call NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num kwarg body Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr Slice SubscriptLoad NameLoad ExtSlice Index AttributeLoad NameLoad attr Slice Slice Num Return Call AttributeLoad NameLoad attr BinOpMult BinOpDiv Num NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num kwarg body Return Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr Slice SubscriptLoad NameLoad ExtSlice Index AttributeLoad NameLoad attr Slice Slice NameLoad decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad Num Num kwarg body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad NameLoad keyword NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpAdd AttributeLoad NameLoad attr BinOpMult Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr NameLoad If UnaryOpNot CompareIs NameLoad NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str decorator_list decorator_list FunctionDef arguments args defaults body Assign TupleStore NameStore NameStore TupleLoad Num Num Assign NameStore Num Assign NameStore Num Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Num NameLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr Assign NameStore BinOpAdd BinOpMult NameLoad Call AttributeLoad NameLoad attr TupleLoad Num Num BinOpDiv Num BinOpAdd Num Call AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad AugAssignDiv NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice Num Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad Num ExtSlice Slice Index AttributeLoad NameLoad attr Assign NameStore BinOpAdd NameLoad BinOpMult Num Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Num Assign NameStore Num Assign NameStore Call NameLoad SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice keyword NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Str NameLoad Str NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpAdd Str Str Assign NameStore Call NameLoad SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice keyword NameLoad keyword NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Str NameLoad Str NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpAdd Str Str decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num Num Num body Assign NameStore Num Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Num NameLoad NameLoad ExtSlice Slice Index AttributeLoad NameLoad attr Assign NameStore BinOpDiv SubscriptLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr BinOpPow NameLoad Num Num ExtSlice Slice Index AttributeLoad NameLoad attr NameLoad Assign NameStore BinOpAdd NameLoad BinOpMult Num Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Num Assign NameStore Num Assign NameStore Call NameLoad SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice keyword NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Str NameLoad Str NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpAdd Str Str Assign NameStore Call NameLoad SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice SubscriptLoad NameLoad ExtSlice Slice NameLoad Slice keyword NameLoad keyword NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Str NameLoad Str NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpAdd Str Str decorator_list If CompareEq NameLoad Str body Expr Call NameLoad Expr Call NameLoad
N/A N/A Kernel Ridge Regression for local non-parametric regression N/A numpy np scipy spatial ssp numpy.testing assert_equal N/A matplotlib.pylab plt plt_closeall N/A N/A n N/A 10 N/A N/A close a number of open matplotlib windows N/A i N/A range n N/A N/A N/A N/A plt close N/A kernel_rbf N/A N/A x y scale N/A 1 kwds N/A N/A dist N/A N/A ssp minkowski_distance_p N/A x N/A N/A N/A N/A np newaxis N/A N/A y N/A N/A N/A np newaxis N/A N/A 2 N/A N/A N/A np exp N/A N/A -0.5 scale dist N/A kernel_euclid N/A N/A x y p N/A 2 kwds N/A N/A N/A N/A ssp minkowski_distance N/A x N/A N/A N/A N/A np newaxis N/A N/A y N/A N/A N/A np newaxis N/A N/A p N/A GaussProcess N/A object N/A N/A class to perform kernel ridge regression (gaussian process)

    Warning: this class is memory intensive, it creates nobs x nobs distance
    matrix and its inverse, where nobs is the number of rows (observations).
    See sparse version for larger number of observations


    Notes
    -----

    Todo:
    * normalize multidimensional x array on demand, either by var or cov
    * add confidence band
    * automatic selection or proposal of smoothing parameters

    Note: this is different from kernel smoothing regression,
       see for example http://en.wikipedia.org/wiki/Kernel_smoother

    In this version of the kernel ridge regression, the training points
    are fitted exactly.
    Needs a fast version for leave-one-out regression, for fitting each
    observation on all the other points.
    This version could be numerically improved for the calculation for many
    different values of the ridge coefficient. see also short summary by
    Isabelle Guyon (ETHZ) in a manuscript KernelRidge.pdf

    Needs verification and possibly additional statistical results or
    summary statistics for interpretation, but this is a problem with
    non-parametric, non-linear methods.

    Reference
    ---------

    Rasmussen, C.E. and C.K.I. Williams, 2006, Gaussian Processes for Machine
    Learning, the MIT Press, www.GaussianProcess.org/gpal, chapter 2

    a short summary of the kernel ridge regression is at
    http://www.ics.uci.edu/~welling/teaching/KernelsICS273B/Kernel-Ridge.pdf
     __init__ N/A N/A self x y kernel scale ridgecoeff N/A None kernel_rbf 0.5 1e-10 kwds N/A N/A 
        Parameters
        ----------
        x : 2d array (N,K)
           data array of explanatory variables, columns represent variables
           rows represent observations
        y : 2d array (N,1) (optional)
           endogenous variable that should be fitted or predicted
           can alternatively be specified as parameter to fit method
        kernel : function, default: kernel_rbf
           kernel: (x1,x2)->kernel matrix is a function that takes as parameter
           two column arrays and return the kernel or distance matrix
        scale : float (optional)
           smoothing parameter for the rbf kernel
        ridgecoeff : float (optional)
           coefficient that is multiplied with the identity matrix in the
           ridge regression

        Notes
        -----
        After initialization, kernel matrix is calculated and if y is given
        as parameter then also the linear regression parameter and the
        fitted or estimated y values, yest, are calculated. yest is available
        as an attribute in this case.

        Both scale and the ridge coefficient smooth the fitted curve.

         N/A N/A self x x N/A N/A self kernel kernel N/A N/A self scale scale N/A N/A self ridgecoeff ridgecoeff N/A N/A self distxsample N/A kernel x x scale scale N/A N/A self Kinv N/A N/A N/A np linalg inv N/A N/A self distxsample N/A N/A N/A np eye N/A N/A self distxsample shape ridgecoeff N/A N/A N/A y None N/A N/A N/A self y y N/A N/A self yest N/A N/A self fit y N/A fit N/A N/A self y N/A N/A N/A fit the training explanatory variables to a sample ouput variable N/A N/A self parest N/A N/A np dot N/A self Kinv y N/A yhat N/A N/A np dot N/A self distxsample N/A self parest N/A yhat N/A predict N/A N/A self x N/A N/A N/A predict new y values for a given array of explanatory variables N/A N/A self xpredict x N/A distxpredict N/A N/A self kernel x N/A self x scale N/A self scale N/A N/A self ypredict N/A N/A np dot distxpredict N/A self parest N/A N/A self ypredict N/A plot N/A N/A self y plt N/A plt N/A N/A some basic plots N/A N/A N/A plt figure N/A N/A N/A plt plot N/A self x N/A self y bo- N/A self x N/A self yest r.- N/A N/A N/A plt title sample (training) points N/A N/A N/A plt figure N/A N/A N/A plt plot N/A self xpredict y bo- N/A self xpredict N/A self ypredict r.- N/A N/A N/A plt title all points N/A N/A example1 N/A N/A N/A N/A N/A N/A m k N/A 500 4 N/A upper 6 N/A scale 10 N/A xs1a N/A N/A N/A np linspace 1 upper m N/A N/A N/A N/A np newaxis N/A xs1 N/A N/A xs1a N/A N/A np ones N/A 1 4 N/A 1 N/A 1.0 N/A N/A np exp N/A N/A N/A np random randn m k N/A xs1 N/A N/A np std N/A xs1 N/A N/A k N/A 0 N/A y1true N/A N/A N/A np sum N/A N/A N/A np sin xs1 N/A N/A np sqrt xs1 1 N/A N/A N/A N/A np newaxis N/A y1 N/A y1true N/A 0.25 N/A N/A N/A np random randn m 1 N/A stride 2 N/A gp1 N/A GaussProcess N/A xs1 N/A N/A stride N/A N/A y1 N/A N/A stride N/A kernel kernel_euclid ridgecoeff 1e-10 N/A yhatr1 N/A N/A gp1 predict xs1 N/A N/A N/A plt figure N/A N/A N/A plt plot y1true y1 bo y1true yhatr1 r. N/A N/A N/A plt title euclid kernel: true y versus noisy y and estimated y N/A N/A N/A plt figure N/A N/A N/A plt plot y1 bo- y1true go- yhatr1 r.- N/A N/A N/A plt title N/A euclid kernel: true (green), noisy (blue) and estimated (red)  observations N/A gp2 N/A GaussProcess N/A xs1 N/A N/A stride N/A N/A y1 N/A N/A stride N/A kernel kernel_rbf scale scale ridgecoeff 0.1 N/A yhatr2 N/A N/A gp2 predict xs1 N/A N/A N/A plt figure N/A N/A N/A plt plot y1true y1 bo y1true yhatr2 r. N/A N/A N/A plt title rbf kernel: true versus noisy (blue) and estimated (red) observations N/A N/A N/A plt figure N/A N/A N/A plt plot y1 bo- y1true go- yhatr2 r.- N/A N/A N/A plt title N/A rbf kernel: true (green), noisy (blue) and estimated (red)  observations N/A example2 N/A N/A m scale stride N/A 100 0.01 2 N/A N/A upper 6 N/A xs1 N/A N/A N/A np linspace 1 upper m N/A N/A N/A N/A np newaxis N/A y1true N/A N/A N/A N/A np sum N/A N/A np sin N/A xs1 2 1 N/A N/A N/A N/A np newaxis xs1 N/A y1 N/A y1true N/A 0.05 N/A N/A N/A np random randn m 1 N/A ridgecoeff 1e-10 N/A gp1 N/A GaussProcess N/A xs1 N/A N/A stride N/A N/A y1 N/A N/A stride N/A kernel kernel_euclid ridgecoeff 1e-10 N/A yhatr1 N/A N/A gp1 predict xs1 N/A N/A N/A plt figure N/A N/A N/A plt plot y1true y1 bo y1true yhatr1 r. N/A N/A N/A plt title euclid kernel: true versus noisy (blue) and estimated (red) observations N/A N/A N/A plt figure N/A N/A N/A plt plot y1 bo- y1true go- yhatr1 r.- N/A N/A N/A plt title N/A euclid kernel: true (green), noisy (blue) and estimated (red)  observations N/A gp2 N/A GaussProcess N/A xs1 N/A N/A stride N/A N/A y1 N/A N/A stride N/A kernel kernel_rbf scale scale ridgecoeff 0.01 N/A yhatr2 N/A N/A gp2 predict xs1 N/A N/A N/A plt figure N/A N/A N/A plt plot y1true y1 bo y1true yhatr2 r. N/A N/A N/A plt title rbf kernel: true versus noisy (blue) and estimated (red) observations N/A N/A N/A plt figure N/A N/A N/A plt plot y1 bo- y1true go- yhatr2 r.- N/A N/A N/A plt title N/A rbf kernel: true (green), noisy (blue) and estimated (red)  observations N/A N/A N/A __name__ __main__ N/A N/A N/A example2 N/A N/A example1