P-V-C/Machine-Learning — Issue #2: Estimation of w_ML in question 1.1
Author: vyakimov
Created: 2012-02-28 17:52:19+00:00
State: open

Nick:
I looked through your code, and I think the estimation of w_ML is wrong. You have:

y <-ginv(t(cols1trainwithone)%*%cols1trainwithone)%*%t(cols1trainwithone)%*%targettrain

where cols1trainwithone is the design matrix.

According to Bishop eq. (3.3), y(x,w)=t(w) %*% phi(x)
according to Bishop eq. (3.15), w= (t(Phi) %*%Phi)^-1%*%t(Phi)*t
and according to Bishop eq (3.17) the Moore-Penrose pseudo-inverse is (t(Phi) %*%Phi)^-1%*%t(Phi) so w=PseudoInverse * t and y=PseudoInverse*t*phi(x)

Let me know if I've made a mistake somewhere.