Neural part done, learn remaining