clean up how we free memory for convnet