unify the code for the cases exact Gram matrix calculated or input