Make sure we are always passing around pre-L2-normalized vectors; drop
all the other vector reprs from RAM!