BUG: Actually use the vnl_vector fill