Actually remove the tmp vectors