Allocate byte slice up-front so we don't have to keep calling append()