Clean up the parser so it no longer passes around the bytes remaining