optimize token stream merge process