hevc: 10bit sequence fix
fix md5 for 10bit sequences
luma is correct with 10bit sequences
fix operation priority order