enc_qsv: cleanup encoding loop, part 2.

Move related code to separate functions whenever
possible; simplify or improve the code where applicable.