transcode: cleanup channel setup and allow upmixing

Allow channel upmixing as we have channel mixer that is able to do it.

Fixes #7261

(cherry picked from commit 6b7dcd7016c6cd46f4a685e5bdadb4e29c5f38d1)
Signed-off-by: Tim Walker <tdskywalker@gmail.com>