Clean up transform dimension height handling

Two issues here:

1. normaliseParams isn't actually normalising width and height for audio
   files. Which means it's handling height=-1 and oversize width/height
   incorrectly.

2. When the limiting factor is height, the code is using the wrong
   formula to calculate the target width, leading to nonsensical values.

Bug: 49921
Change-Id: Ide7208394f8495f5b143cbba215d4645598f1a50
(cherry picked from commit 0d23f78b61f441b0eed85d5758eaea463c504abb)