Use broadcasting to scale back instead of multiplying with a diagonal matrix.