don't try to update the matrix in setMap since it's too early and done l...

...ater in moveTo

Please review.
Some tests are failing but it doesn't seem related to my modificaitons (same failing tests without the modifications).