Don't call asarray() on AffineMatrix objects.

This is a temporary fix for issue #156. We should still try to find out why `asarray` is failing, but this will keep tests from failing in the meantime.