Revert "cache the inverse of the matrix"

This reverts commit 48ffaecbbb976ac07fc670d72ed584ce99ad4686.