Fix memory leaks due to unallocated matrices.