Increase the tolerance on the affine test

Probably wonky due to no normalization.