Switched back to the recursive method of calculating determinant, need to look at other method more closely.  The way it was implemented it only worked for 3x3, so probably a sign issue.