Trying to find right solution to Ax=b solving