fixed check_simplex and added NaN test

#### Summary:
Fix check_simplex (issue #896).

#### Intended Effect:
Make sure check_simplex fails when NaN is passed in.

#### How to Verify:
Run test/unit/math/error_handling/matrix/check_simplex

#### Side Effects:
None.

#### Documentation:
None.

#### Reviewer Suggestions:
Anyone.