MAINT: remove redundant isfinite checks
Merge pull request #3336 from argriffing/redundant-chkfinite

MAINT: remove redundant isfinite checks