MAINT: remove some inf-related warnings from logsumexp

see https://github.com/scipy/scipy/issues/4014
+1 for scipy.lib, which is to become scipy._lib (https://github.com/scipy/scipy/pull/2908/files#diff-e79eb3278b3f95f251c02522330d7b1aR151) at some point in the future.