Be safer about trying to operate on possible strings

Fixes #193