Improve polyset convexity check by tessellating polygons prior to testing. Fixes #1061