calculate points to find the convex polygon vertices