validate rectangle and test invalid rectangles