cross check inferred matrix for rectangles