Use 2 auxiliary arrays to determine what has to be changed when dragging a rectangle.