Accounted for the case when mouseup and mousedown elements are parents