Further refactor ArrayIndexList's remove method