Remove block from hashset after removing entity