Be more selective when deselecting elements