Revert commit 7eec859 ("Do not remove stuff from lists")

When removing a download, Angular doesn't update the download list correctly
when it receives new data unless we pull out the item that should be removed
from the array manually first.