Removing from the index does not result in removing from disk.

Considering the code is still so new I'd prefer it to throw an exception so we can find out why that happened.