Detect change if metadata file is deleted

Closes #191