Restructure how references to deleted files are cleaned up

From #5279. 

Previously, the file was deleted and then removed from any field that referenced it. That could cause issues if the field expected the file to still be around when it's delete hook was run. 

I swapped the order, so the file is removed from fields first, and then deleted.