Remove a reference spec file