Fix CI bug with files being required when they no longer exist