Apply the file manifest filter in the proper place

We should always apply in `set_using_file_manifest()` so the filter overrides whatever argument we're passing in the API request.