StanfordAIMI

/

GREEN-RadPhi2

like 1

Hello,

when I try to run the code demo of GREEN with GREEN-RadPhi2 saved in a local folder:

from green_score import GREEN

model = GREEN(
    model_id_or_path="/path/to/GREEN-RadPhi2",
    do_sample=False,  # should be always False
    batch_size=16,
    return_0_if_no_green_score=True,
    cuda=True
)

refs = [
    "Interstitial opacities without changes."
]
hyps = [
    "Interstitial opacities at bases without changes."
]

mean_green, greens, explanations = model(refs=refs, hyps=hyps)
print("Mean reward for the given examples is: ", mean_green)
print("Array of rewards for the given examples is: ", greens)
print(explanations[0]) # LLM output for first pair

I receive the following error message:

Could not locate the configuration_phi.py inside StanfordAIMI/RadPhi-2.
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
    response.raise_for_status()
  File "/home/user/.local/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/StanfordAIMI/RadPhi-2/resolve/main/configuration_phi.py

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 402, in cached_file
    resolved_file = hf_hub_download(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1325, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1823, in _raise_on_head_call_error
    raise head_call_error
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata
    r = _request_wrapper(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper
    response = _request_wrapper(
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 396, in _request_wrapper
    hf_raise_for_status(response)
  File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 352, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-667179ca-4fbfd09a57c5c8f7747353ee;d2972de1-2308-4818-9c5f-4afb0728feed)

Repository Not Found for url: https://huggingface.co/StanfordAIMI/RadPhi-2/resolve/main/configuration_phi.py.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/path/M3D/green_phi2_script.py", line 6, in <module>
    model = GREEN(
  File "/home/user/.local/lib/python3.10/site-packages/green_score/green.py", line 217, in __init__
    self.model = GREENModel(cuda, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/green_score/green.py", line 42, in __init__
    self.model = AutoModelForCausalLM.from_pretrained(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 524, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 963, in from_pretrained
    config_class = get_class_from_dynamic_module(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 502, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 306, in get_cached_module_file
    resolved_module_file = cached_file(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 425, in cached_file
    raise EnvironmentError(
OSError: StanfordAIMI/RadPhi-2 is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`  

A fix for this is commenting out line 44 trust_remote_code=True in /home/user/.local/lib/python3.10/site-packages/green_score/green.py. Still, we are wondering why this OSError occurs. We have model_id_or_path="/path/to/GREEN-RadPhi2", i.e. have RadPhi2 saved in a local folder, but somehow huggingface tries to get it from the web.

Best regards,
Moritz