phymbert

/

dbrx-16x12b-instruct-q6_k-gguf

like 0

When I try .\convert-hf-to-gguf.py path\to\dbrx-instruct I get:

Traceback (most recent call last):
  File "C:\Python\Python311\Lib\site-packages\transformers\dynamic_module_utils.py", line 596, in resolve_trust_remote_code
    signal.signal(signal.SIGALRM, _raise_timeout_error)
                  ^^^^^^^^^^^^^^
AttributeError: module 'signal' has no attribute 'SIGALRM'. Did you mean: 'SIGABRT'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 2807, in <module>
    main()
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 2794, in main
    model_instance.set_vocab()
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 74, in set_vocab
    self._set_vocab_gpt2()
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 261, in _set_vocab_gpt2
    tokens, toktypes = self.get_basic_vocab()
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 237, in get_basic_vocab
    tokenizer = AutoTokenizer.from_pretrained(self.dir_model)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\transformers\models\auto\tokenization_auto.py", line 809, in from_pretrained
    trust_remote_code = resolve_trust_remote_code(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\site-packages\transformers\dynamic_module_utils.py", line 612, in resolve_trust_remote_code
    raise ValueError(
ValueError: The repository for \dbrx-instruct contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co/\dbrx-instruct.

And when I add trust_remote_code:

Traceback (most recent call last):
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 2807, in <module>
    main()
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 2794, in main
    model_instance.set_vocab()
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 74, in set_vocab
    self._set_vocab_gpt2()
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 261, in _set_vocab_gpt2
    tokens, toktypes = self.get_basic_vocab()
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\llama.cpp\convert-hf-to-gguf.py", line 238, in get_basic_vocab
    vocab_size = self.hparams.get("vocab_size", len(tokenizer.vocab))
                                                    ^^^^^^^^^^^^^^^
AttributeError: 'TiktokenTokenizerWrapper' object has no attribute 'vocab'

I'm using the latest version of llama.cpp.