natong19

/

Mistral-Nemo-Instruct-2407-abliterated

like 8

I was unable to quantize your model with this command:
convert-hf-to-gguf --outtype f16 --outfile ./quants/mistral-nemo-abliterated-fp16.gguf . --verbose
(Note, i renamed the CLI tool name for conveniance, it should work the same)

DEBUG:hf-to-gguf:chkhsh: 63b97e4253352e6f357cc59ea5b583e3a680eaeaf2632188c2b952de2588485e
WARNING:hf-to-gguf:

WARNING:hf-to-gguf:**************************************************************************************
WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized!
WARNING:hf-to-gguf:**          There are 2 possible reasons for this:
WARNING:hf-to-gguf:**          - the model has not been added to convert_hf_to_gguf_update.py yet
WARNING:hf-to-gguf:**          - the pre-tokenization config has changed upstream
WARNING:hf-to-gguf:**          Check your model files and convert_hf_to_gguf_update.py and update them accordingly.
WARNING:hf-to-gguf:** ref:     https://github.com/ggerganov/llama.cpp/pull/6920
WARNING:hf-to-gguf:**
WARNING:hf-to-gguf:** chkhsh:  63b97e4253352e6f357cc59ea5b583e3a680eaeaf2632188c2b952de2588485e
WARNING:hf-to-gguf:**************************************************************************************
WARNING:hf-to-gguf:

Traceback (most recent call last):
  File "/home/james/.local/bin/convert-hf-to-gguf", line 1473, in set_vocab
    self._set_vocab_sentencepiece()
  File "/home/james/.local/bin/convert-hf-to-gguf", line 696, in _set_vocab_sentencepiece
    tokens, scores, toktypes = self._create_vocab_sentencepiece()
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/james/.local/bin/convert-hf-to-gguf", line 713, in _create_vocab_sentencepiece
    raise FileNotFoundError(f"File not found: {tokenizer_path}")
FileNotFoundError: File not found: tokenizer.model

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/james/.local/bin/convert-hf-to-gguf", line 1476, in set_vocab
    self._set_vocab_llama_hf()
  File "/home/james/.local/bin/convert-hf-to-gguf", line 788, in _set_vocab_llama_hf
    vocab = gguf.LlamaHfVocab(self.dir_model)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/james/.local/lib/python3.12/site-packages/gguf/vocab.py", line 362, in __init__
    raise TypeError('Llama 3 must be converted with BpeVocab')
TypeError: Llama 3 must be converted with BpeVocab

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/james/.local/bin/convert-hf-to-gguf", line 4151, in <module>
    main()
  File "/home/james/.local/bin/convert-hf-to-gguf", line 4145, in main
    model_instance.write()
  File "/home/james/.local/bin/convert-hf-to-gguf", line 413, in write
    self.prepare_metadata(vocab_only=False)
  File "/home/james/.local/bin/convert-hf-to-gguf", line 406, in prepare_metadata
    self.set_vocab()
  File "/home/james/.local/bin/convert-hf-to-gguf", line 1479, in set_vocab
    self._set_vocab_gpt2()
  File "/home/james/.local/bin/convert-hf-to-gguf", line 632, in _set_vocab_gpt2
    tokens, toktypes, tokpre = self.get_vocab_base()
                               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/james/.local/bin/convert-hf-to-gguf", line 494, in get_vocab_base
    tokpre = self.get_vocab_base_pre(tokenizer)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/james/.local/bin/convert-hf-to-gguf", line 623, in get_vocab_base_pre
    raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()")
NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre()