Broken files?
ggml_cuda_init: found 1 CUDA devices:
Device 0: Tesla P40, compute capability 6.1, VMM: no
llama_load_model_from_file: using device CUDA0 (Tesla P40) - 24288 MiB free
llama_model_load: error loading model: tensor 'blk.38.ffn_up.weight' data is not within the file bounds, model is corrupted or i"|ЩЦJbllama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "koboldcpp.py", line 5009, in
main(parser.parse_args(),start_server=True)
File "koboldcpp.py", line 4630, in main
loadok = load_model(modelname)
File "koboldcpp.py", line 930, in load_model
ret = handle.load_model(inputs)
OSError: exception: access violation reading 0x00000000000018A4
[13240] Failed to execute script 'koboldcpp' due to unhandled exception!
Trying to load Q6_k in koboldcpp, getting this error. oobabooga release has even older llama.cpp, so it probably will not run there too. I downloaded both parts.
Tried on latest llama.cpp, doesn't work too.
Can you please provide hash for two q6 files so i can check their validity, too long to download from start.
gguf_merge: L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf.part1of2 -> outfile.gguf
gguf_merge: reading metadata L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf.part1of2 ...
gguf_merge: input file does not contain split.count metadata
Tried to merge them and failed.
mradermacher uses splitted files and not GGUF slits so you instead just concatinate them using cat L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf.part1of2 L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf.part2of2 > L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf or even better download them already concatinated from https://hf.tst.eu/model#L3.3-70B-Euryale-v2.3-i1-GGUF
HuggingFace always displays sha256 hashes under https://huggingface.co/mradermacher/L3.3-70B-Euryale-v2.3-i1-GGUF/blob/main/L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf.part1of2 and https://huggingface.co/mradermacher/L3.3-70B-Euryale-v2.3-i1-GGUF/blob/main/L3.3-70B-Euryale-v2.3.i1-Q6_K.gguf.part2of2 which in this case is cdc2cc4358fc52a80bc2185eca0797dab709c77b54ae39a564d311680169b3fc and 9983272a158ec05bdd1fbc056492de396df09ff0a69c8fadb6de1625280d71dc but it really doesn't matter as the issue is you using gguf_merge instead concatinating the files.