Just have to say - this is the best LLM model I have used to date!
Just have to say - this is the best LLM model I have used to date!
Very happy to hear! A new version has undergone big improvements in training, got a two HUGE releases coming these days, keep eye out! :)
Free the true LLaMa inside (the one uninhibited by censorship and denials to fulfill requests)! 🔥😋
Very happy to hear! A new version has undergone big improvements in training, got a two HUGE releases coming these days, keep eye out! :)
Well that be posted here or a new, different link? I'm finding this V1 to be pretty incoherent. Just feels like an old 7B, nothing special, while the vanilla does seem pretty smart.
I want to highlight an ongoing issue with GGUFs' and models trained in bfloat16 such as llama3. https://github.com/ggerganov/llama.cpp/issues/7062
This means that the GGUF models might not behave as expected from the original fine tuning. I'm experimenting further and we are investigating what could be done to fix these issues.
Lexi-Llama-3-8B-Uncensored_F16.gguf
Win10, 64GB, RTX3090
Context window 16384, first hallucination ~11K, still some reasonable answers through 15.5K, non-sense at about 16.4K
Starts out pretty chipper, but at about 6K, answers become much shorter and less chipper. At 15.5K answers are just a couple paragraphs and lots of apologies.
Nice bake.
llama_3_70B.preset.json contents below:
{
"name": "Llama 3 70B Instruct",
"load_params": {
"n_ctx": 16384,
"n_batch": 512,
"rope_freq_base": 1000000,
"rope_freq_scale": 0.85,
"n_gpu_layers": 64,
"use_mlock": true,
"main_gpu": 0,
"tensor_split": [
0
],
"seed": -1,
"f16_kv": true,
"use_mmap": true,
"no_kv_offload": false,
"num_experts_used": 0
},
"inference_params": {
"n_threads": 12,
"n_predict": -1,
"top_k": 40,
"min_p": 0.05,
"top_p": 0.9,
"temp": 0.2,
"repeat_penalty": 1.1,
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"antiprompt": [
"<|start_header_id|>",
"<|eot_id|>"
],
"pre_prompt": "You are a thoughtful, helpful and knowledgeable AI assistant. You provide clear and concise responses, offering gentle guidance when needed, and politely addressing any misconceptions with well-reasoned explanations. No apologies required. We are collaborating together.",
"pre_prompt_suffix": "<|eot_id|>",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"seed": -1,
"tfs_z": 1,
"typical_p": 1,
"repeat_last_n": 64,
"frequency_penalty": 0,
"presence_penalty": 0,
"n_keep": 0,
"logit_bias": {},
"mirostat": 0,
"mirostat_tau": 5,
"mirostat_eta": 0.1,
"memory_f16": true,
"multiline_input": false,
"penalize_nl": true
}
}
@rdtfddgrffdgfdghfghdfujgdhgsf It will be posted on my channel with new link, keep eye out
Will it be here on Hugging Face or do you mean somewhere else? I'm really looking forward to the new version! :-)
@DiamondHunter I'm still in experimentation will see if I make an official release soon. Can't promise when or so but hopefully soon. =)
@DiamondHunter I'm still in experimentation will see if I make an official release soon. Can't promise when or so but hopefully soon. =)
Wow, after waiting for so many days, the llama relay station classifies the user query and matches it to the most suitable LLAMA proxy node. You can come and take a look at this project
@DiamondHunter I'm still in experimentation will see if I make an official release soon. Can't promise when or so but hopefully soon. =)
Wow, after waiting for so many days, the llama relay station classifies the user query and matches it to the most suitable LLAMA proxy node. You can come and take a look at this project
As someone only starting out with AI, what does that mean exactly? After looking at the linked page, I assume better processing of inputs?
@DiamondHunter I'm still in experimentation will see if I make an official release soon. Can't promise when or so but hopefully soon. =)
I have some input that I'd like to discuss privately. How can I contact you? :-)
@DiamondHunter disgrace6161 @ discord