asiansoul/llama-3-Korean-Bllossom-120B-GGUF

🌋🌋 Huggingface Upload Issue

Maximum individual file size is 50.0GB to upload huggingface.

To clear it, Split the file into part_aa, part_ab, part_ac chunks as my "Practical Idea".

After you download this repo on your folder path, command like this.

Download from Huggingface (change your download path, in this case "./")

huggingface-cli download asiansoul/llama-3-Korean-Bllossom-120B-GGUF --local-dir='./'

Merge split files into one gguf file (in this case, run this on "./" download path)

cat part_* > llama-3-korean-bllossom-120b-Q4_K_M.gguf

I thought uploading it as a GGUF rather than a simple original file was for your benefit, so I'm uploading it like this even if it takes a bit of trouble.

Perhaps this will be the first GGUF model to upload such a large GGUF file of over 50GB to huggingface?

Other 120B model for the individual file size is under 50GB, That is why it can be uploaded.

Sometimes we need to use a trick called chunks.

Please wait to upload.....

🇰🇷 About the JayLee "AsianSoul"

"A leader who can make you rich 💵 !!!"

"Prove yourself with actual results, not just saying I know more than you!!!"

About this model storytelling

This is a 128B model based on Bllossom/llama-3-Korean-Bllossom-70B

☕ I started this Korean 120B model merge while drinking an iced Americano at Starbucks referring to other Cognitive Computations 120B.

If you walk around Starbucks in Seoul, Korea, you will see someone creating a merge and an application based on it.

At that time, please come up to me and say "hello".

"Also, if you want to create the Application project you want and provide me with support, I will create the entire architecture for you whatever it is."

🏎️ I am a person whose goal is to turn the great results created by great genius scientists & groups around the world into profitable ones.

My role model is J. Robert Oppenheimer!!!

J. Robert Oppenheimer is highly regarded for his ability to gather and lead a team of brilliant scientists, merging their diverse expertise and efforts towards a common goal. 

Learn more about J. Robert Oppenheimer.

I hope this 120B is a helpful model for your future.

🌍 Collaboration is always welcome 🌍

👊 You can't beat these giant corporations & groups alone and you can never become rich. 

Now we have to come together. 

People who can actually become rich together, let's collaborate with me.!!! 🍸

About Bllossom/llama-3-Korean-Bllossom-70B
- Full model released in Korean over 100GB by Blossom team
- First in Korean! Expansion of Korean vocabulary to over 30,000 words
- Capable of processing Korean context that is approximately 25% longer than Llama3
- Connecting Korean-English knowledge using the Korean-English Parallel Corpus (pre-study)
- Fine tuning using data produced by linguists considering Korean culture and language
- Reinforcement learning

🛰️ About asiansoul/llama-3-Korean-Bllossom-120B-GGUF
- Q4_K_M : On a GPU with 68GB / more OR a CPU with 68G / more memory
- More Quantization ones i hope to upload, but your computer won't be able to handle it then. you know what i mean!!

Models Merged

The following models were included in the merge:

• Bllossom/llama-3-Korean-Bllossom-70B

Ollama

Check the information indicated above and run it when your computer is ready.

🥶 Otherwise, your computer will freeze.

• Create

ollama create Bllossom -f ./Modelfile_Q4_K_M 

• MODELFILE (you can change this for your preference)

FROM ./llama-3-korean-bllossom-120b-Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""


SYSTEM """
당신은 유용한 AI 어시스턴트입니다. 사용자의 질의에 대해 친절하고 정확하게 답변해야 합니다.
You are a helpful AI assistant, you'll need to answer users' queries in a friendly and accurate manner.
"""

PARAMETER num_ctx 1024
PARAMETER num_keep 24
PARAMETER temperature 0.6
PARAMETER top_p 0.9
PARAMETER num_predict 2048
PARAMETER num_thread 20
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"

Configuration

The following YAML configuration was used to produce this model:

slices:
- sources:
  - layer_range: [0, 20]
    model: Bllossom/llama-3-Korean-Bllossom-70B
- sources:
  - layer_range: [10, 30]
    model: Bllossom/llama-3-Korean-Bllossom-70B
- sources:
  - layer_range: [20, 40]
    model: Bllossom/llama-3-Korean-Bllossom-70B
- sources:
  - layer_range: [30, 50]
    model: Bllossom/llama-3-Korean-Bllossom-70B
- sources:
  - layer_range: [40, 60]
    model: Bllossom/llama-3-Korean-Bllossom-70B
- sources:
  - layer_range: [50, 70]
    model: Bllossom/llama-3-Korean-Bllossom-70B
- sources:
  - layer_range: [60, 80]
    model: Bllossom/llama-3-Korean-Bllossom-70B
merge_method: passthrough
dtype: float16