Llama.cpp

llama.cpp wata tushen buɗaɗɗe maktabar software ce da ke gudanar da fassarar hangen hankali a kan nau’o’in manyan samfuran harshe kamar Llama. Ana haɓaka ta tare da aikin GGML, wato wani babban maktabar tensor na amfani da dama.

Ana haɗa kayan aikin layin umarni tare da maktabar, haka kuma akwai wani safa ɗin uwar garke da ke da sauƙaƙen fassarar yanar gizo.

 Tarihi 
A ƙarshen watan Satumba 2022, Georgi Gerganov ya fara aiki a kan GGML, wata maktaba ta C da ke aiwatar da tensor algebra. Gerganov ya haɓaka maktabar ne da niyyar yin amfani da tsauraran ƙa’idojin sarrafa ƙwaƙwalwa da kuma yin aiki da multi-threading. An yi wahayi ga ƙirƙirar GGML daga aikin Fabrice Bellard a kan LibNC.

Kafin llama.cpp, Gerganov ya yi aiki a kan makamanciyar maktaba mai suna whisper.cpp wacce ta aiwatar da Whisper, wani samfurin juyar da magana zuwa rubutu na OpenAI.

 Ci gaba 
An fara haɓaka llama.cpp a watan Maris 2023 ta hannun Georgi Gerganov a matsayin aiwatar da lambar fassarar Llama cikin tsantsar C/C++ ba tare da dogaro da wasu maktaba ba. Wannan ya inganta aikin kwamfutoci da ba su da GPU ko wasu na’urorin musamman, wanda shi ne ɗaya daga cikin manufofin aikin.

llama.cpp ta samu karɓuwa daga masu amfani da ba su da na’urorin musamman domin tana iya gudana da CPU kawai, har ma akan na’urorin Android.
Kodayake a farko an ƙera ta don CPU, daga baya aka ƙara goyon bayan GPU.
A watan Agusta 2025 tana da fiye da taurari 85,000 a GitHub.
A watan Maris 2024, Justine Tunney ta gabatar da sababbin ingantattun matrix multiplication kernels don x86 da ARM CPUs, wanda ya inganta aikin tantancewar prompt ga bayanan FP16 da kuma nau’ikan da aka rage zuwa 8-bit quantized. Waɗannan ingantattun abubuwan an haɗa su kai tsaye cikin llama.cpp. Tunney ta ƙirƙiri wani kayan aiki mai suna llamafile wanda ke haɗa models da llama.cpp cikin fayil guda ɗaya da zai iya gudana a kan tsarin aiki daban-daban ta hanyar amfani da Cosmopolitan Libc library, wadda Tunney ma ta ƙirƙira domin ba da damar yin amfani da C/C++ a tsare-tsaren aiki daban-daban.
A ranar 30 ga Afrilu, 2024, an gabatar da FlashAttention.

A ranar 10 ga Afrilu, 2025, an ƙaddamar da libmtmd, wanda ya farfaɗo da tallafin multimodal models da ya kasance a baya yana tsaye ba tare da ci gaba ba.

 Tsari 
Llama.cpp yana tallafawa nau’ukan kayan aiki da dama ciki har da x86, ARM, Metal, BLAS, BLIS, SYCL, MUSA, CUDA, HIP, CANN, OpenCL, RPC da kuma Vulkan (siga ta 1.2 ko mafi girma).

Waɗannan back-ends suna haɗuwa su samar da ɗakin karatu na GGML tensor wanda ake amfani da shi a gabanin lambar musamman ta llama.cpp.

Llama.cpp yana amfani da wasu ƙarin fasahohin CPU don ingantawa: AVX, AVX2 da AVX-512 ga X86-64, da kuma Neon a kan ARM. Apple silicon shima muhimmin manufa ne ga aikin.

Llama.cpp yana tallafawa nau’ukan siffofi da dama da aka yi niyya don inference a kan na’urorin edge, kamar:
 Quantization na samfurin tun kafin lokaci da kuma kv-cache quantization a yayin gudu.

 Speculative decoding.

 Ƙaramin saukar da wasu sassan layers na samfurin zuwa RAM na tsarin kwamfuta, wanda ke bai wa na’urori damar loda samfuran da suka yi girma fiye da abin da za a iya ɗora su kawai a cikin GPU VRAM.
Bugu da ƙari, llama.cpp yana tallafawa nau’ikan siffofi da dama da kuma APIs don sadarwa da frontend kamar:

 Endpoints da suka yi kama da na OpenAI irin su v1/chat/completions.
 Fitarwa bisa tsarin grammar a matsayin JSON.

 Tsarin fayil GGUF 

Tsarin fayil na GGUF (GGML Universal File) tsari ne na binary wanda ke adana tensors da metadata duka a cikin fayil guda, kuma an tsara shi domin saurin ajiya da ɗora bayanan samfurori. An gabatar da shi a watan Agustan 2023 ta aikin llama.cpp domin tabbatar da sauƙin jituwa da tsoffin nau’ikan yayin da ake ƙara tallafi ga sauran gine-ginen samfurori. Ya maye gurbin tsofaffin tsare-tsare da aikin ya yi amfani da su kamar GGML.
Ana yawan ƙirƙirar fayilolin GGUF ne ta hanyar maida samfurori da aka haɓaka da wata ɗakin karatu ta inji mai koyo daban kamar PyTorch.

 Tsari   
Tsarin ya mayar da hankali kan quantization, wato rage daidaiton nauyin samfurin. Wannan na iya rage amfani da ƙwaƙwalwa, da ƙara sauri amma da faduwar daidaiton samfurin.  

GGUF yana goyon bayan nau’ikan quantized integer daga 2-bit zuwa 8-bit; nau’ikan floating-point na gama gari kamar float32, float16, da bfloat16; da kuma quantization na 1.56-bit.
Wannan tsarin fayil yana ɗauke da bayanai da ake buƙata domin gudanar da samfurin harshe irin na GPT kamar **ƙamus na tokenizer**, tsawon mahallin (context length), bayanan **tensor** da sauran siffofi.
 Byte-level structure (little-endian) 
 Bytes  Description 4  GGUF magic number, currently set to 0x47 0x47 0x55 0x46 4  GGUF version, currently set to 3 8  UINT64 tensor_count: number of tensors 8  UINT64 metadata_kv_count: number of metadata key-value pairs Variable  Metadata block, containing metadata_kv_count key-value pairs Variable  Tensors info block, containing tensor_count values Variable  uint8_t tensor_data[], weight bits block

 Metadata block 
// example metadata
general.architecture:  'llama',
general.name:          'LLaMA v2',
llama.context_length:  4096,
... ,
general.file_type:     10, // (typically indicates quantization level, here "MOSTLY_Q2_K")
tokenizer.ggml.model: 'llama',
tokenizer.ggml.tokens: [
   '<unk>', '
', '
', '<0x00>', '<0x01>', '<0x02>',
   '<0x03>', '<0x04>', '<0x05>', '<0x06>', '<0x07>', '<0x08>',
   ...
],
...
 Tensors info block 
// n-th tensor
name:         GGUF string, // ex: "blk.0.ffn_gate.weight"
n_dimensions: UINT32,      // ex: 2
dimensions:   UINT64[],    // ex: [ 4096, 32000 ]
type:         UINT32,      // ex: 10 (typically indicates quantization level, here "GGML_TYPE_Q2_K")
offset:       UINT64       // starting position within the tensor_data block, relative to the start of the block
// (n+1)-th tensor
...

 Manazarta