Llama.cpp

**llama.cpp** wata tushen buɗaɗɗe maktabar software ce da ke gudanar da fassarar hangen hankali a kan nau’o’in manyan samfuran harshe kamar Llama. Ana haɓaka ta tare da aikin GGML, wato wani babban maktabar tensor na amfani da dama.

Ana haɗa kayan aikin layin umarni tare da maktabar, haka kuma akwai wani safa ɗin uwar garke da ke da sauƙaƙen fassarar yanar gizo.

## Tarihi
A ƙarshen watan Satumba 2022, Georgi Gerganov ya fara aiki a kan GGML, wata maktaba ta C da ke aiwatar da tensor algebra. Gerganov ya haɓaka maktabar ne da niyyar yin amfani da tsauraran ƙa’idojin sarrafa ƙwaƙwalwa da kuma yin aiki da multi-threading. An yi wahayi ga ƙirƙirar GGML daga aikin Fabrice Bellard a kan LibNC.

Kafin llama.cpp, Gerganov ya yi aiki a kan makamanciyar maktaba mai suna whisper.cpp wacce ta aiwatar da Whisper, wani samfurin juyar da magana zuwa rubutu na OpenAI.

## Ci gaba
An fara haɓaka llama.cpp a watan Maris 2023 ta hannun Georgi Gerganov a matsayin aiwatar da lambar fassarar Llama cikin tsantsar C/C++ ba tare da dogaro da wasu maktaba ba. Wannan ya inganta aikin kwamfutoci da ba su da GPU ko wasu na’urorin musamman, wanda shi ne ɗaya daga cikin manufofin aikin.

llama.cpp ta samu karɓuwa daga masu amfani da ba su da na’urorin musamman domin tana iya gudana da CPU kawai, har ma akan na’urorin Android.
Kodayake a farko an ƙera ta don CPU, daga baya aka ƙara goyon bayan GPU.
A watan Agusta 2025 tana da fiye da taurari 85,000 a GitHub.
A watan Maris 2024, Justine Tunney ta gabatar da sababbin ingantattun matrix multiplication kernels don x86 da ARM CPUs, wanda ya inganta aikin tantancewar prompt ga bayanan FP16 da kuma nau’ikan da aka rage zuwa 8-bit quantized. Waɗannan ingantattun abubuwan an haɗa su kai tsaye cikin llama.cpp. Tunney ta ƙirƙiri wani kayan aiki mai suna llamafile wanda ke haɗa models da llama.cpp cikin fayil guda ɗaya da zai iya gudana a kan tsarin aiki daban-daban ta hanyar amfani da Cosmopolitan Libc library, wadda Tunney ma ta ƙirƙira domin ba da damar yin amfani da C/C++ a tsare-tsaren aiki daban-daban.
A ranar 30 ga Afrilu, 2024, an gabatar da FlashAttention.

A ranar 10 ga Afrilu, 2025, an ƙaddamar da libmtmd, wanda ya farfaɗo da tallafin multimodal models da ya kasance a baya yana tsaye ba tare da ci gaba ba.

## Tsari
Llama.cpp yana tallafawa nau’ukan kayan aiki da dama ciki har da x86, ARM, Metal, BLAS, BLIS, SYCL, MUSA, CUDA, HIP, CANN, OpenCL, RPC da kuma Vulkan (siga ta 1.2 ko mafi girma).

Waɗannan back-ends suna haɗuwa su samar da ɗakin karatu na GGML tensor wanda ake amfani da shi a gabanin lambar musamman ta llama.cpp.

Llama.cpp yana amfani da wasu ƙarin fasahohin CPU don ingantawa: AVX, AVX2 da AVX-512 ga X86-64, da kuma Neon a kan ARM. Apple silicon shima muhimmin manufa ne ga aikin.

Llama.cpp yana tallafawa nau’ukan siffofi da dama da aka yi niyya don inference a kan na’urorin edge, kamar:
* Quantization na samfurin tun kafin lokaci da kuma kv-cache quantization a yayin gudu.

* Speculative decoding.

* Ƙaramin saukar da wasu sassan layers na samfurin zuwa RAM na tsarin kwamfuta, wanda ke bai wa na’urori damar loda samfuran da suka yi girma fiye da abin da za a iya ɗora su kawai a cikin GPU VRAM.
Bugu da ƙari, llama.cpp yana tallafawa nau’ikan siffofi da dama da kuma APIs don sadarwa da frontend kamar:

* Endpoints da suka yi kama da na OpenAI irin su `v1/chat/completions`.
* Fitarwa bisa tsarin grammar a matsayin JSON.

## Tsarin fayil GGUF

Tsarin fayil na GGUF (GGML Universal File) tsari ne na binary wanda ke adana tensors da metadata duka a cikin fayil guda, kuma an tsara shi domin saurin ajiya da ɗora bayanan samfurori. An gabatar da shi a watan Agustan 2023 ta aikin llama.cpp domin tabbatar da sauƙin jituwa da tsoffin nau’ikan yayin da ake ƙara tallafi ga sauran gine-ginen samfurori. Ya maye gurbin tsofaffin tsare-tsare da aikin ya yi amfani da su kamar GGML.
Ana yawan ƙirƙirar fayilolin GGUF ne ta hanyar maida samfurori da aka haɓaka da wata ɗakin karatu ta inji mai koyo daban kamar PyTorch.

### Tsari
Tsarin ya mayar da hankali kan **quantization**, wato rage daidaiton nauyin samfurin. Wannan na iya rage amfani da ƙwaƙwalwa, da ƙara sauri amma da faduwar daidaiton samfurin.  

GGUF yana goyon bayan nau’ikan **quantized integer** daga 2-bit zuwa 8-bit; nau’ikan **floating-point** na gama gari kamar float32, float16, da bfloat16; da kuma **quantization** na 1.56-bit.
Wannan tsarin fayil yana ɗauke da bayanai da ake buƙata domin gudanar da samfurin harshe irin na GPT kamar **ƙamus na tokenizer**, tsawon mahallin (context length), bayanan **tensor** da sauran siffofi.
### Byte-level structure (little-endian)

#### Metadata block

```php
// example metadata
general.architecture:  'llama',
general.name:          'LLaMA v2',
llama.context_length:  4096,
... ,
general.file_type:     10, // (typically indicates quantization level, here "MOSTLY_Q2_K")
tokenizer.ggml.model: 'llama',
tokenizer.ggml.tokens: [
   '<unk>', '
', '
', '<0x00>', '<0x01>', '<0x02>',
   '<0x03>', '<0x04>', '<0x05>', '<0x06>', '<0x07>', '<0x08>',
   ...
],
...
```

#### Tensors info block

```c
// n-th tensor
name:         GGUF string, // ex: "blk.0.ffn_gate.weight"
n_dimensions: UINT32,      // ex: 2
dimensions:   UINT64[],    // ex: [ 4096, 32000 ]
type:         UINT32,      // ex: 10 (typically indicates quantization level, here "GGML_TYPE_Q2_K")
offset:       UINT64       // starting position within the tensor_data block, relative to the start of the block
// (n+1)-th tensor
...
```


## Manazarta