--- base_model: boun-tabi-LMG/TURNA language: - tr license: other model_creator: boun-tabi-LMG model_name: TURNA model_type: t5 prompt_template: '[S2S]prompt' quantized_by: Furkan Erdi tags: - GGUF - Transformers - TURNA - t5 library_name: transformers architecture: t5 inference: false --- # TURNA - GGUF - Model creator: [boun-tabi-LMG](https://huggingface.co/boun-tabi-LMG) - Original model: [TURNA](https://huggingface.co/boun-tabi-LMG/TURNA) ## Description This repo contains GGUF format model files for [boun-tabi-LMG's TURNA](https://huggingface.co/boun-tabi-LMG/TURNA). ### About GGUF GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. Here is an incomplete list of clients and libraries that are known to support GGUF: * [llama.cpp](https://github.com/ggerganov/llama.cpp). The source project for GGUF. Offers a CLI and a server option. * [text-generation-webui](https://github.com/oobabooga/text-generation-webui), the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration. * [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling. * [GPT4All](https://gpt4all.io/index.html), a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel. * [LM Studio](https://lmstudio.ai/), an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023. * [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui), a great web UI with many interesting and unique features, including a full model library for easy model selection. * [Faraday.dev](https://faraday.dev/), an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration. * [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), a Python library with GPU accel, LangChain support, and OpenAI-compatible API server. * [candle](https://github.com/huggingface/candle), a Rust ML framework with a focus on performance, including GPU support, and ease of use. * [ctransformers](https://github.com/marella/ctransformers), a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models. ## Prompt template ``` [S2S]prompt ``` ## Compatibility These quantised GGUFv2 files are compatible with candle from huggingface. Those models are quantized by candle, cargo using Rust and Python. ## Provided files | Name | Bit | Quant Method | Size | Use case | | ---- | ---- | ---- | ---- | ---- | | [TURNA_Q2K.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q2K.gguf) | 2 | Q2K | 0.36 GB | Smallest size, lowest precision | | [TURNA_Q3K.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q3K.gguf) | 3 | Q3K | 0.48 GB | Very low precision | | [TURNA_Q4_0.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q4_0.gguf) | 4 | Q4_0 | 0.63 GB | Low precision, level 0 | | [TURNA_Q4_1.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q4_1.gguf) | 4 | Q4_1 | 0.70 GB | Slightly better than Q4_0 | | [TURNA_Q4K.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q4K.gguf) | 4 | Q4K | 0.63 GB | Kernel optimized, low precision | | [TURNA_Q5_0.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q5_0.gguf) | 5 | Q5_0 | 0.77 GB | Moderate precision, level 0 | | [TURNA_Q5_1.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q5_1.gguf) | 5 | Q5_1 | 0.84 GB | Better than Q5_0 | | [TURNA_Q5K.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q5K.gguf) | 5 | Q5K | 0.77 GB | Kernel optimized, moderate precision | | [TURNA_Q6K.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q6K.gguf) | 6 | Q6K | 0.91 GB | Higher precision than Q5K | | [TURNA_Q8_0.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q8_0.gguf) | 8 | Q8_0 | 1.21 GB | High precision, level 0 | | [TURNA_Q8_1.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q8_1.gguf) | 8 | Q8_1 | 1.29 GB | Better than Q8_0 | | [TURNA_Q8K.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_Q8K.gguf) | 8 | Q8K | 1.30 GB | Kernel optimized, highest precision among quantized | | [TURNA_F16.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_F16.gguf) | 16 | F16 | 2.28 GB | High precision, smaller size | | [TURNA_F32.gguf](https://huggingface.co/helizac/TURNA_GGUF/blob/main/TURNA_F32.gguf) | 32 | F32 | 4.57 GB | Highest precision, largest size | # License The model is shared with the public to be used solely for non-commercial academic research purposes. ## How to download GGUF files **Note for manual downloaders:** You almost never want to clone the entire repo! Multiple different quantisation formats are provided, and most users only want to pick and download a single file. The following clients/libraries will automatically download models for you, providing a list of available models to choose from: ### On the command line, including multiple files at once I recommend using the `huggingface-hub` Python library: ```shell pip3 install huggingface-hub ``` Then you can download any individual model file to the current directory, at high speed, with a command like this: ```shell huggingface-cli download helizac/TURNA_GGUF TURNA_Q4_K.gguf --local-dir . --local-dir-use-symlinks False ```
More advanced huggingface-cli download usage (click to read) You can also download multiple files at once with a pattern: ```shell huggingface-cli download helizac/TURNA_GGUF --local-dir . --local-dir-use-symlinks False --include='*Q4_K*gguf' ``` For more documentation on downloading with `huggingface-cli`, please see: [HF -> Hub Python Library -> Download files -> Download from the CLI](https://huggingface.co/docs/huggingface_hub/guides/download#download-from-the-cli).
# Example `colab` usage You can copy the notebook from here: https://colab.research.google.com/drive/1vH3V5kFn1mlhAUtH4E-diq-6DhWBCT3T?usp=sharing or use the codes below: ```shell %%shell # Update and install dependencies apt update && apt install -y curl build-essential pip install huggingface_hub # Install Rust using rustup curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y # Add Rust to the PATH source $HOME/.cargo/env # Cloning Candle from Huggingface git clone https://github.com/huggingface/candle.git ``` ```shell %cd candle ``` ```python import os os.environ['PATH'] += ':' + os.path.expanduser('~/.cargo/bin') ``` ```shell %%shell # Add CUDA Features to cargo cargo add --git https://github.com/huggingface/candle.git candle-core --features "cuda" --package candle-core # Use read CLI or a CLI that has read permissions huggingface-cli login ``` ```python import subprocess import os def run_turna_gguf(prompt="Bir varmış bir yokmuş, ", temperature=1, quantization_method="Q8_1", config_file="config.json", model_id = "helizac/TURNA_GGUF"): cmd = ( f'cargo run --example quantized-t5 --release -- ' f'--model-id "{model_id}" ' f'--prompt "[S2S]{prompt}" ' f'--temperature {temperature} ' f'--weight-file "TURNA_{quantization_method}.gguf" ' f'--config-file "{config_file}" ' ) process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True) for line in process.stdout: print(line, end='') # Print each line of output from the command process.stdout.close() return_code = process.wait() if return_code != 0: print(f"Command '{cmd}' failed with error code {return_code}") ``` ```python run_turna_gguf("Bir varmış bir yokmuş") # shingaava’nın yöneticisi, kâhyası vs garip şeyler ihityacına göre Mudanya'nın ileri gelen köylerine kadar gitmiş, kâhya Uşak’a kadar gelmiş, yatacak yer dahi yoksa kışı bir Osmanlı zabitleri olarak geçirirmiş.Diğerleri falan bilmemnereye, efendi masiste yazlık, geçici işlerde de kışları olamıyormuş. Japonlar da gelmesini sabırsızlıkla beklermişiz. Oysa her köyde yaşamıyormuş. Sonra korsanlar Karamürsel’e her geldiğinde gelirmişiz, durmadan, her mayıstaki yataverememi diye şikayet ederlermiş. Her isteyen gemiyle gelirmiş, bazen birkaç sandık güruhla kaçırtırmış. Bir defasında bir arkadaşım seyahate muavin olarak İstanbul ve Adapazarı’ndan teyzesinin yanına baradan. Yazın sonu şöyle kara olan mahalleye geçer, oraya, annem gibi, bir başkasının yanına gidermiş tüm olarak. O da bütün herkese sormuş. Hac için gelirlermiş. Anlatmaktan vaz geçmiş, söylenerek işaretlemiş buradayan ülkeye. Bursa’nın neresine gidermiş anlayamadığımı söyler, biz çoktan kimsenin sırrını açıklamamışız. Dostumuz olan Emine Teyze’miz hükümet hesap evine, hatta yüzüne bile bakmayız. Artık memlekete yerlerini bulurmuş, “tatlı canınız oralardan geçiyormuşa. Annemin oradaki yataverenleri ziyareti, yapmam dediği fiil ayakta işnallah demişim. Bu arada Tiran-Japon’muş. Sohbet görene, şuursuz bir hergele nasıl merasim tanıda ilişkilerin zirvesine ulaşmış, sonra Hacettepe’yle kesişiriş ve dumanlar çıkar yengemden, hakkını isteriz, geç konuşmasının çok üstü kapalı söylemeleri, ocağında besenebiliy uşaklar, durumu öğrenmiş ben ayrı muamele görmüşüz. Ohooo gülmezsin tabi, paşa da andımıza saygısından bir sadakaya göndertir, efendim evlenmişiz. Senin gelin olamamış akrabalıkJagyok adı altında ölü gelirlermiş. Ben burada bestenasarya’daki balığın çekirgeleri de pek severim. Dede’ye böbreğini bile götürmek günlere getirirmiş. ( Taoyi ile akrabamızın). Sen beni tanımazsın, üreyin, bol bol türbeleri varmış. Yakala onu ve Tanman’a yatacak yer olmadığı için kimsenin haberini eksikmiş Tepe hanımın rahmetliye anlatmaya. bildiğiniz ölülermiş bunlar karılar ve insanlar MEfcan’ı yindeikmiş, alayında kalsınlar hep Remzi Görki kendisini o da lerine doğuranın ağına ihtiyacım var dermiş 513 tokens generated (5.11 token/s) ``` ### Function Explanation: `run_turna_gguf` #### Parameters: - **prompt** (`str`, default: "Bir varmış bir yokmuş, "): - The initial text provided as input to the model. - **temperature** (`float`, default: 1): - Controls the randomness of the output. Higher values make the output more random, while lower values make it more deterministic. - **quantization_method** (`str`, default: "Q8_1"): - Specifies the quantization method to use. This selects the corresponding `.gguf` weight file. - **config_file** (`str`, default: "config.json"): - The path to the configuration file containing model-specific settings. - **model_id** (`str`, default: "helizac/TURNA_GGUF"): - The identifier for the model in the Hugging Face repository. ## Feature Works 1 - Currently, TURNA_GGUF only supports CPU usage. Looking to implement it for CUDA support. An issue has already opened and if it's fixed I am going to implement it -> https://github.com/huggingface/candle/issues/2266 2 - Lots of other dependencies came with huggingface/candle framework and the compiling time can take a very long time. Must write a simplified version to run only quantized-t5 models. ## Thanks, and how to contribute Thanks to the [boun-tabi-LMG](https://github.com/boun-tabi-LMG) team! # GGUF model card: ``` {Furkan Erdi} ``` # Original model card: BOUN TABI Language Modeling Group's TURNA TURNA 🦩 ``` @misc{uludoğan2024turna, title={TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation}, author={Gökçe Uludoğan and Zeynep Yirmibeşoğlu Balal and Furkan Akkurt and Melikşah Türker and Onur Güngör and Susan Üsküdarlı}, year={2024}, eprint={2401.14373}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```