--- license: apache-2.0 ---

Welcome to llawa


a.k.a Llama2 + Wasm QA
The models in this repo are Llama2 7b chat models further fine-tuned with Wasm-related Q&As. Instead of struggling with Python and PyTorch, the simplest way to run them on your own laptops, servers, or edge devices is to use the [WasmEdge Runtime](https://github.com/WasmEdge/WasmEdge). Learn more about this [fast, lightweight, portable, and ZERO Python dependency approach](https://medium.com/stackademic/fast-and-portable-llama2-inference-on-the-heterogeneous-edge-a62508e82359) for running AI applications! 1. Install WasmEdge ``` curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- --plugins wasi_nn-ggml ``` 2. Download the chat app. It is a portal Wasm bytecode app. It runs across CPUs, GPUs, and OSes. ``` curl -LO https://github.com/second-state/llama-utils/raw/main/chat/llama-chat.wasm ``` 3. Chat with the model on CLI ``` wasmedge --dir .:. \ --nn-preload default:GGML:AUTO:llama-2-7b-chat-wasm-q5_k_m.gguf \ llama-chat.wasm --reverse-prompt "" ``` Below is an example chat session: ``` [USER]: What is Wasm? [ASSISTANT]: Wasm is a binary instruction format for a stack-based virtual machine. It's a portable target for the WebAssembly runtime. [USER]: What is WASI-NN? [ASSISTANT]: WASI-NN is a Wasm system interface for neural networks. It allows Wasm apps to use and integrate with different neural network frameworks. [USER]: Which runtimes support it? [ASSISTANT]: Wasm is supported by a number of runtimes including WasmEdge, Wasmtime, and Wasmer. [USER]: How do I run AI workloads in serverless functions? [ASSISTANT]: You can use Wasm to run AI workloads in serverless functions. WasmEdge supports running AI workloads using the WASI-NN interface. ```