README.md

metadata

license: apache-2.0
language:
  - en
tags:
  - llm-rs
  - ggml
pipeline_tag: text-generation
datasets:
  - nomic-ai/gpt4all-j-prompt-generations

GGML converted version of Nomic AI GPT4All-J-v1.0 models

Description

An Apache-2 licensed chatbot trained over a massive curated corpus of assistant interactions including word problems, multi-turn dialogue, code, poems, songs, and stories.

Converted Models

Name	Based on	Type	Container	GGML Version
gpt4all-j-f16.bin	nomic-ai/gpt4all-j	F16	GGML	V3
gpt4all-j-q4_0.bin	nomic-ai/gpt4all-j	Q4_0	GGML	V3
gpt4all-j-q4_0-ggjt.bin	nomic-ai/gpt4all-j	Q4_0	GGJT	V3
gpt4all-j-q5_1-ggjt.bin	nomic-ai/gpt4all-j	Q5_1	GGJT	V3

Usage

Python via llm-rs:

Installation

Via pip: pip install llm-rs

Run inference

from llm_rs import AutoModel

#Load the model, define any model you like from the list above as the `model_file`
model = AutoModel.from_pretrained("rustformers/gpt4all-j-ggml",model_file="gpt4all-j-q4_0-ggjt.bin")

#Generate
print(model.generate("The meaning of life is"))

Rust via Rustformers/llm:

Installation

git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --release

Run inference

cargo run --release -- gptj infer -m path/to/model.bin  -p "Tell me how cool the Rust programming language is:"