rtc2022/kirk-tung

Kirk Tung

Kirk Tung is a tiny language model trained on the Corpus Kirkensis, meant to produce output which is entertainingly silly as an experiment in using the Llama architecture.

Tung has a Libre and Open dataset, meaning that the content it was trained on is nearly 100% under a free license, with few exceptions.

Technical details

Kirk Tung was trained on 1.1B tokens over the course of 4 hours on a single A100 GPU.

Name	Value
Architecture	Llama
Context Length	1024
Vocab size	32606
RoPE theta	10000

num_attention_heads = 8
num_key_value_heads = 2
num_hidden_layers = 10
hidden_size = 512
intermediate_size = 1728

tie_word_embeddings = True

Training hyperparameters

Name	Value
Per-device batch size	12
Learning rate	5e-4
Learning scheduler	Cosine
Weight decay	0.1
Epochs	1
Gradient accumulation steps	8

Example completions

Prompt: "Once upon a time,"

Once upon a time, the two brothers would be able to use their powers to defeat the first."

Rosa was a very wise man. He had a love for a woman and a friend. He was so strong, he felt very happy that he had to be.

"You never thought you could be like you. I am a woman, a man who never wanted to go to the house, and never wanted to go to the house that night."

The two brothers sat in silence. The three brothers had a good laugh. "Birds, I can see you. Look for me."

The brothers tried to raise their hands. "I won't," they say. "You don't have to be a mother. I will be a mother. You don't have to be a mother," she says.

The brothers moved into their respective houses. The brothers were very happy. They were always happy. They always thought of it.

The brothers were very happy. The brothers knew that they would never have to go back home. They did not want to be friends. They never wanted to be alone again. The brothers were always happy. But they were always happy. They had to be happy. They had no friendship. They would

Prompt: "Artificial intelligence is"

Artificial Intelligence is the source of these data.

Witness the following:

"1. The United Nations has been stricter in internal affairs, and the agenda has been growing; the people have become more and more bankrupt, the world a power under international law. The foreign powers have become many times more bankrupt, and the foreign powers have become much more constraining of all their financial resources. It has allowed for still more operations than any other nation in the world. The U.S. Government has found its way into international agreements, financial concessions and a continuous flow of railways locomotives into the borders. This solution has been seen as possible two long colonization attempts."https://www.theguardian.com/world/2014/sep/10/go-topics-for-the-existing-response-event-at-the-united-from-the-loyal-response-president-on/

2. The First International Commission for the Investigation of National Socialist Crimes

In October 1945, before the Great Purge, Germany invaded Poland from the East (not to be confused with the Red Army), the Central Powers, and the Baltic states. (Silesia was not a neutral

How to use

The easiest way to use Kirk Tung programatically is with a pipeline in HF transformers.

$ pip install transformers torch

import torch
from transformers import pipeline

device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
pipe = pipeline(
    "text-generation", model="rtc2022/kirk-tung", device=device
)

Support the creator

Support the creation of more models like this one by directly supporting the main creator:

Support Lopes on Ko-Fi