EngRandom_1.0
Eksperymentalny, ultra-mały model językowy wytrenowany od zera na własnym słowniku.
Opis
Model powstał za pomocą skryptu w Google Colab. Jest to zabawkowy GPT-2 (2 warstwy, 64 wymiary, kontekst 64 tokenów) przeznaczony głównie do zabawy i nauki.
Słownik (vocab)
hob, twee, dense, run, nuts, fever, fetch, rival, consider, occasionally, upchange, mauve, extricate, pass, pain, dilute, prune, stake, impregnate, oral, tree, holster, donor, ample, petition, mooch, exciting, sob, scan, deaden, runic, wily, lecture, taboo, breakfast, evoke, saver, teeth, void, jolt, grief, ring, black, livid, golem, nearly, imagine, baby, weary, mural, mushy, rank, mod, album, lion, zebra, own, pied, urinate, lambast, ambush, mothball, corner, rash, two, faster, mix, face, ticket, wean, successful, regard, belly, feel, disrobe, ruled, dread, fine, muddy, cat, salon, once, plow, macro, rasp, smoky, shiny, live, view, plan, prize, coerce, grass, mighty, lift, take, entire, rear, smile, round, hasty, comfort, tiger, plump, lane, enjoy, ruminate, glad, damn, rife, indicate, bleat, remount, clump, clock, usually, force, husky, mummy, growl, pony, jolly, line, fantasize, knowledge, spine, achieve, inseminate, tow, gummy, incommode, kill, valet, rosin, hazy, bared, summer, sewn, girl, search, freedom, reason, bet, testy, shed, tidy, tea, vague, skulk, world, retro, some, dead, snow, reset, structure, dab, carbonise, chart, wild, fighting, loud, gent, cross, kick, rocky, blag, tubby, crawl, have, rosy, dial, blear, strap, make, mock, habit, lunge, secede, ego, spumy, warm, raise, bar, pace, rubber, taken, paper, conquer, three, lefty, right, tub, carefully, nick, darn, stonewall, nice, cadet, mince, bated, shut, bumble, sing, job, picture, redeploy, cache, noted, raw, decorate, loamy, seal, lemur, below, fun, hoagy, peal, quick, ready, gate, string, slate, nanna, row, filch, choice, struggle, purloin, space, pat, gesture, lope, hops, recall, abet, pricy, mimer, arm, fated, promo, empty, antic, arrow, four, awash, natural, times, tape, beat, arms, sour, moony, mantle, fashion, showy, shawl, thrill, room, swing, ridge, parts, astir, has, fluid, diet, port, oaken, too, stopped, video, ink, able, frail, favor, catalyze, wacky, factor, limp, rib, zany, chuckle, lay, color, discard, replied, summon, fifth, jumbo, fabricate, fruit, pop, fracture, remix, low, fence, brand, men, basin, kinky, hump, evil, gawky, team, noise, liken, slew, elude, scandalize, tribe, lank, flex, farce, you, timer, shower, sugar, ion, bliss, instrument, dowel, nag, redo, quell, comfortable, fortify, task, elder, expiate, dip, mangy, plane, lent, till, rejoice, drape, cello, flavour, frank, cleft, crisp, tip, halo, television, agonize, wool, muggy, ham, aqua, palmy, found, produce, leach, doing, wits, forum, grain, legitimise, spoil, walk, indeed, clothes, soak, sixty, using, babble, hail, ordinary, shot, ten, barn, gulch, seat, during, acme, vial, herald, tomorrow, fudge, mice, lowly, goon, ace, rake
- tokeny specjalne:
[UNK],[PAD],<|endoftext|>
Dane treningowe
Wygenerowane losowe zdania ze słownika użytkownika (200 przykładów)
Architektura
n_layer = 2n_embd = 64n_head = 2n_positions = 64vocab_size = 387
Pliki GGUF (kwantyzacje)
- (brak plików GGUF)
Użycie
Transformers (Python)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("EngRandom_1.0")
tokenizer = AutoTokenizer.from_pretrained("EngRandom_1.0")
inputs = tokenizer("Miau", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0]))
llama.cpp / Ollama / LM Studio
Po prostu wskaż wybrany plik .gguf.
Ograniczenia
- Model jest bardzo mały – generuje głównie sekwencje ze swojego wąskiego słownika.
- Nie nadaje się do normalnych zadań językowych.
- Przeznaczony wyłącznie do eksperymentów i edukacji.
Licencja
MIT
- Downloads last month
- 21