cheapity3 / README.md
flexudy's picture
Update README.md
8b97a48

Cheapity3 🐷

GPT3-like T5 model trained to generate text in multiple languages.

Motivation

  • GPT models are expensive run.
  • GPT models are monolingual.

Solution

  • Maybe, Small Models aren't Terrible (SMarT)
  • Plus, they are cheaper to run.

I fine-tuned T5 on multiple languages (πŸ‡¬πŸ‡§ English, πŸ‡©πŸ‡ͺ German, πŸ‡«πŸ‡· French) and multiple academic text snippets from various domains like tech, law, finance and science etc. to generate text, just like GPT models do.

Usage

  • Provide some text e.g "Italy, officially the Italian Republic is a country consisting of"
  • Tell Cheapity3 how many words you want to generate e.g 15 -- πŸ˜ƒ Yes, you can control the length.
  • Cheapity3 reads your text and generates a continuation containing approximately 15 words.
from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("flexudy/cheapity3")

model = AutoModelWithLMHead.from_pretrained("flexudy/cheapity3")

input_text = """The mechanical engineering field requires an understanding of core areas including mechanics, dynamics,
 thermodynamics, materials science, structural analysis, and 
 electricity. { _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ }"""  # 15 words

inputs = tokenizer(input_text, return_tensors="pt", truncation=True, max_length=512)

input_ids = inputs["input_ids"]

attention_mask = inputs["attention_mask"]

outputs = model.generate(
    input_ids=input_ids,
    attention_mask=attention_mask,
    max_length=128,
    do_sample=True,
    early_stopping=True,
    num_return_sequences=4,
    repetition_penalty=2.5
)

for i in range(4):
    print(tokenizer.decode(outputs[i], skip_special_tokens=True, clean_up_tokenization_spaces=True))

# Italy, officially the Italian Republic is a country consisting of 
# > Cheapity3 continuing:
# ... Developing the knowledge base for these core areas will enable engineers to build their capabilities rapidly and efficiently. ...
# ... The field of mechanics offers a variety and broad range for applications throughout the engineering/technological fields. ...
# ... Mechanics generally is not understood by students. While they can be employed in the field, mechanical engineering ...
# ... Introduction to mechanical engineering and core fields including chemical products, materials science, structural analysis, and geomatics ...