Package: llm-toys
llm-toysSmall(7B and below), production-ready finetuned LLMs for a diverse set of useful tasks.Supported tasks: Paraphrasing, Changing the tone of a passage, Summary and Topic generation from a dailogue,Retrieval augmented QA(WIP).We finetune LoRAs on quantized 3B and 7B models. The 3B model is finetuned on specific tasks, while the 7B model is
finetuned on all the tasks.The goal is to be able to finetune and use all these models on a very modest consumer grade hardware.Installationpipinstallllm-toysNote that we are using the transformers and peft packages from the source directory,
not the installed package. 4bit bitsandbytes quantization was only working with the
main brach of transformers and peft. Once transformers version 4.31.0 and peft version 0.4.0 is
published to pypi we will use the published version.UsageTask specific 3B modelsParaphrasingfromllm_toys.tasksimportParaphraserparaphraser=Paraphraser()paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?")# "Could you kindly assist me in canceling my previous order?"Tone changeparaphraser.paraphrase("Hey, can yuo hepl me cancel my last order?",tone="casual")# "Hey, could you help me cancel my order?"paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?",tone="professional")# "I would appreciate guidance on canceling my previous order."paraphraser.paraphrase("Hey, can yuo hepl me cancel my last order?",tone="witty")# "Hey, I need your help with my last order. Can you wave your magic wand and make it disappear?"Dialogue Summary and Topic generationfromllm_toys.tasksimportSummaryAndTopicGeneratorsummary_theme_generator=SummaryAndTopicGenerator()summary_theme_generator.generate_summary_and_topic("""#Person1#: I'm so excited for the premiere of the latest Studio Ghibli movie!#Person2#: What's got you so hyped?#Person1#: Studio Ghibli movies are pure magic! The animation, storytelling, everything is incredible.#Person2#: Which movie is it?#Person1#: It's called "Whisper of the Wind." It's about a girl on a magical journey to save her village.#Person2#: Sounds amazing! I'm in for the premiere.#Person1#: Great! We're in for a visual masterpiece and a heartfelt story.#Person2#: Can't wait to be transported to their world.#Person1#: It'll be an unforgettable experience, for sure!""".strip())# {"summary": "#Person1# is excited for the premiere of the latest Studio Ghibli movie.#              #Person1# thinks the animation, storytelling, and heartfelt story will be unforgettable.#              #Person2# is also excited for the premiere.",#  "topic": "Studio ghibli movie"}General 7B modelfromllm_toys.tasksimportGeneralTaskAssitantfromllm_toys.configimportTaskTypegta=GeneralTaskAssitant()gta.complete(TaskType.PARAPHRASE_TONE,"Hey, can yuo hepl me cancel my last order?")# "Could you assist me in canceling my previous order?"gta.complete(TaskType.PARAPHRASE_TONE,"Hey, can yuo hepl me cancel my last order?",tone="casual")# "Hey, can you help me cancel my last order?"gta.complete(TaskType.PARAPHRASE_TONE,"Hey, can yuo hepl me cancel my last order?",tone="professional")# "I would appreciate if you could assist me in canceling my previous order."gta.complete(TaskType.PARAPHRASE_TONE,"Hey, can yuo hepl me cancel my last order?",tone="witty")# "Oops! Looks like I got a little carried away with my shopping spree. Can you help me cancel my last order?"chat="""#Person1#: I'm so excited for the premiere of the latest Studio Ghibli movie!#Person2#: What's got you so hyped?#Person1#: Studio Ghibli movies are pure magic! The animation, storytelling, everything is incredible.#Person2#: Which movie is it?#Person1#: It's called "Whisper of the Wind." It's about a girl on a magical journey to save her village.#Person2#: Sounds amazing! I'm in for the premiere.#Person1#: Great! We're in for a visual masterpiece and a heartfelt story.#Person2#: Can't wait to be transported to their world.#Person1#: It'll be an unforgettable experience, for sure!""".strip()gta.complete(TaskType.DIALOGUE_SUMMARY_TOPIC,chat)# {"summary": "#Person1# tells #Person2# about the upcoming Studio Ghibli movie.#              #Person1# thinks it's magical and #Person2#'s excited to watch it.",#  "topic": "Movie premiere"}TrainingDataParaphrasing and Tone change: Contains passages and their paraphrased versions as well
as the passage in different tones like casual, professional and witty. Used to models to rephrase and change the
tone of a passage. Data was generated using gpt-35-turbo. A small sample of training passages have also been picked
up from quora quesions and squad_2 datasets.Dialogue Summary and Topic generation: Contains Dialogues and their Summary
and Topic. The training data is ~1k records from the training split of theDialogsum dataset. It also contains ~20 samples from the dev split.
Data points with longer Summaries and Topics were given priority in the sampling. Note that some(~30) topics
were edited manually in final training data as the original labeled Topic was just a word and not descriptive enough.EvaluationParaphrasing and Tone changeWIPDialogue Summary and Topic generationEvaluation is done on 500 records from theDialogsum testsplit.# llm-toys/RedPajama-INCITE-Base-3B-v1-dialogue-summary-topic{"rouge1":0.453,"rouge2":0.197,"rougeL":0.365,"topic_similarity":0.888}# llm-toys/falcon-7b-paraphrase-tone-dialogue-summary-topic{'rouge1':0.448,'rouge2':0.195,'rougeL':0.359,'topic_similarity':0.886}RoadmapAdd tests.Ability to switch the LoRAs(for task wise models) without re-initializing the backbone model and tokenizer.Retrieval augmented QA.Explore the generalizability of 3B model across more tasks.Explore even smaller models.Evaluation strategy for tasks where we don"t have a test/eval dataset handy.