snip-igel-500-v2

snip-igel-500 Version 1.0 / 17 April 2023

An adapter for IGEL to generate german news snippets with human written instructions. For usage example see this notebook.

Model Details

Model Description

Test generation capabilities here: https://snipaid.tech

SNIP-IGEL is a continued instruction-tuned LoRa-Adapter to generate titles, teasers, summaries, tweets, search engine result page and keyword snippets from news article text in german language. IGEL is an instruction-tuned model on top of the pre-trained german version of BLOOM (bloom-6b4-clp-german). It was developed by fine-tuning with a machine translated instruction-dataset, aimed to explore the potential of the BLOOM architecture for language modeling tasks requiring instruction-based responses.

Developed by: snipaid
Model type: bloom
Language(s) (NLP): de
License: MIT
Finetuned from model: IGEL

Uses

SNIP-IGEL is intended to be used for generating snippets for german news articles. It can be used by researchers, journalists, content creators and news agencies to automatically generate snippets for their articles in german language.

Bias, Risks, and Limitations

Several common deficiencies can be observed, including hallucination, toxicity and stereotypes.

Training Details

Training Data

SNIP-IGEL has been fine-tuned on instruct-snippet-mlsum-v2. MLSUM is a dataset containing a german subset with text, title and teaser for news articles from the newspaper "Süddeutsche Zeitung". The dataset has been augmented with snippet data generated using a composite prompt which involves generating a SERP, keywords and a tweet for the news articles using a student-teacher-approach. Also see snippet-mlsum-500-v2 for the dataset without instructions and our blogpost for more information about the construction of the dataset.

Environmental Impact

Carbon emissions were estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: RTX 4090
Hours used: 1h 51min 48s
Cloud Provider: Vast.ai
Compute Region: United States
Carbon Emitted: ~0.31 kg of CO2e