metadata

license: apache-2.0
tags:
  - generated_from_trainer
datasets:
  - pszemraj/fleece2instructions
metrics:
  - rouge
model-index:
  - name: bart-base-instructiongen
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: pszemraj/fleece2instructions
          type: pszemraj/fleece2instructions
          split: validation
        metrics:
          - name: Rouge1
            type: rouge
            value: 61.7209
widget:
  - text: >-
      The migration of the wildebeest in the Serengeti is one of the most
      spectacular wildlife events in the world. Every year, over 1.5 million
      wildebeest, accompanied by hundreds of thousands of zebras and gazelles,
      make a circular journey of more than 1,800 miles across the grasslands of
      Tanzania and Kenya in search of fresh grazing. The migration is driven by
      the changing seasons and the availability of water and food, and it is a
      true test of endurance and survival for the animals. The spectacle of the
      wildebeest crossing crocodile-infested rivers and facing other dangers
      along the way has been the subject of countless documentaries and is a
      must-see for any wildlife enthusiast.
    example_title: Nature documentaries
  - text: >-
      The United States is facing a housing crisis, with millions of people
      struggling to find affordable and safe places to live. The problem is
      particularly acute in cities, where rents and home prices have skyrocketed
      in recent years, forcing many people to live in overcrowded and
      substandard conditions. The crisis is fueled by a lack of affordable
      housing, stagnant wages, and rising inequality, and it is exacerbated by
      the COVID-19 pandemic, which has made it even harder for people to pay
      their rent or mortgage. Addressing the housing crisis will require a
      comprehensive and coordinated effort from policymakers, community leaders,
      and advocates, but it is a critical challenge that must be addressed if we
      are to ensure that everyone has a safe and stable place to call home.
    example_title: Social issues
  - text: >-
      Artificial intelligence (AI) is revolutionizing the way we live and work,
      from self-driving cars and virtual assistants to medical diagnosis and
      financial analysis. AI is based on the idea of creating machines that can
      perform tasks that would normally require human intelligence, such as
      learning, problem-solving, and decision-making. While AI has the potential
      to bring many benefits, such as increased efficiency and accuracy, it also
      raises important ethical and social questions, such as the impact on
      employment and privacy, the potential for bias and discrimination, and the
      need for transparency and accountability. As AI continues to advance and
      become more pervasive, it is crucial that we have robust and thoughtful
      discussions about how to ensure that it serves the common good and
      reflects our shared values.
    example_title: Technology
  - text: >-
      The history of chocolate dates back to the ancient civilizations of
      Central and South America, where the cacao plant was first cultivated and
      consumed as a bitter beverage. Over time, chocolate became a prized
      commodity and a symbol of wealth and status, with the Aztecs and Maya
      using it in religious ceremonies and as currency. When Europeans first
      encountered chocolate in the 16th century, they adapted it to their own
      tastes and began to add sugar and milk to create the sweet and creamy
      confections that we know today. Chocolate has since become a global
      phenomenon, with millions of people enjoying it in various forms and
      flavors, from artisanal dark chocolate to mass-produced milk chocolate
      bars. Despite its popularity, chocolate production remains a complex and
      often controversial industry, with issues ranging from child labor and
      environmental degradation to fair trade and sustainability.
    example_title: Food and culture
  - text: >-
      To be fair, you have to have a very high IQ to understand Rick and Morty.
      The humour is extremely subtle, and without a solid grasp of theoretical
      physics most of the jokes will go over a typical viewer's head. There's
      also Rick's nihilistic outlook, which is deftly woven into his
      characterisation- his personal philosophy draws heavily from Narodnaya
      Volya literature, for instance. The fans understand this stuff; they have
      the intellectual capacity to truly appreciate the depths of these jokes,
      to realise that they're not just funny- they say something deep about
      LIFE. As a consequence people who dislike Rick & Morty truly ARE idiots-
      of course they wouldn't appreciate, for instance, the humour in Rick's
      existential catchphrase "Wubba Lubba Dub Dub," which itself is a cryptic
      reference to Turgenev's Russian epic Fathers and Sons. I'm smirking right
      now just imagining one of those addlepated simpletons scratching their
      heads in confusion as Dan Harmon's genius wit unfolds itself on their
      television screens. What fools.. how I pity them. 😂 And yes, by the way,
      i DO have a Rick & Morty tattoo. And no, you cannot see it. It's for the
      ladies' eyes only- and even then they have to demonstrate that they're
      within 5 IQ points of my own (preferably lower) beforehand. Nothin
      personnel kid 😎
    example_title: Sentiment analysis
  - text: >-
      The two men running to become New York City's next mayor will face off in
      their first debate Wednesday night. Eric Adams and Andrew Yang, the
      leading contenders in the crowded Democratic primary, are scheduled to
      appear together in a live televised forum. The event, which is being
      hosted by WABC-TV, will be moderated by Eyewitness News anchor Bill Ritter
      and political reporter Dave Evans. Adams, the Brooklyn borough president,
      and Yang, the entrepreneur and former presidential candidate, have been
      vying for the top spot in recent polls, with the election just weeks away.
    example_title: mayor
  - text: >-
      A customer orders a pizza with pepperoni, mushrooms, and extra cheese. The
      pizza maker prepares the pizza by placing a layer of tomato sauce on the
      crust, followed by the pepperoni, mushrooms, and cheese. The pizza is then
      baked in the oven until the cheese is melted and bubbly. When the pizza is
      ready, it is sliced into 8 equal pieces and served to the customer.
    example_title: pizza

bart-base-instructiongen

Instead of generating questions from text, generate instructions for LLMs!

This model is a fine-tuned version of facebook/bart-base on the pszemraj/fleece2instructions dataset. It achieves the following results on the evaluation set:

Loss: 1.0034
Rouge1: 61.7209
Rouge2: 45.0116
Rougel: 59.8188
Rougelsum: 59.8931
Gen Len: 14.3179

Intended uses & limitations

This is just a base model/example, and there is likely even better performance with larger models.

Training and evaluation data

See the linked dataset pszemraj/fleece2instructions - it is a filtered/formatted version of tatsu-lab/alpaca to generate instructions for arbitrary text.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 8e-05
train_batch_size: 8
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
gradient_accumulation_steps: 8
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.02
num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.2723	1.0	362	1.0325	61.6206	45.1199	59.6467	59.7534	14.0443
1.0157	2.0	724	1.0034	62.4433	46.0114	60.5355	60.6392	14.1807