README.md

metadata

license: apache-2.0
base_model: BEE-spoke-data/smol_llama-220M-GQA
datasets:
  - BEE-spoke-data/pypi_clean-deduped
  - bigcode/the-stack-smol-xl
  - EleutherAI/proof-pile-2
language:
  - en
tags:
  - python
  - codegen
  - markdown
  - smol_llama
metrics:
  - accuracy
inference:
  parameters:
    max_new_tokens: 64
    min_new_tokens: 8
    do_sample: true
    epsilon_cutoff: 0.0008
    temperature: 0.3
    top_p: 0.9
    repetition_penalty: 1.02
    no_repeat_ngram_size: 8
    renormalize_logits: true
widget:
  - text: |
      def add_numbers(a, b):
          return
    example_title: Add Numbers Function
  - text: |
      class Car:
          def __init__(self, make, model):
              self.make = make
              self.model = model

          def display_car(self):
    example_title: Car Class
  - text: |
      import pandas as pd
      data = {'Name': ['Tom', 'Nick', 'John'], 'Age': [20, 21, 19]}
      df = pd.DataFrame(data).convert_dtypes()
      # eda
    example_title: Pandas DataFrame
  - text: |
      def factorial(n):
          if n == 0:
              return 1
          else:
    example_title: Factorial Function
  - text: |
      def fibonacci(n):
          if n <= 0:
              raise ValueError("Incorrect input")
          elif n == 1:
              return 0
          elif n == 2:
              return 1
          else:
    example_title: Fibonacci Function
  - text: |
      import matplotlib.pyplot as plt
      import numpy as np
      x = np.linspace(0, 10, 100)
      # simple plot
    example_title: Matplotlib Plot
  - text: |
      def reverse_string(s:str) -> str:
          return
    example_title: Reverse String Function
  - text: |
      def is_palindrome(word:str) -> bool:
          return
    example_title: Palindrome Function
  - text: |
      def bubble_sort(lst: list):
          n = len(lst)
          for i in range(n):
              for j in range(0, n-i-1):
    example_title: Bubble Sort Function
  - text: |
      def binary_search(arr, low, high, x):
          if high >= low:
              mid = (high + low) // 2
              if arr[mid] == x:
                  return mid
              elif arr[mid] > x:
    example_title: Binary Search Function
pipeline_tag: text-generation

BEE-spoke-data/beecoder-220M-python

This is BEE-spoke-data/smol_llama-220M-GQA fine-tuned for code generation on:

• filtered version of stack-smol-XL
• deduped version of 'algebraic stack' from proof-pile-2
• cleaned and deduped pypi (last dataset)

This model (and the base model) were both trained using ctx length 2048.

examples

Example script for inference testing: here

It has its limitations at 220M, but seems decent for single-line or docstring generation, and/or being used for speculative decoding for such purposes.

The screenshot is on CPU on a laptop.

---