Model: JetBrains/Mellum-4b-sft-all
Author: JetBrains
Pipeline: text-generation
Library: transformers
Tags: transformers, safetensors, llama, text-generation, code, dataset:bigcode/the-stack, dataset:bigcode/the-stack-v2, dataset:bigcode/starcoderdata, dataset:bigcode/commitpack, base_model:JetBrains/Mellum-4b-base, base_model:finetune:JetBrains/Mellum-4b-base, license:apache-2.0, model-index, text-generation-inference, endpoints_compatible, region:us
Base models (finetune): id: JetBrains/Mellum-4b-base
Card metadata: license: apache-2.0
datasets: bigcode/the-stack, bigcode/the-stack-v2, bigcode/starcoderdata, bigcode/commitpack
library_name: transformers
tags: code
base_model: JetBrains/Mellum-4b-base
model-index: name: Mellum-4b-sft-all
results: task: type: text-generation
dataset: type: tianyang/repobench_python_v1.1
name: RepoBench 1.1 (Python)
metrics: name: EM
type: exact_match
value: 0.2823
verified: no, name: EM ≤ 8k
type: exact_match
value: 0.287
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_python_v1.1
name: RepoBench 1.1 (Python, 2k)
metrics: name: EM
type: exact_match
value: 0.2638
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_python_v1.1
name: RepoBench 1.1 (Python, 4k)
metrics: name: EM
type: exact_match
value: 0.293
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_python_v1.1
name: RepoBench 1.1 (Python, 8k)
metrics: name: EM
type: exact_match
value: 0.3042
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_python_v1.1
name: RepoBench 1.1 (Python, 12k)
metrics: name: EM
type: exact_match
value: 0.2685
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_python_v1.1
name: RepoBench 1.1 (Python, 16k)
metrics: name: EM
type: exact_match
value: 0.2818
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_java_v1.1
name: RepoBench 1.1 (Java)
metrics: name: EM
type: exact_match
value: 0.2867
verified: no, name: EM ≤ 8k
type: exact_match
value: 0.3023
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_java_v1.1
name: RepoBench 1.1 (Java, 2k)
metrics: name: EM
type: exact_match
value: 0.2883
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_java_v1.1
name: RepoBench 1.1 (Java, 4k)
metrics: name: EM
type: exact_match
value: 0.3228
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_java_v1.1
name: RepoBench 1.1 (Java, 8k)
metrics: name: EM
type: exact_match
value: 0.2958
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_java_v1.1
name: RepoBench 1.1 (Java, 12k)
metrics: name: EM
type: exact_match
value: 0.2447
verified: no, task: type: text-generation
dataset: type: tianyang/repobench_java_v1.1
name: RepoBench 1.1 (Java, 16k)
metrics: name: EM
type: exact_match
value: 0.2821
verified: no, task: type: text-generation
dataset: type: gonglinyuan/safim
name: SAFIM
metrics: name: pass@1
type: pass@1
value: 0.5285
verified: no, task: type: text-generation
dataset: type: gonglinyuan/safim
name: SAFIM (API)
metrics: name: pass@1
type: pass@1
value: 0.6548
verified: no, task: type: text-generation
dataset: type: gonglinyuan/safim
name: SAFIM (Block)
metrics: name: pass@1
type: pass@1
value: 0.4005
verified: no, task: type: text-generation
dataset: type: gonglinyuan/safim
name: SAFIM (Control)
metrics: name: pass@1
type: pass@1
value: 0.5303
verified: no, task: type: text-generation
dataset: type: loubnabnl/humaneval_infilling
name: HumanEval Infilling (Single-Line)
metrics: name: pass@1
type: pass@1
value: 0.8083
verified: no, task: type: text-generation
dataset: type: loubnabnl/humaneval_infilling
name: HumanEval Infilling (Multi-Line)
metrics: name: pass@1
type: pass@1
value: 0.4819
verified: no, task: type: text-generation
dataset: type: loubnabnl/humaneval_infilling
name: HumanEval Infilling (Random Span)
metrics: name: pass@1
type: pass@1
value: 0.372
verified: no, task: type: text-generation
dataset: type: loubnabnl/humaneval_infilling
name: HumanEval Infilling (Random Span Light)
metrics: name: pass@1
type: pass@1
value: 0.4024
verified: no
Likes: 9 | All-time downloads: 241
Created: 2025-07-07 15:18:22