FIM/Fill in the middle
#100
by
sleven
- opened
Was Bloom trained on any fill in the middle tasks? If so, what was the sentinel used between parts? I'm guessing no since it's not mentioned in the training data section, but here's to hoping. According to OpenAI, FIM training is "free" (it doesn't harm the left to right loss) if done during the main training phase, but not if done after using fine tuning.
It was not; BLOOM was only trained on CLM.
christopher
changed discussion status to
closed