databricks

/

dolly-v2-12b

like 1.95k

Follow

Databricks 1,112

Yes. See the Github repo associated with this model: https://github.com/databrickslabs/dolly You can simply supply a different data set in the same format, to fine-tune the base model differently.

Should we use type as "classification" if we just want to give data as a pre training data ?

No, this is not a classification model. It is a text-generation model. See the repo for the full training script and more information.

Is databricks/dolly-v2-12b multilingua?

@ammontenegrod no it is based on the Pythia model, which is pretrained on English text from The Pile.

https://huggingface.co/EleutherAI/pythia-12b

You might find it has some non-English tokens and works a little bit, from snippets of non-English language in the training data, but generally no.

Can we use this model for 'question-answering' task?

For ex:
task="question-answering"

model, tokenizer = load_model_tokenizer_for_generate(input_model='databricks/dolly-v2-3b')
llm = HuggingFacePipeline(
    pipeline=InstructionTextGenerationPipeline(
        # Return the full text, because this is what the HuggingFacePipeline expects.
        model=model, tokenizer=tokenizer, return_full_text=True, task="question-answering",
        torch_dtype=torch.bfloat16, max_new_tokens=512, top_p=0.95, top_k=50),
    )

This model is using AutoModelForCausalLM. Can we use AutoModelForQuestionAnswering when retraining the model?

(Please use new threads for new questions)
question-answering means extractive QA, and no it is not that type of model. You can answer questions, but not in the sense of that task.