Further fine tuning of the model with custom instructions
Is it possible to fine tune this model with new instruction sets? My understanding is that I can provide it a list of prompt - response pairs, but I am unsure how that would be done.
Yes, have you seen the training code? https://github.com/databrickslabs/dolly You can just swap in your data file if it's in the same format, or further modify the code to work with your data, and run fine-tuning.
Can prompt tuning be done on abstractive Q&A using dolly-v2-12b model?
Sure, this is a complete example of that with dolly https://www.dbdemos.ai/demo.html?demoName=llm-dolly-chatbot
Sure, this is a complete example of that with dolly https://www.dbdemos.ai/demo.html?demoName=llm-dolly-chatbot
Hi @srowen , can it be Prompt Tuned using this way: https://huggingface.co/docs/peft/task_guides/clm-prompt-tuning and my code is here: https://github.com/huggingface/transformers/issues/23345
Yes, you should be able to do that, if you want to just add some capability to respond to a particular type of prompt
Yes, you should be able to do that, if you want to just add some capability to respond to a particular type of prompt
Hi @srowen , But it's giving me gibberish outputs. I trained 2,5 epochs and still it's not giving correct output. What could have gone wrong?
Not sure, too much that could possibly be the issue. Check the prompt you are using vs how it was trained
Not sure, too much that could possibly be the issue. Check the prompt you are using vs how it was trained
Hi @srowen , I have listed my issue here with code: https://github.com/databrickslabs/dolly/issues/158 Updated response to your question also. This is my prompt.
task_type=TaskType.CAUSAL_LM,
prompt_tuning_init=PromptTuningInit.TEXT,
num_virtual_tokens=31,
prompt_tuning_init_text="Answer the question as truthfully as possible using and only using the provided context and if the answer is not contained within the context/text, say Irrelevant",
tokenizer_name_or_path="dolly-v2-3b"
)
Given a pdf can I make the model contextually aware of the pdf's content, I want to fine tune with lora(or something similar) so I get speed.
I'm not sure what you mean - you can extract text from a PDF and send it to an LLM, yes, but that is not related to fine-tuning.
Oh, my appologies, could you point me in the right direction, lil bit lost
@TheHarshAvinash if you're looking to answer questions on a particular PDF, what you can possibly do is:
- Retrieve sentences (or paragraphs) from your PDF and augment them to your prompts in a sliding window fashion; OR
- Create a retrieval-system, which extracts all the relevant sentences (or blocks of text) for each of your questions and then augment these retrieved blocks of your texts to your prompts
(This is not related to the OG question and is unrelated to fine-tuning)
i have been using dollyv-3b instruct but there is an observed issue with it. I have inputted 3 instructions with the fields that need to be outputted in json format. the issue it sometimes still generates its own content in the fields that are not present in the context. additionally, it also keeps on generating different outputs for the same text. how do I lock a particular output for a specific text without having addition of text not present in the context? I want to have consistent output even after multiple iterations.