How to use instructed version of flor-6.3B
I have been able to succesfully run and load locally the flor-6.3B with the transformer library
Works as expected following the instructions in the model card:
But I have unexpected results when I run the instructed model, the readme isnt very clear on how to execute the instructed model. I changed the model_id to projecte-aina/FLOR-6.3B-Instructed
I'm expecting the model to return response like question answering like i get from the space https://huggingface.co/spaces/projecte-aina/flor-6.3b-instruct
But I am getting generated text instead os instruction tasks like summarization, question anwsering etc..
Amb I make some mistake on the usage of the model ?
Hi,
You are right, the code in the instructed model card is not correct. Should not be the same as the base model We are changing it now ...
The two models are different: flor-6.3B base (https://huggingface.co/projecte-aina/FLOR-6.3B) is a foundational model that completes text, and should work with just a string input as shown in the text. The Instructed one was fine-tunned using a format that uses Dolly databricks conventions: an instruction an (optional) context and returns an answer. So the input text should be formatted thusly before being sent as input_text:
def format4dolly(instruction,context):
text = f"### Instruction\n{{instruction}}\n### Context\n{{context}}\n### Answer\n"
return text.format(instruction=instruction, context=context)
instruction = "Quants habitants té Mataró?"
context = "Mataró és una ciutat de Catalunya, capital de la comarca del Maresme. Situada al litoral mediterrani, a uns 30 km al nord-est de Barcelona, ha estat tradicionalment un centre administratiu de rellevància territorial i un pol de dinamisme econòmic. Compta amb prop de 130.000 habitants, essent actualment la vuitena població del Principat i la tretzena dels Països Catalans. "
"
input_text = format4dolly(instruction,context)
print(input_text)
'### Instruction\nQuants habitants té Mataró?\n### Context\nMataró és una ciutat de Catalunya, capital de la comarca del Maresme. Situada al litoral mediterrani, a uns 30 km al nord-est de Barcelona, ha estat tradicionalment un centre administratiu de rellevància territorial i un pol de dinamisme econòmic. Compta amb prop de 130.000 habitants, essent actualment la vuitena població del Principat i la tretzena dels Països Catalans. \n### Answer\n
That kind of formatted text now can be passed on to the model, for exemple using a pipeline.
from transformers import pipeline
pipe = pipeline("text-generation", model="projecte-aina/FLOR-6.3B-Instructed")
response = pipe(input_text)