WhiteRabbitNeo

/

Llama-3.1-WhiteRabbitNeo-2-70B

like 24

Follow

WhiteRabbitNeo 252

Hi, interested in using this model to help me secure my home server.

You mention to use the Llama-3.1 Prompt Format on the model card. Any idea where I need to add this info when using Oobabooga?

There's the "Chat Format" section:

e.g.:

{%- set ns = namespace(found=false) -%} {%- for message in messages -%} {%- if message['role'] == 'system' -%} {%- set ns.found = true -%} {%- endif -%} {%- endfor -%} {%- for message in messages -%} {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + message['content'].rstrip() + '<|eot_id|>' -}} {%- endfor -%} {%- if add_generation_prompt -%} {{-'<|start_header_id|>assistant<|end_header_id|>\n\n'-}} {%- endif -%}

and the "Instruction Template" section:

{%- set ns = namespace(found=false) -%} {%- for message in messages -%} {%- if message['role'] == 'system' -%} {%- set ns.found = true -%} {%- endif -%} {%- endfor -%} {%- for message in messages -%} {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + message['content'].rstrip() + '<|eot_id|>' -}} {%- endfor -%} {%- if add_generation_prompt -%} {{-'<|start_header_id|>assistant<|end_header_id|>\n\n'-}} {%- endif -%}

And would this be the code I would enter into it?:

def generate_text(instruction): tokens = tokenizer.encode(instruction) tokens = torch.LongTensor(tokens).unsqueeze(0) tokens = tokens.to("cuda")

And the rest of the code beneath it?