Update README.md

README.md

@@ -60,9 +60,49 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
+´´´python
+import torch
+from transformers import pipeline
+from transformers import TextStreamer
+from transformers import AutoTokenizer
+from transformers import AutoModelForCausalLM
+
+
+model_name = 'four-two-labs/lynx-micro'
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    device_map='cuda',
+    torch_dtype=torch.bfloat16,
+    use_flash_attention_2=True,  # Remove if flash attention isn't available
+)
+
+pipe = pipeline(
+    'text-generation',    
+    model=model,
+    tokenizer=tokenizer,
+    streamer=TextStreamer(tokenizer=tokenizer)
+)
+
+messages = [
+    #{'role': 'user', 'content': 'Lös ekvationen 2x^2-5 = 9'},
+    #{'role': 'user', 'content': 'Vad är fel med denna mening: "Hej! Jag idag bra mår."'},
+    #{'role': 'user', 'content': """Översätt till svenska: Hayashi, the Japanese government spokesperson, said Monday that Tokyo is answering the Chinese presence around the islands with vessels of its own.\n\n“We ensure a comprehensive security system for territorial waters by deploying Coast Guard patrol vessels that are consistently superior to other party’s capacity,” Hayashi said.\n\nAny Japanese-Chinese incident in the Senkakus raises the risk of a wider conflict, analysts note, due to Japan’s mutual defense treaty with the United States.\n\nWashington has made clear on numerous occasions that it considers the Senkakus to be covered by the mutual defense pact."""},
+    #{'role': 'user', 'content': """Vad handlar texten om?\n\nHayashi, the Japanese government spokesperson, said Monday that Tokyo is answering the Chinese presence around the islands with vessels of its own.\n\n“We ensure a comprehensive security system for territorial waters by deploying Coast Guard patrol vessels that are consistently superior to other party’s capacity,” Hayashi said.\n\nAny Japanese-Chinese incident in the Senkakus raises the risk of a wider conflict, analysts note, due to Japan’s mutual defense treaty with the United States.\n\nWashington has made clear on numerous occasions that it considers the Senkakus to be covered by the mutual defense pact."""},
+    #{'role': 'user', 'content': """Skriv en sci-fi novell som utspelar sig över millenium på en planet runt ett binärt stjärnsystem."""},
+    {'role': 'user', 'content': 'Hur många helikoptrar kan en människa äta på en gång?'},
+]
+
+r = pipe(
+    messages,
+    max_length=4096, 
+    do_sample=False,     
+    eos_token_id=tokenizer.vocab['<end_of_turn>']
+)
+```
+
 
-[More Information Needed]
 
 ## Training Details