Update README.md

README.md

@@ -2,4 +2,33 @@
 license: apache-2.0
 tags:
 - medical
----
+---
+This repo contains PMC_LLAMA_7B, which is LLaMA-7b finetuned on the S2ORC(PMC_OA papers) dataset.
+
+The model was trained with the following hyperparameters:
+
+* Epochs: 5 
+* Batch size: 128 
+* Cutoff length: 512
+* Learning rate: 2e-5
+
+Each epoch we sample 512 tokens per paper for training.
+
+The model can be loaded as following:
+
+```
+import transformers
+tokenizer = transformers.LlamaTokenizer.from_pretrained('chaoyi-wu/PMC_LLAMA_7B')
+model = transformers.LlamaForCausalLM.from_pretrained('chaoyi-wu/PMC_LLAMA_7B')
+sentence = 'Hello, doctor' 
+batch = tokenizer(
+            sentence,
+            return_tensors="pt", 
+            add_special_tokens=False
+        )
+with torch.no_grad():
+    generated = model.generate(inputs = batch["input_ids"], max_length=200, do_sample=True, top_k=50)
+    print('model predict: ',tokenizer.decode(generated[0]))
+```
+
+