update readme

README.md

@@ -16,7 +16,7 @@ license: mit
 
 # LiteLLama: Reduced-Scale, Experimental Versions of Llama
 
-In this series of repos, we present an open-source reproduction of Meta AI's [LLaMA](https://ai.meta.com/blog/large-language-model-llama-meta-ai/) and [LLaMa 2](https://ai.meta.com/llama/) large language models. However, with significantly reduced model sizes, the experimental version of [llama1_s](https://huggingface.co/ahxt/llama1_s_1.8B_experimental) has 1.8B parameters, and the experimental version of [llama2_xs](https://huggingface.co/ahxt/llama2_xs_460M_experimental) has 460M parameters. ('s' stands for small, while 'xs' denotes extra small).
+In this series of repos, we present an open-source reproduction of Meta AI's [LLaMa 2](https://ai.meta.com/llama/). However, with significantly reduced model sizes, [LiteLlama-460M-1T](https://huggingface.co/ahxt/LiteLlama-460M-1T) has 460M parameters trained with 1T tokens.
 
 
 ## Dataset and Tokenization
@@ -26,7 +26,7 @@ We train our models on part of [RedPajama](https://www.together.xyz/blog/redpaja
 
 The model was trained with ~1T tokens (0.98T). num of tokens = steps*length*batch_size=499679*1024*192=98240888832≈0.98T.
 
-The training curve is at https://wandb.ai/ahxt/llama2_xs_460M_training_loss/reports/reduced_train_loss-23-09-05-20-25-43---Vmlldzo1MzIwNDUx?accessToken=x2ch3n30jo77p1x8y7q9js4h4d8zpjtz1tzot4xxullyefixp4jwt7au2q37k2q6
+The training curve is at this [WandB project](https://wandb.ai/ahxt/llama2_xs_460M_training_loss/reports/reduced_train_loss-23-09-05-20-25-43---Vmlldzo1MzIwNDUx?accessToken=x2ch3n30jo77p1x8y7q9js4h4d8zpjtz1tzot4xxullyefixp4jwt7au2q37k2q6).
 
 ### Using with HuggingFace Transformers
 The experimental checkpoints can be directly loaded by [Transformers](https://huggingface.co/transformers/) library. The following code snippet shows how to load the our experimental model and generate text with it. 
@@ -51,7 +51,7 @@ print( tokenizer.decode(tokens[0].tolist(), skip_special_tokens=True) )
 
 ## Evaluation
 
-# We evaluate our models on the MMLU task.
+### We evaluate our models on the MMLU task.
 
 | Models | #parameters |zero-shot |  5-shot |
 | --- | --- | --- | --- |
@@ -61,7 +61,7 @@ print( tokenizer.decode(tokens[0].tolist(), skip_special_tokens=True) )
 | LiteLlama-460M-1T           | 0.46B | 21.13 | 26.39 |
 
 
-# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+### [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
 Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ahxt__llama2_xs_460M_experimental)
 
 | Metric                | Value                     |
@@ -79,8 +79,7 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
 
 
 ## Contact
-This experimental version is developed by:
-[Xiaotian Han](https://ahxt.github.io/) from Texas A&M University. The model is released 
+This model is developed by [Xiaotian Han](https://ahxt.github.io/) from Texas A&M University and released under MIT License.