update readme

README.md

@@ -13,16 +13,15 @@ AutoConfig.register("moduleformer", ModuleFormerConfig)
 AutoModelForCausalLM.register(ModuleFormerConfig, ModuleFormerForCausalLM)
 AutoModelForSequenceClassification.register(ModuleFormerConfig, ModuleFormerForSequenceClassification)
 
-tokenizer = AutoTokenizer.from_pretrained('ibm/MoLM-350M-4B')
-model = AutoModelForCausalLM.from_pretrained('ibm/MoLM-350M-4B')
+tokenizer = AutoTokenizer.from_pretrained('ibm/MoLM-700M-8B')
+model = AutoModelForCausalLM.from_pretrained('ibm/MoLM-700M-8B')
 ```
 
 **Model Details**
-MoLM-350M-4B is a MoE-based language models. It has 4 billion parameters, but each input token only use 350M parameteres during its inference. Thus, it's computationally equivelant to a 350M dense model.
-MoLM-700M-4B has 4 billion parameters and computationally equivelant to a 700M dense model.
-MoLM-700M-8B has 8 billion parameters and computationally equivelant to a 700M dense model.
-Both models are trained on 300 billion tokens from publicly available sources, with a learning rate of 3.0 x 10<sup>-4</sup> and a global batch-size of 3M tokens.
-
+MoLM-350M-4B is a MoE-based language model. It has 4 billion parameters, but each input token only activates 350M parameters. Thus, it's computationally equivalent to a 350M dense model. 
+MoLM-700M-4B has 4 billion parameters and is computationally equivalent to a 700M dense model. 
+MoLM-700M-8B has 8 billion parameters and is computationally equivalent to a 700M dense model. All models are trained on 300 billion tokens from publicly available sources.
+All models are trained on 300 billion tokens from publicly available sources, with a learning rate of 3.0 x 10<sup>-4</sup> and a global batch-size of 3M tokens.
 **Model Developers** IBM
 
 **Variations** MoLM comes in two different parameter sizes — 4B and 8B. The 4B models has two variants with different computation cost — 350M and 700M.