Update README.md

README.md

@@ -1,7 +1,16 @@
 ---
 language:
 - en
+tags:
+- finance
 ---
+# Disclaimer
+The current model is trained from randomly initialized weights due to some computational and data obstacles. Therefore, the context captured by the models as well as the word semantics are not really good. The tokenizer in this version is also trained from scratch.
+
+We're training the model again with more care of the semantics to enhance the semantics of words. To this end, we initialize the embedding layers (i.e., `word_embeddings`, `position_embeddings`, `token_type_embeddings`, and `LayerNorm`) with the pre-trained embeddings from FinBERT (https://huggingface.co/yiyanghkust/finbert-tone). Accordingly, we use the same tokenizer as that of this model.
+
+Furthermore, the model is trained longer (10 epochs). The new pre-trained model weights will be updated as soon as the training and validation are completed.
+
 # Introduction
 This is the implementation of the BERT model using the LongNet structure (paper: https://arxiv.org/pdf/2307.02486.pdf). 
 
@@ -30,7 +39,7 @@ sys.path.append('/LongFinBERT-base')
 
 * Load the pre-trained tokenizer, model configuration, and model weights
 ```
-from model import LongBERT
+from model import LongBERTModel
 from custom_config import LongBERTConfig
 from tokenizer import LongBERTTokenizer
 
@@ -38,7 +47,7 @@ backbone = 'minhtriphan/LongFinBERT-base'
 
 tokenizer = LongBERTTokenizer.from_pretrained(backbone)
 config = LongBERTConfig.from_pretrained(backbone)
-model = LongBERT.from_pretrained(backbone)
+model = LongBERTModel.from_pretrained(backbone)
 ```
 
 # Model usage