Update README.md

README.md

@@ -4,6 +4,11 @@ language:
 tags:
 - finance
 ---
+# Introduction
+This is the implementation of the BERT model using the LongNet structure (paper: https://arxiv.org/pdf/2307.02486.pdf). 
+
+The model is pre-trained with 10-K/Q filings of US firms from 1994 to 2008. Filings from 2009 to 2013 are used for model validation, and filings from 2013 to 2018 are used for model testing.
+
 # Disclaimer
 ~The current model is trained from randomly initialized weights due to some computational and data obstacles. Therefore, the context captured by the models as well as the word semantics are not really good. The tokenizer in this version is also trained from scratch.~
 
@@ -28,11 +33,6 @@ The experiments are implemented with an NVIDIA A100-SXM4-40GB. Batch size of 1.
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61d2d2993c2083e1c08af221/9-PCSONEVOTzZgPuaPSzo.png)
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61d2d2993c2083e1c08af221/q4zLyOQkZ4phmMKPiddSa.png)
 
-# Introduction
-This is the implementation of the BERT model using the LongNet structure (paper: https://arxiv.org/pdf/2307.02486.pdf). 
-
-The model is pre-trained with 10-K/Q filings of US firms from 1994 to 2008. Filings from 2009 to 2013 are used for model validation, and filings from 2013 to 2018 are used for model testing.
-
 # Training code
 https://github.com/minhtriphan/LongFinBERT-base/tree/main