Taishi-N324 commited on
Commit
8b3dc4b
1 Parent(s): 8ec7523

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -2
README.md CHANGED
@@ -40,7 +40,7 @@ We are excited to share the release schedule for our latest models:
40
  ![logo](./logo.png)
41
 
42
  This repository provides large language models developed by [TokyoTech-LLM](https://tokyotech-llm.github.io/).
43
- Read our [blog post](https://zenn.dev/tokyotech_lm/articles/d6cb3a8fdfc907) or our [paper](https://www.anlp.jp/proceedings/annual_meeting/2024/pdf_dir/A8-5.pdf)
44
 
45
  ## Model Details
46
 
@@ -224,7 +224,7 @@ The following datasets were used for continual pre-training.
224
 
225
  - [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
226
  - [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
227
- - Swallow Corpus
228
  - [The Pile](https://huggingface.co/datasets/EleutherAI/pile)
229
 
230
 
@@ -265,3 +265,15 @@ Here are the team members:
265
  - [Rio Yokota](https://twitter.com/rioyokota)
266
  - [Kazuki Fujii](https://twitter.com/okoge_kaz)
267
  - [Taishi Nakamura](https://twitter.com/Setuna7777_2)
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  ![logo](./logo.png)
41
 
42
  This repository provides large language models developed by [TokyoTech-LLM](https://tokyotech-llm.github.io/).
43
+ Read our [blog post](https://zenn.dev/tokyotech_lm/articles/d6cb3a8fdfc907) or our [paper](https://arxiv.org/abs/2404.17790)
44
 
45
  ## Model Details
46
 
 
224
 
225
  - [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
226
  - [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
227
+ - [Swallow Corpus](https://arxiv.org/abs/2404.17733)
228
  - [The Pile](https://huggingface.co/datasets/EleutherAI/pile)
229
 
230
 
 
265
  - [Rio Yokota](https://twitter.com/rioyokota)
266
  - [Kazuki Fujii](https://twitter.com/okoge_kaz)
267
  - [Taishi Nakamura](https://twitter.com/Setuna7777_2)
268
+
269
+ ## How to cite
270
+ ```
271
+ @misc{fujii2024continual,
272
+ title={Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities},
273
+ author={Kazuki Fujii and Taishi Nakamura and Mengsay Loem and Hiroki Iida and Masanari Ohi and Kakeru Hattori and Hirai Shota and Sakae Mizuki and Rio Yokota and Naoaki Okazaki},
274
+ year={2024},
275
+ eprint={2404.17790},
276
+ archivePrefix={arXiv},
277
+ primaryClass={cs.CL}
278
+ }
279
+ ```