Update README.md
Browse files
README.md
CHANGED
@@ -50,10 +50,13 @@ You can download the full details of our grid search for all models in all tasks
|
|
50 |
|
51 |
For the XL-Sum task, we choose our best run for each model using the eval set. We use the official evaluation script from XL-Sum, which uses the stemmer function, which may show better results than papers that don't use the stemmer function. The official XL-Sum paper uses a stemmer function.
|
52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
53 |
|
54 |
-
# Continual Pre-Training of ArabicT5 with T5x
|
55 |
-
if you want to continue pre-training ArabicT5 on your own data, we have uploaded the raw t5x checkpoint to this link https://huggingface.co/sultan/ArabicT5-49GB-base/blob/main/arabict5_49GB_base_t5x.tar.gz
|
56 |
-
We will soon share a tutorial on how you can do that for free with Kaggle TPU
|
57 |
|
58 |
|
59 |
|
@@ -63,6 +66,12 @@ We will soon share a tutorial on how you can do that for free with Kaggle TPU
|
|
63 |
|
64 |
[COLAB]: https://colab.research.google.com/assets/colab-badge.svg
|
65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
## GitHub Page
|
67 |
|
68 |
https://github.com/salrowili/ArabicT5
|
|
|
50 |
|
51 |
For the XL-Sum task, we choose our best run for each model using the eval set. We use the official evaluation script from XL-Sum, which uses the stemmer function, which may show better results than papers that don't use the stemmer function. The official XL-Sum paper uses a stemmer function.
|
52 |
|
53 |
+
# FineTuning our efficient ArabicT5-49GB-Small model with Torch on 3070 laptop GPU ###
|
54 |
+
|
55 |
+
If you are running your code on a laptop GPU (e.g., a gaming laptop) or limited GPU memory, we recommended using our ArabicT5-49GB-Small model, which was the only model from the list that we were able to run on 3070 Laptop card with a batch size of 8. We manage to achieve an F1 score of 85.391 (slightly better than our FLAX code ) on the TyDi QA task. See the notebook below for reference :
|
56 |
+
|
57 |
+
[![Open In Colab][COLAB]](https://colab.research.google.com/github/salrowili/ArabicT5/blob/main/ArabicT5_49GB_Small_on_3070_Laptop_GPU.ipynb)
|
58 |
+
|
59 |
|
|
|
|
|
|
|
60 |
|
61 |
|
62 |
|
|
|
66 |
|
67 |
[COLAB]: https://colab.research.google.com/assets/colab-badge.svg
|
68 |
|
69 |
+
|
70 |
+
# Continual Pre-Training of ArabicT5 with T5x
|
71 |
+
if you want to continue pre-training ArabicT5 on your own data, we have uploaded the raw t5x checkpoint to this link https://huggingface.co/sultan/ArabicT5-49GB-base/blob/main/arabict5_49GB_base_t5x.tar.gz
|
72 |
+
We will soon share a tutorial on how you can do that for free with Kaggle TPU
|
73 |
+
|
74 |
+
|
75 |
## GitHub Page
|
76 |
|
77 |
https://github.com/salrowili/ArabicT5
|