Large files lfs tracking

.gitattributes

@@ -25,3 +25,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zstandard filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+data/preprocessed/shortened_abstracts_hu_2021_09_01.txt filter=lfs diff=lfs merge=lfs -text
+data/preprocessed/shortened_abstracts_hu_2021_09_01_embedded.pt filter=lfs diff=lfs merge=lfs -text
+data/raw/short-abstracts_lang=hu.ttl filter=lfs diff=lfs merge=lfs -text

README.md

@@ -61,7 +61,7 @@ Model facts:
 
 To reproduce the precalculated embedding use the notebook in `notebooks/QA_retrieval_precalculate_embeddings.ipynb`, with GPU in Google Colab.
 
-Known bug: the precalculated embeddings contain an extra random tensor in the beginning, thus the total size of 466529 (one more than the number of raw sentences). This is corrected by substracting 1 from the index of the most similar embedding, to find the corresponding raw sentence.
+Known bug: the precalculated embeddings contain an extra tensor at the end, which is the empty newline at the end of the text file, this last index should be ignored
 
 ## Search top-k matches
 

data/preprocessed/shortened_abstracts_hu_2021_09_01.txt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c7a591796d7ced733adfebf1b7c162a66084b5e27d506946cfb4d6749bc02d3
+size 130726428

data/preprocessed/shortened_abstracts_hu_2021_09_01_embedded.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69cdcfc6da9e5cff770eaf5deeddfb84162a61b23954bc7585b5f65f98b25866
+size 716589291

data/raw/short-abstracts_lang=hu.ttl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36d487bb7be95e2a8313fc381cd461967ee16063f64cd4e34a68b338b91cfcd4
+size 180856895