How can I get large corpus dataset (over 200 Millions of records) in a tsv file format to encode with intfloat/e5-large-v2 as an embedding model ?

#15

by liorf95 - opened Apr 17

Apr 17

The tsv file format should be as follows:
id text title
1 Alice likes adventures. Alice
2 Bob likes reading books. Bob
3 Charlie is a free-spirited artist. Charlie

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment