Spaces:

taesiri
/

ClaudeReadsArxiv

Running

display number of tokens

by obi77 - opened Oct 4, 2023

Oct 4, 2023

Could you please consider adding a feature that displays the number of tokens in the paper? It would be really helpful to have this information to know of we could use the same paper with other language models. Thank you!

taesiri

Owner Oct 5, 2023

Hello @obi77

Yeah, that's a cool suggestion. The reason I haven't added this yet is that the last time I checked, tokenizer for Claude-2.0 was not available, or something like that (if I recall correctly). I will take a look into this soon.

Artyom2003

Mar 6

This comment has been hidden

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment