Spaces:

kotstantinovskii
/

YSDA_arxiv_classification

Runtime error

YSDA_arxiv_classification / tokenizer.py

Upload tokenizer.py

c57f5ec about 2 years ago

No virus

248 Bytes

	import streamlit as st
	from transformers import DistilBertTokenizerFast


	@st.cache(allow_output_mutation=True)
	def get_tokenizer(num_labels=153):
	return DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased', num_labels=num_labels)