Spaces:

sonalkum
/

GAMA

Running on Zero

GAMA / utils /test_module /custom_tokenization.py

Sonal Kumar

first commit

ed7a497 7 months ago

88 Bytes

	from transformers import BertTokenizer


	class CustomTokenizer(BertTokenizer):
	pass