semvec/pypi-packages huggingface dataset:
pycopy-tokenize 0.1
tokenize module for Pycopy
License: MIT

This is a module for the standard library of Pycopy project:
https://github.com/pfalcon/pycopy . This module was implemented from
scratch specifically to target Pycopy, with efficient and lean design
in mind. Note that this module is likely work in progress and likely
supports just a subset of CPython’s corresponding module. Please help
with the development if you are interested in this module.