tokenize (English, verb)
expansion: tokenize (third-person singular simple present tokenizes, present participle tokenizing, simple past and past participle tokenized)
Senses:
topics: computing
engineering
mathematics
natural-sciences
physical-sciences
sciences
glosses: To reduce to a token or set of tokens by lexical analysis.
tags: transitive
topics: computing
engineering
mathematics
natural-sciences
physical-sciences
sciences
glosses: To substitute sensitive data with meaningless placeholders.
tags: transitive
glosses: To treat as a token minority.
tags: transitive
Forms:
form: tokenizes
tags: present
singular
third-person
form: tokenizing
tags: participle
present
form: tokenized
tags: participle
past
form: tokenized
tags: past
form: tokenise
tags: alternative