tokenize

About 0.00 uses per million words.

Meaning

1. verb To reduce to a token or set of tokens by lexical analysis.
2. verb To substitute sensitive data with meaningless placeholders.
3. verb To treat as a token minority.

Similar words

tokenise detokenize disassociate analyze unpack delexicalize fragmentize minimalize atomize linguisticize

Often followed by

the a and

Word family

tokenized tokenizes tokenizing