tokenize()
Updated 10 Nov 2026
tokenize(code, language?): Token[][]
Defined in: tokenizer.ts:1459
High-level tokenizer breaking code into per-line token arrays.
Parameters
code
string
language?
string
Returns
Token[][]
Md2Docx Md2DocxUpdated 10 Nov 2026
tokenize(code, language?): Token[][]
Defined in: tokenizer.ts:1459
High-level tokenizer breaking code into per-line token arrays.
string
string
Token[][]