splitTokensIntoLines()
Updated 10 Nov 2026
splitTokensIntoLines(tokens): Token[][]
Defined in: tokenizer.ts:1437
Splits flat tokens across newlines (CRLF, LF, or standalone CR) into per-line token arrays.
Parameters
tokens
Token[]
Returns
Token[][]