Variable SIMPLE_TOKENIZER_SEP_REConst

SIMPLE_TOKENIZER_SEP_RE: RegExp = ...

Regex used by simpleTokenizer to split input into tokens.

  • Matches one or more non-word characters or underscores