Delimiters

Functions findw, findt, tok, str.gett and str.findreplace have a delim parameter.

 

delim can be:

 

With these functions, blanks are: space, new line, tab, all characters with codes 0-31.

 

With these functions, word characters are:

 

In Unicode mode, can be used only ASCII characters. Non ASCII characters are not delimiters, even if explicitly specified.

 

Table of delimiters

delim also can be a table of delimiters. Use flag 0x100. The table must be a 256-byte array (usually str variable). Each byte is for a character whose code is equal to the array indice. For delimiter characters, set the bytes to 1, for others 0. For example, if delim[32] is 1, then space (code 32) is delimiter. Always set delim[0] to 1. In Unicode mode, don't use non ASCII characters as delimiters.

 

In the example, we create table of delimiters and use it with tok function. Delimiters will be all blanks (characters with values < 33), ", < and >.

 

str d.all(256 2 0)
d.set(1 0 33)
d[34]=1; d['<']=1; d['>']=1
 
str s="one.one ''two two'' <three + three>"
lpstr s1 s2 s3
int n=tok(s &s1 3 d 1|4|64|256)
 Now s1 is "one.one",
 s2 is "two two",
 s3 is "three + three"