Page History
...
| Code Block |
|---|
Tokens = SRP_String("TokenizeCode", String, WhitespaceOptions, IncludeComments) |
Parameters
| Parameter | Description |
|---|---|
| String | A string of BASIC+ code. (REQUIRED) |
| WhitespaceOptions | Whitespace tokenizing options. (OPTIONAL, Default="None") |
| IncludeComments | TRUE to include comments, FALSE to omit them. (OPTIONAL, Default=0) |
Returns
An @FM delimited array of strings, each one a BASIC+ token.
...
This service takes a string of BASIC+ code and returns an @FM delimited array of tokens. A token is a discrete element of code: a string, an operator, whitespace, comment, identifier, etc. This was written to support the SRP Precompiler, which parses BASIC+ code in order to provide language enhancements. This routine does all the work of breaking up the code into tokens for easier analysis. There are some options to control To use this service, pass your BASIC+ code to the String parameter. The remaining parameters provide some options for controlling the output.
WhitespaceOptions: This parameter supports one of three values: "None", "AllWhitespace", or "LineBreaksOnly". The default option is "None", in which case, whitespace tokens are never included in the final array. "AllWhitespace" has the exact opposite effect: all whitespace tokens are included in the output. "LineBreaksOnly" will omit whitespace composed of spaces and tabs but include carriage return and/or line feeds. This last option could be useful for detecting line breaks in code while still ignoring all other whitespace.
...
| Code Block |
|---|
// Sample code
Code = "A = B + C; // This is a simple expression"
// Output will be: "A":@FM:"=":@FM:"B":@FM:"+":@FM:"C":@FM:";"
Tokens = SRP_String("TokenizeCode")
// Output will be: "A":@FM:" ":@FM:"=":@FM:" ":@FM:"B":@FM:" ":@FM:"+"::@FM:" "@FM:"C":@FM:";":@FM:" ":@FM:\0D0A\
Tokens = SRP_String("TokenizeCode", "AllWhitespace")
// Output will be: "A":@FM:"=":@FM:"B":@FM:"+":@FM:"C":@FM:";":@FM:\0D0A\
Tokens = SRP_String("TokenizeCode", "LineBreaksOnly")
// Output will be: "A":@FM:"=":@FM:"B":@FM:"+":@FM:"C":@FM:";":@FM:"// This is a simple expression"
Tokens = SRP_String("TokenizeCode", "None", 1) |
See Also
