new Lexer()
Converts source code into a token
stream.
- Source:
- Lexer.js, line 13
Methods
-
getErrors() → {Array}
-
Returns errors found during lexing
- Source:
- Lexer.js, line 41
Returns:
Errors- Type
- Array
-
lex(source) → {Array}
-
Traverses the source code input to verify and build a token stream.
Parameters:
Name Type Description source
String The source code to convert into tokens - Source:
- Lexer.js, line 24
Returns:
An array of tokens- Type
- Array
-
<inner> findToken(str) → {String}
-
Finds a token at the beginning of the source input.
Parameters:
Name Type Description str
String A chunk of source code to be tested for a token at the start. - Source:
- Lexer.js, line 114
Returns:
A token constant that represents the type of token found.- Type
- String
-
<inner> getTokenLength(tokenType) → {Integer}
-
Gets the length of a token.
Parameters:
Name Type Description tokenType
String Type of token to find the length of - Source:
- Lexer.js, line 135
Returns:
length of token or 1 if token is not found- Type
- Integer
-
<inner> process(src) → {Array}
-
Each recursive itteration will have one less token removed from the front of the source.
Parameters:
Name Type Description src
String Source code to process. - Source:
- Lexer.js, line 51
Returns:
Stream of tokens found in source.- Type
- Array