Source files:
Properties | |
| bool | PreferCustomLiterals [get, set] |
| If this flag is true, all literals except plain strings and true/false/null are stored as CustomLiteral, bypassing number parsing so that all original characters are preserved if the output is written back to text. More... | |
 Properties inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token > | |
| int | SpacesPerTab [get, set] |
| Number of spaces per tab, for the purpose of computing IndentLevel. Initial value: 4 More... | |
| UString | IndentString [get] |
| Gets a string slice that holds the spaces or tabs that were used to indent the current line. More... | |
| int | IndentLevel [get] |
| Gets the number of spaces that were used to indent the current line, where a tab counts as rounding up to the next multiple of SpacesPerTab spaces. More... | |
| new LexerSourceFile< CharSrc > | SourceFile [get] |
| Token | Current [get] |
| Properties inherited from Loyc.Syntax.Lexing.ILexer< Token > | |
| ISourceFile | SourceFile [get] |
| The file being lexed. More... | |
| IMessageSink | ErrorSink [get, set] |
| Event handler for errors. More... | |
| int | IndentLevel [get] |
| Indentation level of the current line. This is updated after scanning the first whitespaces on a new line, and may be reset to zero when NextToken() returns a newline. More... | |
| UString | IndentString [get] |
| Gets a string slice that holds the spaces or tabs that were used to indent the current line. More... | |
| int | LineNumber [get] |
| Current line number (1 for the first line). More... | |
| int | InputPosition [get] |
| Current input position (an index into SourceFile.Text). More... | |
| Properties inherited from Loyc.Syntax.IHasFileName | |
| string | FileName [get] |
Public Member Functions | |
| Les3Lexer (string text, string fileName="") | |
| Les3Lexer (ICharSource text, string fileName, IMessageSink sink, int startPosition=0) | |
| override Maybe< Token > | NextToken () |
| Scans the next token and returns information about it. More... | |
| bool | TDQStringLine () |
| bool | TSQStringLine () |
| bool | MLCommentLine (ref int nested) |
| Public Member Functions inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token > | |
| override void | Reset (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true) |
| Reinitializes the object. This method is called by the constructor. More... | |
| Public Member Functions inherited from Loyc.Syntax.IIndexToLine | |
| ILineColumnFile | IndexToLine (int index) |
| Returns the position in a source file of the specified index. More... | |
Static Public Member Functions | |
| static object | ParseLiteral (Symbol typeMarker, UString unescapedText, out string syntaxError) |
| static TokenType | GetOperatorTokenType (UString op) |
Under the assumption that op is a sequence of punctuation marks that forms a legal operator, this method decides its TokenType. More... | |
Protected Member Functions | |
| UString | Text () |
| UString | Text (int startPosition) |
| sealed override void | AfterNewline () |
| override bool | SupportDotIndents () |
| The LES and EC# languages support "dot indents", which are lines that start with a dot (.) followed by a tab or spaces. If you overload this method to return true, then AfterNewline() and Reset will count dot indents as part of the indentation at the beginning of each line; otherwise, only spaces and tabs will be counted. More... | |
| UString | ParseStringValue (bool parseNeeded, bool isTripleQuoted) |
| UString | UnescapeSQStringValue (bool parseNeeded) |
| Symbol | ParseOp (out TokenType type) |
| Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token > | |
| BaseILexer (CharSrc charSrc, string fileName="", int inputPosition=0) | |
| override void | AfterNewline () |
| The lexer must call this method exactly once after it advances past each newline, even inside comments and strings. This method keeps the BaseLexer<C>.LineNumber, BaseLexer<C>.LineStartAt, IndentString and IndentLevel properties updated. More... | |
| void | AfterNewline (bool ignoreIndent, bool skipIndent) |
| void | ScanIndent (bool skipSpaces=true) |
| Scans indentation at the beginning of a line and updates the IndentLevel and IndentString properties. This function is called automatically by AfterNewline(), but should be called manually on the very first line of the file. More... | |
Protected fields | |
| TokenType | _type |
| NodeStyle | _style |
| int | _startPosition |
| UString | _textValue |
| Dictionary< Pair< UString, bool >, Symbol > | _typeMarkers = new Dictionary<Pair<UString, bool>, Symbol>() |
| Dictionary< UString, Pair< Symbol, TokenType > > | _opCache = new Dictionary<UString, Pair<Symbol, TokenType>>() |
| Protected fields inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token > | |
| int | _indentLevel |
| Maybe< Token > | _current |
| The token that will be returned from the Current property. More... | |
Member Function Documentation
◆ GetOperatorTokenType()
|
inlinestatic |
Under the assumption that op is a sequence of punctuation marks that forms a legal operator, this method decides its TokenType.
References Loyc.UString.Length.
◆ NextToken()
Scans the next token and returns information about it.
- Returns
- The next token, or null at the end of the source file.
Implements Loyc.Syntax.Lexing.ILexer< Token >.
References Loyc.Syntax.Lexing.ILexer< Token >.InputPosition, and Loyc.Syntax.Lexing.Spaces.
◆ SupportDotIndents()
|
inlineprotectedvirtual |
The LES and EC# languages support "dot indents", which are lines that start with a dot (.) followed by a tab or spaces. If you overload this method to return true, then AfterNewline() and Reset will count dot indents as part of the indentation at the beginning of each line; otherwise, only spaces and tabs will be counted.
A dot indent has the syntax ('.' ('\t' | ' '+))*. This indentation style is recognized only if a dot is the first character on a line. Each pair of dot+(tab/spaces) prior to the first non-space token is counted the same way as a tab character (\t). Dot indents are useful for posting source code on "bad" blog software or forums that do not preseve indentation.
Reimplemented from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >.
Property Documentation
◆ PreferCustomLiterals
|
getset |
If this flag is true, all literals except plain strings and true/false/null are stored as CustomLiteral, bypassing number parsing so that all original characters are preserved if the output is written back to text.
