Enhanced C#
Language of your choice: library documentation
Properties | Public Member Functions | Static Public Member Functions | Protected Member Functions | Static Protected Member Functions | Protected fields | Protected static fields | List of all members
Loyc.Syntax.Les.Les3Lexer Class Reference

Source files:
Inheritance diagram for Loyc.Syntax.Les.Les3Lexer:
Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token > Loyc.Syntax.Lexing.ILexer< Token > Loyc.Syntax.IIndexToLine

Properties

bool PreferCustomLiterals [get, set]
 If this flag is true, all literals except plain strings and true/false/null are stored as CustomLiteral, bypassing number parsing so that all original characters are preserved if the output is written back to text. More...
 
- Properties inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
int SpacesPerTab [get, set]
 Number of spaces per tab, for the purpose of computing IndentLevel. Initial value: 4 More...
 
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
 
int IndentLevel [get]
 Gets the number of spaces that were used to indent the current line, where a tab counts as rounding up to the next multiple of SpacesPerTab spaces. More...
 
new LexerSourceFile< CharSrc > SourceFile [get]
 
Token Current [get]
 
- Properties inherited from Loyc.Syntax.Lexing.ILexer< Token >
ISourceFile SourceFile [get]
 The file being lexed. More...
 
IMessageSink ErrorSink [get, set]
 Event handler for errors. More...
 
int IndentLevel [get]
 Indentation level of the current line. This is updated after scanning the first whitespaces on a new line, and may be reset to zero when NextToken() returns a newline. More...
 
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
 
int LineNumber [get]
 Current line number (1 for the first line). More...
 
int InputPosition [get]
 Current input position (an index into SourceFile.Text). More...
 
- Properties inherited from Loyc.Syntax.IIndexToLine
string FileName [get]
 Gets the file name used in results returned by IndexToLine(int). More...
 

Public Member Functions

 Les3Lexer (string text, string fileName="")
 
 Les3Lexer (ICharSource text, string fileName, IMessageSink sink, int startPosition=0)
 
override Maybe< TokenNextToken ()
 Scans the next token and returns information about it. More...
 
bool TDQStringLine ()
 
bool TSQStringLine ()
 
bool MLCommentLine (ref int nested)
 
- Public Member Functions inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
override void Reset (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 Reinitializes the object. This method is called by the constructor. More...
 
- Public Member Functions inherited from Loyc.Syntax.IIndexToLine
SourcePos IndexToLine (int index)
 Returns the position in a source file of the specified index. More...
 

Static Public Member Functions

static object ParseLiteral (UString typeMarker, UString parsedText, bool isNumericLiteral, out string syntaxError)
 

Protected Member Functions

UString Text ()
 
sealed override void AfterNewline ()
 
override bool SupportDotIndents ()
 The LES and EC# languages support "dot indents", which are lines that start with a dot (.) followed by a tab or spaces. If you overload this method to return true, then AfterNewline() and Reset will count dot indents as part of the indentation at the beginning of each line; otherwise, only spaces and tabs will be counted. More...
 
string ParseStringValue (bool parseNeeded, bool isTripleQuoted)
 
object ParseSQStringValue (bool parseNeeded)
 
Symbol ParseOp (out TokenType type)
 
- Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
 BaseILexer (CharSrc charSrc, string fileName="", int inputPosition=0)
 
override void AfterNewline ()
 The lexer must call this method exactly once after it advances past each newline, even inside comments and strings. This method keeps the BaseLexer<C>.LineNumber, BaseLexer<C>.LineStartAt, IndentString and IndentLevel properties updated. More...
 
void AfterNewline (bool ignoreIndent, bool skipIndent)
 
void ScanIndent (bool skipSpaces=true)
 Scans indentation at the beginning of a line and updates the IndentLevel and IndentString properties. This function is called automatically by AfterNewline(), but should be called manually on the very first line of the file. More...
 

Static Protected Member Functions

static Dictionary< UString, Func< UString, object > > InitLiteralParsers ()
 

Protected fields

TokenType _type
 
NodeStyle _style
 
int _startPosition
 
Dictionary< UString, Pair< Symbol, TokenType > > _opCache = new Dictionary<UString, Pair<Symbol, TokenType>>()
 
- Protected fields inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
int _indentLevel
 
Maybe< Token_current
 The token that will be returned from the Current property. More...
 

Protected static fields

static Dictionary< UString, Func< UString, object > > LiteralParsers = InitLiteralParsers()
 
static readonly Symbol _AtAt = GSymbol.Get("@@")
 

Member Function Documentation

override Maybe<Token> Loyc.Syntax.Les.Les3Lexer.NextToken ( )
inline

Scans the next token and returns information about it.

Returns
The next token, or null at the end of the source file.

Implements Loyc.Syntax.Lexing.ILexer< Token >.

References Loyc.Syntax.Lexing.Spaces.

override bool Loyc.Syntax.Les.Les3Lexer.SupportDotIndents ( )
inlineprotectedvirtual

The LES and EC# languages support "dot indents", which are lines that start with a dot (.) followed by a tab or spaces. If you overload this method to return true, then AfterNewline() and Reset will count dot indents as part of the indentation at the beginning of each line; otherwise, only spaces and tabs will be counted.

A dot indent has the syntax ('.' ('' | ' '+))*. This indentation style is recognized only if a dot is the first character on a line. Each pair of dot+(tab/spaces) prior to the first non-space token is counted the same way as a tab character (). Dot indents are useful for posting source code on "bad" blog software or forums that do not preseve indentation.

Reimplemented from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >.

References Loyc.G.BoxedFalse, Loyc.G.BoxedTrue, Loyc.Collections.CG.Cache(), Loyc.Syntax.CodeSymbols.Dot, Loyc.Syntax.CodeSymbols.Not, Loyc.UString.Slice(), Loyc.UString.StartsWith(), Loyc.UString.Substring(), Loyc.Syntax.ParseHelpers.TryParseDouble(), Loyc.Syntax.ParseHelpers.TryParseUInt(), and Loyc.Syntax.Les.Les2Lexer.UnescapeQuotedString().

Property Documentation

bool Loyc.Syntax.Les.Les3Lexer.PreferCustomLiterals
getset

If this flag is true, all literals except plain strings and true/false/null are stored as CustomLiteral, bypassing number parsing so that all original characters are preserved if the output is written back to text.