Enhanced C#
Language of your choice: library documentation
Properties | Public Member Functions | Static Public Member Functions | Static Protected Member Functions | Protected static fields | List of all members
Loyc.Syntax.Les.Les3Lexer Class Reference

Source files:
Inheritance diagram for Loyc.Syntax.Les.Les3Lexer:
Loyc.Syntax.Les.Les2Lexer Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token > Loyc.Syntax.Lexing.ILexer< Token > Loyc.ICloneable< Les2Lexer > Loyc.Syntax.IIndexToLine

Properties

bool PreferCustomLiterals [get, set]
 If this flag is true, all literals except plain strings and true/false/null are stored as CustomLiteral, bypassing number parsing so that all original characters are preserved if the output is written back to text. More...
 
- Properties inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
int SpacesPerTab [get, set]
 Number of spaces per tab, for the purpose of computing IndentLevel. Initial value: 4 More...
 
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
 
int IndentLevel [get]
 Gets the number of spaces that were used to indent the current line, where a tab counts as rounding up to the next multiple of SpacesPerTab spaces. More...
 
new LexerSourceFile< CharSrc > SourceFile [get]
 
Token Current [get]
 
- Properties inherited from Loyc.Syntax.Lexing.ILexer< Token >
ISourceFile SourceFile [get]
 The file being lexed. More...
 
IMessageSink ErrorSink [get, set]
 Event handler for errors. More...
 
int IndentLevel [get]
 Indentation level of the current line. This is updated after scanning the first whitespaces on a new line, and may be reset to zero when NextToken() returns a newline. More...
 
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
 
int LineNumber [get]
 Current line number (1 for the first line). More...
 
int InputPosition [get]
 Current input position (an index into SourceFile.Text). More...
 
- Properties inherited from Loyc.Syntax.IIndexToLine
string FileName [get]
 Gets the file name used in results returned by IndexToLine(int). More...
 

Public Member Functions

 Les3Lexer (string text, string fileName="")
 
 Les3Lexer (ICharSource text, string fileName, IMessageSink sink, int startPosition=0)
 
override Maybe< TokenNextToken ()
 Scans the next token and returns information about it. More...
 
new bool TDQStringLine ()
 
new bool TSQStringLine ()
 
new bool MLCommentLine (ref int nested)
 
- Public Member Functions inherited from Loyc.Syntax.Les.Les2Lexer
 Les2Lexer (UString text, IMessageSink errorSink)
 
 Les2Lexer (ICharSource text, string fileName, IMessageSink sink, int startPosition=0)
 
override void Reset (ICharSource source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 
Les2Lexer Clone ()
 
override Maybe< TokenNextToken ()
 Scans the next token and returns information about it. More...
 
bool TDQStringLine ()
 
bool TSQStringLine ()
 
bool MLCommentLine (ref int nested)
 
- Public Member Functions inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
override void Reset (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 Reinitializes the object. This method is called by the constructor. More...
 
- Public Member Functions inherited from Loyc.Syntax.IIndexToLine
SourcePos IndexToLine (int index)
 Returns the position in a source file of the specified index. More...
 
- Public Member Functions inherited from Loyc.ICloneable< Les2Lexer >
Clone ()
 

Static Public Member Functions

static object ParseLiteral (UString typeMarker, UString parsedText, bool isNumericLiteral, out string syntaxError)
 
- Static Public Member Functions inherited from Loyc.Syntax.Les.Les2Lexer
static string UnescapeQuotedString (ref UString sourceText, Action< int, string > onError, UString indentation=default(UString), bool les3TQIndents=false)
 Parses a normal or triple-quoted string that still includes the quotes. Supports quote types '\'', '"' and '`'. More...
 
static void UnescapeQuotedString (ref UString sourceText, Action< int, string > onError, StringBuilder sb, UString indentation=default(UString), bool les3TQIndents=false)
 Parses a normal or triple-quoted string that still includes the quotes (see documentation of the first overload) into a StringBuilder. More...
 
static bool UnescapeString (ref UString sourceText, char quoteType, bool isTripleQuoted, Action< int, string > onError, StringBuilder sb, UString indentation=default(UString), bool les3TQIndents=false)
 Parses a normal or triple-quoted string whose starting quotes have been stripped out. If triple-quote parsing was requested, stops parsing at three quote marks; otherwise, stops parsing at a single end-quote or newline. More...
 
static string ParseIdentifier (ref UString source, Action< int, string > onError, out bool checkForNamedLiteral)
 Parses an LES-style identifier such as foo, , <tt>foo or &ndash;punctuation–. More...
 
static object ParseNumberCore (UString source, bool isNegative, int numberBase, bool isFloat, Symbol typeSuffix, out string error)
 Parses the digits of a literal (integer or floating-point), not including the radix prefix (0x, 0b) or type suffix (F, D, L, etc.) More...
 

Static Protected Member Functions

static Dictionary< UString, Func< UString, object > > InitLiteralParsers ()
 

Protected static fields

static Dictionary< UString, Func< UString, object > > LiteralParsers = InitLiteralParsers()
 
static readonly Symbol _AtAt = GSymbol.Get("@@")
 
- Protected static fields inherited from Loyc.Syntax.Les.Les2Lexer
static Symbol _sub = GSymbol.Get("-")
 
static Symbol _U = GSymbol.Get("U")
 
static Symbol _L = GSymbol.Get("L")
 
static Symbol _UL = GSymbol.Get("UL")
 
static Symbol _Z = GSymbol.Get("Z")
 
static Symbol _F = GSymbol.Get("F")
 
static Symbol _D = GSymbol.Get("D")
 
static Symbol _M = GSymbol.Get("M")
 

Additional Inherited Members

- Public fields inherited from Loyc.Syntax.Les.Les2Lexer
bool AllowNestedComments = true
 
bool SkipValueParsing = false
 Used for syntax highlighting, which doesn't care about token values. This option causes the Token.Value to be set to a default, like '\0' for single-quoted strings and 0 for numbers. Operator names are still parsed. More...
 
- Protected Member Functions inherited from Loyc.Syntax.Les.Les2Lexer
override void Error (int lookaheadIndex, string message, params object[] args)
 
UString Text ()
 
sealed override void AfterNewline ()
 
override bool SupportDotIndents ()
 The LES and EC# languages support "dot indents", which are lines that start with a dot (.) followed by a tab or spaces. If you overload this method to return true, then AfterNewline() and Reset will count dot indents as part of the indentation at the beginning of each line; otherwise, only spaces and tabs will be counted. More...
 
object ParseSQStringValue ()
 
Symbol ParseBQStringValue ()
 
object ParseStringValue (bool isTripleQuoted, bool les3TQIndents=false)
 
string ParseStringCore (bool isTripleQuoted, bool les3TQindents=false)
 
object ParseIdValue (bool isFancy)
 
object ParseSymbolValue (bool lesv3=false)
 
Symbol IdToSymbol (UString ustr)
 
object ParseNumberValue (int numberEndPosition)
 
object ParseNormalOp ()
 
- Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
 BaseILexer (CharSrc charSrc, string fileName="", int inputPosition=0)
 
override void AfterNewline ()
 The lexer must call this method exactly once after it advances past each newline, even inside comments and strings. This method keeps the BaseLexer<C>.LineNumber, BaseLexer<C>.LineStartAt, IndentString and IndentLevel properties updated. More...
 
void AfterNewline (bool ignoreIndent, bool skipIndent)
 
void ScanIndent (bool skipSpaces=true)
 Scans indentation at the beginning of a line and updates the IndentLevel and IndentString properties. This function is called automatically by AfterNewline(), but should be called manually on the very first line of the file. More...
 
- Protected fields inherited from Loyc.Syntax.Les.Les2Lexer
bool _isFloat
 
NodeStyle _style
 
int _numberBase
 
Symbol _typeSuffix
 
TokenType _type
 
object _value
 
int _startPosition
 
Dictionary< UString, Symbol_idCache = new Dictionary<UString,Symbol>()
 
- Protected fields inherited from Loyc.Syntax.Lexing.BaseILexer< ICharSource, Token >
int _indentLevel
 
Maybe< Token_current
 The token that will be returned from the Current property. More...
 

Member Function Documentation

override Maybe<Token> Loyc.Syntax.Les.Les3Lexer.NextToken ( )
inline

Scans the next token and returns information about it.

Returns
The next token, or null at the end of the source file.

Implements Loyc.Syntax.Lexing.ILexer< Token >.

References Loyc.Syntax.Lexing.Spaces.

Property Documentation

bool Loyc.Syntax.Les.Les3Lexer.PreferCustomLiterals
getset

If this flag is true, all literals except plain strings and true/false/null are stored as CustomLiteral, bypassing number parsing so that all original characters are preserved if the output is written back to text.