Enhanced C#
Language of your choice: library documentation
Properties | Public Member Functions | Static Public Member Functions | Protected Member Functions | List of all members
Loyc.Syntax.Lexing.LexerSource< CharSrc > Class Template Reference

An implementation of the LLLPG Lexer API, used with the LLLPG options inputSource and inputClass. More...


Source file:
Inheritance diagram for Loyc.Syntax.Lexing.LexerSource< CharSrc >:
Loyc.Syntax.Lexing.LexerSourceWorkaround< CharSrc > Loyc.Syntax.Lexing.ILllpgLexerApi< int > Loyc.Syntax.Lexing.BaseLexer< CharSrc > Loyc.Syntax.IIndexToLine Loyc.Syntax.IHasFileName Loyc.Syntax.IHasFileName

Remarks

An implementation of the LLLPG Lexer API, used with the LLLPG options inputSource and inputClass.

A synonym for LexerSource<C> where C is ICharSource.

This derived class simply makes public all of the LLLPG APIs which are marked protected in BaseLexer<CharSrc>.

LLLPG(lexer(inputSource(src), inputClass(LexerSource))) { static rule int ParseInt(string input) { var src = (LexerSource<UString>)input; @[ (d:='0'..'9' {$result = $result * 10 + (d - '0');})+ ]; } }

Template Parameters
CharSrcA class that implements ICharSource. In order to write lexers that can accept any source of characters, set CharSrc=ICharSource. For maximum performance when parsing strings (or to avoid memory allocation), set CharSrc=UString (UString is a wrapper around System.String that, among other things, implements ICharSource; please note that C# will implicitly convert normal strings to UString for you).
Type Constraints
CharSrc :ICharSource 

Properties

new int LA0 [get]
 
new CharSrc CharSource [get]
 
new string FileName [get]
 
new int InputPosition [get, protected set]
 
new LexerSourceFile< CharSrc > SourceFile [get]
 
new int LineStartAt [get]
 
- Properties inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
IMessageSink?? ErrorSink [get, set]
 Gets or sets the object to which error messages are sent. The default object is LogExceptionErrorSink, which throws an exception if an error occurs. More...
 
int LA0 [get]
 
CharSrc CharSource [get]
 
string FileName [get]
 
int InputPosition [get, protected set]
 
LexerSourceFile< CharSrc > SourceFile [get]
 
int LineNumber [get]
 Current line number. Starts at 1 for the first line, unless derived class changes it. More...
 
int LineStartAt [get]
 Index at which the current line started. More...
 
- Properties inherited from Loyc.Syntax.IHasFileName
string FileName [get]
 

Public Member Functions

 LexerSource (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 Initializes LexerSource. More...
 
virtual new void Reset (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 Reinitializes the object. This method is called by the constructor.
 
new void Reset ()
 
new int LA (int i)
 
new void Skip ()
 
virtual new void AfterNewline ()
 The lexer must call this method exactly once after it advances past each newline, even inside comments and strings. This method keeps the LineNumber and LineStartAt properties updated.
 
new void Newline ()
 
new int MatchAny ()
 
new int Match (HashSet< int > set)
 
new int Match (int a)
 
new int Match (int a, int b)
 
new int Match (int a, int b, int c)
 
new int Match (int a, int b, int c, int d)
 
new int MatchRange (int aLo, int aHi)
 
new int MatchRange (int aLo, int aHi, int bLo, int bHi)
 
new int MatchExcept ()
 
new int MatchExcept (HashSet< int > set)
 
new int MatchExcept (int a)
 
new int MatchExcept (int a, int b)
 
new int MatchExcept (int a, int b, int c)
 
new int MatchExcept (int a, int b, int c, int d)
 
new int MatchExceptRange (int aLo, int aHi)
 
new int MatchExceptRange (int aLo, int aHi, int bLo, int bHi)
 
new bool TryMatch (HashSet< int > set)
 
new bool TryMatch (int a)
 
new bool TryMatch (int a, int b)
 
new bool TryMatch (int a, int b, int c)
 
new bool TryMatch (int a, int b, int c, int d)
 
new bool TryMatchRange (int aLo, int aHi)
 
new bool TryMatchRange (int aLo, int aHi, int bLo, int bHi)
 
new bool TryMatchExcept ()
 
new bool TryMatchExcept (HashSet< int > set)
 
new bool TryMatchExcept (int a)
 
new bool TryMatchExcept (int a, int b)
 
new bool TryMatchExcept (int a, int b, int c)
 
new bool TryMatchExcept (int a, int b, int c, int d)
 
new bool TryMatchExceptRange (int aLo, int aHi)
 
new bool TryMatchExceptRange (int aLo, int aHi, int bLo, int bHi)
 
virtual new void Check (bool expectation, string expectedDescr="")
 
virtual new void Error (int lookaheadIndex, string format)
 This method is called to handle errors that occur during lexing. More...
 
virtual new void Error (int lookaheadIndex, string format, params object[] args)
 This method is called to format and handle errors that occur during lexing. The default implementation sends errors to ErrorSink, which, by default, throws a FormatException. More...
 
new void PrintChar (int c, StringBuilder sb)
 
 LexerSource (ICharSource source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 
- Public Member Functions inherited from Loyc.Syntax.Lexing.LexerSourceWorkaround< CharSrc >
 LexerSourceWorkaround (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 
- Public Member Functions inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
 BaseLexer (CharSrc chars, string fileName="", int inputPosition=0, bool newSourceFile=true)
 Initializes BaseLexer. More...
 
LineColumnFile IndexToLine (int charIndex)
 Returns the position in a source file of the specified index. More...
 
- Public Member Functions inherited from Loyc.Syntax.Lexing.ILllpgLexerApi< int >
Token MatchRange (int aLo, int aHi)
 
Token MatchRange (int aLo, int aHi, int bLo, int bHi)
 
Token MatchExceptRange (int aLo, int aHi)
 
Token MatchExceptRange (int aLo, int aHi, int bLo, int bHi)
 
bool TryMatchRange (int aLo, int aHi)
 
bool TryMatchRange (int aLo, int aHi, int bLo, int bHi)
 
bool TryMatchExceptRange (int aLo, int aHi)
 
bool TryMatchExceptRange (int aLo, int aHi, int bLo, int bHi)
 

Static Public Member Functions

static new HashSet< int > NewSet (params int[] items)
 
static new HashSet< int > NewSetOfRanges (params int[] ranges)
 
static operator LexerSource< CharSrc > (CharSrc str)
 
static operator LexerSource (string str)
 

Protected Member Functions

override void Error_Renamed (int lookaheadIndex, string format, params object[] args)
 
override void Error_Renamed (int lookaheadIndex, string format)
 
- Protected Member Functions inherited from Loyc.Syntax.Lexing.LexerSourceWorkaround< CharSrc >
override void Error (int lookaheadIndex, string format)
 This method is called to handle errors that occur during lexing. More...
 
override void Error (int lookaheadIndex, string format, params object[] args)
 This method is called to format and handle errors that occur during lexing. The default implementation sends errors to ErrorSink, which, by default, throws a FormatException. More...
 
- Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
void Reset ()
 
int LA (int i)
 
void Skip ()
 Increments InputPosition. Called by LLLPG when prediction already verified the input (and caller doesn't save LA(0)) More...
 
 BaseLexer (ICharSource source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 
void Newline ()
 Default newline parser that matches '
' or '\r' unconditionally. More...
 
void Spaces ()
 Skips past any spaces at the current position. Equivalent to rule Spaces @[ (' '|'\t')* ] in LLLPG. More...
 
int MatchAny ()
 
int Match (HashSet< int > set)
 
int Match (int a)
 
int Match (int a, int b)
 
int Match (int a, int b, int c)
 
int Match (int a, int b, int c, int d)
 
int MatchRange (int aLo, int aHi)
 
int MatchRange (int aLo, int aHi, int bLo, int bHi)
 
int MatchExcept ()
 
int MatchExcept (HashSet< int > set)
 
int MatchExcept (int a)
 
int MatchExcept (int a, int b)
 
int MatchExcept (int a, int b, int c)
 
int MatchExcept (int a, int b, int c, int d)
 
int MatchExceptRange (int aLo, int aHi)
 
int MatchExceptRange (int aLo, int aHi, int bLo, int bHi)
 
bool TryMatch (HashSet< int > set)
 
bool TryMatch (int a)
 
bool TryMatch (int a, int b)
 
bool TryMatch (int a, int b, int c)
 
bool TryMatch (int a, int b, int c, int d)
 
bool TryMatchRange (int aLo, int aHi)
 
bool TryMatchRange (int aLo, int aHi, int bLo, int bHi)
 
bool TryMatchExcept ()
 
bool TryMatchExcept (HashSet< int > set)
 
bool TryMatchExcept (int a)
 
bool TryMatchExcept (int a, int b)
 
bool TryMatchExcept (int a, int b, int c)
 
bool TryMatchExcept (int a, int b, int c, int d)
 
bool TryMatchExceptRange (int aLo, int aHi)
 
bool TryMatchExceptRange (int aLo, int aHi, int bLo, int bHi)
 
virtual object IndexToPositionObject (int charIndex)
 
virtual void MatchError (bool inverted, params int[] ranges)
 
virtual void MatchError (bool inverted, IList< int > ranges)
 Handles an error that occurs during Match(), MatchExcept(), MatchRange() or MatchExceptRange() More...
 
virtual void Error (bool inverted, HashSet< int > set)
 
string RangesToString (IList< int > ranges)
 Converts a list of character ranges to a string, e.g. for input list {'*','*','a','z'}, the output is "'*' 'a'..'z'". More...
 
void PrintChar (int c, StringBuilder sb)
 Prints a character as a string, e.g. 'a' -> "'a'", with the special value -1 representing EOF, so PrintChar(-1, ...) == "EOF". More...
 

Additional Inherited Members

- Public static fields inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
static readonly IMessageSink LogExceptionErrorSink
 Throws LogException when it receives an error. Non-errors are sent to MessageSink.Default. More...
 
static readonly IMessageSink FormatExceptionErrorSink
 
- Static Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
static HashSet< int > NewSet (params int[] items)
 
static HashSet< int > NewSetOfRanges (params int[] ranges)
 
- Protected fields inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
int CachedBlockSize = 512
 
int _lineStartAt
 
int _lineNumber = 1
 

Constructor & Destructor Documentation

◆ LexerSource()

Loyc.Syntax.Lexing.LexerSource< CharSrc >.LexerSource ( CharSrc  source,
string  fileName = "",
int  inputPosition = 0,
bool  newSourceFile = true 
)
inline

Initializes LexerSource.

Parameters
sourceA source of characters, e.g. UString.
fileNameA file name associated with the characters, which will be used for error reporting.
inputPositionA location to start lexing (normally 0). Careful: If you're starting to lex in the middle of the file, the BaseLexer<C>.LineNumber still starts at 1, and (if newSourceFile is true) the SourceFile object may or may not discover line breaks prior to the starting point, depending on how it is used.
newSourceFileWhether to create a LexerSourceFile<C> object (an implementation of ISourceFile) to keep track of line boundaries. The SourceFile property will point to this object, and it will be null if this parameter is false. Using 'false' will avoid memory allocation, but prevent you from mapping character positions to line numbers and vice versa. However, this object will still keep track of the current BaseLexer<C>.LineNumber and LineStartAt (the index where the current line started) when this parameter is false.

Member Function Documentation

◆ Error() [1/2]

virtual new void Loyc.Syntax.Lexing.LexerSource< CharSrc >.Error ( int  lookaheadIndex,
string  message 
)
inlinevirtual

This method is called to handle errors that occur during lexing.

Parameters
lookaheadIndexIndex where the error occurred, relative to the current InputPosition (i.e. InputPosition + lookaheadIndex is the position of the error).
messageAn error message, not including the error location.

Reimplemented from Loyc.Syntax.Lexing.BaseLexer< CharSrc >.

Referenced by Loyc.Syntax.Lexing.LexerSource< Loyc.UString >.Error().

◆ Error() [2/2]

virtual new void Loyc.Syntax.Lexing.LexerSource< CharSrc >.Error ( int  lookaheadIndex,
string  format,
params object[]  args 
)
inlinevirtual

This method is called to format and handle errors that occur during lexing. The default implementation sends errors to ErrorSink, which, by default, throws a FormatException.

Parameters
lookaheadIndexIndex where the error occurred, relative to the current InputPosition (i.e. InputPosition + lookaheadIndex is the position of the error).
formatAn error description with argument placeholders.
argsArguments to insert into the error message.

Reimplemented from Loyc.Syntax.Lexing.BaseLexer< CharSrc >.