An implementation of the LLLPG Lexer API, used with the LLLPG options inputSource and inputClass. More...

Source file:

/Core/Loyc.Syntax/Lexing/LexerSource.cs

Inheritance diagram for Loyc.Syntax.Lexing.LexerSource< CharSrc >:

Remarks

An implementation of the LLLPG Lexer API, used with the LLLPG options inputSource and inputClass.

A synonym for LexerSource<C> where C is ICharSource.

This derived class simply makes public all of the LLLPG APIs which are marked protected in BaseLexer<CharSrc>.

LLLPG(lexer(inputSource(src), inputClass(LexerSource))) { static rule int ParseInt(string input) { var src = (LexerSource<UString>)input; @[ (d:='0'..'9' {$result = $result * 10 + (d - '0');})+ ]; } }

Template Parameters

CharSrc A class that implements ICharSource. In order to write lexers that can accept any source of characters, set CharSrc=ICharSource. For maximum performance when parsing strings (or to avoid memory allocation), set CharSrc=UString (UString is a wrapper around System.String that, among other things, implements ICharSource; please note that C# will implicitly convert normal strings to UString for you).

Type Constraints

CharSrc

:

ICharSource

Properties
new int	LA0 `[get]`

new CharSrc	CharSource `[get]`

new string	FileName `[get]`

new int	InputPosition `[get, protected set]`

new LexerSourceFile< CharSrc >	SourceFile `[get]`

new int	LineStartAt `[get]`

Properties inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
IMessageSink??	ErrorSink `[get, set]`
	Gets or sets the object to which error messages are sent. The default object is LogExceptionErrorSink, which throws an exception if an error occurs. More...

int	LA0 `[get]`

CharSrc	CharSource `[get]`

string	FileName `[get]`

int	InputPosition `[get, protected set]`

LexerSourceFile< CharSrc >	SourceFile `[get]`

int	LineNumber `[get]`
	Current line number. Starts at 1 for the first line, unless derived class changes it. More...

int	LineStartAt `[get]`
	Index at which the current line started. More...

Properties inherited from Loyc.Syntax.IHasFileName
string	FileName `[get]`

Public Member Functions
	LexerSource (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
	Initializes LexerSource. More...

virtual new void	Reset (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)
	Reinitializes the object. This method is called by the constructor.

new void	Reset ()

new int	LA (int i)

new void	Skip ()

virtual new void	AfterNewline ()
	The lexer must call this method exactly once after it advances past each newline, even inside comments and strings. This method keeps the LineNumber and LineStartAt properties updated.

new void	Newline ()

new int	MatchAny ()

new int	Match (HashSet< int > set)

new int	Match (int a)

new int	Match (int a, int b)

new int	Match (int a, int b, int c)

new int	Match (int a, int b, int c, int d)

new int	MatchRange (int aLo, int aHi)

new int	MatchRange (int aLo, int aHi, int bLo, int bHi)

new int	MatchExcept ()

new int	MatchExcept (HashSet< int > set)

new int	MatchExcept (int a)

new int	MatchExcept (int a, int b)

new int	MatchExcept (int a, int b, int c)

new int	MatchExcept (int a, int b, int c, int d)

new int	MatchExceptRange (int aLo, int aHi)

new int	MatchExceptRange (int aLo, int aHi, int bLo, int bHi)

new bool	TryMatch (HashSet< int > set)

new bool	TryMatch (int a)

new bool	TryMatch (int a, int b)

new bool	TryMatch (int a, int b, int c)

new bool	TryMatch (int a, int b, int c, int d)

new bool	TryMatchRange (int aLo, int aHi)

new bool	TryMatchRange (int aLo, int aHi, int bLo, int bHi)

new bool	TryMatchExcept ()

new bool	TryMatchExcept (HashSet< int > set)

new bool	TryMatchExcept (int a)

new bool	TryMatchExcept (int a, int b)

new bool	TryMatchExcept (int a, int b, int c)

new bool	TryMatchExcept (int a, int b, int c, int d)

new bool	TryMatchExceptRange (int aLo, int aHi)

new bool	TryMatchExceptRange (int aLo, int aHi, int bLo, int bHi)

virtual new void	Check (bool expectation, string expectedDescr="")

virtual new void	Error (int lookaheadIndex, string format)
	This method is called to handle errors that occur during lexing. More...

virtual new void	Error (int lookaheadIndex, string format, params object[] args)
	This method is called to format and handle errors that occur during lexing. The default implementation sends errors to ErrorSink, which, by default, throws a FormatException. More...

new void	PrintChar (int c, StringBuilder sb)

	LexerSource (ICharSource source, string fileName="", int inputPosition=0, bool newSourceFile=true)

Public Member Functions inherited from Loyc.Syntax.Lexing.LexerSourceWorkaround< CharSrc >
	LexerSourceWorkaround (CharSrc source, string fileName="", int inputPosition=0, bool newSourceFile=true)

Public Member Functions inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
	BaseLexer (CharSrc chars, string fileName="", int inputPosition=0, bool newSourceFile=true)
	Initializes BaseLexer. More...

LineColumnFile	IndexToLine (int charIndex)
	Returns the position in a source file of the specified index. More...

Public Member Functions inherited from Loyc.Syntax.Lexing.ILllpgLexerApi< int >
Token	MatchRange (int aLo, int aHi)

Token	MatchRange (int aLo, int aHi, int bLo, int bHi)

Token	MatchExceptRange (int aLo, int aHi)

Token	MatchExceptRange (int aLo, int aHi, int bLo, int bHi)

bool	TryMatchRange (int aLo, int aHi)

bool	TryMatchRange (int aLo, int aHi, int bLo, int bHi)

bool	TryMatchExceptRange (int aLo, int aHi)

bool	TryMatchExceptRange (int aLo, int aHi, int bLo, int bHi)

Static Public Member Functions
static new HashSet< int >	NewSet (params int[] items)

static new HashSet< int >	NewSetOfRanges (params int[] ranges)

static	operator LexerSource< CharSrc > (CharSrc str)

static	operator LexerSource (string str)

Protected Member Functions
override void	Error_Renamed (int lookaheadIndex, string format, params object[] args)

override void	Error_Renamed (int lookaheadIndex, string format)

Protected Member Functions inherited from Loyc.Syntax.Lexing.LexerSourceWorkaround< CharSrc >
override void	Error (int lookaheadIndex, string format)
	This method is called to handle errors that occur during lexing. More...

override void	Error (int lookaheadIndex, string format, params object[] args)
	This method is called to format and handle errors that occur during lexing. The default implementation sends errors to ErrorSink, which, by default, throws a FormatException. More...

Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
void	Reset ()

int	LA (int i)

void	Skip ()
	Increments InputPosition. Called by LLLPG when prediction already verified the input (and caller doesn't save LA(0)) More...

	BaseLexer (ICharSource source, string fileName="", int inputPosition=0, bool newSourceFile=true)

void	Newline ()
	Default newline parser that matches ' ' or '\r' unconditionally. More...

void	Spaces ()
	Skips past any spaces at the current position. Equivalent to `rule Spaces @[ (' '\|'\t')* ]` in LLLPG. More...

int	MatchAny ()

int	Match (HashSet< int > set)

int	Match (int a)

int	Match (int a, int b)

int	Match (int a, int b, int c)

int	Match (int a, int b, int c, int d)

int	MatchRange (int aLo, int aHi)

int	MatchRange (int aLo, int aHi, int bLo, int bHi)

int	MatchExcept ()

int	MatchExcept (HashSet< int > set)

int	MatchExcept (int a)

int	MatchExcept (int a, int b)

int	MatchExcept (int a, int b, int c)

int	MatchExcept (int a, int b, int c, int d)

int	MatchExceptRange (int aLo, int aHi)

int	MatchExceptRange (int aLo, int aHi, int bLo, int bHi)

bool	TryMatch (HashSet< int > set)

bool	TryMatch (int a)

bool	TryMatch (int a, int b)

bool	TryMatch (int a, int b, int c)

bool	TryMatch (int a, int b, int c, int d)

bool	TryMatchRange (int aLo, int aHi)

bool	TryMatchRange (int aLo, int aHi, int bLo, int bHi)

bool	TryMatchExcept ()

bool	TryMatchExcept (HashSet< int > set)

bool	TryMatchExcept (int a)

bool	TryMatchExcept (int a, int b)

bool	TryMatchExcept (int a, int b, int c)

bool	TryMatchExcept (int a, int b, int c, int d)

bool	TryMatchExceptRange (int aLo, int aHi)

bool	TryMatchExceptRange (int aLo, int aHi, int bLo, int bHi)

virtual object	IndexToPositionObject (int charIndex)

virtual void	MatchError (bool inverted, params int[] ranges)

virtual void	MatchError (bool inverted, IList< int > ranges)
	Handles an error that occurs during Match(), MatchExcept(), MatchRange() or MatchExceptRange() More...

virtual void	Error (bool inverted, HashSet< int > set)

string	RangesToString (IList< int > ranges)
	Converts a list of character ranges to a string, e.g. for input list {'','','a','z'}, the output is "'*' 'a'..'z'". More...

void	PrintChar (int c, StringBuilder sb)
	Prints a character as a string, e.g. `'a' -> "'a'"`, with the special value -1 representing EOF, so PrintChar(-1, ...) == "EOF". More...

Additional Inherited Members
Public static fields inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
static readonly IMessageSink	LogExceptionErrorSink
	Throws LogException when it receives an error. Non-errors are sent to MessageSink.Default. More...

static readonly IMessageSink	FormatExceptionErrorSink

Static Protected Member Functions inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
static HashSet< int >	NewSet (params int[] items)

static HashSet< int >	NewSetOfRanges (params int[] ranges)

Protected fields inherited from Loyc.Syntax.Lexing.BaseLexer< CharSrc >
int	CachedBlockSize = 512

int	_lineStartAt

int	_lineNumber = 1

Constructor & Destructor Documentation

◆ LexerSource()

Loyc.Syntax.Lexing.LexerSource< CharSrc >.LexerSource	(	CharSrc	source,
		string	fileName = `""`,
		int	inputPosition = `0`,
		bool	newSourceFile = `true`
	)

inline

Initializes LexerSource.

Parameters

source	A source of characters, e.g. UString.
fileName	A file name associated with the characters, which will be used for error reporting.
inputPosition	A location to start lexing (normally 0). Careful: If you're starting to lex in the middle of the file, the BaseLexer<C>.LineNumber still starts at 1, and (if `newSourceFile` is true) the SourceFile object may or may not discover line breaks prior to the starting point, depending on how it is used.
newSourceFile	Whether to create a LexerSourceFile<C> object (an implementation of ISourceFile) to keep track of line boundaries. The SourceFile property will point to this object, and it will be null if this parameter is false. Using 'false' will avoid memory allocation, but prevent you from mapping character positions to line numbers and vice versa. However, this object will still keep track of the current BaseLexer<C>.LineNumber and LineStartAt (the index where the current line started) when this parameter is false.

Member Function Documentation

◆ Error() [1/2]

virtual new void Loyc.Syntax.Lexing.LexerSource< CharSrc >.Error	(	int	lookaheadIndex,
		string	message
	)

inlinevirtual

This method is called to handle errors that occur during lexing.

Parameters

lookaheadIndex	Index where the error occurred, relative to the current InputPosition (i.e. InputPosition + lookaheadIndex is the position of the error).
message	An error message, not including the error location.

Reimplemented from Loyc.Syntax.Lexing.BaseLexer< CharSrc >.

Referenced by Loyc.Syntax.Lexing.LexerSource< Loyc.UString >.Error().

◆ Error() [2/2]

virtual new void Loyc.Syntax.Lexing.LexerSource< CharSrc >.Error	(	int	lookaheadIndex,
		string	format,
		params object[]	args
	)

inlinevirtual

This method is called to format and handle errors that occur during lexing. The default implementation sends errors to ErrorSink, which, by default, throws a FormatException.

Parameters

lookaheadIndex	Index where the error occurred, relative to the current InputPosition (i.e. InputPosition + lookaheadIndex is the position of the error).
format	An error description with argument placeholders.
args	Arguments to insert into the error message.

Reimplemented from Loyc.Syntax.Lexing.BaseLexer< CharSrc >.

Remarks

Properties

Public Member Functions

Static Public Member Functions

Protected Member Functions

Additional Inherited Members

Constructor & Destructor Documentation

◆ LexerSource()

Member Function Documentation

◆ Error() [1/2]

◆ Error() [2/2]