Enhanced C#
Language of your choice: library documentation
Properties | Public Member Functions | List of all members
Loyc.Syntax.Lexing.TriviaSaver Class Reference

A lexer wrapper that saves whitespace tokens into a list (TriviaList). More...

Source file:
Inheritance diagram for Loyc.Syntax.Lexing.TriviaSaver:
Loyc.Syntax.Lexing.LexerWrapper< Token > Loyc.Syntax.Lexing.ILexer< Token > Loyc.Syntax.IIndexToLine


A lexer wrapper that saves whitespace tokens into a list (TriviaList).

Typically used with

See also



IListAndListSource< TokenTriviaList [get]
- Properties inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
ILexer< TokenLexer [get, set]
ISourceFile SourceFile [get]
virtual IMessageSink ErrorSink [get, set]
int IndentLevel [get]
UString IndentString [get]
int LineNumber [get]
int InputPosition [get]
string FileName [get]
- Properties inherited from Loyc.Syntax.Lexing.ILexer< Token >
ISourceFile SourceFile [get]
 The file being lexed. More...
IMessageSink ErrorSink [get, set]
 Event handler for errors. More...
int IndentLevel [get]
 Indentation level of the current line. This is updated after scanning the first whitespaces on a new line, and may be reset to zero when NextToken() returns a newline. More...
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
int LineNumber [get]
 Current line number (1 for the first line). More...
int InputPosition [get]
 Current input position (an index into SourceFile.Text). More...
- Properties inherited from Loyc.Syntax.IIndexToLine
string FileName [get]
 Gets the file name used in results returned by IndexToLine(int). More...

Public Member Functions

 TriviaSaver (ILexer< Token > lexer, int newlineTypeInt=int.MinValue)
 Initializer. More...
override Maybe< TokenNextToken ()
 Returns the next (postprocessed) token. This method should set the _current field to the returned value. More...
- Public Member Functions inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
 LexerWrapper (ILexer< Token > sourceLexer)
SourcePos IndexToLine (int index)
 Returns the position in a source file of the specified index. More...
virtual void Reset ()

Additional Inherited Members

- Protected Member Functions inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
void WriteError (int index, string msg, params object[] args)
- Protected fields inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
Maybe< Token_current

Constructor & Destructor Documentation

Loyc.Syntax.Lexing.TriviaSaver.TriviaSaver ( ILexer< Token lexer,
int  newlineTypeInt = int.MinValue 


lexerLexer to wrap.
newlineTypeIntThis wrapper filters out and saves whitespace tokens (where Token.Value is WhitespaceTag.Value) but in some languages, newlines are not considered whitespace but they still need to be saved in the trivia list. If the Token.TypeInt equals this value, the token is saved but NOT filtered out.

Member Function Documentation

override Maybe<Token> Loyc.Syntax.Lexing.TriviaSaver.NextToken ( )

Returns the next (postprocessed) token. This method should set the _current field to the returned value.

Implements Loyc.Syntax.Lexing.LexerWrapper< Token >.