Enhanced C#
Language of your choice: library documentation
Properties | Public Member Functions | List of all members
Loyc.Syntax.Lexing.TriviaSaver Class Reference

A lexer wrapper that saves whitespace tokens into a list (TriviaList). More...

Source file:
Inheritance diagram for Loyc.Syntax.Lexing.TriviaSaver:
Loyc.Syntax.Lexing.LexerWrapper< Token > Loyc.Syntax.Lexing.ILexer< Token > Loyc.Syntax.IIndexToLine Loyc.Syntax.IHasFileName


A lexer wrapper that saves whitespace tokens into a list (TriviaList).

This wrapper filters out all whitespace tokens (where Token.Value is WhitespaceTag.Value) and saves them in a list. It is typically used with

See also



IListAndListSource< TokenTriviaList [get]
- Properties inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
ILexer< TokenLexer [get, set]
ISourceFile SourceFile [get]
virtual IMessageSink ErrorSink [get, set]
int IndentLevel [get]
UString IndentString [get]
int LineNumber [get]
int InputPosition [get]
string FileName [get]
- Properties inherited from Loyc.Syntax.Lexing.ILexer< Token >
ISourceFile SourceFile [get]
 The file being lexed. More...
IMessageSink ErrorSink [get, set]
 Event handler for errors. More...
int IndentLevel [get]
 Indentation level of the current line. This is updated after scanning the first whitespaces on a new line, and may be reset to zero when NextToken() returns a newline. More...
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
int LineNumber [get]
 Current line number (1 for the first line). More...
int InputPosition [get]
 Current input position (an index into SourceFile.Text). More...
- Properties inherited from Loyc.Syntax.IHasFileName
string FileName [get]

Public Member Functions

 TriviaSaver (ILexer< Token > lexer, int newlineTypeInt=int.MinValue)
 Initializer. More...
override Maybe< TokenNextToken ()
 Returns the next (postprocessed) token. This method should set the _current field to the returned value. More...
- Public Member Functions inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
 LexerWrapper (ILexer< Token > sourceLexer)
ILineColumnFile IndexToLine (int index)
 Returns the position in a source file of the specified index. More...
virtual void Reset ()

Additional Inherited Members

- Protected Member Functions inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
void WriteError (int index, string msg, params object[] args)
- Protected fields inherited from Loyc.Syntax.Lexing.LexerWrapper< Token >
Maybe< Token_current

Constructor & Destructor Documentation

◆ TriviaSaver()

Loyc.Syntax.Lexing.TriviaSaver.TriviaSaver ( ILexer< Token lexer,
int  newlineTypeInt = int.MinValue 


lexerLexer to wrap.
newlineTypeIntIn some languages, newlines are not considered whitespace but they still need to be saved in the trivia list. If the Token.TypeInt equals this value, the token is saved but NOT filtered out.

Member Function Documentation

◆ NextToken()

override Maybe<Token> Loyc.Syntax.Lexing.TriviaSaver.NextToken ( )

Returns the next (postprocessed) token. This method should set the _current field to the returned value.

Implements Loyc.Syntax.Lexing.LexerWrapper< Token >.