General-purpose code generator that supports any language with a finite number of input symbols represented by LNode expressions. This is the code generator helper for LLLPG parser {...}. More...

Source file:

/Main/LLLPG/CodeGenHelpers/GeneralCodeGenHelper.cs

Inheritance diagram for Loyc.LLParserGenerator.GeneralCodeGenHelper:

Remarks

General-purpose code generator that supports any language with a finite number of input symbols represented by LNode expressions. This is the code generator helper for LLLPG parser {...}.

To use, assign a new instance of this class to LLParserGenerator.CodeGenHelper

This code generator operates on sets of LNodes. It assumes that every expression in a set is a unique terminal; for example, it assumes that the expressions 123 and Foo represent two different terminals. The expected data type of each terminal is given to the constructor (the default is int).

Public fields
LNode	LaType
	Specifies the data type of LA0 and lookahead variables. More...

LNode	SetType
	Specifies the data type for large terminal sets (default: HashSet<T>). More...

bool	AllowSwitch
	Specified whether this class is allowed to generate C# switch() statements. More...

LNode	MatchCast
	If MatchCast is set, a cast to this type is added when calling Match() or NewSet() or set.Contains(). More...

Public static fields
static readonly LNode	EOF = F_.Id("EOF")

Properties
bool	CastLA `[get, set]`
	Whether to cast the result of LA0 and LA(i) to LaType (default: true) More...

override IPGTerminalSet	EmptySet `[get]`

Properties inherited from Loyc.LLParserGenerator.CodeGenHelperBase
LNode	InputSource `[get, set]`
	Specifies an object or class on which LLLPG APIs such as Match() and LA() should be called. More...

LNode	InputClass `[get, set]`
	Specifies a class or namespace to use when calling static functions. There is only one currently: NewSet(), which applies only to . More...

LNode	TerminalType `[get, set]`
	The type returned from Match() methods. More...

LNode	ListType `[get, set]`
	Gets or sets the type of lists created with the +: operator (default: List<T>). The identifier "T" should appear in the expression; it will be replaced with the type of items in the list. More...

LNode	ListInitializer `[get, set]`
	Gets or sets the initializer expression for lists created with the +: operator (default: new List<T>()). The identifier "T" should appear in the expression; it will be replaced with the type of items in the list. More...

bool	NoCheckByDefault `[get, set]`
	If true, calls to `Check()` are suppressed when AndPred.CheckErrorMessage is null. More...

abstract IPGTerminalSet	EmptySet `[get]`

virtual int	BaseCostForSwitch `[get]`
	Used to help decide whether a "switch" or an if-else chain will be used for prediction. This is the starting cost of a switch (the starting cost of an if-else chain is set to zero). More...

Properties inherited from Loyc.LLParserGenerator.IPGCodeGenHelper
IPGTerminalSet	EmptySet `[get]`
	Returns an empty set of the appropriate type for the kind of parser being generated by this code. More...

LNode	TerminalType `[get]`
	Type of variables auto-declared when you use labels in your grammar (e.g. x:Foo (list+:Bar)*) More...

Public Member Functions
	GeneralCodeGenHelper (LNode laType=null, bool allowSwitch=true)

	GeneralCodeGenHelper (LNode laType, LNode setType=null, bool allowSwitch=true)

override Pred	CodeToTerminalPred (LNode expr, ref string errorMsg)
	Creates a terminal predicate from a code expression. More...

override string	Example (IPGTerminalSet set_)
	Returns an example of an item in the set. If the example is a character, it should be surrounded by single quotes. More...

override LNode	GenerateMatchExpr (IPGTerminalSet set_, bool savingResult, bool recognizerMode)
	Generate code to match a set, e.g. `MatchRange('a', 'z');` or `MatchExcept(' ', '\r'); }`. If the set is too complex, a declaration for it is created in classBody. More...

override LNode	LAType ()
	Returns the data type of LA(k) More...

override LNode	LA (int k)
	Generates code to read LA(k). More...

Public Member Functions inherited from Loyc.LLParserGenerator.CodeGenHelperBase
void	SetListInitializer (LNode varDecl)
	Sets ListType and/or ListInitializer based on an expression. A statement like `Type x = expr` sets `ListType = Type` and `ListInitializer = expr`; A statement like `Type x` just sets `ListType = Type`; and any other expression `expr` sets `ListInitializer = expr`. More...

LNode	ResolveAlias (LNode expr)
	Returns the node for an alias. If the specified node is not an alias, returns the same node unchanged. More...

virtual LNode	VisitInput (LNode stmt, IMessageSink sink)
	In case the IPGCodeGenHelper is interested, the LLLPG macro calls this method on each statement in the body of the macro (as a preprocessing step, before LLLPG looks at it). No action is required. More...

virtual IPGTerminalSet	Optimize (IPGTerminalSet set, IPGTerminalSet dontcare)
	Simplifies the specified set, if possible, so that GenerateTest() can generate simpler code for an if-else chain in a prediction tree. More...

virtual ? char	ExampleChar (IPGTerminalSet set)
	Returns an example of a character in the set, or null if this is not a set of characters or if EOF is the only member of the set. More...

virtual void	Begin (WList< LNode > classBody, ISourceFile sourceFile)
	Before the parser generator generates code, it calls this method. More...

virtual void	BeginRule (Rule rule)
	Notifies the snippet generator that code generation is starting for a new rule. More...

virtual void	Done ()
	LLParserGenerator calls this method to notify the snippet generator that code generation is complete. More...

virtual LNode	GenerateTest (IPGTerminalSet set, LNode laVar)
	Generates code to test whether the terminal denoted 'laVar' is in the set. More...

virtual LNode	GenerateSkip (bool savingResult)
	Returns `(Skip())`, or `(MatchAny())` if the result is to be saved. More...

virtual LNode	GenerateAndPredCheck (AndPred andPred, LNode code, int li)
	Generate code to check an and-predicate during or after prediction, e.g. &!{foo} becomes !(foo) during prediction and Check(!(foo)); afterward. More...

virtual LNode	GenerateMatch (IPGTerminalSet set, bool savingResult, bool recognizerMode)
	Generate code to match a set, e.g. `MatchRange('a', 'z');` or `MatchExcept(' ', '\r'); }`. If the set is too complex, a declaration for it is created in the `classBody` which was passed to `Begin()`. More...

virtual LNode	ErrorBranch (IPGTerminalSet covered, int laIndex)
	Generates code for the default error branch of prediction (called when there is no explicit error branch). More...

virtual bool	ShouldGenerateSwitch (IPGTerminalSet[] sets, MSet< int > casesToInclude, bool hasErrorBranch)
	Decides whether to use a switch() and for which cases, using BaseCostForSwitch and GetRelativeCostForSwitch. More...

virtual LNode	GenerateSwitch (IPGTerminalSet[] branchSets, LNode[] branchCode, MSet< int > casesToInclude, LNode defaultBranch, LNode laVar)
	Generates a switch statement with the specified branches where branchCode[i] is the code to run if the input is in the set branchSets[i]. More...

virtual LNode	CreateRuleMethod (Rule rule, LNodeList methodBody)
	Generates the method for a rule, given the method's contents. More...

LNode	CreateTryWrapperForRecognizer (Rule rule)
	See IPGCodeGenHelper.CreateTryWrapperForRecognizer for more information. More...

virtual LNode	CallRule (RuleRef rref, bool recognizerMode)
	Generates code to call a rule based on `rref.Rule.Name` and `rref.Params`. More...

virtual LNode	CallTryRecognizer (RuleRef rref, int lookahead)
	Generates a call to the Try_Scan_() function that wraps around a Scan_() recognizer. Called while generating code for an and-pred. More...

virtual LNode	GetListType (LNode type)
	Gets the list type for elements of the specified type (e.g. List<type>) More...

virtual LNode	MakeInitializedVarDecl (LNode type, bool wantList, Symbol varName)
	Gets a variable declaration for the specified type, e.g. if type is `Foo` and `wantList == true` and `varName.Name == "x"`, the statement returned might be `List<Foo> x = new List<Foo>();` More...

Public Member Functions inherited from Loyc.LLParserGenerator.IPGCodeGenHelper

Protected Member Functions
override LNode	GenerateTest (IPGTerminalSet set_, LNode subject, Symbol setName)
	Generates code to test whether a terminal is in the set. More...

override LNode	GenerateSetDecl (IPGTerminalSet set_, Symbol setName)
	Generates a declaration for a variable that holds the set. More...

override int	GetRelativeCostForSwitch (IPGTerminalSet set_)
	Used to help decide whether a "switch" or an if statement will be used to handle a prediction tree, and if so which branches. This method should calculate the "cost of switch" (which generally represents a code size penalty, as there is a separate case for every element of the set) and the "cost of if" (which generally represents a speed penalty) and return the difference (so that positive numbers favor "switch" and negative numbers favor "if".) More...

override IEnumerable< LNode >	GetCases (IPGTerminalSet set_)
	Gets the literals or symbols to use for switch cases of a set (just the values, not including the case labels.) More...

Protected Member Functions inherited from Loyc.LLParserGenerator.CodeGenHelperBase
virtual Symbol	GenerateSetName (Rule currentRule)

virtual Symbol	GenerateSetDecl (IPGTerminalSet set)

virtual LNode	ApiCall (Symbol apiName, params LNode[] args)
	Returns an LNode representing a call to the specified LLLPG API. For example, if the user used a "inputSource=input" option, then `ApiCall(_Match, F.Literal('7'))` would generate a node that represents `input.Match('7')`. More...

virtual LNode	ApiCall (Symbol apiName, IEnumerable< LNode > args, bool isStatic=false)
	Returns an LNode representing a call to the specified LLLPG API. For example, if the user used a "inputSource=input" option, then `ApiCall(_Match, F.Literal('7'))` would generate a node that represents `input.Match('7')`. More...

virtual LNode	ApiType (LNode typeName)

virtual LNode	DefaultOf (LNode type, bool wantList)

Protected static fields
static readonly Symbol	_Symbol = GSymbol.Get("Symbol")

static readonly Symbol	_HashSet = GSymbol.Get("HashSet")

static readonly Symbol	_NewSet = GSymbol.Get("NewSet")

Protected static fields inherited from Loyc.LLParserGenerator.CodeGenHelperBase
static readonly Symbol	_Skip = GSymbol.Get("Skip")

static readonly Symbol	_MatchAny = GSymbol.Get("MatchAny")

static readonly Symbol	_Match = GSymbol.Get("Match")

static readonly Symbol	_MatchExcept = GSymbol.Get("MatchExcept")

static readonly Symbol	_MatchRange = GSymbol.Get("MatchRange")

static readonly Symbol	_MatchExceptRange = GSymbol.Get("MatchExceptRange")

static readonly Symbol	_TryMatch = GSymbol.Get("TryMatch")

static readonly Symbol	_TryMatchExcept = GSymbol.Get("TryMatchExcept")

static readonly Symbol	_TryMatchRange = GSymbol.Get("TryMatchRange")

static readonly Symbol	_TryMatchExceptRange = GSymbol.Get("TryMatchExceptRange")

static readonly Symbol	_LA = GSymbol.Get("LA")

static readonly Symbol	_LA0 = GSymbol.Get("LA0")

static readonly Symbol	_Check = GSymbol.Get("Check")

static readonly Symbol	_Error = GSymbol.Get("Error")

static readonly Symbol	_underscore = GSymbol.Get("_")

static readonly Symbol	_alias = GSymbol.Get("alias")

static readonly Symbol	_T = GSymbol.Get("T")

Additional Inherited Members
Static Protected Member Functions inherited from Loyc.LLParserGenerator.CodeGenHelperBase
static bool	EndMayBeReachable (LNode stmt)

Protected fields inherited from Loyc.LLParserGenerator.CodeGenHelperBase
int	_setNameCounter = 0

LNodeFactory	F

WList< LNode >	_classBody

Rule	_currentRule

Dictionary< LNode, LNode >	_definedAliases = new Dictionary<LNode, LNode>()

Member Function Documentation

◆ CodeToTerminalPred()

override Pred Loyc.LLParserGenerator.GeneralCodeGenHelper.CodeToTerminalPred	(	LNode	expr,
		ref string	errorMsg
	)

inlinevirtual

Creates a terminal predicate from a code expression.

Parameters

expr	A expression provided by the user, such as `"a string"`, a `Token.Type`, or a `value..range`. `expr` will not be a call to the inversion operator #~ (that's handled internally using IPGTerminalSet.Inverted()). This method also handles the "any token" input, which is an underscore by convention (_).
errorMsg	An error message to display. If the method returns null, the LLLPG macro shows this as an error; if this method does not return null, the message (if provided) is shown as a warning.

Returns: If successful, a terminal predicate; otherwise null.

Implements Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.Syntax.CodeSymbols.Dot, Loyc.Syntax.LNode.Equals(), Loyc.Syntax.LNode.Name, Loyc.Syntax.CodeSymbols.Of, and Loyc.LLParserGenerator.CodeGenHelperBase.ResolveAlias().

◆ Example()

override string Loyc.LLParserGenerator.GeneralCodeGenHelper.Example ( IPGTerminalSet set )

inlinevirtual

Returns an example of an item in the set. If the example is a character, it should be surrounded by single quotes.

This helps produce error messages in LLLPG.

Implements Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.Syntax.ParsingMode.Expressions.

◆ GenerateMatchExpr()

override LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateMatchExpr	(	IPGTerminalSet	set,
		bool	savingResult,
		bool	recognizerMode
	)

inlinevirtual

Generate code to match a set, e.g. MatchRange('a', 'z'); or MatchExcept(' ', '\r'); }. If the set is too complex, a declaration for it is created in classBody.

Implements Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.LLParserGenerator.CodeGenHelperBase.ApiCall(), and Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateSetDecl().

◆ GenerateSetDecl()

override LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateSetDecl	(	IPGTerminalSet	set,
		Symbol	setName
	)

inlineprotectedvirtual

Generates a declaration for a variable that holds the set.

For example, if setName is foo, a set such as [aeiouy] might use an external declaration such as

HashSet<int> foo = NewSet('a', 'e', 'i', 'o', 'u', 'y');

Implements Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.Syntax.CodeSymbols.Cast, Loyc.LLParserGenerator.GeneralCodeGenHelper.MatchCast, and Loyc.LLParserGenerator.GeneralCodeGenHelper.SetType.

Referenced by Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateMatchExpr().

◆ GenerateTest()

override LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateTest	(	IPGTerminalSet	set,
		LNode	subject,
		Symbol	setName
	)

inlineprotectedvirtual

Generates code to test whether a terminal is in the set.

Parameters

subject	Represents the variable to be tested.
setName	Names an external set variable to use for the test.

Returns: A test expression such as (la0 >= '0' && '9' >= la0), or null if an external setName is needed and was not provided.

At first, LLParserGenerator calls this method with setName == null. If it returns null, it calls the method a second time, giving the name of an external variable in which the set is held (see GenerateSetDecl(IPGTerminalSet)).

For example, if the subject is @la0, the test for a simple set like [a-z?] might be something like (la0 >= 'a' && 'z' >= la0) || la0 == '?'. When the setName is foo, the test might be foo.Contains(la0) instead.

Implements Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.Syntax.CodeSymbols.Cast, Loyc.Syntax.CodeSymbols.Eq, Loyc.LLParserGenerator.CodeGenHelperBase.InputClass, Loyc.LLParserGenerator.GeneralCodeGenHelper.LaType, Loyc.LLParserGenerator.GeneralCodeGenHelper.MatchCast, Loyc.Syntax.CodeSymbols.Not, Loyc.Syntax.CodeSymbols.NotEq, and Loyc.Syntax.CodeSymbols.Or.

◆ GetCases()

override IEnumerable<LNode> Loyc.LLParserGenerator.GeneralCodeGenHelper.GetCases ( IPGTerminalSet set )

inlineprotectedvirtual

Gets the literals or symbols to use for switch cases of a set (just the values, not including the case labels.)

Reimplemented from Loyc.LLParserGenerator.CodeGenHelperBase.

◆ GetRelativeCostForSwitch()

override int Loyc.LLParserGenerator.GeneralCodeGenHelper.GetRelativeCostForSwitch ( IPGTerminalSet set )

inlineprotectedvirtual

Used to help decide whether a "switch" or an if statement will be used to handle a prediction tree, and if so which branches. This method should calculate the "cost of switch" (which generally represents a code size penalty, as there is a separate case for every element of the set) and the "cost of if" (which generally represents a speed penalty) and return the difference (so that positive numbers favor "switch" and negative numbers favor "if".)

If the set is inverted, return a something like -1000000 to ensure 'switch' is not used for that set.

Reimplemented from Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.LLParserGenerator.GeneralCodeGenHelper.AllowSwitch.

◆ LA()

override LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.LA ( int k )

inlinevirtual

Generates code to read LA(k).

Reimplemented from Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.Syntax.CodeSymbols.Cast, Loyc.LLParserGenerator.GeneralCodeGenHelper.CastLA, and Loyc.LLParserGenerator.GeneralCodeGenHelper.LaType.

◆ LAType()

override LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.LAType ( )

inlinevirtual

Returns the data type of LA(k)

Implements Loyc.LLParserGenerator.CodeGenHelperBase.

References Loyc.LLParserGenerator.GeneralCodeGenHelper.LaType.

Member Data Documentation

◆ AllowSwitch

bool Loyc.LLParserGenerator.GeneralCodeGenHelper.AllowSwitch

Specified whether this class is allowed to generate C# switch() statements.

C# switch() only allows constant values as cases. If the token values are not constants (e.g. if they are symbols), you'll have to disable switch generation.

Referenced by Loyc.LLParserGenerator.GeneralCodeGenHelper.GetRelativeCostForSwitch().

◆ LaType

LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.LaType

Specifies the data type of LA0 and lookahead variables.

Referenced by Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateTest(), Loyc.LLParserGenerator.GeneralCodeGenHelper.LA(), and Loyc.LLParserGenerator.GeneralCodeGenHelper.LAType().

◆ MatchCast

LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.MatchCast

If MatchCast is set, a cast to this type is added when calling Match() or NewSet() or set.Contains().

This requires some explanation because it's a bit subtle. I made the decision to implement BaseParser<Token> with Match(...) methods that accept integers, e.g. Match(int a, int b, int c). I could have parameterized BaseParser and its Match methods on the token type (e.g. BaseParser(Token,TokenType)) but unfortunately this lowers performance because if BaseParser doesn't know that TokenType is an integer or an enum, it requires three virtual method calls to compare the current token with a, b and c (also, note that C# prohibits "enum" as a generic constraint for reasons unknown).

To avoid this performance snag, BaseParser just assumes that the token type is an integer. Of course, the derived class will still use named enum values. If the enum type is called TT, Match(TT.A, TT.B, TT.C) produces a C# compiler error, so LLLPG needs to generate a cast to int: Match((int) TT.A, (int) TT.B, (int) TT.C). That's what this option is for. When you set this option, it inserts a cast to the specified type. Normally you'll set it to #int32.

When using this option, LaType should still be the enum type rather than #int32.

Referenced by Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateSetDecl(), and Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateTest().

◆ SetType

LNode Loyc.LLParserGenerator.GeneralCodeGenHelper.SetType

Specifies the data type for large terminal sets (default: HashSet<T>).

Referenced by Loyc.LLParserGenerator.GeneralCodeGenHelper.GenerateSetDecl().

Property Documentation

◆ CastLA

bool Loyc.LLParserGenerator.GeneralCodeGenHelper.CastLA

getset

Whether to cast the result of LA0 and LA(i) to LaType (default: true)

Referenced by Loyc.LLParserGenerator.GeneralCodeGenHelper.LA().

Remarks

Public fields

Public static fields

Properties

Public Member Functions

Protected Member Functions

Protected static fields

Additional Inherited Members

Member Function Documentation

◆ CodeToTerminalPred()

◆ Example()

◆ GenerateMatchExpr()

◆ GenerateSetDecl()

◆ GenerateTest()

◆ GetCases()

◆ GetRelativeCostForSwitch()

◆ LA()

◆ LAType()

Member Data Documentation

◆ AllowSwitch

◆ LaType

◆ MatchCast

◆ SetType

Property Documentation

◆ CastLA