Enhanced C#
Language of your choice: library documentation
Public fields | Public Member Functions | List of all members
Loyc.Syntax.Lexing.TokenTree Class Reference

A list of Token structures along with the ISourceFile object that represents the source file that the tokens came from. More...


Source file:
Inheritance diagram for Loyc.Syntax.Lexing.TokenTree:
Loyc.Collections.DList< Token > Loyc.Collections.IListSource< Token > Loyc.ICloneable< TokenTree >

Remarks

A list of Token structures along with the ISourceFile object that represents the source file that the tokens came from.

This class is called TokenTree because certain kinds of tokens used by some parsers are formed into trees by using TokenTree as the type of the Token.Value of certain tokens. Specifically, the LES and EC# parsers expect open-bracket and open-brace tokens ('(', '[' and '{') to have a child TokenTree that contains all the tokens within a pair of brackets or braces. Typically this tree is not created directly by the lexer, but by a helper class (TokensToTree).

Caution: this class is mutable, even though TokenTrees are sometimes stored in LNodes which are supposed to be immutable. Please do not modify token trees that are stored inside LNodes.

Public fields

readonly ISourceFile File
 

Public Member Functions

 TokenTree (ISourceFile file, int capacity)
 
 TokenTree (ISourceFile file, ICollectionAndReadOnly< Token > items)
 
 TokenTree (ISourceFile file, IReadOnlyCollection< Token > items)
 
 TokenTree (ISourceFile file, ICollection< Token > items)
 
 TokenTree (ISourceFile file, IEnumerable< Token > items)
 
 TokenTree (ISourceFile file, Token[] items)
 
 TokenTree (ISourceFile file)
 
new TokenTree Clone ()
 Gets a deep (recursive) clone of the token tree. More...
 
TokenTree Clone (bool deep)
 
override string ToString ()
 
string ToString (Func< Token, string > toStringStrategy=null)
 
override bool Equals (object obj)
 
bool Equals (TokenTree other)
 Compares the elements of the token tree for equality. More...
 
override int GetHashCode ()
 
VList< LNodeToLNodes ()
 Converts this list of Token to a list of LNode. More...
 
DList< TokenFlatten ()
 Converts a token tree back to a plain list. More...
 
- Public Member Functions inherited from Loyc.Collections.DList< Token >
 DList (int capacity)
 
 DList (IReadOnlyCollection< T > items)
 
 DList (ICollection< T > items)
 
 DList (ICollectionAndReadOnly< T > items)
 
 DList (IEnumerable< T > items)
 
 DList (T[] items)
 
int IndexOf (T item)
 
void PushLast (ICollection< T > items)
 
void PushLast (IEnumerable< T > items)
 
void PushLast (IReadOnlyCollection< T > items)
 
void PushLast (ICollectionAndReadOnly< T > items)
 
void PushLast (T item)
 
void PushFirst (T item)
 
void PopLast (int amount)
 
void PopFirst (int amount)
 
void Insert (int index, T item)
 
void InsertRange (int index, ICollection< T > items)
 
void InsertRange (int index, IReadOnlyCollection< T > items)
 
void InsertRange (int index, ICollectionAndReadOnly< T > items)
 
void InsertRange (int index, IEnumerable< T > e)
 
void AddRange (ICollectionAndReadOnly< T > c)
 
void AddRange (ICollection< T > c)
 
void AddRange (IReadOnlyCollection< T > s)
 
void AddRange (IEnumerable< T > e)
 
void RemoveAt (int index)
 
void RemoveRange (int index, int amount)
 
int RemoveAll (Predicate< T > condition)
 Removes the all the elements that match the conditions defined by the specified predicate. More...
 
bool TrySet (int index, T value)
 
TryGet (int index, out bool fail)
 
void Add (T item)
 An alias for PushLast(). More...
 
new int Add (object obj)
 
void Clear ()
 
bool Contains (T item)
 
void CopyTo (T[] array, int arrayIndex)
 
void CopyTo (int sourceIndex, T[] destination, int destinationIndex, int subcount)
 
void CopyTo (Array array, int arrayIndex)
 
bool Remove (T item)
 
new void Remove (object obj)
 
IEnumerator< T > GetEnumerator ()
 
Maybe< T > TryPopFirst ()
 
Maybe< T > TryPeekFirst ()
 
Maybe< T > TryPopLast ()
 
Maybe< T > TryPeekLast ()
 
int BinarySearch (T k, Comparer< T > comp)
 
int BinarySearch< K > (K k, Func< T, K, int > comp)
 
void Resize (int newSize)
 
DList< T > Clone ()
 
DList< T > CopySection (int start, int subcount)
 
void Sort (Comparison< T > comp)
 
void Sort (int index, int count, Comparison< T > comp)
 
ListSlice< T > Slice (int start, int count)
 
- Public Member Functions inherited from Loyc.Collections.IListSource< Token >
TryGet (int index, out bool fail)
 Gets the item at the specified index, and does not throw an exception on failure. More...
 
IRange< T > Slice (int start, int count=int.MaxValue)
 Returns a sub-range of this list. More...
 
- Public Member Functions inherited from Loyc.ICloneable< TokenTree >
Clone ()
 

Additional Inherited Members

- Properties inherited from Loyc.Collections.DList< Token >
int Capacity [get, set]
 
this[int index] [get, set]
 
this[int index, T defaultValue] [get]
 
int Count [get]
 
bool IsReadOnly [get]
 
First [get, set]
 
Last [get, set]
 
bool IsEmpty [get]
 
bool IsFixedSize [get]
 
bool IsSynchronized [get]
 
object SyncRoot [get]
 
- Protected fields inherited from Loyc.Collections.DList< Token >
InternalDList< T > _dlist
 

Member Function Documentation

new TokenTree Loyc.Syntax.Lexing.TokenTree.Clone ( )
inline
bool Loyc.Syntax.Lexing.TokenTree.Equals ( TokenTree  other)
inline

Compares the elements of the token tree for equality.

Because LNodes are compared by value and not by reference, and LNodes can contain TokenTrees, TokenTrees should also be compared by value.

DList<Token> Loyc.Syntax.Lexing.TokenTree.Flatten ( )
inline

Converts a token tree back to a plain list.

References Loyc.Collections.DList< T >.Add().

VList<LNode> Loyc.Syntax.Lexing.TokenTree.ToLNodes ( )
inline

Converts this list of Token to a list of LNode.

See Token.ToLNode(ISourceFile) for more information.