Repository: lark-parser/lark

Issue:
Syntactic sugar: Tree.find_token()
**Suggestion**
Add new method `Tree.find_type(typ: str) -> Iterator[Token]` as a companion for `Tree.find_data()`:
```python
for term in tree.find_type('TERM'):
    ...
```

**Describe alternatives you've considered**
Right now this can be achieved by
```python
for term in tree.find_pred(lambda node: isinstance(node, Token) and node.type == 'TERM'):
    ...
```
which doesn't fit Black's line width and will look even uglier in real cases.

**Additional context**
I can provide PR if you find this enhancement reasonable.

The method proposed seems to be a logical extension of existing API:
```
Tree => find_data()
Token => find_type()
```

Discussion:
@makukha: Sorry for bothering, just started using Lark, and see missing pieces (as of my taste). Handy API already made many packages super-popular (like click, requests etc.)@erezsh: Actually you need to use `scan_values()`

But, yeah, I think that's a good one to add. But let's call it find_token

Solution patch:
diff --git a/.gitignore b/.gitignore
index 26275e4b..d4e64180 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,6 +1,7 @@
 *.pyc
 *.pyo
 /.tox
+/lark.egg-info/**
 /lark_parser.egg-info/**
 tags
 .vscode
diff --git a/docs/grammar.md b/docs/grammar.md
index 6cf80240..14179d0c 100644
--- a/docs/grammar.md
+++ b/docs/grammar.md
@@ -20,13 +20,13 @@ EBNF is basically a short-hand for common BNF patterns.
 
 Optionals are expanded:
 
-```ebnf
+```ruby
   a b? c    ->    (a c | a b c)
 ```
 
 Repetition is extracted into a recursion:
 
-```ebnf
+```ruby
   a: b*    ->    a: _b_tag
                  _b_tag: (_b_tag b)?
 ```
@@ -35,11 +35,11 @@ And so on.
 
 Lark grammars are composed of a list of definitions and directives, each on its own line. A definition is either a named rule, or a named terminal, with the following syntax, respectively:
 
-```c
-  rule: <EBNF EXPRESSION>
+```html
+  rule: <EBNF-EXPRESSION>
       | etc.
 
-  TERM: <EBNF EXPRESSION>   // Rules aren't allowed
+  TERM: <EBNF-EXPRESSION>   // Rules aren't allowed
 ```
 
 
@@ -82,18 +82,18 @@ Templates are expanded when preprocessing the grammar.
 
 Definition syntax:
 
-```ebnf
+```javascript
   my_template{param1, param2, ...}: <EBNF EXPRESSION>
 ```
 
 Use syntax:
 
-```ebnf
+```javascript
 some_rule: my_template{arg1, arg2, ...}
 ```
 
 Example:
-```ebnf
+```javascript
 _separated{x, sep}: x (sep x)*  // Define a sequence of 'x sep x sep x ...'
 
 num_list: "[" _separated{NUMBER, ","} "]"   // Will match "[1, 2, 3]" etc.
@@ -173,9 +173,9 @@ Tree('start', [Token('A', 'ab')])
 
 This is happening because Python's regex engine always returns the best matching option. There is no way to access the alternatives.
 
-If you find yourself in this situation, the recommended solution is to use rules instead.
+If you find yourself in this situation, the recommended solution is to either use the "dynamic_complete" lexer, or use rules instead.
 
-Example:
+Example using rules:
 
 ```python
 >>> p = Lark("""start: (a | b)+
@@ -191,6 +191,25 @@ _ambig
     b   b
 ```
 
+Example using dynamic-complete:
+
+```python
+>>> g = """
+...     start: (A | B)+
+...     A    : "a" | "ab"
+...     B    : "b"
+... """
+>>> p = Lark(g, ambiguity="explicit", lexer="dynamic_complete")
+>>> rich.print(p.parse("ab"))
+_ambig
+├── start
+│   └── ab
+└── start
+    ├── a
+    └── b
+```
+
+(note: the dynamic-complete lexer can significantly affect the performance of the parser)
 
 ## Rules
 
@@ -265,7 +284,7 @@ COMMENT: "#" /[^\n]/*
 
 Allows one to import terminals and rules from lark grammars.
 
-When importing rules, all their dependencies will be imported into a namespace, to avoid collisions. It's not possible to override their dependencies (e.g. like you would when inheriting a class).
+When importing rules, all their dependencies will be imported into a namespace, to avoid collisions. To override any of their dependencies (e.g. like you would override methods when inheriting a class), use the ``%override`` directive.
 
 **Syntax:**
 ```html
@@ -276,11 +295,11 @@ When importing rules, all their dependencies will be imported into a namespace,
 %import <module> (<TERM1>, <TERM2>, <rule1>, <rule2>)
 ```
 
-If the module path is absolute, Lark will attempt to load it from the built-in directory (which currently contains `common.lark`, `python.lark`, and `unicode.lark`).
+If the module path is absolute, Lark will attempt to load it from the built-in directory (which currently contains `common.lark`, `lark.lark`, `python.lark`, and `unicode.lark`).
 
 If the module path is relative, such as `.path.to.file`, Lark will attempt to load it from the current working directory. Grammars must have the `.lark` extension.
 
-The rule or terminal can be imported under another name with the `->` syntax.
+The rule or terminal can be imported under another name (an alias) with the `->` syntax.
 
 **Example:**
 ```perl
@@ -288,7 +307,7 @@ The rule or terminal can be imported under another name with the `->` syntax.
 
 %import .terminals_file (A, B, C)
 
-%import .rules_file.rulea -> ruleb
+%import .rules_file.rule_a -> rule_b
 ```
 
 Note that `%ignore` directives cannot be imported. Imported rules will abide by the `%ignore` directives declared in the main grammar.
@@ -328,4 +347,4 @@ Can also be used to implement a plugin system where a core grammar is extended b
 %extend NUMBER: /0x\w+/
 ```
 
-For both `%extend` and `%override`, there is not requirement for a rule/terminal to come from another file, but that is probably the most common usecase
+For both `%extend` and `%override`, there is not requirement for a rule/terminal to come from another file, but that is probably the most common use-case.
diff --git a/lark/grammar.py b/lark/grammar.py
index 1d226d9e..016496bd 100644
--- a/lark/grammar.py
+++ b/lark/grammar.py
@@ -16,7 +16,8 @@ def __init__(self, name: str) -> None:
         self.name = name
 
     def __eq__(self, other):
-        assert isinstance(other, Symbol), other
+        if not isinstance(other, Symbol):
+            return NotImplemented
         return self.is_term == other.is_term and self.name == other.name
 
     def __ne__(self, other):
diff --git a/lark/parse_tree_builder.py b/lark/parse_tree_builder.py
index e3a41718..1acad4d3 100644
--- a/lark/parse_tree_builder.py
+++ b/lark/parse_tree_builder.py
@@ -320,7 +320,7 @@ def f(children):
 
 def apply_visit_wrapper(func, name, wrapper):
     if wrapper is _vargs_meta or wrapper is _vargs_meta_inline:
-        raise NotImplementedError("Meta args not supported for internal transformer")
+        raise NotImplementedError("Meta args not supported for internal transformer; use YourTransformer().transform(parser.parse()) instead")
 
     @wraps(func)
     def f(children):
diff --git a/lark/tree.py b/lark/tree.py
index 76f8738e..9dccadd7 100644
--- a/lark/tree.py
+++ b/lark/tree.py
@@ -3,8 +3,10 @@
 
 from typing import List, Callable, Iterator, Union, Optional, Generic, TypeVar, TYPE_CHECKING
 
+from .lexer import Token
+
 if TYPE_CHECKING:
-    from .lexer import TerminalDef, Token
+    from .lexer import TerminalDef
     try:
         import rich
     except ImportError:
@@ -171,6 +173,16 @@ def find_data(self, data: str) -> 'Iterator[Tree[_Leaf_T]]':
 
 ###}
 
+    def find_token(self, token_type: str) -> Iterator[_Leaf_T]:
+        """Returns all tokens whose type equals the given token_type.
+
+        This is a recursive function that will find tokens in all the subtrees.
+
+        Example:
+            >>> term_tokens = tree.find_token('TERM')
+        """
+        return self.scan_values(lambda v: isinstance(v, Token) and v.type == token_type)
+
     def expand_kids_by_data(self, *data_values):
         """Expand (inline) children with any of the given data values. Returns True if anything changed"""
         changed = False
diff --git a/lark/visitors.py b/lark/visitors.py
index 0e051ed9..b6db3ea0 100644
--- a/lark/visitors.py
+++ b/lark/visitors.py
@@ -523,7 +523,7 @@ def v_args(inline: bool = False, meta: bool = False, tree: bool = False, wrapper
 
     Parameters:
         inline (bool, optional): Children are provided as ``*args`` instead of a list argument (not recommended for very long lists).
-        meta (bool, optional): Provides two arguments: ``meta`` and ``children`` (instead of just the latter)
+        meta (bool, optional): Provides two arguments: ``meta`` and ``children`` (instead of just the latter); ``meta`` isn't available for transformers supplied to Lark using the ``transformer`` parameter (aka internal transformers).
         tree (bool, optional): Provides the entire tree as the argument, instead of the children.
         wrapper (function, optional): Provide a function to decorate all methods.
 


Test patch:
diff --git a/tests/test_grammar.py b/tests/test_grammar.py
index 624b0799..52425db5 100644
--- a/tests/test_grammar.py
+++ b/tests/test_grammar.py
@@ -6,7 +6,7 @@
 from lark import Lark, Token, Tree, ParseError, UnexpectedInput
 from lark.load_grammar import GrammarError, GRAMMAR_ERRORS, find_grammar_errors, list_grammar_imports
 from lark.load_grammar import FromPackageLoader
-
+from lark.grammar import Symbol
 
 class TestGrammar(TestCase):
     def setUp(self):
@@ -296,8 +296,11 @@ def test_line_breaks(self):
         p.parse('ab')
 
 
+    def test_symbol_eq(self):
+        a = None
+        b = Symbol("abc")
 
-
+        self.assertNotEqual(a, b)
 
 
 if __name__ == '__main__':
diff --git a/tests/test_trees.py b/tests/test_trees.py
index 1f69869e..55fdae91 100644
--- a/tests/test_trees.py
+++ b/tests/test_trees.py
@@ -17,6 +17,11 @@
 class TestTrees(TestCase):
     def setUp(self):
         self.tree1 = Tree('a', [Tree(x, y) for x, y in zip('bcd', 'xyz')])
+        self.tree2 = Tree('a', [
+            Tree('b', [Token('T', 'x')]),
+            Tree('c', [Token('T', 'y')]),
+            Tree('d', [Tree('z', [Token('T', 'zz'), Tree('zzz', 'zzz')])]),
+        ])
 
     def test_eq(self):
         assert self.tree1 == self.tree1
@@ -48,6 +53,11 @@ def test_iter_subtrees_topdown(self):
         nodes = list(self.tree1.iter_subtrees_topdown())
         self.assertEqual(nodes, expected)
 
+    def test_find_token(self):
+        expected = [Token('T', 'x'), Token('T', 'y'), Token('T', 'zz')]
+        tokens = list(self.tree2.find_token('T'))
+        self.assertEqual(tokens, expected)
+
     def test_visitor(self):
         class Visitor1(Visitor):
             def __init__(self):