Module Expr Str Import alias ImportFrom alias alias ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str TryExcept body Assign NameStore Call NameLoad NameLoad handlers ExceptHandler type NameLoad name NameStore body Print BinOpMod Str TupleLoad NameLoad NameLoad Return NameLoad Assign AttributeStore NameLoad attr Dict Assign AttributeStore NameLoad attr Dict Assign NameStore Num For NameStore NameLoad body AugAssignAdd NameStore Num Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad If BoolOpAnd UnaryOpNot NameLoad Call AttributeLoad NameLoad attr body Print BinOpMod Str TupleLoad NameLoad NameLoad Call AttributeLoad NameLoad attr orelse Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad Assert CompareNotIn NameLoad AttributeLoad NameLoad attr Assert CompareNotIn NameLoad AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str TryExcept body Assign NameStore Call NameLoad NameLoad handlers ExceptHandler type NameLoad name NameStore body Print BinOpMod Str TupleLoad NameLoad NameLoad Return NameLoad Assign NameStore Num Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Dict Assign NameStore ListLoad While Call AttributeLoad NameLoad attr Str body While Call AttributeLoad NameLoad attr Str body Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore NameStore Call NameLoad NameLoad Call AttributeLoad NameLoad attr Assign NameStore ListLoad For NameStore Call NameLoad NameLoad body Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore Call NameLoad NameLoad Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign SubscriptStore NameLoad Index TupleLoad NameLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore Call NameLoad NameLoad Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Call NameLoad NameLoad TupleLoad NameLoad NameLoad Assign NameStore ListLoad For NameStore Call NameLoad NameLoad body Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore NameStore Call NameLoad NameLoad Call AttributeLoad NameLoad attr Assign NameStore SubscriptLoad NameLoad Index TupleLoad NameLoad NameLoad Assert CompareEq NameLoad Call NameLoad NameLoad TupleLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign NameStore Dict Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Num For NameStore Call NameLoad NameLoad body Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr Num Assign TupleStore NameStore NameStore NameStore NameStore Call NameLoad NameLoad Call AttributeLoad NameLoad attr Num Num Num Num Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index NameLoad NameLoad TupleLoad NameLoad NameLoad Assert CompareEq SubscriptLoad AttributeLoad NameLoad attr Index NameLoad NameLoad TupleLoad NameLoad NameLoad Assert CompareEq NameLoad Num TupleLoad NameLoad NameLoad Assign NameStore SubscriptLoad NameLoad Index NameLoad Assert CompareEq NameLoad Call NameLoad NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Dict Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Num For TupleStore NameStore NameStore Call NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad For NameStore Call NameLoad Num body If BinOpBitAnd NameLoad BinOpLShift Num NameLoad body Assign SubscriptStore NameLoad Index BinOpAdd BinOpMult NameLoad Num NameLoad Num Assign SubscriptStore NameLoad Index NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore ListLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Num For NameStore Call NameLoad NameLoad body Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad If CompareEq NameLoad Str body Assign NameStore NameLoad orelse Assign NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Num Assert CompareEq NameLoad Call NameLoad AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Num Assert CompareEq NameLoad Call NameLoad AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str NameLoad Assert NameLoad TupleLoad NameLoad NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Num Assert CompareIn NameLoad AttributeLoad NameLoad attr TupleLoad NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr Assert CompareEq NameLoad Str TupleLoad NameLoad NameLoad TryExcept body Assign TupleStore NameStore NameStore TupleLoad BinOpAdd NameLoad Num Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass orelse Assert Num TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign AttributeStore NameLoad attr Dict Assign AttributeStore NameLoad attr Dict For TupleStore NameStore TupleStore NameStore NameStore Call NameLoad AttributeLoad NameLoad attr body If BoolOpAnd CompareEq NameLoad AttributeLoad NameLoad attr CompareIsNot NameLoad NameLoad body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad orelse If CompareIs NameLoad NameLoad body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad decorator_list decorator_list
N/A N/A Convert graminit.[ch] spit out by pgen to Python code.

Pgen is the Python parser generator.  It is useful to quickly create a
parser from a grammar file in Python's grammar notation.  But I don't
want my parsers to be written in C (yet), so I'm translating the
parsing tables to Python data structures and writing a Python parse
engine.

Note that the token numbers are constants determined by the standard
Python tokenizer.  The standard token module defines these numbers and
their names (the names are not used much).  The token numbers are
hardcoded into the Python tokenizer and into pgen.  A Python
implementation of the Python tokenizer is also available, in the
standard tokenize module.

On the other hand, symbol numbers (representing the grammar's
non-terminals) are assigned by pgen based on the actual grammar
input.

Note: this module is pretty much obsolete; the pgen module generates
equivalent grammar tables directly from the Grammar.txt input file
without having to invoke the Python pgen C program.

 N/A re pgen2 grammar token Converter N/A N/A grammar Grammar N/A N/A Grammar subclass that reads classic pgen output files.

    The run() method reads the tables as produced by the pgen parser
    generator, typically contained in two C files, graminit.h and
    graminit.c.  The other methods are for internal use only.

    See the base class for more documentation.

     run N/A N/A self graminit_h graminit_c N/A N/A N/A Load the grammar tables from the text files written by pgen. N/A N/A N/A self parse_graminit_h graminit_h N/A N/A N/A self parse_graminit_c graminit_c N/A N/A N/A self finish_off N/A parse_graminit_h N/A N/A self filename N/A N/A N/A Parse the .h file written by pgen.  (Internal)

        This file is a sequence of #define statements defining the
        nonterminals of the grammar as numbers.  We build two tables
        mapping the numbers to names and back.

         N/A N/A N/A f N/A open filename N/A N/A N/A IOError N/A err N/A N/A N/A Can't open %s: %s N/A filename err N/A False N/A N/A self symbol2number N/A N/A N/A self number2symbol N/A N/A lineno 0 N/A line f N/A N/A lineno 1 N/A mo N/A N/A re match ^#define\s+(\w+)\s+(\d+)$ line N/A N/A N/A mo N/A N/A line strip N/A N/A N/A %s(%s): can't parse %s N/A filename lineno N/A N/A line strip N/A N/A N/A symbol number N/A N/A mo groups N/A number N/A int number N/A N/A symbol N/A self symbol2number N/A N/A number N/A self number2symbol N/A N/A N/A self symbol2number N/A symbol number N/A N/A N/A self number2symbol N/A number symbol N/A True N/A parse_graminit_c N/A N/A self filename N/A N/A N/A Parse the .c file written by pgen.  (Internal)

        The file looks as follows.  The first two lines are always this:

        #include "pgenheaders.h"
        #include "grammar.h"

        After that come four blocks:

        1) one or more state definitions
        2) a table defining dfas
        3) a table defining labels
        4) a struct defining the grammar

        A state definition has the following form:
        - one or more arc arrays, each of the form:
          static arc arcs_<n>_<m>[<k>] = {
                  {<i>, <j>},
                  ...
          };
        - followed by a state array, of the form:
          static state states_<s>[<t>] = {
                  {<k>, arcs_<n>_<m>},
                  ...
          };

         N/A N/A N/A f N/A open filename N/A N/A N/A IOError N/A err N/A N/A N/A Can't open %s: %s N/A filename err N/A False N/A lineno 0 N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line #include "pgenheaders.h"
 N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line #include "grammar.h"
 N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A allarcs N/A N/A states N/A N/A N/A N/A line startswith static arc  N/A N/A N/A N/A line startswith static arc  N/A N/A mo N/A N/A re match static arc arcs_(\d+)_(\d+)\[(\d+)\] = {$ line N/A mo N/A lineno line N/A N/A n m k N/A map int N/A N/A mo groups N/A arcs N/A N/A _ N/A range k N/A N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+{(\d+), (\d+)},$ line N/A mo N/A lineno line N/A N/A i j N/A map int N/A N/A mo groups N/A N/A N/A arcs append N/A i j N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line };
 N/A lineno line N/A N/A allarcs N/A N/A n m arcs N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match static state states_(\d+)\[(\d+)\] = {$ line N/A mo N/A lineno line N/A N/A s t N/A map int N/A N/A mo groups N/A N/A s N/A len states N/A lineno line N/A state N/A N/A _ N/A range t N/A N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+{(\d+), arcs_(\d+)_(\d+)},$ line N/A mo N/A lineno line N/A N/A k n m N/A map int N/A N/A mo groups N/A arcs N/A allarcs N/A N/A n m N/A N/A k N/A len arcs N/A lineno line N/A N/A N/A state append arcs N/A N/A N/A states append state N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line };
 N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A self states states N/A dfas N/A N/A mo N/A N/A re match static dfa dfas\[(\d+)\] = {$ line N/A mo N/A lineno line N/A ndfas N/A int N/A N/A mo group 1 N/A i N/A range ndfas N/A N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+{(\d+), "(\w+)", (\d+), (\d+), states_(\d+),$ line N/A mo N/A lineno line N/A symbol N/A N/A mo group 2 N/A N/A number x y z N/A map int N/A N/A mo group 1 3 4 5 N/A N/A N/A N/A self symbol2number N/A symbol number N/A lineno line N/A N/A N/A N/A self number2symbol N/A number symbol N/A lineno line N/A N/A x 0 N/A lineno line N/A state N/A states N/A z N/A N/A y N/A len state N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+("(?:\\\d\d\d)*")},$ line N/A mo N/A lineno line N/A first N/A N/A rawbitset N/A eval N/A N/A mo group 1 N/A N/A i c N/A enumerate rawbitset N/A N/A byte N/A ord c N/A j N/A range 8 N/A N/A N/A byte N/A 1 j N/A N/A N/A first N/A N/A N/A i 8 j 1 N/A N/A dfas N/A number N/A state first N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line };
 N/A lineno line N/A N/A self dfas dfas N/A labels N/A N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match static label labels\[(\d+)\] = {$ line N/A mo N/A lineno line N/A nlabels N/A int N/A N/A mo group 1 N/A i N/A range nlabels N/A N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+{(\d+), (0|"\w+")},$ line N/A mo N/A lineno line N/A N/A x y N/A N/A mo groups N/A x N/A int x N/A N/A y 0 N/A N/A y None N/A N/A y N/A eval y N/A N/A N/A labels append N/A x y N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line };
 N/A lineno line N/A N/A self labels labels N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line grammar _PyParser_Grammar = {
 N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+(\d+),$ line N/A mo N/A lineno line N/A ndfas N/A int N/A N/A mo group 1 N/A N/A ndfas N/A len N/A self dfas N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line 	dfas,
 N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+{(\d+), labels},$ line N/A mo N/A lineno line N/A nlabels N/A int N/A N/A mo group 1 N/A N/A nlabels N/A len N/A self labels N/A lineno line N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A mo N/A N/A re match \s+(\d+)$ line N/A mo N/A lineno line N/A start N/A int N/A N/A mo group 1 N/A N/A start N/A self number2symbol N/A lineno line N/A N/A self start start N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A line };
 N/A lineno line N/A N/A N/A N/A lineno line N/A N/A lineno 1 N/A N/A f next N/A N/A N/A StopIteration N/A N/A N/A N/A 0 N/A lineno line N/A finish_off N/A N/A self N/A N/A N/A Create additional useful structures.  (Internal). N/A N/A self keywords N/A N/A N/A self tokens N/A N/A N/A ilabel N/A type value N/A enumerate N/A self labels N/A N/A N/A N/A type N/A token NAME N/A value None N/A N/A N/A N/A self keywords N/A value ilabel N/A N/A N/A value None N/A N/A N/A N/A self tokens N/A type ilabel N/A N/A