Module Expr Str ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias alias alias alias ImportFrom alias alias alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias alias ImportFrom alias alias ImportFrom alias alias ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr ListLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list Call NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return IfExp AttributeLoad NameLoad attr ListComp BinOpAdd Str NameLoad comprehension NameStore AttributeLoad AttributeLoad NameLoad attr attr ListLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore BinOpAdd BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr ListLoad For NameStore AttributeLoad AttributeLoad NameLoad attr attr body If CompareIn NameLoad NameLoad body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad orelse If BoolOpAnd AttributeLoad NameLoad attr CompareIn NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Call AttributeLoad NameLoad attr NameLoad orelse Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Assign TupleStore NameStore AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Assign AttributeStore AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign AttributeStore AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad kwarg body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr keyword Num Expr Call NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr IfExp CompareIsNot NameLoad NameLoad BinOpDiv Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr keyword Str Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad kwarg body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Num Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad keyword NameLoad keyword NameLoad Call NameLoad NameLoad NameLoad NameLoad Assign NameStore Call NameLoad GeneratorExp TupleLoad NameLoad SubscriptLoad SubscriptLoad NameLoad Index NameLoad Slice Num comprehension NameStore AttributeLoad NameLoad attr Assign NameStore Call NameLoad GeneratorExp TupleLoad NameLoad SubscriptLoad SubscriptLoad NameLoad Index NameLoad Slice Num comprehension NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Num Num Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad keyword NameLoad keyword NameLoad Assign NameStore DictComp BinOpAdd Str NameLoad NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr orelse Assign NameStore Dict Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad Call NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad If CompareIsNot NameLoad NameLoad body AugAssignMult NameStore NameLoad For TupleStore NameStore NameStore BinOpAdd Call NameLoad Call AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr keyword NameLoad For NameStore BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Call AttributeLoad SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num attr keyword BinOpAdd NameLoad Str Return BinOpAdd BinOpAdd ListLoad NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return BinOpAdd BinOpAdd ListLoad Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad kwarg body Expr Str Return SubscriptLoad Call AttributeLoad NameLoad attr NameLoad keyword NameLoad NameLoad Index Num decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad kwarg body Expr Str Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad Call NameLoad NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Call NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore IfExp UnaryOpNot NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore IfExp AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad Dict Str NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad Call NameLoad NameLoad NameLoad NameLoad NameLoad Assign NameStore Dict If AttributeLoad NameLoad attr body Assign NameStore DictComp SubscriptLoad NameLoad Slice Num SubscriptLoad NameLoad Index NameLoad comprehension NameStore NameLoad Assign NameStore Call NameLoad Call NameLoad AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad keyword NameLoad NameLoad Return BinOpAdd BinOpAdd BinOpAdd BinOpAdd NameLoad ListLoad NameLoad Call NameLoad Call AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad NameLoad attr ListLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad NameLoad attr attr decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam defaults body Assign NameStore BinOpAdd BinOpAdd AttributeLoad NameLoad attr ListLoad Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Return NameLoad decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam defaults body Assign NameStore BinOpAdd BinOpAdd AttributeLoad NameLoad attr ListLoad Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Return NameLoad decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body If CompareIn NameLoad BinOpAdd BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareEq NameLoad Str body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If BoolOpAnd AttributeLoad NameLoad attr CompareIn NameLoad AttributeLoad NameLoad attr body Return Call AttributeLoad AttributeLoad NameLoad attr attr SubscriptLoad NameLoad Slice Num Return Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults vararg kwarg body Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad NameLoad NameLoad keyword Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword NameLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad DictComp BinOpAdd Str NameLoad NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Return ListComp SubscriptLoad NameLoad Index NameLoad comprehension NameStore AttributeLoad AttributeLoad NameLoad attr attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults kwarg body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list Call NameLoad keyword ListLoad Str Str FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam defaults kwarg body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad decorator_list Call NameLoad NameLoad ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad If UnaryOpNot NameLoad body Assign NameStore Call NameLoad AttributeLoad NameLoad attr If UnaryOpNot NameLoad body Assign NameStore Call NameLoad AttributeLoad NameLoad attr If UnaryOpNot NameLoad body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr keyword NameLoad If UnaryOpNot NameLoad body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr If UnaryOpNot NameLoad body Assign NameStore AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr ListLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Str Assign AttributeStore AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Str Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults kwarg body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr DictComp NameLoad SubscriptLoad NameLoad Index NameLoad comprehension NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list Call NameLoad keyword ListLoad Str FunctionDef arguments args NameParam NameParam defaults body If CompareEq NameLoad Str body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad orelse If CompareEq NameLoad Str body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad orelse If CompareEq NameLoad Str body Return AttributeLoad NameLoad attr Return Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad decorator_list Call NameLoad NameLoad ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Expr Str Pass decorator_list NameLoad decorator_list Call NameLoad NameLoad ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list Call NameLoad keyword ListLoad Str FunctionDef arguments args NameParam NameParam defaults body Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr TupleLoad NameLoad AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body If CompareEq NameLoad Str body Return AttributeLoad NameLoad attr Return Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad NameLoad NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults Num kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr ListLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword BinOpSub AttributeLoad NameLoad attr Num decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Num Assign NameStore Call AttributeLoad NameLoad attr TupleLoad NameLoad Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad Return Call AttributeLoad Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr attr keyword Num decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad keyword BinOpSub AttributeLoad NameLoad attr Num decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return UnaryOpUSub Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword BinOpSub AttributeLoad NameLoad attr Num decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return BinOpMult AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr TupleLoad NameLoad keyword Str decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body If CompareEq NameLoad Str body Return Num Return Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list Call NameLoad keyword ListLoad Str FunctionDef arguments args NameParam NameParam defaults body Return NameLoad decorator_list Call NameLoad keyword ListLoad Str FunctionDef arguments args NameParam NameParam defaults body If CompareEq NameLoad Str body Return AttributeLoad NameLoad attr Return Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad NameLoad kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad NameLoad keyword AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr ListLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Assert CompareEq AttributeLoad NameLoad attr Num Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body If CompareEq NameLoad Str body Return AttributeLoad NameLoad attr Return Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults kwarg body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr ListLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults vararg kwarg body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad NameLoad attr attr decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam defaults vararg kwarg body Return Call AttributeLoad AttributeLoad NameLoad attr attr keyword NameLoad NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad NameLoad attr attr decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam defaults vararg kwarg body Return NameLoad decorator_list Call NameLoad keyword ListLoad FunctionDef arguments args NameParam NameParam defaults vararg kwarg body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad kwarg body Assign NameStore ListComp NameLoad comprehension NameStore AttributeLoad AttributeLoad NameLoad attr attr CompareNotIn Str NameLoad Expr Call AttributeLoad NameLoad attr Str Call NameLoad NameLoad If NameLoad body Assign NameStore Call NameLoad NameLoad NameLoad keyword NameLoad keyword Str orelse Assign NameStore Call NameLoad NameLoad keyword Str Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad NameLoad NameLoad decorator_list decorator_list
N/A N/A Sequence generation framework.

Recurrent networks are often used to generate/model sequences.
Examples include language modelling, machine translation, handwriting
synthesis, etc.. A typical pattern in this context is that
sequence elements are generated one often another, and every generated
element is fed back into the recurrent network state. Sometimes
also an attention mechanism is used to condition sequence generation
on some structured input like another sequence or an image.

This module provides :class:`SequenceGenerator` that builds a sequence
generating network from three main components:

* a core recurrent transition, e.g. :class:`~blocks.bricks.recurrent.LSTM`
  or :class:`~blocks.bricks.recurrent.GatedRecurrent`

* a readout component that can produce sequence elements using
  the network state and the information from the attention mechanism

* an attention mechanism (see :mod:`~blocks.bricks.attention` for
  more information)

Implementation-wise :class:`SequenceGenerator` fully relies on
:class:`BaseSequenceGenerator`. At the level of the latter an
attention is mandatory, moreover it must be a part of the recurrent
transition (see :class:`~blocks.bricks.attention.AttentionRecurrent`).
To simulate optional attention, :class:`SequenceGenerator` wraps the
pure recurrent network in :class:`FakeAttentionRecurrent`.

 abc ABCMeta abstractmethod collections OrderedDict six add_metaclass theano tensor blocks.bricks Initializable Random Bias NDimensionalSoftmax blocks.bricks.base application Brick lazy blocks.bricks.parallel Fork Merge blocks.bricks.lookup LookupTable blocks.bricks.recurrent recurrent blocks.bricks.attention AbstractAttentionRecurrent AttentionRecurrent blocks.roles add_role COST blocks.utils dict_union dict_subset BaseSequenceGenerator N/A Initializable N/A N/A A generic sequence generator.

    This class combines two components, a readout network and an
    attention-equipped recurrent transition, into a context-dependent
    sequence generator. Third component must be also given which
    forks feedback from the readout network to obtain inputs for the
    transition.

    The class provides two methods: :meth:`generate` and :meth:`cost`. The
    former is to actually generate sequences and the latter is to compute
    the cost of generating given sequences.

    The generation algorithm description follows.

    **Definitions and notation:**

    * States :math:`s_i` of the generator are the states of the transition
      as specified in `transition.state_names`.

    * Contexts of the generator are the contexts of the
      transition as specified in `transition.context_names`.

    * Glimpses :math:`g_i` are intermediate entities computed at every
      generation step from states, contexts and the previous step glimpses.
      They are computed in the transition's `apply` method when not given
      or by explicitly calling the transition's `take_glimpses` method. The
      set of glimpses considered is specified in
      `transition.glimpse_names`.

    * Outputs :math:`y_i` are produced at every step and form the output
      sequence. A generation cost :math:`c_i` is assigned to each output.

    **Algorithm:**

    1. Initialization.

       .. math::

           y_0 = readout.initial\_outputs(contexts)\\
           s_0, g_0 = transition.initial\_states(contexts)\\
           i = 1\\

       By default all recurrent bricks from :mod:`~blocks.bricks.recurrent`
       have trainable initial states initialized with zeros. Subclass them
       or :class:`~blocks.bricks.recurrent.BaseRecurrent` directly to get
       custom initial states.

    2. New glimpses are computed:

       .. math:: g_i = transition.take\_glimpses(
           s_{i-1}, g_{i-1}, contexts)

    3. A new output is generated by the readout and its cost is
       computed:

       .. math::

            f_{i-1} = readout.feedback(y_{i-1}) \\
            r_i = readout.readout(f_{i-1}, s_{i-1}, g_i, contexts) \\
            y_i = readout.emit(r_i) \\
            c_i = readout.cost(r_i, y_i)

       Note that the *new* glimpses and the *old* states are used at this
       step. The reason for not merging all readout methods into one is
       to make an efficient implementation of :meth:`cost` possible.

    4. New states are computed and iteration is done:

       .. math::

           f_i = readout.feedback(y_i) \\
           s_i = transition.compute\_states(s_{i-1}, g_i,
                fork.apply(f_i), contexts) \\
           i = i + 1

    5. Back to step 2 if the desired sequence
       length has not been yet reached.

    | A scheme of the algorithm described above follows.

    .. image:: /_static/sequence_generator_scheme.png
            :height: 500px
            :width: 500px

    ..

    Parameters
    ----------
    readout : instance of :class:`AbstractReadout`
        The readout component of the sequence generator.
    transition : instance of :class:`AbstractAttentionRecurrent`
        The transition component of the sequence generator.
    fork : :class:`.Brick`
        The brick to compute the transition's inputs from the feedback.

    See Also
    --------
    :class:`.Initializable` : for initialization parameters

    :class:`SequenceGenerator` : more user friendly interface to this\
        brick

     __init__ N/A N/A self readout transition fork language_model N/A None kwargs N/A N/A N/A N/A N/A super BaseSequenceGenerator self __init__ kwargs N/A N/A self readout readout N/A N/A self transition transition N/A N/A self fork fork N/A N/A self language_model language_model N/A N/A self children N/A N/A self readout N/A self fork N/A self transition N/A N/A self language_model N/A N/A N/A N/A N/A self children append N/A self language_model N/A N/A lazy _state_names N/A N/A self N/A N/A N/A N/A N/A N/A self transition compute_states outputs N/A property _context_names N/A N/A self N/A N/A N/A N/A N/A N/A self transition apply contexts N/A property _glimpse_names N/A N/A self N/A N/A N/A N/A N/A N/A self transition take_glimpses outputs N/A property _lm_state_names N/A N/A self N/A N/A N/A N/A N/A self language_model N/A N/A lm_ name N/A name N/A N/A self language_model _state_names N/A N/A property _push_allocation_config N/A N/A self N/A N/A N/A N/A N/A N/A self transition push_allocation_config N/A transition_sources N/A N/A N/A self _state_names N/A self _context_names N/A self _glimpse_names N/A N/A N/A self readout source_dims N/A N/A name N/A N/A self readout source_names N/A N/A N/A name transition_sources N/A N/A N/A N/A N/A N/A self readout source_dims append N/A N/A N/A self transition get_dim name N/A N/A N/A N/A self language_model N/A name N/A self _lm_state_names N/A N/A N/A N/A N/A N/A self readout source_dims append N/A N/A self get_dim name N/A N/A N/A N/A N/A self readout get_dim name N/A N/A N/A N/A self readout push_allocation_config N/A N/A feedback_name N/A N/A N/A self readout feedback outputs N/A N/A N/A self fork input_dim N/A N/A N/A self readout get_dim feedback_name N/A N/A N/A self fork output_dims N/A N/A N/A self transition get_dims N/A N/A N/A self fork apply outputs N/A cost N/A N/A self application_call outputs mask N/A None kwargs N/A N/A Returns the average cost over the minibatch.

        The cost is computed by averaging the sum of per token costs for
        each sequence over the minibatch.

        .. warning::
            Note that, the computed cost can be problematic when batches
            consist of vastly different sequence lengths.

        Parameters
        ----------
        outputs : :class:`~tensor.TensorVariable`
            The 3(2) dimensional tensor containing output sequences.
            The axis 0 must stand for time, the axis 1 for the
            position in the batch.
        mask : :class:`~tensor.TensorVariable`
            The binary matrix identifying fake outputs.

        Returns
        -------
        cost : :class:`~tensor.Variable`
            Theano variable for cost, computed by summing over timesteps
            and then averaging over the minibatch.

        Notes
        -----
        The contexts are expected as keyword arguments.

        Adds average cost per sequence element `AUXILIARY` variable to
        the computational graph with name ``per_sequence_element``.

         N/A costs N/A N/A self cost_matrix outputs mask mask kwargs N/A cost N/A N/A tensor mean N/A N/A costs sum axis 0 N/A N/A add_role cost COST N/A N/A N/A application_call add_auxiliary_variable N/A N/A mask None N/A N/A N/A costs sum N/A N/A mask sum N/A N/A costs mean name per_sequence_element N/A cost N/A application evaluate N/A N/A self application_call outputs mask N/A None kwargs N/A N/A batch_size N/A N/A outputs shape N/A 1 N/A states N/A dict_subset kwargs N/A self _state_names must_have False N/A contexts N/A dict_subset kwargs N/A self _context_names must_have False N/A feedback N/A N/A N/A self readout feedback outputs N/A inputs N/A N/A N/A self fork apply feedback as_dict True N/A results N/A N/A N/A self transition apply mask mask return_initial_states True as_dict True N/A dict_union inputs states contexts N/A states N/A OrderedDict N/A N/A name N/A N/A results N/A name N/A -1 N/A name N/A self _state_names N/A glimpses N/A OrderedDict N/A N/A name N/A N/A results N/A name N/A 1 N/A name N/A self _glimpse_names N/A feedback N/A N/A tensor roll feedback 1 0 N/A feedback N/A N/A tensor set_subtensor N/A feedback N/A 0 N/A N/A N/A self readout feedback N/A N/A N/A self readout initial_outputs batch_size N/A N/A self language_model N/A N/A lm_states N/A N/A N/A self language_model evaluate outputs outputs mask mask as_dict True N/A lm_states N/A N/A lm_ name value N/A N/A name value N/A N/A lm_states items N/A N/A lm_states N/A N/A readouts N/A N/A N/A self readout readout feedback feedback N/A dict_union lm_states states glimpses contexts N/A costs N/A N/A N/A self readout cost readouts outputs N/A N/A mask None N/A N/A costs mask N/A N/A name variable N/A N/A list N/A N/A glimpses items N/A list N/A N/A states items N/A N/A N/A N/A application_call add_auxiliary_variable N/A N/A variable copy name name N/A name N/A N/A self _state_names N/A self _glimpse_names N/A N/A N/A N/A application_call add_auxiliary_variable N/A N/A N/A N/A results N/A name N/A -1 copy name N/A name _final_value N/A N/A N/A N/A costs N/A N/A states values N/A N/A glimpses values N/A application evaluate_outputs N/A N/A self N/A N/A N/A N/A N/A N/A costs N/A self _state_names N/A self _glimpse_names N/A N/A N/A evaluate property outputs cost_matrix N/A N/A self outputs mask N/A None kwargs N/A N/A Returns generation costs for output sequences.

        See Also
        --------
        :meth:`cost` : Scalar cost.

         N/A N/A N/A N/A self evaluate outputs mask mask kwargs N/A 0 N/A application generate N/A N/A self outputs dont_generate_new_outputs N/A False kwargs N/A N/A A sequence generation step.

        Parameters
        ----------
        outputs : :class:`~tensor.TensorVariable`
            The outputs from the previous step.
        dont_generate_new_outputs : bool, optional
            If ``True``, the previous outputs are used instead
            of generated ones. It is a temporary hack for ASRU.

        Notes
        -----
        The contexts, previous states and glimpses are expected as keyword
        arguments.

         N/A states N/A dict_subset kwargs N/A self _state_names N/A contexts N/A dict_subset kwargs N/A self _context_names must_have False N/A glimpses N/A dict_subset kwargs N/A self _glimpse_names N/A lm_states N/A dict_subset kwargs N/A self _lm_state_names N/A next_glimpses N/A N/A N/A self transition take_glimpses as_dict True N/A dict_union states glimpses contexts N/A next_readouts N/A N/A N/A self readout readout feedback N/A N/A N/A self readout feedback outputs N/A dict_union states next_glimpses contexts lm_states N/A next_outputs N/A N/A dont_generate_new_outputs N/A N/A N/A self readout emit next_readouts outputs N/A next_costs N/A N/A N/A self readout cost next_readouts next_outputs N/A next_feedback N/A N/A N/A self readout feedback next_outputs N/A next_inputs N/A N/A self fork N/A N/A N/A self fork apply next_feedback as_dict True N/A feedback next_feedback N/A next_states N/A N/A N/A self transition compute_states as_list True N/A dict_union next_inputs states next_glimpses contexts N/A next_lm_states N/A N/A N/A self language_model N/A N/A unmangled_lm_states N/A N/A name N/A 3 N/A lm_states N/A name N/A name lm_states N/A next_lm_states N/A OrderedDict N/A zip N/A self _lm_state_names N/A N/A N/A self language_model generate next_outputs dont_generate_new_outputs True iterate False unmangled_lm_states N/A N/A N/A N/A N/A next_states N/A next_outputs N/A list N/A N/A next_glimpses values N/A list N/A N/A next_lm_states values N/A next_costs N/A recurrent generate_delegate N/A N/A self N/A N/A N/A N/A N/A self transition apply N/A N/A generate delegate generate_states N/A N/A self N/A N/A N/A result N/A N/A N/A self _state_names N/A outputs N/A self _glimpse_names N/A N/A self language_model N/A N/A N/A N/A result extend N/A self _lm_state_names N/A result N/A N/A N/A generate property states generate_outputs N/A N/A self N/A N/A N/A result N/A N/A N/A self _state_names N/A outputs N/A self _glimpse_names N/A N/A self language_model N/A N/A N/A N/A result extend N/A self _lm_state_names N/A N/A N/A result append costs N/A result N/A N/A N/A generate property outputs get_dim N/A N/A self name N/A N/A N/A N/A name N/A N/A N/A self _state_names N/A self _context_names N/A self _glimpse_names N/A N/A N/A N/A N/A self transition get_dim name N/A N/A name outputs N/A N/A N/A N/A N/A self readout get_dim name N/A N/A N/A self language_model N/A name N/A self _lm_state_names N/A N/A N/A N/A N/A self language_model get_dim N/A name N/A 3 N/A N/A N/A N/A super BaseSequenceGenerator self get_dim name N/A initial_states N/A N/A self batch_size N/A args kwargs N/A N/A state_dict N/A dict N/A N/A N/A self transition initial_states batch_size as_dict True args kwargs outputs N/A N/A N/A self readout initial_outputs batch_size N/A N/A self language_model N/A N/A lm_initial_states N/A N/A N/A self language_model initial_states batch_size as_dict True args kwargs N/A state_dict N/A dict_union state_dict N/A N/A lm_ name state N/A N/A name state N/A N/A lm_initial_states items N/A N/A N/A state_dict N/A state_name N/A state_name N/A N/A self generate states N/A application initial_states_outputs N/A N/A self N/A N/A N/A N/A N/A self generate states N/A N/A N/A initial_states property outputs N/A AbstractReadout N/A Initializable N/A N/A The interface for the readout component of a sequence generator.

    The readout component of a sequence generator is a bridge between
    the core recurrent network and the output sequence.

    Parameters
    ----------
    source_names : list
        A list of the source names (outputs) that are needed for the
        readout part e.g. ``['states']`` or
        ``['states', 'weighted_averages']`` or ``['states', 'feedback']``.
    readout_dim : int
        The dimension of the readout.

    Attributes
    ----------
    source_names : list
    readout_dim : int

    See Also
    --------
    :class:`BaseSequenceGenerator` : see how exactly a readout is used

    :class:`Readout` : the typically used readout brick

     __init__ N/A N/A self source_names readout_dim N/A kwargs N/A N/A N/A self source_names source_names N/A N/A self readout_dim readout_dim N/A N/A N/A N/A super AbstractReadout self __init__ kwargs N/A N/A lazy allocation N/A source_names readout_dim emit N/A N/A self readouts N/A N/A N/A Produce outputs from readouts.

        Parameters
        ----------
        readouts : :class:`~theano.Variable`
            Readouts produced by the :meth:`readout` method of
            a `(batch_size, readout_dim)` shape.

         N/A N/A abstractmethod cost N/A N/A self readouts outputs N/A N/A N/A Compute generation cost of outputs given readouts.

        Parameters
        ----------
        readouts : :class:`~theano.Variable`
            Readouts produced by the :meth:`readout` method
            of a `(..., readout dim)` shape.
        outputs : :class:`~theano.Variable`
            Outputs whose cost should be computed. Should have as many
            or one less dimensions compared to `readout`. If readout has
            `n` dimensions, first `n - 1` dimensions of `outputs` should
            match with those of `readouts`.

         N/A N/A abstractmethod costs N/A N/A self readouts N/A N/A N/A Cost matrix for all outputs.

        For beam search.

         N/A N/A abstractmethod initial_outputs N/A N/A self batch_size N/A N/A N/A Compute initial outputs for the generator's first step.

        In the notation from the :class:`BaseSequenceGenerator`
        documentation this method should compute :math:`y_0`.

         N/A N/A abstractmethod readout N/A N/A self N/A kwargs N/A N/A Compute the readout vector from states, glimpses, etc.

        Parameters
        ----------
        \*\*kwargs: dict
            Contains sequence generator states, glimpses,
            contexts and feedback from the previous outputs.

         N/A N/A abstractmethod feedback N/A N/A self outputs N/A N/A N/A Feeds outputs back to be used as inputs of the transition. N/A N/A abstractmethod N/A N/A add_metaclass ABCMeta Readout N/A AbstractReadout N/A N/A Readout brick with separated emitter and feedback parts.

    :class:`Readout` combines a few bits and pieces into an object
    that can be used as the readout component in
    :class:`BaseSequenceGenerator`. This includes an emitter brick,
    to which :meth:`emit`, :meth:`cost` and :meth:`initial_outputs`
    calls are delegated, a feedback brick to which :meth:`feedback`
    functionality is delegated, and a pipeline to actually compute
    readouts from all the sources (see the `source_names` attribute
    of :class:`AbstractReadout`).

    The readout computation pipeline is constructed from `merge` and
    `post_merge` brick, whose responsibilites are described in the
    respective docstrings.

    Parameters
    ----------
    emitter : an instance of :class:`AbstractEmitter`
        The emitter component.
    feedback_brick : an instance of :class:`AbstractFeedback`
        The feedback component.
    merge : :class:`.Brick`, optional
        A brick that takes the sources given in `source_names` as an input
        and combines them into a single output. If given, `merge_prototype`
        cannot be given.
    merge_prototype : :class:`.FeedForward`, optional
        If `merge` isn't given, the transformation given by
        `merge_prototype` is applied to each input before being summed. By
        default a :class:`.Linear` transformation without biases is used.
        If given, `merge` cannot be given.
    post_merge : :class:`.Feedforward`, optional
        This transformation is applied to the merged inputs. By default
        :class:`.Bias` is used.
    merged_dim : int, optional
        The input dimension of `post_merge` i.e. the output dimension of
        `merge` (or `merge_prototype`). If not give, it is assumed to be
        the same as `readout_dim` (i.e. `post_merge` is assumed to not
        change dimensions).
    \*\*kwargs : dict
        Passed to the parent's constructor.

    See Also
    --------
    :class:`BaseSequenceGenerator` : see how exactly a readout is used

    :class:`AbstractEmitter`, :class:`AbstractFeedback`

     __init__ N/A N/A self emitter feedback_brick merge merge_prototype post_merge merged_dim N/A None None None None None None kwargs N/A N/A N/A N/A N/A super Readout self __init__ kwargs N/A N/A emitter N/A N/A emitter N/A TrivialEmitter N/A self readout_dim N/A N/A feedback_brick N/A N/A feedback_brick N/A TrivialFeedback N/A self readout_dim N/A N/A merge N/A N/A merge N/A Merge input_names N/A self source_names prototype merge_prototype N/A N/A post_merge N/A N/A post_merge N/A Bias dim N/A self readout_dim N/A N/A merged_dim N/A N/A merged_dim N/A self readout_dim N/A N/A self emitter emitter N/A N/A self feedback_brick feedback_brick N/A N/A self merge merge N/A N/A self post_merge post_merge N/A N/A self merged_dim merged_dim N/A N/A self children N/A N/A self emitter N/A self feedback_brick N/A self merge N/A self post_merge N/A _push_allocation_config N/A N/A self N/A N/A N/A N/A N/A self emitter readout_dim N/A N/A self get_dim readouts N/A N/A N/A self feedback_brick output_dim N/A N/A self get_dim outputs N/A N/A N/A self merge input_names N/A self source_names N/A N/A N/A self merge input_dims N/A self source_dims N/A N/A N/A self merge output_dim N/A self merged_dim N/A N/A N/A self post_merge input_dim N/A self merged_dim N/A N/A N/A self post_merge output_dim N/A self readout_dim N/A readout N/A N/A self N/A kwargs N/A N/A merged N/A N/A N/A self merge apply N/A name N/A kwargs N/A name N/A name N/A N/A self merge input_names N/A merged N/A N/A N/A self post_merge apply merged N/A merged N/A application emit N/A N/A self readouts N/A N/A N/A N/A N/A N/A self emitter emit readouts N/A application cost N/A N/A self readouts outputs N/A N/A N/A N/A N/A N/A self emitter cost readouts outputs N/A application costs N/A N/A self readouts N/A N/A N/A N/A N/A N/A self emitter costs readouts N/A application initial_outputs N/A N/A self batch_size N/A N/A N/A N/A N/A N/A self emitter initial_outputs batch_size N/A application feedback N/A N/A self outputs N/A N/A N/A N/A N/A N/A self feedback_brick feedback outputs N/A N/A application outputs N/A feedback get_dim N/A N/A self name N/A N/A N/A N/A name outputs N/A N/A N/A N/A N/A self emitter get_dim name N/A N/A N/A name feedback N/A N/A N/A N/A N/A self feedback_brick get_dim name N/A N/A N/A name readouts N/A N/A N/A self readout_dim N/A N/A N/A N/A super Readout self get_dim name N/A N/A AbstractEmitter N/A Brick N/A N/A The interface for the emitter component of a readout.

    Attributes
    ----------
    readout_dim : int
        The dimension of the readout. Is given by the
        :class:`Readout` brick when allocation configuration
        is pushed.

    See Also
    --------
    :class:`Readout`

    :class:`SoftmaxEmitter` : for integer outputs

    Notes
    -----
    An important detail about the emitter cost is that it will be
    evaluated with inputs of different dimensions so it has to be
    flexible enough to handle this. The two ways in which it can be
    applied are:

        1. In :meth:BaseSequenceGenerator.cost_matrix where it will
        be applied to the whole sequence at once.

        2. In :meth:BaseSequenceGenerator.generate where it will be
        applied to only one step of the sequence.

     emit N/A N/A self readouts N/A N/A N/A Implements the respective method of :class:`Readout`. N/A N/A abstractmethod cost N/A N/A self readouts outputs N/A N/A N/A Implements the respective method of :class:`Readout`. N/A N/A abstractmethod initial_outputs N/A N/A self batch_size N/A N/A N/A Implements the respective method of :class:`Readout`. N/A N/A abstractmethod N/A N/A add_metaclass ABCMeta AbstractFeedback N/A Brick N/A N/A The interface for the feedback component of a readout.

    See Also
    --------
    :class:`Readout`

    :class:`LookupFeedback` for integer outputs

     feedback N/A N/A self outputs N/A N/A N/A Implements the respective method of :class:`Readout`. N/A N/A abstractmethod N/A N/A add_metaclass ABCMeta TrivialEmitter N/A AbstractEmitter N/A N/A An emitter for the trivial case when readouts are outputs.

    Parameters
    ----------
    readout_dim : int
        The dimension of the readout.

    Notes
    -----
    By default :meth:`cost` always returns zero tensor.

     __init__ N/A N/A self readout_dim N/A kwargs N/A N/A N/A N/A N/A super TrivialEmitter self __init__ kwargs N/A N/A self readout_dim readout_dim N/A N/A lazy allocation N/A readout_dim emit N/A N/A self readouts N/A N/A N/A readouts N/A application cost N/A N/A self readouts outputs N/A N/A N/A N/A N/A tensor zeros_like outputs N/A application initial_outputs N/A N/A self batch_size N/A N/A N/A N/A N/A tensor zeros N/A batch_size N/A self readout_dim N/A application get_dim N/A N/A self name N/A N/A N/A N/A name outputs N/A N/A N/A self readout_dim N/A N/A N/A N/A super TrivialEmitter self get_dim name N/A N/A SoftmaxEmitter N/A AbstractEmitter Initializable Random N/A N/A A softmax emitter for the case of integer outputs.

    Interprets readout elements as energies corresponding to their indices.

    Parameters
    ----------
    initial_output : int or a scalar :class:`~theano.Variable`
        The initial output.

     __init__ N/A N/A self initial_output N/A 0 kwargs N/A N/A N/A N/A N/A super SoftmaxEmitter self __init__ kwargs N/A N/A self initial_output initial_output N/A N/A self softmax N/A NDimensionalSoftmax N/A N/A self children N/A N/A self softmax N/A probs N/A N/A self readouts N/A N/A N/A N/A N/A N/A self softmax apply readouts extra_ndim N/A N/A readouts ndim 2 N/A application emit N/A N/A self readouts N/A N/A N/A probs N/A N/A self probs readouts N/A batch_size N/A N/A probs shape N/A 0 N/A pvals_flat N/A N/A probs reshape N/A batch_size -1 N/A generated N/A N/A N/A self theano_rng multinomial pvals pvals_flat N/A N/A N/A N/A N/A generated reshape N/A probs shape argmax axis -1 N/A application cost N/A N/A self readouts outputs N/A N/A N/A N/A N/A N/A self softmax categorical_cross_entropy outputs readouts extra_ndim N/A N/A readouts ndim 2 N/A application costs N/A N/A self readouts N/A N/A N/A N/A N/A N/A N/A self softmax log_probabilities readouts extra_ndim N/A N/A readouts ndim 2 N/A application initial_outputs N/A N/A self batch_size N/A N/A N/A N/A N/A self initial_output N/A N/A tensor ones N/A batch_size dtype int64 N/A application get_dim N/A N/A self name N/A N/A N/A N/A name outputs N/A N/A 0 N/A N/A N/A N/A super SoftmaxEmitter self get_dim name N/A N/A TrivialFeedback N/A AbstractFeedback N/A N/A A feedback brick for the case when readout are outputs. __init__ N/A N/A self output_dim N/A kwargs N/A N/A N/A N/A N/A super TrivialFeedback self __init__ kwargs N/A N/A self output_dim output_dim N/A N/A lazy allocation N/A output_dim feedback N/A N/A self outputs N/A N/A N/A outputs N/A N/A application outputs N/A feedback get_dim N/A N/A self name N/A N/A N/A N/A name feedback N/A N/A N/A self output_dim N/A N/A N/A N/A super TrivialFeedback self get_dim name N/A N/A LookupFeedback N/A AbstractFeedback Initializable N/A N/A A feedback brick for the case when readout are integers.

    Stores and retrieves distributed representations of integers.

     __init__ N/A N/A self num_outputs feedback_dim N/A None None kwargs N/A N/A N/A N/A N/A super LookupFeedback self __init__ kwargs N/A N/A self num_outputs num_outputs N/A N/A self feedback_dim feedback_dim N/A N/A self lookup N/A LookupTable num_outputs feedback_dim weights_init N/A self weights_init N/A N/A self children N/A N/A self lookup N/A _push_allocation_config N/A N/A self N/A N/A N/A N/A N/A self lookup length N/A self num_outputs N/A N/A N/A self lookup dim N/A self feedback_dim N/A feedback N/A N/A self outputs N/A N/A N/A N/A N/A self output_dim 0 N/A N/A N/A N/A self lookup apply outputs N/A application get_dim N/A N/A self name N/A N/A N/A N/A name feedback N/A N/A N/A self feedback_dim N/A N/A N/A N/A super LookupFeedback self get_dim name N/A N/A FakeAttentionRecurrent N/A AbstractAttentionRecurrent Initializable N/A N/A Adds fake attention interface to a transition.

    :class:`BaseSequenceGenerator` requires its transition brick to support
    :class:`~blocks.bricks.attention.AbstractAttentionRecurrent` interface,
    that is to have an embedded attention mechanism.  For the cases when no
    attention is required (e.g.  language modeling or encoder-decoder
    models), :class:`FakeAttentionRecurrent` is used to wrap a usual
    recurrent brick. The resulting brick has no glimpses and simply
    passes all states and contexts to the wrapped one.

    .. todo::

        Get rid of this brick and support attention-less transitions
        in :class:`BaseSequenceGenerator`.

     __init__ N/A N/A self transition N/A kwargs N/A N/A N/A N/A N/A super FakeAttentionRecurrent self __init__ kwargs N/A N/A self transition transition N/A N/A self state_names N/A N/A transition apply states N/A N/A self context_names N/A N/A transition apply contexts N/A N/A self glimpse_names N/A N/A N/A self children N/A N/A self transition N/A apply N/A N/A self N/A args kwargs N/A N/A N/A N/A N/A self transition apply args kwargs N/A application apply_delegate N/A N/A self N/A N/A N/A N/A N/A self transition apply N/A N/A apply delegate compute_states N/A N/A self N/A args kwargs N/A N/A N/A N/A N/A self transition apply iterate False args kwargs N/A application compute_states_delegate N/A N/A self N/A N/A N/A N/A N/A self transition apply N/A N/A compute_states delegate take_glimpses N/A N/A self N/A args kwargs N/A N/A None N/A N/A application outputs N/A initial_states N/A N/A self batch_size N/A args kwargs N/A N/A N/A N/A N/A self transition initial_states batch_size args kwargs N/A application initial_states_outputs N/A N/A self N/A N/A N/A N/A N/A N/A self transition apply states N/A N/A N/A initial_states property outputs get_dim N/A N/A self name N/A N/A N/A N/A N/A N/A self transition get_dim name N/A N/A SequenceGenerator N/A BaseSequenceGenerator N/A N/A A more user-friendly interface for :class:`BaseSequenceGenerator`.

    Parameters
    ----------
    readout : instance of :class:`AbstractReadout`
        The readout component for the sequence generator.
    transition : instance of :class:`.BaseRecurrent`
        The recurrent transition to be used in the sequence generator.
        Will be combined with `attention`, if that one is given.
    attention : object, optional
        The attention mechanism to be added to ``transition``,
        an instance of
        :class:`~blocks.bricks.attention.AbstractAttention`.
    add_contexts : bool
        If ``True``, the
        :class:`.AttentionRecurrent` wrapping the
        `transition` will add additional contexts for the attended and its
        mask.
    \*\*kwargs : dict
        All keywords arguments are passed to the base class. If `fork`
        keyword argument is not provided, :class:`.Fork` is created
        that forks all transition sequential inputs without a "mask"
        substring in them.

     __init__ N/A N/A self readout transition attention add_contexts N/A None True kwargs N/A N/A normal_inputs N/A name N/A name N/A N/A transition apply sequences N/A mask name N/A N/A N/A kwargs setdefault fork N/A Fork normal_inputs N/A attention N/A N/A transition N/A AttentionRecurrent transition attention add_contexts add_contexts name att_trans N/A N/A transition N/A FakeAttentionRecurrent transition name with_fake_attention N/A N/A N/A N/A super SequenceGenerator self __init__ readout transition kwargs N/A N/A