neulab/docprompting-conala huggingface dataset:
class ast.IfExp(test, body, orelse)  
An expression such as a if b else c. Each field holds a single node, so in the following example, all three are Name nodes. >>> print(ast.dump(ast.parse('a if b else c', mode='eval'), indent=4))
Expression(
    body=IfExp(
        test=Name(id='b', ctx=Load()),
        body=Name(id='a', ctx=Load()),
        orelse=Name(id='c', ctx=Load())))
class ast.Import(names)  
An import statement. names is a list of alias nodes. >>> print(ast.dump(ast.parse('import x,y,z'), indent=4))
Module(
    body=[
        Import(
            names=[
                alias(name='x'),
                alias(name='y'),
                alias(name='z')])],
    type_ignores=[])
class ast.ImportFrom(module, names, level)  
Represents from x import y. module is a raw string of the ‘from’ name, without any leading dots, or None for statements such as from . import foo. level is an integer holding the level of the relative import (0 means absolute import). >>> print(ast.dump(ast.parse('from y import x,y,z'), indent=4))
Module(
    body=[
        ImportFrom(
            module='y',
            names=[
                alias(name='x'),
                alias(name='y'),
                alias(name='z')],
            level=0)],
    type_ignores=[])
ast.increment_lineno(node, n=1)  
Increment the line number and end line number of each node in the tree starting at node by n. This is useful to “move code” to a different location in a file.
class ast.UAdd  
class ast.USub  
class ast.Not  
class ast.Invert  
Unary operator tokens. Not is the not keyword, Invert is the ~ operator. >>> print(ast.dump(ast.parse('not x', mode='eval'), indent=4))
Expression(
    body=UnaryOp(
        op=Not(),
        operand=Name(id='x', ctx=Load())))
ast.iter_child_nodes(node)  
Yield all direct child nodes of node, that is, all fields that are nodes and all items of fields that are lists of nodes.
ast.iter_fields(node)  
Yield a tuple of (fieldname, value) for each field in node._fields that is present on node.
class ast.JoinedStr(values)  
An f-string, comprising a series of FormattedValue and Constant nodes. >>> print(ast.dump(ast.parse('f"sin({a}) is {sin(a):.3}"', mode='eval'), indent=4))
Expression(
    body=JoinedStr(
        values=[
            Constant(value='sin('),
            FormattedValue(
                value=Name(id='a', ctx=Load()),
                conversion=-1),
            Constant(value=') is '),
            FormattedValue(
                value=Call(
                    func=Name(id='sin', ctx=Load()),
                    args=[
                        Name(id='a', ctx=Load())],
                    keywords=[]),
                conversion=-1,
                format_spec=JoinedStr(
                    values=[
                        Constant(value='.3')]))]))
class ast.keyword(arg, value)  
A keyword argument to a function call or class definition. arg is a raw string of the parameter name, value is a node to pass in.
class ast.Lambda(args, body)  
lambda is a minimal function definition that can be used inside an expression. Unlike FunctionDef, body holds a single node. >>> print(ast.dump(ast.parse('lambda x,y: ...'), indent=4))
Module(
    body=[
        Expr(
            value=Lambda(
                args=arguments(
                    posonlyargs=[],
                    args=[
                        arg(arg='x'),
                        arg(arg='y')],
                    kwonlyargs=[],
                    kw_defaults=[],
                    defaults=[]),
                body=Constant(value=Ellipsis)))],
    type_ignores=[])
class ast.List(elts, ctx)  
class ast.Tuple(elts, ctx)  
A list or tuple. elts holds a list of nodes representing the elements. ctx is Store if the container is an assignment target (i.e. (x,y)=something), and Load otherwise. >>> print(ast.dump(ast.parse('[1, 2, 3]', mode='eval'), indent=4))
Expression(
    body=List(
        elts=[
            Constant(value=1),
            Constant(value=2),
            Constant(value=3)],
        ctx=Load()))
>>> print(ast.dump(ast.parse('(1, 2, 3)', mode='eval'), indent=4))
Expression(
    body=Tuple(
        elts=[
            Constant(value=1),
            Constant(value=2),
            Constant(value=3)],
        ctx=Load()))
ast.literal_eval(node_or_string)  
Safely evaluate an expression node or a string containing a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None. This can be used for safely evaluating strings containing Python values from untrusted sources without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing.  Warning It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler.   Changed in version 3.2: Now allows bytes and set literals.   Changed in version 3.9: Now supports creating empty sets with 'set()'.
class ast.Name(id, ctx)  
A variable name. id holds the name as a string, and ctx is one of the following types.
class ast.NamedExpr(target, value)   A named expression. This AST node is produced by the assignment expressions operator (also known as the walrus operator). As opposed to the Assign node in which the first argument can be multiple nodes, in this case both target and value must be single nodes. >>> print(ast.dump(ast.parse('(x := 4)', mode='eval'), indent=4))
Expression(
    body=NamedExpr(
        target=Name(id='x', ctx=Store()),
        value=Constant(value=4)))
class ast.NodeTransformer  
A NodeVisitor subclass that walks the abstract syntax tree and allows modification of nodes. The NodeTransformer will walk the AST and use the return value of the visitor methods to replace or remove the old node. If the return value of the visitor method is None, the node will be removed from its location, otherwise it is replaced with the return value. The return value may be the original node in which case no replacement takes place. Here is an example transformer that rewrites all occurrences of name lookups (foo) to data['foo']: class RewriteName(NodeTransformer):

    def visit_Name(self, node):
        return Subscript(
            value=Name(id='data', ctx=Load()),
            slice=Constant(value=node.id),
            ctx=node.ctx
        )
 Keep in mind that if the node you’re operating on has child nodes you must either transform the child nodes yourself or call the generic_visit() method for the node first. For nodes that were part of a collection of statements (that applies to all statement nodes), the visitor may also return a list of nodes rather than just a single node. If NodeTransformer introduces new nodes (that weren’t part of original tree) without giving them location information (such as lineno), fix_missing_locations() should be called with the new sub-tree to recalculate the location information: tree = ast.parse('foo', mode='eval')
new_tree = fix_missing_locations(RewriteName().visit(tree))
 Usually you use the transformer like this: node = YourTransformer().visit(node)
class ast.NodeVisitor  
A node visitor base class that walks the abstract syntax tree and calls a visitor function for every node found. This function may return a value which is forwarded by the visit() method. This class is meant to be subclassed, with the subclass adding visitor methods.  
visit(node)  
Visit a node. The default implementation calls the method called self.visit_classname where classname is the name of the node class, or generic_visit() if that method doesn’t exist. 
  
generic_visit(node)  
This visitor calls visit() on all children of the node. Note that child nodes of nodes that have a custom visitor method won’t be visited unless the visitor calls generic_visit() or visits them itself. 
 Don’t use the NodeVisitor if you want to apply changes to nodes during traversal. For this a special visitor exists (NodeTransformer) that allows modifications.  Deprecated since version 3.8: Methods visit_Num(), visit_Str(), visit_Bytes(), visit_NameConstant() and visit_Ellipsis() are deprecated now and will not be called in future Python versions. Add the visit_Constant() method to handle all constant nodes.
generic_visit(node)  
This visitor calls visit() on all children of the node. Note that child nodes of nodes that have a custom visitor method won’t be visited unless the visitor calls generic_visit() or visits them itself.
visit(node)  
Visit a node. The default implementation calls the method called self.visit_classname where classname is the name of the node class, or generic_visit() if that method doesn’t exist.
ast.parse(source, filename='<unknown>', mode='exec', *, type_comments=False, feature_version=None)  
Parse the source into an AST node. Equivalent to compile(source,
filename, mode, ast.PyCF_ONLY_AST). If type_comments=True is given, the parser is modified to check and return type comments as specified by PEP 484 and PEP 526. This is equivalent to adding ast.PyCF_TYPE_COMMENTS to the flags passed to compile(). This will report syntax errors for misplaced type comments. Without this flag, type comments will be ignored, and the type_comment field on selected AST nodes will always be None. In addition, the locations of # type:
ignore comments will be returned as the type_ignores attribute of Module (otherwise it is always an empty list). In addition, if mode is 'func_type', the input syntax is modified to correspond to PEP 484 “signature type comments”, e.g. (str, int) -> List[str]. Also, setting feature_version to a tuple (major, minor) will attempt to parse using that Python version’s grammar. Currently major must equal to 3. For example, setting feature_version=(3, 4) will allow the use of async and await as variable names. The lowest supported version is (3, 4); the highest is sys.version_info[0:2].  Warning It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler.   Changed in version 3.8: Added type_comments, mode='func_type' and feature_version.
class ast.Pass  
A pass statement. >>> print(ast.dump(ast.parse('pass'), indent=4))
Module(
    body=[
        Pass()],
    type_ignores=[])
ast.PyCF_ALLOW_TOP_LEVEL_AWAIT  
Enables support for top-level await, async for, async with and async comprehensions.  New in version 3.8.
ast.PyCF_ONLY_AST  
Generates and returns an abstract syntax tree instead of returning a compiled code object.
ast.PyCF_TYPE_COMMENTS  
Enables support for PEP 484 and PEP 526 style type comments (# type: <type>, # type: ignore <stuff>).  New in version 3.8.
class ast.Raise(exc, cause)  
A raise statement. exc is the exception object to be raised, normally a Call or Name, or None for a standalone raise. cause is the optional part for y in raise x from y. >>> print(ast.dump(ast.parse('raise x from y'), indent=4))
Module(
    body=[
        Raise(
            exc=Name(id='x', ctx=Load()),
            cause=Name(id='y', ctx=Load()))],
    type_ignores=[])
class ast.Return(value)  
A return statement. >>> print(ast.dump(ast.parse('return 4'), indent=4))
Module(
    body=[
        Return(
            value=Constant(value=4))],
    type_ignores=[])
class ast.Set(elts)  
A set. elts holds a list of nodes representing the set’s elements. >>> print(ast.dump(ast.parse('{1, 2, 3}', mode='eval'), indent=4))
Expression(
    body=Set(
        elts=[
            Constant(value=1),
            Constant(value=2),
            Constant(value=3)]))
class ast.Slice(lower, upper, step)  
Regular slicing (on the form lower:upper or lower:upper:step). Can occur only inside the slice field of Subscript, either directly or as an element of Tuple. >>> print(ast.dump(ast.parse('l[1:2]', mode='eval'), indent=4))
Expression(
    body=Subscript(
        value=Name(id='l', ctx=Load()),
        slice=Slice(
            lower=Constant(value=1),
            upper=Constant(value=2)),
        ctx=Load()))
class ast.Starred(value, ctx)  
A *var variable reference. value holds the variable, typically a Name node. This type must be used when building a Call node with *args. >>> print(ast.dump(ast.parse('a, *b = it'), indent=4))
Module(
    body=[
        Assign(
            targets=[
                Tuple(
                    elts=[
                        Name(id='a', ctx=Store()),
                        Starred(
                            value=Name(id='b', ctx=Store()),
                            ctx=Store())],
                    ctx=Store())],
            value=Name(id='it', ctx=Load()))],
    type_ignores=[])
class ast.Subscript(value, slice, ctx)  
A subscript, such as l[1]. value is the subscripted object (usually sequence or mapping). slice is an index, slice or key. It can be a Tuple and contain a Slice. ctx is Load, Store or Del according to the action performed with the subscript. >>> print(ast.dump(ast.parse('l[1:2, 3]', mode='eval'), indent=4))
Expression(
    body=Subscript(
        value=Name(id='l', ctx=Load()),
        slice=Tuple(
            elts=[
                Slice(
                    lower=Constant(value=1),
                    upper=Constant(value=2)),
                Constant(value=3)],
            ctx=Load()),
        ctx=Load()))
class ast.Try(body, handlers, orelse, finalbody)  
try blocks. All attributes are list of nodes to execute, except for handlers, which is a list of ExceptHandler nodes. >>> print(ast.dump(ast.parse("""
... try:
...    ...
... except Exception:
...    ...
... except OtherException as e:
...    ...
... else:
...    ...
... finally:
...    ...
... """), indent=4))
Module(
    body=[
        Try(
            body=[
                Expr(
                    value=Constant(value=Ellipsis))],
            handlers=[
                ExceptHandler(
                    type=Name(id='Exception', ctx=Load()),
                    body=[
                        Expr(
                            value=Constant(value=Ellipsis))]),
                ExceptHandler(
                    type=Name(id='OtherException', ctx=Load()),
                    name='e',
                    body=[
                        Expr(
                            value=Constant(value=Ellipsis))])],
            orelse=[
                Expr(
                    value=Constant(value=Ellipsis))],
            finalbody=[
                Expr(
                    value=Constant(value=Ellipsis))])],
    type_ignores=[])
class ast.UnaryOp(op, operand)  
A unary operation. op is the operator, and operand any expression node.
ast.unparse(ast_obj)  
Unparse an ast.AST object and generate a string with code that would produce an equivalent ast.AST object if parsed back with ast.parse().  Warning The produced code string will not necessarily be equal to the original code that generated the ast.AST object (without any compiler optimizations, such as constant tuples/frozensets).   Warning Trying to unparse a highly complex expression would result with RecursionError.   New in version 3.9.
ast.walk(node)  
Recursively yield all descendant nodes in the tree starting at node (including node itself), in no specified order. This is useful if you only want to modify nodes in place and don’t care about the context.
class ast.While(test, body, orelse)  
A while loop. test holds the condition, such as a Compare node. >> print(ast.dump(ast.parse("""
... while x:
...    ...
... else:
...    ...
... """), indent=4))
Module(
    body=[
        While(
            test=Name(id='x', ctx=Load()),
            body=[
                Expr(
                    value=Constant(value=Ellipsis))],
            orelse=[
                Expr(
                    value=Constant(value=Ellipsis))])],
    type_ignores=[])
class ast.With(items, body, type_comment)  
A with block. items is a list of withitem nodes representing the context managers, and body is the indented block inside the context.  
type_comment  
type_comment is an optional string with the type annotation as a comment.
class ast.withitem(context_expr, optional_vars)  
A single context manager in a with block. context_expr is the context manager, often a Call node. optional_vars is a Name, Tuple or List for the as foo part, or None if that isn’t used. >>> print(ast.dump(ast.parse("""\
... with a as b, c as d:
...    something(b, d)
... """), indent=4))
Module(
    body=[
        With(
            items=[
                withitem(
                    context_expr=Name(id='a', ctx=Load()),
                    optional_vars=Name(id='b', ctx=Store())),
                withitem(
                    context_expr=Name(id='c', ctx=Load()),
                    optional_vars=Name(id='d', ctx=Store()))],
            body=[
                Expr(
                    value=Call(
                        func=Name(id='something', ctx=Load()),
                        args=[
                            Name(id='b', ctx=Load()),
                            Name(id='d', ctx=Load())],
                        keywords=[]))])],
    type_ignores=[])
class ast.Yield(value)  
class ast.YieldFrom(value)  
A yield or yield from expression. Because these are expressions, they must be wrapped in a Expr node if the value sent back is not used. >>> print(ast.dump(ast.parse('yield x'), indent=4))
Module(
    body=[
        Expr(
            value=Yield(
                value=Name(id='x', ctx=Load())))],
    type_ignores=[])

>>> print(ast.dump(ast.parse('yield from x'), indent=4))
Module(
    body=[
        Expr(
            value=YieldFrom(
                value=Name(id='x', ctx=Load())))],
    type_ignores=[])
class asynchat.async_chat  
This class is an abstract subclass of asyncore.dispatcher. To make practical use of the code you must subclass async_chat, providing meaningful collect_incoming_data() and found_terminator() methods. The asyncore.dispatcher methods can be used, although not all make sense in a message/response context. Like asyncore.dispatcher, async_chat defines a set of events that are generated by an analysis of socket conditions after a select() call. Once the polling loop has been started the async_chat object’s methods are called by the event-processing framework with no action on the part of the programmer. Two class attributes can be modified, to improve performance, or possibly even to conserve memory.  
ac_in_buffer_size  
The asynchronous input buffer size (default 4096). 
  
ac_out_buffer_size  
The asynchronous output buffer size (default 4096). 
 Unlike asyncore.dispatcher, async_chat allows you to define a FIFO queue of producers. A producer need have only one method, more(), which should return data to be transmitted on the channel. The producer indicates exhaustion (i.e. that it contains no more data) by having its more() method return the empty bytes object. At this point the async_chat object removes the producer from the queue and starts using the next producer, if any. When the producer queue is empty the handle_write() method does nothing. You use the channel object’s set_terminator() method to describe how to recognize the end of, or an important breakpoint in, an incoming transmission from the remote endpoint. To build a functioning async_chat subclass your input methods collect_incoming_data() and found_terminator() must handle the data that the channel receives asynchronously. The methods are described below.
ac_in_buffer_size  
The asynchronous input buffer size (default 4096).
ac_out_buffer_size  
The asynchronous output buffer size (default 4096).
async_chat.close_when_done()  
Pushes a None on to the producer queue. When this producer is popped off the queue it causes the channel to be closed.
async_chat.collect_incoming_data(data)  
Called with data holding an arbitrary amount of received data. The default method, which must be overridden, raises a NotImplementedError exception.
async_chat.discard_buffers()  
In emergencies this method will discard any data held in the input and/or output buffers and the producer queue.
async_chat.found_terminator()  
Called when the incoming data stream matches the termination condition set by set_terminator(). The default method, which must be overridden, raises a NotImplementedError exception. The buffered input data should be available via an instance attribute.
async_chat.push(data)  
Pushes data on to the channel’s queue to ensure its transmission. This is all you need to do to have the channel write the data out to the network, although it is possible to use your own producers in more complex schemes to implement encryption and chunking, for example.
async_chat.push_with_producer(producer)  
Takes a producer object and adds it to the producer queue associated with the channel. When all currently-pushed producers have been exhausted the channel will consume this producer’s data by calling its more() method and send the data to the remote endpoint.
async_chat.set_terminator(term)  
Sets the terminating condition to be recognized on the channel. term may be any of three types of value, corresponding to three different ways to handle incoming protocol data.   
term Description   
string Will call found_terminator() when the string is found in the input stream  
integer Will call found_terminator() when the indicated number of characters have been received  
None The channel continues to collect data forever   Note that any data following the terminator will be available for reading by the channel after found_terminator() is called.
asyncio — Asynchronous I/O  Hello World! import asyncio

async def main():
    print('Hello ...')
    await asyncio.sleep(1)
    print('... World!')

# Python 3.7+
asyncio.run(main())
  asyncio is a library to write concurrent code using the async/await syntax. asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc. asyncio is often a perfect fit for IO-bound and high-level structured network code. asyncio provides a set of high-level APIs to:  
run Python coroutines concurrently and have full control over their execution; perform network IO and IPC; control subprocesses; distribute tasks via queues; 
synchronize concurrent code;  Additionally, there are low-level APIs for library and framework developers to:  create and manage event loops, which provide asynchronous APIs for networking, running subprocesses, handling OS signals, etc; implement efficient protocols using transports; 
bridge callback-based libraries and code with async/await syntax.  Reference High-level APIs  Coroutines and Tasks Streams Synchronization Primitives Subprocesses Queues Exceptions  Low-level APIs  Event Loop Futures Transports and Protocols Policies Platform Support  Guides and Tutorials  High-level API Index Low-level API Index Developing with asyncio   Note The source code for asyncio can be found in Lib/asyncio/.
class asyncio.AbstractChildWatcher  
 
add_child_handler(pid, callback, *args)  
Register a new child handler. Arrange for callback(pid, returncode, *args) to be called when a process with PID equal to pid terminates. Specifying another callback for the same process replaces the previous handler. The callback callable must be thread-safe. 
  
remove_child_handler(pid)  
Removes the handler for process with PID equal to pid. The function returns True if the handler was successfully removed, False if there was nothing to remove. 
  
attach_loop(loop)  
Attach the watcher to an event loop. If the watcher was previously attached to an event loop, then it is first detached before attaching to the new loop. Note: loop may be None. 
  
is_active()  
Return True if the watcher is ready to use. Spawning a subprocess with inactive current child watcher raises RuntimeError.  New in version 3.8.  
  
close()  
Close the watcher. This method has to be called to ensure that underlying resources are cleaned-up.
add_child_handler(pid, callback, *args)  
Register a new child handler. Arrange for callback(pid, returncode, *args) to be called when a process with PID equal to pid terminates. Specifying another callback for the same process replaces the previous handler. The callback callable must be thread-safe.