Module ImportFrom alias Expr Str Assign NameStore Str Assign NameStore ListLoad Str ImportFrom alias Import alias Import alias Import alias Import alias Import alias Import alias ImportFrom alias ImportFrom alias alias alias alias alias alias alias alias alias ImportFrom alias ImportFrom alias alias ImportFrom alias alias alias Assign NameStore Call AttributeLoad NameLoad attr NameLoad ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad AttributeLoad NameLoad attr Num Num Num BinOpMult Num Num Num Num BinOpMult Num Num NameLoad Num NameLoad NameLoad NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If BoolOpAnd CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr CompareEq AttributeLoad NameLoad attr Str body Expr Call AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Num IfExp UnaryOpNot NameLoad NameLoad Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr IfExp BoolOpOr NameLoad AttributeLoad NameLoad attr Call NameLoad AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr If AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareIsNot AttributeLoad NameLoad attr NameLoad body Assign TupleStore NameStore NameStore NameStore AttributeLoad NameLoad attr If UnaryOpNot AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Str Call AttributeLoad Str attr Call AttributeLoad NameLoad attr NameLoad Raise NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad Str attr keyword AttributeLoad AttributeLoad NameLoad attr attr keyword AttributeLoad AttributeLoad NameLoad attr attr keyword Call NameLoad Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str With AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad NameLoad attr If Call NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str Call NameLoad NameLoad For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore ListLoad If CompareIsNot AttributeLoad NameLoad attr NameLoad body Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr For NameStore NameLoad body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad NameLoad attr While NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr If UnaryOpNot NameLoad body Break Expr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Dict For NameStore Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body If CompareNotIn AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Assign SubscriptStore AttributeLoad NameLoad attr Index AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad NameLoad AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str FunctionDef arguments args defaults body If UnaryOpNot AttributeLoad NameLoad attr body Return ListLoad Return ListComp AttributeLoad NameLoad attr comprehension NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args defaults body Expr Call AttributeLoad NameLoad attr If CompareIsNot AttributeLoad NameLoad attr NameLoad body For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr decorator_list While AttributeLoad NameLoad attr body Assign NameStore Call NameLoad Expr Call NameLoad If CompareEq Call NameLoad NameLoad Num body Assign AttributeStore NameLoad attr NameLoad orelse For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str If UnaryOpNot BoolOpOr Call NameLoad NameLoad NameLoad CompareIs NameLoad NameLoad body Raise Call NameLoad Str Call NameLoad NameLoad If UnaryOpNot BoolOpOr Call NameLoad NameLoad NameLoad CompareIs NameLoad NameLoad body Raise Call NameLoad Str Call NameLoad NameLoad If UnaryOpNot AttributeLoad NameLoad attr body Raise Call NameLoad Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Assign NameStore AttributeLoad Call AttributeLoad NameLoad attr NameLoad NameLoad attr Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr NameLoad If AttributeLoad NameLoad attr body While NameLoad body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr TryExcept body Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr keyword Num Break handlers ExceptHandler type NameLoad body Continue Assert CompareIs NameLoad NameLoad If CompareIsNot NameLoad NameLoad body Raise NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad NameLoad body Expr Str TryExcept body Return Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad NameLoad handlers ExceptHandler type NameLoad body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore NameLoad While UnaryOpNot NameLoad body Assign NameStore AttributeLoad AttributeLoad SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index AttributeLoad NameLoad attr attr attr If CompareIn NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad attr NameLoad Assign NameStore NameLoad orelse Assign NameStore NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Num For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Call NameLoad NameLoad AttributeLoad AttributeLoad NameLoad attr attr FunctionDef arguments args NameParam NameParam defaults body Expr Str Return GeneratorExp NameLoad comprehension TupleStore NameStore NameStore Call NameLoad AttributeLoad NameLoad attr comprehension TupleStore NameStore NameStore Call NameLoad NameLoad CompareEq NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr BinOpMult Num Call NameLoad NameLoad AugAssignAdd AttributeStore NameLoad attr Call NameLoad NameLoad For NameStore NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list TryExcept body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareEq AttributeLoad NameLoad attr Num body Expr Call NameLoad NameLoad Return Assign NameStore ListLoad For TupleStore NameStore NameStore Call NameLoad AttributeLoad NameLoad attr body For TupleStore NameStore NameStore Call NameLoad NameLoad body If CompareEq AttributeLoad NameLoad attr Num body Expr Call NameLoad AttributeLoad SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index NameLoad attr Continue If CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad Str attr NameLoad NameLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call SubscriptLoad NameLoad Index AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr GeneratorExp TupleLoad NameLoad NameLoad comprehension NameStore Call NameLoad NameLoad NameLoad handlers ExceptHandler type NameLoad name NameStore body Expr Call AttributeLoad NameLoad attr Str AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Expr Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Expr Call AttributeLoad NameLoad attr Str Assign NameStore ListComp TupleLoad NameLoad NameLoad comprehension TupleStore NameStore NameStore Call NameLoad AttributeLoad NameLoad attr comprehension TupleStore NameStore NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr AttributeLoad NameLoad attr Call NameLoad NameLoad AttributeLoad AttributeLoad NameLoad attr attr If NameLoad body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr BinOpDiv AttributeLoad NameLoad attr Num Expr Call AttributeLoad NameLoad attr BinOpMult Num Call NameLoad NameLoad For TupleStore NameStore NameStore NameLoad body Assign NameStore CompareIn Call NameLoad NameLoad TupleLoad NameLoad NameLoad For NameStore AttributeLoad NameLoad attr body If BoolOpOr NameLoad CompareGtE AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad orelse AugAssignAdd AttributeStore NameLoad attr Num Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr Str While Call NameLoad GeneratorExp Call AttributeLoad NameLoad attr comprehension NameStore Call NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Num Expr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr attr Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body FunctionDef arguments args defaults body While AttributeLoad NameLoad attr body TryExcept body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad handlers ExceptHandler type NameLoad body Assign AttributeStore AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Break Expr Call AttributeLoad NameLoad attr Str AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Expr Call AttributeLoad NameLoad attr Str AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad Str attr AttributeLoad AttributeLoad NameLoad attr attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr attr NameLoad keyword NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body With AttributeLoad NameLoad attr body AugAssignAdd AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Num AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return CompareGt AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr With AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad NameLoad attr Num If CompareGtE Call NameLoad AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr body If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad body Expr Str Expr Call AttributeLoad NameLoad attr NameLoad With AttributeLoad NameLoad attr body Assign NameStore ListLoad Assign NameStore Num While CompareGt Call NameLoad AttributeLoad NameLoad attr Num body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Num If BoolOpAnd NameLoad CompareIsNot AttributeLoad NameLoad attr NameLoad body If CompareGt Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad Call AttributeLoad Str attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If CompareIsNot AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr TupleLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Num Continue orelse If CompareGt BinOpAdd NameLoad Call NameLoad NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr Break Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AugAssignAdd NameStore Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad If NameLoad body Expr Call AttributeLoad NameLoad attr BinOpMult Num Call NameLoad NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareLt Call NameLoad AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr body With AttributeLoad NameLoad attr body If CompareLt Call NameLoad AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr If CompareGt NameLoad Num body Expr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpDiv NameLoad Num decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareGtE Call NameLoad AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr body With AttributeLoad NameLoad attr body If CompareGtE Call NameLoad AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr If AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr orelse Raise Call NameLoad Str AttributeLoad AttributeLoad NameLoad attr attr decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr TupleLoad NameLoad NameLoad decorator_list NameLoad decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Return decorator_list NameLoad decorator_list
N/A __future__ division N/A 
Author: Emmett Butler, Keith Bourgoin
 N/A __license__ 
Copyright 2015 Parse.ly, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
 N/A __all__ N/A Producer collections deque N/A logging N/A platform N/A sys N/A threading N/A traceback N/A weakref common CompressionType exceptions ERROR_CODES KafkaException InvalidMessageSize MessageSizeTooLarge NoBrokersAvailableError NotLeaderForPartition ProducerQueueFullError ProducerStoppedException SocketDisconnectedError partitioners random_partitioner protocol Message ProduceRequest utils.compat iteritems itervalues Empty N/A log N/A N/A logging getLogger __name__ Producer N/A object N/A N/A Implements asynchronous producer logic similar to the JVM driver.

    It creates a thread of execution for each broker that is the leader of
    one or more of its topic's partitions. Each of these threads (which may
    use `threading` or some other parallelism implementation like `gevent`)
    is associated with a queue that holds the messages that are waiting to be
    sent to that queue's broker.
     __init__ N/A N/A self cluster topic partitioner compression max_retries retry_backoff_ms required_acks ack_timeout_ms max_queued_messages min_queued_messages linger_ms block_on_queue_full max_request_size sync delivery_reports auto_start N/A random_partitioner N/A CompressionType NONE 3 100 1 N/A 10 1000 100000 70000 N/A 5 1000 True 1000012 False False True N/A N/A Instantiate a new AsyncProducer

        :param cluster: The cluster to which to connect
        :type cluster: :class:`pykafka.cluster.Cluster`
        :param topic: The topic to which to produce messages
        :type topic: :class:`pykafka.topic.Topic`
        :param partitioner: The partitioner to use during message production
        :type partitioner: :class:`pykafka.partitioners.BasePartitioner`
        :param compression: The type of compression to use.
        :type compression: :class:`pykafka.common.CompressionType`
        :param max_retries: How many times to attempt to produce a given batch of
            messages before raising an error. Allowing retries will potentially change
            the ordering of records because if two records are sent to a single partition,
            and the first fails and is retried but the second succeeds, then the second
            record may appear first. If you want to completely disallow message
            reordering, use `sync=True`.
        :type max_retries: int
        :param retry_backoff_ms: The amount of time (in milliseconds) to
            back off during produce request retries.
        :type retry_backoff_ms: int
        :param required_acks: The number of other brokers that must have
            committed the data to their log and acknowledged this to the leader
            before a request is considered complete
        :type required_acks: int
        :param ack_timeout_ms: The amount of time (in milliseconds) to wait for
            acknowledgment of a produce request.
        :type ack_timeout_ms: int
        :param max_queued_messages: The maximum number of messages the producer
            can have waiting to be sent to the broker. If messages are sent
            faster than they can be delivered to the broker, the producer will
            either block or throw an exception based on the preference specified
            with block_on_queue_full.
        :type max_queued_messages: int
        :param min_queued_messages: The minimum number of messages the producer
            can have waiting in a queue before it flushes that queue to its
            broker (must be greater than 0). This paramater can be used to
            control the number of messages sent in one batch during async
            production. This parameter is automatically overridden to 1
            when `sync=True`.
        :type min_queued_messages: int
        :param linger_ms: This setting gives the upper bound on the delay for
            batching: once the producer gets min_queued_messages worth of
            messages for a broker, it will be sent immediately regardless of
            this setting.  However, if we have fewer than this many messages
            accumulated for this partition we will 'linger' for the specified
            time waiting for more records to show up. linger_ms=0 indicates no
            lingering.
        :type linger_ms: int
        :param block_on_queue_full: When the producer's message queue for a
            broker contains max_queued_messages, we must either stop accepting
            new messages (block) or throw an error. If True, this setting
            indicates we should block until space is available in the queue.
            If False, we should throw an error immediately.
        :type block_on_queue_full: bool
        :param max_request_size:
            The maximum size of a request in bytes. This is also effectively a
            cap on the maximum record size. Note that the server has its own
            cap on record size which may be different from this. This setting
            will limit the number of record batches the producer will send in a
            single request to avoid sending huge requests.
        :type max_request_size: int
        :param sync: Whether calls to `produce` should wait for the message to
            send before returning.  If `True`, an exception will be raised from
            `produce()` if delivery to kafka failed.
        :type sync: bool
        :param delivery_reports: If set to `True`, the producer will maintain a
            thread-local queue on which delivery reports are posted for each
            message produced.  These must regularly be retrieved through
            `get_delivery_report()`, which returns a 2-tuple of
            :class:`pykafka.protocol.Message` and either `None` (for success)
            or an `Exception` in case of failed delivery to kafka.
            This setting is ignored when `sync=True`.
        :type delivery_reports: bool
        :param auto_start: Whether the producer should begin communicating
            with kafka after __init__ is complete. If false, communication
            can be started with `start()`.
        :type auto_start: bool
         N/A N/A self _cluster cluster N/A N/A self _topic topic N/A N/A self _partitioner partitioner N/A N/A self _compression compression N/A N/A N/A N/A self _compression N/A CompressionType SNAPPY N/A N/A platform python_implementation PyPy N/A N/A N/A N/A log warning Caution: python-snappy segfaults when attempting to compress large messages under PyPy N/A N/A self _max_retries max_retries N/A N/A self _retry_backoff_ms retry_backoff_ms N/A N/A self _required_acks required_acks N/A N/A self _ack_timeout_ms ack_timeout_ms N/A N/A self _max_queued_messages max_queued_messages N/A N/A self _min_queued_messages N/A max 1 N/A N/A sync min_queued_messages 1 N/A N/A self _linger_ms linger_ms N/A N/A self _block_on_queue_full block_on_queue_full N/A N/A self _max_request_size max_request_size N/A N/A self _synchronous sync N/A N/A self _worker_exception None N/A N/A self _worker_trace_logged False N/A N/A self _owned_brokers None N/A N/A self _delivery_reports N/A N/A delivery_reports N/A self _synchronous N/A _DeliveryReportQueue N/A N/A self _cluster handler N/A _DeliveryReportNone N/A N/A self _auto_start auto_start N/A N/A self _running False N/A N/A self _update_lock N/A N/A N/A N/A self _cluster handler Lock N/A N/A self _auto_start N/A N/A N/A N/A self start N/A __del__ N/A N/A self N/A N/A N/A N/A N/A log debug N/A N/A Finalising {} format self N/A N/A N/A self stop N/A _raise_worker_exceptions N/A N/A self N/A N/A N/A Raises exceptions encountered on worker threads N/A N/A N/A self _worker_exception None N/A N/A N/A _ ex tb N/A self _worker_exception N/A N/A N/A self _worker_trace_logged N/A N/A N/A self _worker_trace_logged True N/A N/A N/A log error Exception encountered in worker thread:
%s N/A N/A join N/A N/A traceback format_tb tb N/A ex N/A __repr__ N/A N/A self N/A N/A N/A N/A N/A <{module}.{name} at {id_}> format module N/A N/A self __class__ __module__ name N/A N/A self __class__ __name__ id_ N/A hex N/A id self N/A __enter__ N/A N/A self N/A N/A N/A Context manager entry point - start the producer N/A self N/A __exit__ N/A N/A self exc_type exc_value traceback N/A N/A N/A Context manager exit point - stop the producer N/A N/A N/A self stop N/A start N/A N/A self N/A N/A N/A Set up data structures and start worker threads N/A N/A N/A self _running N/A N/A N/A N/A self _setup_owned_brokers N/A N/A self _running True N/A N/A N/A self _raise_worker_exceptions N/A _update N/A N/A self N/A N/A N/A Update the producer and cluster after an ERROR_CODE

        Also re-produces messages that were in queues at the time the update
        was triggered
         N/A N/A self _update_lock N/A N/A N/A N/A N/A self _cluster update N/A queued_messages N/A N/A self _setup_owned_brokers N/A N/A len queued_messages N/A N/A N/A N/A log debug Re-producing %d queued messages after update N/A len queued_messages N/A message queued_messages N/A N/A N/A N/A self _produce message N/A _setup_owned_brokers N/A N/A self N/A N/A N/A Instantiate one OwnedBroker per broker

        If there are already OwnedBrokers instantiated, safely stop and flush them
        before creating new ones.
         N/A queued_messages N/A N/A N/A N/A self _owned_brokers None N/A N/A brokers N/A list N/A N/A N/A self _owned_brokers keys N/A broker brokers N/A N/A owned_broker N/A N/A N/A self _owned_brokers pop broker N/A N/A N/A owned_broker stop N/A True N/A N/A batch N/A N/A owned_broker flush N/A self _linger_ms N/A self _max_request_size N/A N/A batch N/A N/A N/A N/A N/A queued_messages extend batch N/A N/A self _owned_brokers N/A N/A partition N/A N/A N/A N/A self _topic partitions values N/A N/A N/A N/A N/A partition leader id N/A self _owned_brokers N/A N/A N/A N/A self _owned_brokers N/A N/A N/A partition leader id N/A OwnedBroker self N/A partition leader N/A queued_messages N/A stop N/A N/A self N/A N/A N/A Mark the producer as stopped, and wait until all messages to be sent get_queue_readers N/A N/A N/A N/A N/A N/A N/A self _owned_brokers N/A N/A N/A N/A N/A N/A owned_broker _queue_reader_worker N/A owned_broker N/A N/A N/A self _owned_brokers values N/A owned_broker running N/A stop_owned_brokers N/A N/A N/A N/A N/A N/A N/A self _wait_all N/A N/A N/A self _owned_brokers None N/A N/A owned_broker N/A N/A N/A self _owned_brokers values N/A N/A N/A N/A owned_broker stop N/A N/A N/A self _running N/A N/A queue_readers N/A get_queue_readers N/A N/A stop_owned_brokers N/A N/A N/A len queue_readers 0 N/A N/A N/A self _running False N/A N/A queue_reader queue_readers N/A N/A N/A N/A queue_reader join N/A produce N/A N/A self message partition_key N/A None N/A N/A Produce a message.

        :param message: The message to produce (use None to send null)
        :type message: bytes
        :param partition_key: The key to use when deciding which partition to send this
            message to
        :type partition_key: bytes
         N/A N/A N/A N/A isinstance partition_key bytes N/A partition_key None N/A N/A N/A TypeError Producer.produce accepts a bytes object as partition_key, but it got '%s' N/A type partition_key N/A N/A N/A N/A isinstance message bytes N/A message None N/A N/A N/A TypeError Producer.produce accepts a bytes object as message, but it got '%s' N/A type message N/A N/A N/A self _running N/A N/A N/A ProducerStoppedException N/A partitions N/A list N/A N/A N/A N/A self _topic partitions values N/A partition_id N/A N/A N/A self _partitioner partitions partition_key id N/A msg N/A Message value message partition_key partition_key partition_id partition_id delivery_report_q N/A N/A self _delivery_reports queue N/A N/A N/A self _produce msg N/A N/A self _synchronous N/A N/A True N/A N/A N/A N/A N/A N/A self _cluster handler sleep N/A N/A N/A N/A reported_msg exc N/A N/A self get_delivery_report timeout 1 N/A N/A N/A N/A Empty N/A N/A N/A N/A reported_msg msg N/A N/A exc None N/A N/A exc N/A N/A N/A self _raise_worker_exceptions N/A get_delivery_report N/A N/A self block timeout N/A True None N/A N/A Fetch delivery reports for messages produced on the current thread

        Returns 2-tuples of a `pykafka.protocol.Message` and either `None`
        (for successful deliveries) or `Exception` (for failed deliveries).
        This interface is only available if you enabled `delivery_reports` on
        init (and you did not use `sync=True`)

        :param block: Whether to block on dequeueing a delivery report
        :type block: bool
        :param timeout: How long (in seconds) to block before returning None
        ;type timeout: int
         N/A N/A N/A N/A N/A N/A N/A self _delivery_reports queue get block timeout N/A N/A N/A AttributeError N/A N/A N/A KafkaException Delivery-reporting is disabled N/A _produce N/A N/A self message N/A N/A N/A Enqueue a message for the relevant broker

        :param message: Message with valid `partition_id`, ready to be sent
        :type message: `pykafka.protocol.Message`
         N/A success False N/A N/A success N/A N/A leader_id N/A N/A N/A N/A N/A self _topic partitions N/A N/A message partition_id leader id N/A N/A leader_id N/A self _owned_brokers N/A N/A N/A N/A N/A N/A self _owned_brokers N/A leader_id enqueue message N/A success True N/A N/A success False N/A _send_request N/A N/A self message_batch owned_broker N/A N/A N/A Send the produce request to the broker and handle the response.

        :param message_batch: An iterable of messages to send
        :type message_batch: iterable of `pykafka.protocol.Message`
        :param owned_broker: The broker to which to send the request
        :type owned_broker: :class:`pykafka.producer.OwnedBroker`
         N/A req N/A ProduceRequest compression_type N/A self _compression required_acks N/A self _required_acks timeout N/A self _ack_timeout_ms N/A N/A req delivered 0 N/A msg message_batch N/A N/A N/A N/A req add_message msg N/A N/A self _topic name N/A msg partition_id N/A N/A N/A log debug Sending %d messages to broker %d N/A len message_batch N/A N/A owned_broker broker id _get_partition_msgs N/A N/A partition_id req N/A N/A N/A Get all the messages for the partitions from the request. N/A N/A mset N/A N/A topic partitions N/A iteritems N/A req msets N/A N/A p_id mset N/A iteritems partitions N/A p_id partition_id N/A mark_as_delivered N/A N/A message_batch N/A N/A N/A N/A N/A owned_broker increment_messages_pending N/A -1 N/A len message_batch N/A N/A req delivered N/A len message_batch N/A msg message_batch N/A N/A N/A N/A N/A self _delivery_reports put msg N/A N/A N/A N/A response N/A N/A N/A owned_broker broker produce_messages req N/A N/A N/A self _required_acks 0 N/A N/A N/A mark_as_delivered message_batch N/A N/A to_retry N/A N/A N/A topic partitions N/A iteritems N/A response topics N/A N/A N/A partition presponse N/A iteritems partitions N/A N/A N/A N/A presponse err 0 N/A N/A N/A mark_as_delivered N/A N/A N/A N/A req msets N/A topic N/A partition messages N/A N/A N/A N/A presponse err N/A NotLeaderForPartition ERROR_CODE N/A N/A N/A N/A self _update N/A info N/A N/A Produce request for {}/{} to {}:{} failed with error code {}. format topic partition N/A N/A owned_broker broker host N/A N/A owned_broker broker port N/A presponse err N/A N/A N/A log warning info N/A exc N/A N/A ERROR_CODES N/A N/A presponse err info N/A N/A N/A to_retry extend N/A N/A mset exc N/A mset N/A _get_partition_msgs partition req N/A N/A N/A SocketDisconnectedError N/A exc N/A N/A N/A N/A log warning Broker %s:%s disconnected. Retrying. N/A N/A owned_broker broker host N/A N/A owned_broker broker port N/A N/A N/A N/A N/A self _update N/A N/A N/A NoBrokersAvailableError N/A N/A N/A N/A log warning No brokers available N/A to_retry N/A N/A mset exc N/A N/A topic partitions N/A iteritems N/A req msets N/A N/A p_id mset N/A iteritems partitions N/A N/A N/A log debug N/A N/A Successfully sent {}/{} messages to broker {} format N/A req delivered N/A len message_batch N/A N/A owned_broker broker id N/A to_retry N/A N/A N/A N/A N/A N/A self _cluster handler sleep N/A N/A self _retry_backoff_ms 1000 N/A N/A N/A owned_broker increment_messages_pending N/A -1 N/A len to_retry N/A N/A mset exc to_retry N/A N/A non_recoverable N/A N/A type exc N/A InvalidMessageSize MessageSizeTooLarge N/A msg N/A mset messages N/A N/A N/A non_recoverable N/A N/A msg produce_attempt N/A self _max_retries N/A N/A N/A N/A N/A self _delivery_reports put msg exc N/A N/A N/A log error N/A Message not delivered!! %r exc N/A N/A N/A msg produce_attempt 1 N/A N/A N/A self _produce msg N/A _wait_all N/A N/A self N/A N/A N/A Block until all pending messages are sent

        "Pending" messages are those that have been used in calls to `produce`
        and have not yet been dequeued and sent to the broker
         N/A N/A N/A log info Blocking until all messages are sent N/A N/A any N/A N/A N/A q message_is_pending N/A q N/A itervalues N/A self _owned_brokers N/A N/A N/A N/A N/A N/A self _cluster handler sleep 0.3 N/A N/A N/A self _raise_worker_exceptions N/A N/A OwnedBroker N/A object N/A N/A An abstraction over a broker connected to by the producer

    An OwnedBroker object contains thread-synchronization primitives
    and message queue corresponding to a single broker for this producer.

    :ivar lock: The lock used to control access to shared resources for this
        queue
    :ivar flush_ready: A condition variable that indicates that the queue is
        ready to be flushed via requests to the broker
    :ivar slot_available: A condition variable that indicates that there is
        at least one position free in the queue for a new message
    :ivar queue: The message queue for this broker. Contains messages that have
        been supplied as arguments to `produce()` waiting to be sent to the
        broker.
    :type queue: collections.deque
    :ivar messages_pending: A counter indicating how many messages have been
        enqueued for this broker and not yet sent in a request.
    :type messages_pending: int
    :ivar producer: The producer to which this OwnedBroker instance belongs
    :type producer: :class:`pykafka.producer.AsyncProducer`
    :param auto_start: Whether the OwnedBroker should start flushing all
        waiting messages and send to kafka after __init__ is complete. If
        false, communication can be started with `start()`.
    :type auto_start: bool
     __init__ N/A N/A self producer broker auto_start N/A True N/A N/A N/A self producer N/A N/A weakref proxy producer N/A N/A self broker broker N/A N/A self lock N/A N/A N/A N/A N/A self producer _cluster handler RLock N/A N/A self flush_ready N/A N/A N/A N/A N/A self producer _cluster handler Event N/A N/A self slot_available N/A N/A N/A N/A N/A self producer _cluster handler Event N/A N/A self queue N/A deque N/A N/A self messages_pending 0 N/A N/A self running True N/A N/A self _auto_start auto_start N/A N/A self _auto_start N/A N/A N/A N/A self start N/A start N/A N/A self N/A N/A queue_reader N/A N/A N/A N/A N/A N/A self running N/A N/A N/A N/A batch N/A N/A self flush N/A N/A self producer _linger_ms N/A N/A self producer _max_request_size N/A batch N/A N/A N/A N/A N/A self producer _send_request batch self N/A N/A N/A Exception N/A N/A N/A N/A self producer _worker_exception N/A N/A sys exc_info N/A N/A N/A N/A log info Worker exited for broker %s:%s N/A N/A self broker host N/A N/A self broker port N/A N/A N/A N/A log info Starting new produce worker for broker %s N/A N/A self broker id N/A name N/A N/A pykafka.OwnedBroker.queue_reader for broker {} format N/A N/A self broker id N/A N/A self _queue_reader_worker N/A N/A N/A N/A N/A self producer _cluster handler spawn queue_reader name name N/A stop N/A N/A self N/A N/A N/A N/A self running False N/A increment_messages_pending N/A N/A self amnt N/A N/A N/A N/A self lock N/A N/A N/A self messages_pending amnt N/A N/A self messages_pending N/A max 0 N/A self messages_pending N/A message_is_pending N/A N/A self N/A N/A N/A 
        Indicates whether there are currently any messages that have been
            `produce()`d and not yet sent to the broker
         N/A N/A N/A self messages_pending 0 N/A enqueue N/A N/A self message N/A N/A N/A Push message onto the queue

        :param message: The message to push onto the queue
        :type message: `pykafka.protocol.Message`
         N/A N/A N/A self _wait_for_slot_available N/A N/A self lock N/A N/A N/A N/A N/A self queue appendleft message N/A N/A N/A self increment_messages_pending 1 N/A N/A N/A len N/A self queue N/A N/A self producer _min_queued_messages N/A N/A N/A N/A N/A N/A self flush_ready is_set N/A N/A N/A N/A N/A self flush_ready set N/A flush N/A N/A self linger_ms max_request_size release_pending N/A False N/A N/A Pop messages from the end of the queue

        :param linger_ms: How long (in milliseconds) to wait for the queue
            to contain messages before flushing
        :type linger_ms: int
        :param max_request_size: The max size should each batch of messages
            should be in bytes
        :type max_request_size: int
        :param release_pending: Whether to decrement the messages_pending
            counter when the queue is flushed. True means that the messages
            popped from the queue will be discarded unless re-enqueued
            by the caller.
        :type release_pending: bool
         N/A N/A N/A self _wait_for_flush_ready linger_ms N/A N/A self lock N/A N/A batch N/A N/A batch_size_in_bytes 0 N/A N/A N/A len N/A self queue 0 N/A N/A peeked_message N/A N/A self queue N/A -1 N/A N/A peeked_message N/A N/A peeked_message value None N/A N/A N/A N/A len peeked_message max_request_size N/A N/A exc N/A MessageSizeTooLarge N/A N/A Message size larger than max_request_size: {} format max_request_size N/A N/A N/A log warning exc N/A message N/A N/A N/A self queue pop N/A N/A N/A peeked_message delivery_report_q None N/A N/A N/A N/A N/A peeked_message delivery_report_q put N/A message exc N/A N/A N/A self increment_messages_pending -1 N/A N/A N/A N/A N/A batch_size_in_bytes N/A len peeked_message max_request_size N/A N/A N/A N/A log debug max_request_size reached. producing batch N/A N/A N/A N/A self flush_ready set N/A N/A message N/A N/A N/A self queue pop N/A batch_size_in_bytes N/A len message N/A N/A N/A batch append message N/A release_pending N/A N/A N/A N/A self increment_messages_pending N/A -1 N/A len batch N/A N/A N/A N/A N/A self slot_available is_set N/A N/A N/A N/A N/A self slot_available set N/A batch N/A _wait_for_flush_ready N/A N/A self linger_ms N/A N/A N/A Block until the queue is ready to be flushed

        If the queue does not contain at least one message after blocking for
        `linger_ms` milliseconds, return.

        :param linger_ms: How long (in milliseconds) to wait for the queue
            to contain messages before returning
        :type linger_ms: int
         N/A N/A N/A len N/A self queue N/A N/A self producer _min_queued_messages N/A N/A N/A self lock N/A N/A N/A N/A len N/A self queue N/A N/A self producer _min_queued_messages N/A N/A N/A N/A N/A self flush_ready clear N/A N/A linger_ms 0 N/A N/A N/A N/A N/A self flush_ready wait N/A linger_ms 1000 N/A _wait_for_slot_available N/A N/A self N/A N/A N/A Block until the queue has at least one slot not containing a message N/A N/A N/A len N/A self queue N/A N/A self producer _max_queued_messages N/A N/A N/A self lock N/A N/A N/A N/A len N/A self queue N/A N/A self producer _max_queued_messages N/A N/A N/A N/A N/A self slot_available clear N/A N/A N/A self producer _block_on_queue_full N/A N/A N/A N/A N/A self slot_available wait N/A N/A N/A ProducerQueueFullError Queue full for broker %d N/A N/A self broker id N/A N/A _DeliveryReportQueue N/A N/A threading local N/A N/A Helper that instantiates a new report queue on every calling thread __init__ N/A N/A self handler N/A N/A N/A N/A self queue N/A N/A handler Queue N/A put N/A N/A msg exc N/A None N/A N/A N/A N/A N/A msg delivery_report_q put N/A msg exc N/A staticmethod N/A _DeliveryReportNone N/A object N/A N/A Stand-in for when _DeliveryReportQueue has been disabled __init__ N/A N/A self N/A N/A N/A N/A self queue None N/A put N/A N/A msg exc N/A None N/A N/A N/A staticmethod N/A