Module ImportFrom alias ImportFrom alias Import alias identifier Import alias Assign NameStore Call AttributeLoad NameLoad attr NameLoad ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Num decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str AugAssignAdd AttributeStore NameLoad attr NameLoad AugAssignAdd AttributeStore NameLoad attr NameLoad AugAssignAdd AttributeStore NameLoad attr BinOpMult Num BinOpSub NameLoad AttributeLoad NameLoad attr AugAssignAdd AttributeStore NameLoad attr BinOpMult Num BinOpSub NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareEq AttributeLoad NameLoad attr Num body Return NameLoad orelse Return BinOpDiv Call NameLoad AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If CompareEq AttributeLoad NameLoad attr Num body Return NameLoad orelse Return BinOpDiv Call NameLoad AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore Num Assign NameStore Num Assign NameStore Num Assign NameStore Num FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Dict Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr Dict Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body While NameLoad body If CompareIsNot AttributeLoad NameLoad attr NameLoad body Return Assign NameStore Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr BinOpDiv AttributeLoad NameLoad attr Num AttributeLoad NameLoad attr NameLoad TryFinally body Expr Yield NameLoad finalbody Assign AttributeStore NameLoad attr NameLoad TryExcept body Assign NameStore Yield Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr handlers ExceptHandler body Expr Call AttributeLoad NameLoad attr Str If CompareIs NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str Return decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Assign NameStore Yield Call AttributeLoad NameLoad attr Str keyword NameLoad keyword TupleLoad Str For NameStore NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr SubscriptLoad NameLoad Index Str If UnaryOpNot AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareIs NameLoad NameLoad body Assign NameStore Call NameLoad NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr If CompareIsNot NameLoad NameLoad body Assign NameStore Call NameLoad BinOpMult NameLoad NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore Yield Call AttributeLoad NameLoad attr Str keyword Dict Str NameLoad keyword Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Yield Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore BinOpSub NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr ListLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Return Call AttributeLoad NameLoad attr Str Dict Str Str NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr ListComp NameLoad comprehension NameStore AttributeLoad NameLoad attr CompareNotEq NameLoad NameLoad Return Call AttributeLoad NameLoad attr Str keyword Dict Str NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Str keyword Dict Str NameLoad keyword Dict Str NameLoad decorator_list decorator_list
N/A _base SQLBaseStore twisted.internet defer N/A ujson json N/A logging N/A logger N/A N/A logging getLogger __name__ BackgroundUpdatePerformance N/A object N/A N/A Tracks the how long a background update is taking to update its items __init__ N/A N/A self name N/A N/A N/A N/A self name name N/A N/A self total_item_count 0 N/A N/A self total_duration_ms 0 N/A N/A self avg_item_count 0 N/A N/A self avg_duration_ms 0 N/A update N/A N/A self item_count duration_ms N/A N/A N/A Update the stats after doing an update N/A N/A self total_item_count item_count N/A N/A self total_duration_ms duration_ms N/A N/A self avg_item_count N/A 0.1 N/A item_count N/A self avg_item_count N/A N/A self avg_duration_ms N/A 0.1 N/A duration_ms N/A self avg_duration_ms N/A average_items_per_ms N/A N/A self N/A N/A N/A An estimate of how long it takes to do a single update.
        Returns:
            A duration in ms as a float
         N/A N/A N/A self total_item_count 0 N/A N/A None N/A N/A N/A N/A float N/A self avg_item_count N/A float N/A self avg_duration_ms N/A total_items_per_ms N/A N/A self N/A N/A N/A An estimate of how long it takes to do a single update.
        Returns:
            A duration in ms as a float
         N/A N/A N/A self total_item_count 0 N/A N/A None N/A N/A N/A N/A float N/A self total_item_count N/A float N/A self total_duration_ms N/A N/A BackgroundUpdateStore N/A SQLBaseStore N/A N/A  Background updates are updates to the database that run in the
    background. Each update processes a batch of data at once. We attempt to
    limit the impact of each update by monitoring how long each batch takes to
    process and autotuning the batch size.
     N/A MINIMUM_BACKGROUND_BATCH_SIZE 100 N/A DEFAULT_BACKGROUND_BATCH_SIZE 100 N/A BACKGROUND_UPDATE_INTERVAL_MS 1000 N/A BACKGROUND_UPDATE_DURATION_MS 100 __init__ N/A N/A self hs N/A N/A N/A N/A N/A N/A super BackgroundUpdateStore self __init__ hs N/A N/A self _background_update_performance N/A N/A N/A self _background_update_queue N/A N/A N/A self _background_update_handlers N/A N/A N/A self _background_update_timer None N/A start_doing_background_updates N/A N/A self N/A N/A N/A True N/A N/A N/A N/A self _background_update_timer None N/A N/A N/A sleep N/A N/A defer Deferred N/A N/A self _background_update_timer N/A N/A N/A self _clock call_later N/A N/A self BACKGROUND_UPDATE_INTERVAL_MS 1000.0 N/A sleep callback None N/A N/A N/A N/A sleep N/A N/A N/A self _background_update_timer None N/A N/A N/A result N/A N/A N/A self do_background_update N/A self BACKGROUND_UPDATE_DURATION_MS N/A N/A N/A N/A N/A N/A logger exception Error doing update N/A N/A result None N/A N/A N/A N/A logger info No more background updates to do. Unscheduling background update task. N/A N/A N/A defer inlineCallbacks do_background_update N/A N/A self desired_duration_ms N/A N/A N/A Does some amount of work on a background update
        Args:
            desired_duration_ms(float): How long we want to spend
                updating.
        Returns:
            A deferred that completes once some amount of work is done.
            The deferred will have a value of None if there is currently
            no more work to do.
         N/A N/A N/A self _background_update_queue N/A N/A updates N/A N/A N/A self _simple_select_list background_updates keyvalues None retcols N/A update_name N/A update updates N/A N/A N/A N/A N/A self _background_update_queue append N/A update N/A update_name N/A N/A N/A self _background_update_queue N/A N/A N/A N/A defer returnValue None N/A update_name N/A N/A N/A self _background_update_queue pop 0 N/A N/A N/A N/A self _background_update_queue append update_name N/A update_handler N/A N/A self _background_update_handlers N/A update_name N/A performance N/A N/A N/A self _background_update_performance get update_name N/A N/A performance None N/A N/A performance N/A BackgroundUpdatePerformance update_name N/A N/A N/A self _background_update_performance N/A update_name performance N/A items_per_ms N/A N/A performance average_items_per_ms N/A N/A items_per_ms None N/A N/A batch_size N/A int N/A desired_duration_ms items_per_ms N/A batch_size N/A max batch_size N/A self MINIMUM_BACKGROUND_BATCH_SIZE N/A N/A batch_size N/A self DEFAULT_BACKGROUND_BATCH_SIZE N/A progress_json N/A N/A N/A self _simple_select_one_onecol background_updates keyvalues N/A update_name update_name retcol progress_json N/A progress N/A N/A json loads progress_json N/A time_start N/A N/A N/A self _clock time_msec N/A items_updated N/A N/A update_handler progress batch_size N/A time_stop N/A N/A N/A self _clock time_msec N/A duration_ms N/A time_stop time_start N/A N/A N/A logger info Updating %r. Updated %r items in %rms. (total_rate=%r/ms, current_rate=%r/ms, total_updated=%r) update_name items_updated duration_ms N/A N/A performance total_items_per_ms N/A N/A performance average_items_per_ms N/A performance total_item_count N/A N/A N/A performance update items_updated duration_ms N/A N/A N/A defer returnValue N/A len N/A self _background_update_performance N/A N/A defer inlineCallbacks register_background_update_handler N/A N/A self update_name update_handler N/A N/A N/A Register a handler for doing a background update.

        The handler should take two arguments:

        * A dict of the current progress
        * An integer count of the number of items to update in this batch.

        The handler should return a deferred integer count of items updated.
        The hander is responsible for updating the progress of the update.

        Args:
            update_name(str): The name of the update that this code handles.
            update_handler(function): The function that does the update.
         N/A N/A N/A self _background_update_handlers N/A update_name update_handler N/A start_background_update N/A N/A self update_name progress N/A N/A N/A Starts a background update running.

        Args:
            update_name: The update to set running.
            progress: The initial state of the progress of the update.

        Returns:
            A deferred that completes once the task has been added to the
            queue.
         N/A N/A self _background_update_queue N/A N/A progress_json N/A N/A json dumps progress N/A N/A N/A self _simple_insert background_updates N/A update_name progress_json update_name progress_json N/A _end_background_update N/A N/A self update_name N/A N/A N/A Removes a completed background update task from the queue.

        Args:
            update_name(str): The name of the completed task to remove
        Returns:
            A deferred that completes once the task is removed.
         N/A N/A self _background_update_queue N/A name N/A name N/A self _background_update_queue N/A name update_name N/A N/A N/A self _simple_delete_one background_updates keyvalues N/A update_name update_name N/A _background_update_progress_txn N/A N/A self txn update_name progress N/A N/A N/A Update the progress of a background update

        Args:
            txn(cursor): The transaction.
            update_name(str): The name of the background update task
            progress(dict): The progress of the update.
         N/A progress_json N/A N/A json dumps progress N/A N/A N/A self _simple_update_one_txn txn background_updates keyvalues N/A update_name update_name updatevalues N/A progress_json progress_json N/A N/A