Module Expr Str Import alias Import alias Import alias Import alias TryExcept body Import alias ImportFrom alias handlers ExceptHandler type NameLoad body ImportFrom alias ImportFrom alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias identifier ImportFrom alias ImportFrom alias ImportFrom alias identifier Assign NameStore ListLoad AttributeLoad NameLoad attr Assign NameStore ListLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Num FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad For NameStore Call AttributeLoad NameLoad attr NameLoad body If CompareEq NameLoad AttributeLoad NameLoad attr body Continue Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareIn NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str NameLoad Continue If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body AugAssignAdd NameStore Call NameLoad NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr NameLoad NameLoad orelse If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad If CompareNotEq NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad NameLoad AugAssignAdd NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad orelse Expr Call AttributeLoad NameLoad attr Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return Num If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return Num Return Call NameLoad NameLoad NameLoad NameLoad Str decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad NameLoad AugAssignAdd AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call NameLoad NameLoad Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If AttributeLoad NameLoad attr body Return Str If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If NameLoad body Return NameLoad Assign AttributeStore NameLoad attr NameLoad AugAssignSub NameStore AttributeLoad AttributeLoad NameLoad attr attr While BoolOpAnd UnaryOpNot AttributeLoad NameLoad attr UnaryOpNot AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If UnaryOpNot NameLoad body Assign AttributeStore NameLoad attr NameLoad AugAssignAdd AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Break Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If NameLoad body AugAssignAdd AttributeStore NameLoad attr NameLoad If CompareGt Call NameLoad AttributeLoad NameLoad attr NameLoad body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Slice NameLoad Assign AttributeStore NameLoad attr SubscriptLoad AttributeLoad NameLoad attr Slice NameLoad orelse Assign NameStore AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Str If UnaryOpNot NameLoad body Assign AttributeStore NameLoad attr NameLoad Return BinOpAdd Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad NameLoad NameLoad NameLoad NameLoad TryFinally body TryExcept body Assign NameStore Dict Str Num Assign NameStore Call AttributeLoad NameLoad attr Str Str keyword NameLoad keyword NameLoad If CompareNotEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Return NameLoad handlers ExceptHandler type AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr Str NameLoad Return NameLoad finalbody Expr Call AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad NameLoad TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type AttributeLoad NameLoad attr body Pass Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Return Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore AttributeLoad NameLoad attr For NameStore Call AttributeLoad NameLoad attr NameLoad body If CompareEq NameLoad AttributeLoad NameLoad attr body Continue Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareIn NameLoad NameLoad body Continue If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore Call NameLoad NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad If UnaryOpNot NameLoad body Return NameLoad Continue If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Continue Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad If CompareNotEq NameLoad NameLoad body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore AttributeLoad NameLoad attr Assign NameStore TupleLoad NameLoad NameLoad TryExcept body Assign NameStore SubscriptLoad NameLoad Index NameLoad handlers ExceptHandler type NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad NameLoad If UnaryOpNot Call NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad body Return NameLoad Expr Call AttributeLoad NameLoad attr NameLoad orelse Expr Call AttributeLoad NameLoad attr Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Dict Assign NameStore ListLoad Assign NameStore Call NameLoad NameLoad NameLoad NameLoad Str NameLoad NameLoad If UnaryOpNot NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad NameLoad With Call NameLoad NameLoad Str NameStore body For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr BinOpAdd Call AttributeLoad NameLoad attr Str Str Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Index Num Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str NameLoad With Call NameLoad NameLoad Str NameStore body Assign NameStore Call AttributeLoad NameLoad attr While NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Index Num If UnaryOpNot NameLoad body Break Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore BinOpAdd AttributeLoad NameLoad attr NameLoad While NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad If NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr orelse Assign NameStore Str Assign NameStore Call AttributeLoad NameLoad attr If NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad If CompareNotEq Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Str NameLoad AttributeLoad NameLoad attr Return NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass For NameStore Call AttributeLoad NameLoad attr NameLoad body If CompareIn NameLoad NameLoad body Continue Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad orelse If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Expr Call NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad NameLoad Assign NameStore ListLoad If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore ListComp Call AttributeLoad Call AttributeLoad NameLoad attr Str attr Str comprehension NameStore Call NameLoad NameLoad Str For NameStore NameLoad body If UnaryOpNot Call AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr Str NameLoad Continue If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Expr Call AttributeLoad NameLoad attr Str NameLoad Continue If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Expr Call AttributeLoad NameLoad attr Str NameLoad Continue Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Str NameLoad TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass TryExcept body Expr Call NameLoad NameLoad NameLoad Str handlers ExceptHandler type NameLoad body Return NameLoad If UnaryOpNot NameLoad body If Call AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr With Call NameLoad NameLoad Str NameStore body Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad With Call NameLoad NameLoad Str NameStore body Return Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad If UnaryOpNot Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Return NameLoad Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr TryExcept body If Call AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass decorator_list
N/A N/A Utilities for load rebalancing. N/A os N/A shutil N/A StringIO N/A zlib N/A N/A N/A urllib3 urllib3 connectionpool N/A N/A N/A ImportError N/A requests.packages urllib3 requests.packages.urllib3 connectionpool N/A logging grr.lib data_store grr.lib utils grr.lib.data_stores common grr.lib.rdfvalues data_server rdf_data_server grr.server.data_server constants grr.server.data_server store grr.server.data_server utils sutils N/A COPY_EXCEPTIONS N/A N/A store BASE_MAP_SUBJECT N/A MOVE_EXCEPTIONS N/A N/A constants TRANSACTION_FILENAME N/A constants REMOVE_FILENAME N/A COMPRESSION_LEVEL 3 _RecComputeRebalanceSize N/A N/A mapping server_id dspath subpath N/A N/A N/A Recursively compute the size of files that need to be moved. N/A total 0 N/A fulldir N/A N/A utils JoinPath dspath subpath N/A comp N/A N/A os listdir fulldir N/A N/A N/A comp N/A constants REBALANCE_DIRECTORY N/A N/A N/A path N/A N/A utils JoinPath fulldir comp N/A N/A name unused_extension N/A N/A N/A os path splitext comp N/A N/A name COPY_EXCEPTIONS N/A N/A N/A N/A logging info Skip %s comp N/A N/A N/A N/A N/A os path isdir path N/A N/A total N/A _RecComputeRebalanceSize mapping server_id dspath N/A N/A utils JoinPath subpath comp N/A N/A N/A N/A N/A os path isfile path N/A N/A key N/A N/A common MakeDestinationKey subpath name N/A where N/A N/A sutils MapKeyToServer mapping key N/A N/A where server_id N/A N/A N/A N/A logging info Need to move %s from %d to %d path server_id where N/A total N/A N/A N/A os path getsize path N/A N/A N/A N/A logging info File %s stays here path N/A total N/A ComputeRebalanceSize N/A N/A mapping server_id N/A N/A N/A Compute size of files that need to be moved. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A 0 N/A N/A N/A N/A N/A os path isdir loc N/A N/A 0 N/A N/A _RecComputeRebalanceSize mapping server_id loc N/A FileCopyWrapper N/A object N/A N/A Wraps the database file for post'ing it to the server. __init__ N/A N/A self rebalance directory filename fullpath N/A N/A N/A filesize N/A N/A N/A os path getsize fullpath N/A filecopy N/A N/A rdf_data_server DataServerFileCopy rebalance_id N/A rebalance id directory directory filename filename size filesize N/A filecopy_str N/A N/A filecopy SerializeToString N/A N/A self header N/A N/A N/A sutils SIZE_PACKER pack N/A len filecopy_str N/A N/A self header filecopy_str N/A N/A self header N/A N/A StringIO StringIO N/A self header N/A N/A self fp N/A open fullpath rb N/A N/A self compressor N/A N/A zlib compressobj COMPRESSION_LEVEL N/A N/A self buffered N/A N/A self end_of_file False N/A N/A self end_of_stream False N/A N/A self read_header True N/A read N/A N/A self blocksize N/A N/A N/A Returns data back to the HTTP post request. N/A N/A self end_of_stream N/A N/A N/A N/A self read_header N/A N/A ret N/A N/A N/A self header read blocksize N/A ret N/A N/A ret N/A N/A self read_header False N/A blocksize N/A N/A sutils SIZE_PACKER size N/A N/A N/A N/A self buffered N/A N/A self end_of_file N/A N/A raw N/A N/A N/A self fp read blocksize N/A N/A raw N/A N/A N/A self end_of_file True N/A N/A self buffered N/A N/A N/A self compressor flush N/A N/A compressed N/A N/A N/A self compressor compress raw N/A compressed N/A N/A N/A self buffered compressed N/A N/A N/A len N/A self buffered blocksize N/A N/A ret N/A N/A self buffered N/A blocksize N/A N/A self buffered N/A N/A self buffered N/A blocksize N/A N/A ret N/A self buffered N/A N/A self buffered N/A N/A ret N/A N/A N/A self end_of_stream True N/A N/A N/A N/A N/A sutils SIZE_PACKER pack N/A len ret ret N/A close N/A N/A self N/A N/A N/A N/A N/A N/A self fp close N/A N/A N/A N/A self header close N/A N/A _SendFileToServer N/A N/A pool fullpath subpath basename rebalance N/A N/A N/A Sends a specific data store file to the server. N/A fp N/A FileCopyWrapper rebalance subpath basename fullpath N/A N/A N/A N/A N/A headers N/A Content-Length 0 N/A res N/A N/A pool urlopen POST /rebalance/copy-file headers headers body fp N/A N/A N/A res status N/A constants RESPONSE_OK N/A N/A False N/A N/A N/A N/A N/A urllib3 exceptions MaxRetryError N/A N/A N/A N/A logging warning Failed to send file %s fullpath N/A False N/A N/A N/A N/A fp close N/A True N/A _GetTransactionDirectory N/A N/A database_dir rebalance_id N/A N/A N/A dirname N/A N/A common ConvertStringToFilename rebalance_id N/A tempdir N/A N/A utils JoinPath database_dir N/A constants REBALANCE_DIRECTORY dirname N/A tempdir N/A _CreateDirectory N/A N/A database_dir rebalance_id N/A N/A N/A tempdir N/A _GetTransactionDirectory database_dir rebalance_id N/A N/A N/A N/A N/A os makedirs tempdir N/A N/A N/A N/A os error N/A N/A N/A tempdir N/A _FileWithRemoveList N/A N/A database_dir rebalance N/A N/A N/A tempdir N/A _CreateDirectory database_dir N/A rebalance id N/A N/A N/A utils JoinPath tempdir N/A constants REMOVE_FILENAME N/A _RecCopyFiles N/A N/A rebalance server_id dspath subpath pool_cache removed_list N/A N/A N/A Recursively send files for moving to the required data server. N/A fulldir N/A N/A utils JoinPath dspath subpath N/A mapping N/A rebalance mapping N/A comp N/A N/A os listdir fulldir N/A N/A N/A comp N/A constants REBALANCE_DIRECTORY N/A N/A N/A path N/A N/A utils JoinPath fulldir comp N/A N/A name unused_extension N/A N/A N/A os path splitext comp N/A N/A name COPY_EXCEPTIONS N/A N/A N/A N/A N/A N/A os path isdir path N/A N/A result N/A _RecCopyFiles rebalance server_id dspath N/A N/A utils JoinPath subpath comp pool_cache removed_list N/A N/A result N/A N/A False N/A N/A N/A N/A N/A N/A os path isfile path N/A N/A N/A key N/A N/A common MakeDestinationKey subpath name N/A where N/A N/A sutils MapKeyToServer mapping key N/A N/A where server_id N/A N/A server N/A N/A mapping servers N/A where N/A addr N/A server address N/A port N/A server port N/A key N/A addr port N/A N/A N/A pool N/A pool_cache N/A key N/A N/A N/A KeyError N/A N/A pool N/A N/A connectionpool HTTPConnectionPool addr port port N/A N/A pool_cache N/A key pool N/A N/A N/A logging info Need to move %s from %d to %d key server_id where N/A N/A N/A _SendFileToServer pool path subpath comp rebalance N/A N/A False N/A N/A N/A removed_list append path N/A N/A N/A N/A logging info File %s stays here path N/A True N/A CopyFiles N/A N/A rebalance server_id N/A N/A N/A Copies data store files to the corresponding data servers. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A True N/A N/A N/A N/A N/A os path isdir loc N/A N/A True N/A pool_cache N/A N/A removed_list N/A N/A ok N/A _RecCopyFiles rebalance server_id loc pool_cache removed_list N/A N/A ok N/A N/A False N/A remove_file N/A _FileWithRemoveList loc rebalance N/A N/A open remove_file w fp N/A N/A f removed_list N/A N/A N/A N/A fp write N/A N/A N/A f encode utf8 N/A True N/A SaveTemporaryFile N/A N/A fp N/A N/A N/A Store incoming database file in a temporary directory. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A False N/A N/A N/A N/A N/A os path isdir loc N/A N/A False N/A filecopy_len_str N/A N/A fp read N/A N/A sutils SIZE_PACKER size N/A filecopy_len N/A N/A N/A N/A sutils SIZE_PACKER unpack filecopy_len_str N/A 0 N/A filecopy N/A N/A rdf_data_server DataServerFileCopy N/A N/A fp read filecopy_len N/A rebdir N/A _CreateDirectory loc N/A filecopy rebalance_id N/A filedir N/A N/A utils JoinPath rebdir N/A filecopy directory N/A N/A N/A N/A N/A os makedirs filedir N/A N/A N/A OSError N/A N/A N/A filepath N/A N/A utils JoinPath filedir N/A filecopy filename N/A N/A N/A logging info Writing to file %s filepath N/A N/A open filepath wb wp N/A N/A decompressor N/A N/A zlib decompressobj N/A True N/A N/A block_len_str N/A N/A fp read N/A N/A sutils SIZE_PACKER size N/A block_len N/A N/A N/A N/A sutils SIZE_PACKER unpack block_len_str N/A 0 N/A N/A block_len N/A N/A N/A block N/A N/A fp read block_len N/A to_decompress N/A N/A decompressor unconsumed_tail block N/A to_decompress N/A N/A decompressed N/A N/A decompressor decompress to_decompress N/A decompressed N/A N/A N/A N/A wp write decompressed N/A to_decompress N/A decompressor unconsumed_tail N/A N/A to_decompress N/A remaining N/A N/A decompressor flush N/A remaining N/A N/A N/A N/A wp write remaining N/A N/A N/A N/A N/A os path getsize filepath N/A filecopy size N/A N/A N/A N/A logging error Size of file %s is not %d filepath N/A filecopy size N/A False N/A True N/A _RecMoveFiles N/A N/A tempdir dspath subpath N/A N/A N/A fulltempdir N/A N/A utils JoinPath tempdir subpath N/A fulldsdir N/A N/A utils JoinPath dspath subpath N/A N/A N/A N/A N/A os makedirs fulldsdir N/A N/A N/A OSError N/A N/A N/A fname N/A N/A os listdir fulltempdir N/A N/A N/A fname MOVE_EXCEPTIONS N/A N/A N/A temppath N/A N/A utils JoinPath fulltempdir fname N/A N/A N/A N/A os path isfile temppath N/A N/A newpath N/A N/A utils JoinPath fulldsdir fname N/A N/A N/A logging info Moving file %s to %s temppath newpath N/A N/A N/A os rename temppath newpath N/A N/A N/A N/A N/A os path isdir temppath N/A N/A N/A _RecMoveFiles tempdir dspath N/A N/A utils JoinPath subpath fname N/A MoveFiles N/A N/A rebalance is_master N/A N/A N/A Commit the received files into the database. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A False N/A N/A N/A N/A N/A os path isdir loc N/A N/A False N/A tempdir N/A _CreateDirectory loc N/A rebalance id N/A remove_file N/A _FileWithRemoveList loc rebalance N/A to_remove N/A N/A N/A N/A N/A os path exists remove_file N/A N/A to_remove N/A N/A N/A N/A N/A line decode utf8 rstrip N/A line N/A open remove_file r N/A fname to_remove N/A N/A N/A N/A N/A fname startswith loc N/A N/A N/A N/A logging warning Wrong file to remove: %s fname N/A N/A N/A N/A N/A N/A os path exists fname N/A N/A N/A N/A logging warning File does not exist: %s fname N/A N/A N/A N/A N/A N/A os path isfile fname N/A N/A N/A N/A logging warning Not a file: %s fname N/A N/A N/A N/A os unlink fname N/A N/A N/A logging info Removing file %s fname N/A N/A N/A N/A N/A os unlink remove_file N/A N/A N/A OSError N/A N/A N/A N/A N/A N/A _RecMoveFiles tempdir loc N/A N/A N/A OSError N/A N/A False N/A N/A is_master N/A N/A N/A N/A tempdir startswith loc N/A N/A N/A N/A shutil rmtree tempdir N/A True N/A SaveCommitInformation N/A N/A rebalance N/A N/A N/A Save rebalance object to file. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A False N/A N/A N/A N/A N/A os path isdir loc N/A N/A False N/A tempdir N/A _CreateDirectory loc N/A rebalance id N/A tempfile N/A N/A utils JoinPath tempdir N/A constants TRANSACTION_FILENAME N/A N/A open tempfile wb fp N/A N/A N/A N/A fp write N/A N/A rebalance SerializeToString N/A True N/A DeleteCommitInformation N/A N/A rebalance N/A N/A N/A Remove file with rebalance information. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A False N/A N/A N/A N/A N/A os path isdir loc N/A N/A False N/A tempdir N/A _GetTransactionDirectory loc N/A rebalance id N/A tempfile N/A N/A utils JoinPath tempdir N/A constants TRANSACTION_FILENAME N/A N/A N/A N/A N/A os unlink tempfile N/A N/A N/A OSError N/A N/A N/A True N/A GetCommitInformation N/A N/A transid N/A N/A N/A Read transaction information from stored file. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A False N/A N/A N/A N/A N/A os path isdir loc N/A N/A False N/A tempdir N/A _GetTransactionDirectory loc transid N/A tempfile N/A N/A utils JoinPath tempdir N/A constants TRANSACTION_FILENAME N/A N/A N/A N/A N/A os path exists tempfile N/A N/A None N/A N/A N/A N/A N/A os path isfile tempfile N/A N/A None N/A N/A open tempfile rb fp N/A N/A N/A N/A rdf_data_server DataServerRebalance N/A N/A fp read N/A RemoveDirectory N/A N/A rebalance N/A N/A N/A Remove temporary directory of the given rebalance object. N/A loc N/A N/A N/A data_store DB Location N/A N/A N/A N/A N/A os path exists loc N/A N/A False N/A N/A N/A N/A N/A os path isdir loc N/A N/A False N/A tempdir N/A _GetTransactionDirectory loc N/A rebalance id N/A N/A N/A N/A N/A tempdir startswith loc N/A N/A N/A N/A shutil rmtree tempdir N/A N/A N/A OSError N/A N/A N/A