Module Expr Str Assign NameStore Str Import alias Import alias Import alias Import alias Import alias Import alias ImportFrom alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias FunctionDef arguments args defaults body Expr Str Return Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign TupleStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr Str If BoolOpOr UnaryOpNot NameLoad UnaryOpNot Call NameLoad NameLoad body Return NameLoad orelse Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str If Call AttributeLoad NameLoad attr Str body Return SubscriptLoad NameLoad Slice Num orelse If Call AttributeLoad NameLoad attr Str body Return IfExp CompareEq NameLoad Str Str NameLoad orelse If CompareEq NameLoad Str body Return NameLoad orelse Return BinOpAdd NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Yield Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword NameLoad keyword Num Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad NameLoad NameLoad Expr Call NameLoad BoolOpAnd CompareGt Call NameLoad NameLoad Num CompareEq SubscriptLoad NameLoad Index Num NameLoad decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad Num body Expr Str Assign NameStore Call NameLoad Num NameLoad Assign NameStore ListLoad Assign NameStore NameLoad While UnaryOpNot NameLoad body Assign NameStore Yield Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad If CompareLt Call NameLoad NameLoad NameLoad body Break Assign NameStore SubscriptLoad NameLoad Index Num Expr Call NameLoad NameLoad decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad body Expr Str Assign NameStore Call NameLoad Assign NameStore ListLoad While NameLoad body Assign TupleStore NameStore NameStore Yield Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad keyword NameLoad keyword NameLoad keyword Num Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad If CompareLt BinOpAdd Call NameLoad NameLoad Call NameLoad NameLoad Num body Break Assign NameStore Str If CompareGt Call NameLoad NameLoad Num body Assign NameStore Call NameLoad NameLoad SubscriptLoad NameLoad Index Num If CompareGt Call NameLoad NameLoad Num body Assign NameStore Call NameLoad NameLoad SubscriptLoad NameLoad Index Num Expr Call NameLoad TupleLoad Call NameLoad Call NameLoad NameLoad NameLoad decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Yield Call AttributeLoad NameLoad attr NameLoad NameLoad Str keyword Call NameLoad NameLoad Expr Call NameLoad NameLoad decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Yield Call AttributeLoad NameLoad attr NameLoad NameLoad keyword Call NameLoad NameLoad Expr Call NameLoad NameLoad decorator_list AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Yield Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad If BoolOpAnd NameLoad Call AttributeLoad NameLoad attr Str body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr orelse Assign NameStore NameLoad Expr Call NameLoad NameLoad decorator_list AttributeLoad NameLoad attr
N/A N/A Object store high level utilities.

 N/A __author__ marc@emailscrubbed.com (Marc Berhault) N/A cStringIO N/A gzip N/A logging N/A subprocess N/A sys N/A traceback functools partial tornado gen options viewfinder.backend.base main viewfinder.backend.db db_client viewfinder.backend.storage.object_store ObjectStore ListBuckets N/A N/A N/A N/A N/A Return the list of bucket names initialized in the object store. N/A N/A N/A ObjectStore ListInstances N/A IsBucket N/A N/A name N/A N/A N/A Returns true if there is a bucket under this name in the object store. N/A N/A N/A ObjectStore HasInstance name N/A ParseFullPath N/A N/A full_path N/A N/A N/A Parse a full path and return a (bucket, relative_path). If the bucket is not registered, return None. N/A N/A bucket _ path N/A N/A full_path partition / N/A N/A N/A bucket N/A N/A IsBucket bucket N/A N/A None N/A N/A N/A bucket path N/A PrefixFromPattern N/A N/A pattern N/A N/A N/A Given a path pattern, return the prefix to use in ListKeys.
  If a pattern ends with '*', the prefix is the pattern without the '*', otherwise, we format is as a directory.
   N/A N/A N/A pattern endswith * N/A N/A N/A pattern N/A -1 N/A N/A N/A N/A pattern endswith / N/A N/A N/A N/A pattern / pattern N/A N/A N/A pattern N/A N/A pattern N/A N/A N/A pattern / N/A FileExists N/A N/A store filename callback N/A N/A N/A Returns true if a given S3 key exists. N/A ret N/A N/A N/A gen Task N/A store ListKeys prefix filename maxkeys 1 N/A N/A N/A logging info N/A File: %s -> ret: %r N/A filename ret N/A N/A callback N/A N/A N/A len ret 0 N/A N/A ret N/A 0 filename N/A N/A gen engine ListAllKeys N/A N/A store callback prefix marker batch_size N/A None None 1000 N/A N/A List all keys (repeatedly call ListKeys). N/A batch_size N/A min 1000 batch_size N/A keys N/A N/A done False N/A N/A done N/A N/A new_keys N/A N/A N/A gen Task N/A store ListKeys prefix prefix marker marker maxkeys batch_size N/A N/A N/A keys extend new_keys N/A N/A N/A len new_keys batch_size N/A N/A N/A marker N/A new_keys N/A -1 N/A N/A callback keys N/A N/A gen engine ListAllCommonPrefixes N/A N/A store delimiter callback prefix marker N/A None None N/A N/A List all common prefixes (repeatedly call ListCommonPrefixes). N/A prefixes N/A set N/A keys N/A N/A True N/A N/A N/A new_prefixes new_keys N/A N/A N/A gen Task N/A store ListCommonPrefixes delimiter prefix prefix marker marker maxkeys 1000 N/A prefixes N/A N/A prefixes union N/A set new_prefixes N/A N/A N/A keys extend new_keys N/A N/A N/A N/A len new_prefixes N/A len new_keys 1000 N/A N/A N/A marker N/A N/A N/A len new_prefixes 0 N/A N/A marker N/A max marker N/A new_prefixes N/A -1 N/A N/A N/A len new_keys 0 N/A N/A marker N/A max marker N/A new_keys N/A -1 N/A N/A callback N/A N/A sorted N/A list prefixes keys N/A N/A gen engine ListFilesAndDirs N/A N/A store pattern callback N/A N/A N/A List all subdirectories and files in a directory. Not recursive. Returns (subdirs, files). N/A result N/A N/A N/A gen Task ListAllCommonPrefixes store / prefix N/A PrefixFromPattern pattern N/A N/A callback result N/A N/A gen engine ListRecursively N/A N/A store pattern callback N/A N/A N/A Recursively list all files matching 'pattern'. This does not return directories. N/A results N/A N/A N/A gen Task ListAllKeys store prefix N/A PrefixFromPattern pattern N/A N/A callback results N/A N/A gen engine GetFileContents N/A N/A store path callback auto_gunzip N/A True N/A N/A Get the contents of a file in S3. If the extension is .gz, gunzip it first. N/A buf N/A N/A N/A gen Task N/A store Get path N/A N/A auto_gunzip N/A N/A path endswith .gz N/A N/A iobuffer N/A N/A cStringIO StringIO buf N/A gzipIO N/A N/A gzip GzipFile rb fileobj iobuffer N/A contents N/A N/A gzipIO read N/A N/A N/A iobuffer close N/A N/A contents buf N/A N/A callback contents N/A N/A gen engine