Module Expr Str Import alias Import alias Import alias Import alias Import alias Assign NameStore BinOpMult Num Num FunctionDef arguments args defaults body Expr Str TryExcept body Assign NameStore Call AttributeLoad NameLoad attr Str If CompareEq NameLoad Num body Raise Call NameLoad Return Call NameLoad BinOpFloorDiv NameLoad Num Num handlers ExceptHandler body Return Num decorator_list Assign NameStore Call NameLoad FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str If CompareIs NameLoad NameLoad body Assign NameStore Lambda arguments args NameParam defaults NameLoad Assign NameStore ListLoad For TupleStore NameStore NameStore Call NameLoad NameLoad body TryExcept body Assign NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass orelse Expr Call AttributeLoad NameLoad attr TupleLoad Call NameLoad NameLoad NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad While NameLoad body Assign TupleStore NameStore NameStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad Expr Yield NameLoad TryExcept body Assign NameStore Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass orelse Expr Call AttributeLoad NameLoad attr NameLoad TupleLoad Call NameLoad NameLoad NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Call NameLoad NameLoad If CompareEq Call NameLoad NameLoad Num body Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num NameLoad orelse Assign NameStore Call NameLoad NameLoad Str NameLoad TryFinally body Assign NameStore ListLoad TryFinally body For NameStore NameLoad body Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad Str NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad NameLoad finalbody For NameStore NameLoad body TryExcept body Expr Call AttributeLoad NameLoad attr handlers ExceptHandler body Pass finalbody Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str For NameStore NameLoad body TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler body Pass decorator_list FunctionDef arguments args NameParam defaults ListLoad body Expr Str If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad orelse Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Assign NameStore Num While NameLoad body Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Str BinOpMod Str TupleLoad NameLoad Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Yield NameLoad AugAssignAdd NameStore Num decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad body Expr Str If CompareLtE NameLoad Num body Raise Call NameLoad Str If CompareIs NameLoad NameLoad body Assign NameStore Call NameLoad Assign NameStore Call NameLoad NameLoad If CompareLtE Call NameLoad NameLoad Num body Raise Call NameLoad Str While NameLoad body Assign NameStore SubscriptLoad NameLoad Slice NameLoad Delete SubscriptDel NameLoad Slice NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call NameLoad NameLoad NameLoad keyword NameLoad If NameLoad body Expr Call NameLoad NameLoad Expr Yield NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad body Expr Str Assign NameStore Call NameLoad NameLoad If UnaryOpNot NameLoad body Expr Call AttributeLoad Call NameLoad NameLoad Str attr orelse If CompareIs NameLoad NameLoad body Assign NameStore Call NameLoad While CompareGt Call NameLoad NameLoad NameLoad body Assign NameStore Call NameLoad Call NameLoad NameLoad NameLoad keyword NameLoad keyword NameLoad keyword NameLoad Assign NameStore NameLoad Expr Call NameLoad NameLoad NameLoad keyword NameLoad If NameLoad body Expr Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad Num ListLoad NameLoad body Assign NameStore Call NameLoad NameLoad Assign NameStore ListLoad Assign NameStore Call NameLoad NameLoad Str NameLoad TryFinally body TryFinally body Assign NameStore Call NameLoad NameLoad While NameLoad body Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad NameLoad If UnaryOpNot NameLoad body Break Expr Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad Str NameLoad TryFinally body Expr Call AttributeLoad NameLoad attr NameLoad finalbody Expr Call AttributeLoad NameLoad attr finalbody Expr Call AttributeLoad NameLoad attr Expr Call NameLoad NameLoad NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad finalbody Expr Call NameLoad NameLoad decorator_list
N/A N/A Functions to sort large files.

The functions in this module were originally downloaded from the
following URL:

    http://code.activestate.com/recipes/466302/

It was apparently submitted by Nicolas Lehuen on Tue, 17 Jan 2006.
According to the terms of service of that website, the code is usable
under the MIT license.

 N/A os N/A shutil N/A heapq N/A itertools N/A tempfile N/A BUFSIZE N/A 64 1024 get_default_max_merge N/A N/A N/A N/A N/A Return the default maximum number of files to merge at once. N/A N/A N/A _SC_OPEN_MAX N/A N/A os sysconf SC_OPEN_MAX N/A N/A _SC_OPEN_MAX -1 N/A N/A N/A ValueError N/A N/A min N/A _SC_OPEN_MAX 2 100 N/A N/A N/A N/A 50 N/A N/A DEFAULT_MAX_MERGE N/A get_default_max_merge merge N/A N/A iterables key N/A None N/A N/A Merge (in the sense of mergesort) ITERABLES.

  Generate the output in order.  If KEY is specified, it should be a
  function that returns the sort key. N/A N/A key None N/A N/A key N/A N/A N/A x N/A x N/A values N/A N/A N/A index iterable N/A enumerate iterables N/A N/A N/A N/A iterator N/A iter iterable N/A value N/A N/A iterator next N/A N/A N/A StopIteration N/A N/A N/A N/A N/A N/A values append N/A N/A key value index value iterator N/A N/A N/A heapq heapify values N/A values N/A N/A N/A k index value iterator N/A N/A heapq heappop values N/A N/A value N/A N/A N/A value N/A N/A iterator next N/A N/A N/A StopIteration N/A N/A N/A N/A N/A N/A heapq heappush values N/A N/A key value index value iterator N/A merge_files_onepass N/A N/A input_filenames output_filename key N/A None N/A N/A Merge a number of input files into one output file.

  This is a merge in the sense of mergesort; namely, it is assumed
  that the input files are each sorted, and (under that assumption)
  the output file will also be sorted. N/A input_filenames N/A list input_filenames N/A N/A N/A len input_filenames 1 N/A N/A N/A N/A shutil move N/A input_filenames N/A 0 output_filename N/A N/A output_file N/A file output_filename wb BUFSIZE N/A N/A N/A chunks N/A N/A N/A N/A input_filename input_filenames N/A N/A N/A N/A chunks append N/A open input_filename rb BUFSIZE N/A N/A N/A output_file writelines N/A merge chunks key N/A N/A chunk chunks N/A N/A N/A N/A N/A N/A chunk close N/A N/A N/A N/A N/A N/A N/A N/A output_file close N/A _try_delete_files N/A N/A filenames N/A N/A N/A Try to remove the named files.  Ignore errors. N/A filename filenames N/A N/A N/A N/A N/A N/A os remove filename N/A N/A N/A N/A N/A tempfile_generator N/A N/A tempdirs N/A N/A N/A N/A Yield filenames of temporary files. N/A tempdirs N/A N/A tempdirs N/A N/A itertools cycle tempdirs N/A N/A tempdirs N/A N/A itertools repeat N/A N/A tempfile gettempdir N/A i 0 N/A True N/A N/A N/A fd filename N/A N/A tempfile mkstemp N/A sort%06i- N/A i N/A N/A tempdirs next False N/A N/A N/A os close fd N/A N/A filename N/A i 1 N/A _merge_file_generation N/A N/A input_filenames delete_inputs key max_merge tempfiles N/A None DEFAULT_MAX_MERGE None N/A N/A Merge multiple input files into fewer output files.

  This is a merge in the sense of mergesort; namely, it is assumed
  that the input files are each sorted, and (under that assumption)
  the output file will also be sorted.  At most MAX_MERGE input files
  will be merged at once, to avoid exceeding operating system
  restrictions on the number of files that can be open at one time.

  If DELETE_INPUTS is True, then the input files will be deleted when
  they are no longer needed.

  If temporary files need to be used, they will be created using the
  specified TEMPFILES tempfile generator.

  Generate the names of the output files. N/A N/A max_merge 1 N/A N/A N/A ValueError max_merge must be greater than one N/A N/A tempfiles None N/A N/A tempfiles N/A tempfile_generator N/A filenames N/A list input_filenames N/A N/A N/A len filenames 1 N/A N/A N/A ValueError It makes no sense to merge a single file N/A filenames N/A N/A group N/A filenames N/A max_merge N/A N/A filenames N/A max_merge N/A group_output N/A N/A tempfiles next N/A N/A merge_files_onepass group group_output key key N/A delete_inputs N/A N/A N/A _try_delete_files group N/A N/A group_output N/A merge_files N/A N/A input_filenames output_filename key delete_inputs max_merge tempfiles N/A None False DEFAULT_MAX_MERGE None N/A N/A Merge a number of input files into one output file.

  This is a merge in the sense of mergesort; namely, it is assumed
  that the input files are each sorted, and (under that assumption)
  the output file will also be sorted.  At most MAX_MERGE input files
  will be merged at once, to avoid exceeding operating system
  restrictions on the number of files that can be open at one time.

  If DELETE_INPUTS is True, then the input files will be deleted when
  they are no longer needed.

  If temporary files need to be used, they will be created using the
  specified TEMPFILES tempfile generator. N/A filenames N/A list input_filenames N/A N/A filenames N/A N/A N/A N/A N/A open output_filename wb close N/A N/A N/A tempfiles None N/A N/A tempfiles N/A tempfile_generator N/A N/A N/A len filenames max_merge N/A N/A filenames N/A list N/A _merge_file_generation filenames delete_inputs key key max_merge max_merge tempfiles tempfiles N/A delete_inputs True N/A N/A merge_files_onepass filenames output_filename key key N/A delete_inputs N/A N/A N/A _try_delete_files filenames N/A sort_file N/A N/A input output key buffer_size tempdirs max_merge N/A None 32000 N/A DEFAULT_MAX_MERGE N/A N/A tempfiles N/A tempfile_generator tempdirs N/A filenames N/A N/A input_file N/A file input rb BUFSIZE N/A N/A N/A N/A N/A input_iterator N/A iter input_file N/A True N/A N/A current_chunk N/A list N/A N/A itertools islice input_iterator buffer_size N/A N/A current_chunk N/A N/A N/A N/A N/A current_chunk sort key key N/A filename N/A N/A tempfiles next N/A N/A N/A filenames append filename N/A f N/A open filename w+b BUFSIZE N/A N/A N/A N/A N/A f writelines current_chunk N/A N/A N/A N/A f close N/A N/A N/A N/A input_file close N/A N/A merge_files filenames output key key delete_inputs True max_merge max_merge tempfiles tempfiles N/A N/A N/A _try_delete_files filenames N/A