Module ImportFrom alias Import alias Import alias Import alias ImportFrom alias Import alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias alias ImportFrom alias alias ImportFrom alias ImportFrom alias ImportFrom alias alias alias alias ImportFrom alias alias ImportFrom alias alias alias alias alias alias ImportFrom alias ImportFrom alias Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str Str Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str Str Assign NameStore Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str Call NameLoad BinOpDiv BinOpMult NameLoad NameLoad Num ClassDef bases NameLoad NameLoad NameLoad body Expr Str FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword Str decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad Num NameLoad Num Num NameLoad body Expr Str For NameStore Call NameLoad NameLoad body TryFinally body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Expr Call NameLoad NameLoad keyword NameLoad keyword NameLoad With Call NameLoad NameLoad Str NameStore body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad keyword NameLoad keyword NameLoad Assign AttributeStore NameLoad attr NameLoad TryExcept body With Call NameLoad NameLoad NameStore body Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass Assign AttributeStore NameLoad attr NameLoad TryExcept body With Call NameLoad NameLoad NameStore body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num handlers ExceptHandler type NameLoad name NameStore body Assign NameStore AttributeLoad NameLoad attr TryExcept body With Call NameLoad NameLoad NameStore body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass Assign AttributeStore NameLoad attr NameLoad Assign NameStore Num While CompareNotEq NameLoad Num body Assign AttributeStore NameLoad attr CompareGt Call AttributeLoad NameLoad attr Num TryExcept body With Call NameLoad NameLoad NameStore body Expr Call AttributeLoad NameLoad attr Assign NameStore Num handlers ExceptHandler type NameLoad name NameStore body Assign NameStore AttributeLoad NameLoad attr If CompareGt NameLoad Num body Expr Call AttributeLoad NameLoad attr AugAssignAdd NameStore Num TryExcept body With Call NameLoad NameLoad NameStore body Expr Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass orelse Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad With Call NameLoad NameLoad Str NameStore body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad finalbody Expr Call AttributeLoad NameLoad attr ListLoad Call NameLoad Str Str NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr TryFinally body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str finalbody Expr Call AttributeLoad NameLoad attr decorator_list NameLoad NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr TryFinally body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str finalbody Expr Call AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr TryFinally body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str finalbody Expr Call AttributeLoad NameLoad attr decorator_list NameLoad NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str keyword Num keyword NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str keyword Num keyword Num decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr TryFinally body Expr Call AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr keyword Str finalbody Expr Call AttributeLoad NameLoad attr decorator_list NameLoad Call AttributeLoad NameLoad attr Str Assign NameStore Num FunctionDef arguments args NameParam defaults body For NameStore Call NameLoad AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Expr Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Expr Call NameLoad NameLoad With Call NameLoad NameLoad Str NameStore body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body For NameStore Call NameLoad AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Expr Call NameLoad NameLoad Expr Call NameLoad NameLoad Expr Call NameLoad NameLoad Expr Call NameLoad NameLoad With Call NameLoad NameLoad Str NameStore body With Call NameLoad NameLoad NameStore body With Call NameLoad NameLoad NameStore body Expr Call NameLoad NameLoad NameLoad NameLoad Assign NameStore BinOpAdd Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr With Call NameLoad NameLoad Str NameStore body Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body For NameStore Call NameLoad AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Expr Call NameLoad NameLoad keyword Num keyword NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assert CompareGt NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad Num NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call NameLoad NameLoad NameLoad Assign NameStore Call NameLoad NameLoad Str Expr Call NameLoad NameLoad NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad Call NameLoad NameLoad Str attr Assign NameStore SubscriptLoad Call AttributeLoad Call NameLoad NameLoad Str attr Slice NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body For NameStore Call NameLoad AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Str Expr Call NameLoad NameLoad Assign NameStore Call AttributeLoad Call NameLoad NameLoad Str attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call NameLoad NameLoad Num NameLoad Print BinOpMod Str TupleLoad NameLoad NameLoad Assert CompareLt NameLoad NameLoad Assert CompareEq SubscriptLoad NameLoad Index NameLoad Str Assert CompareGtE NameLoad Num decorator_list FunctionDef arguments args NameParam defaults body Return BinOpMod Str Call NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return BinOpMod Str Call NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body With Call NameLoad NameLoad Str NameStore body Return Call AttributeLoad NameLoad attr decorator_list decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad NameLoad body With Call NameLoad NameLoad Str NameStore body For NameStore Call NameLoad NameLoad body Assign NameStore BinOpAdd Call AttributeLoad Str attr GeneratorExp Call AttributeLoad NameLoad attr Str comprehension NameStore Call NameLoad BinOpSub NameLoad Num Str Expr Call AttributeLoad NameLoad attr NameLoad decorator_list
N/A __future__ absolute_import N/A unittest N/A os N/A random uuid uuid4 N/A logging N/A subprocess toil resolveEntryPoint toil.batchSystems.parasolTestSupport ParasolTestSupport toil.common Toil ToilConfigException toil.job Job JobException toil.lib.bioio getLogLevelString toil.batchSystems.mesos.test MesosTestSupport toil.test.sort.lib merge sort copySubRangeOfFile getMidPoint toil.test.sort.sort setup sortMemory toil.test ToilTest needs_aws needs_mesos needs_azure needs_parasol needs_gridengine toil.jobStores.abstractJobStore JobStoreCreationException toil.leader FailedJobsException N/A log N/A N/A logging getLogger __name__ N/A defaultLineLen N/A int N/A N/A N/A os environ get TOIL_TEST_SORT_LINE_LEN 10 N/A defaultLines N/A int N/A N/A N/A os environ get TOIL_TEST_SORT_LINES 10 N/A defaultN N/A int N/A N/A N/A os environ get TOIL_TEST_SORT_N N/A str N/A N/A defaultLineLen defaultLines 5 SortTest N/A ToilTest MesosTestSupport ParasolTestSupport N/A N/A 
    Tests Toil by sorting a file in parallel on various combinations of job stores and batch
    systems.
     setUp N/A N/A self N/A N/A N/A N/A N/A N/A super SortTest self setUp N/A N/A self tempDir N/A N/A self _createTempDir purpose tempDir N/A _toilSort N/A N/A self jobStore batchSystem lines N testNo lineLen retryCount badWorker downCheckpoints N/A defaultLines defaultN 1 defaultLineLen 2 0.5 False N/A N/A 
        Generate a file consisting of the given number of random lines, each line of the given
        length. Sort the file with Toil by splitting the file recursively until each part is less
        than the given number of bytes, sorting each part and merging them back together. Then
        verify the result.

        :param jobStore: a job store string

        :param batchSystem: the name of the batch system

        :param lines: the number of random lines to generate

        :param N: the size in bytes of each split

        :param testNo: the number of repeats of this test

        :param lineLen: the length of each random line in the file
         N/A test N/A xrange testNo N/A N/A N/A N/A options N/A N/A N/A Job Runner getDefaultOptions jobStore N/A N/A options logLevel N/A getLogLevelString N/A N/A options retryCount retryCount N/A N/A options batchSystem batchSystem N/A N/A options clean never N/A N/A options badWorker badWorker N/A N/A options badWorkerFailInterval 0.05 N/A tempSortFile N/A N/A N/A os path join N/A self tempDir fileToSort.txt N/A N/A makeFileToSort tempSortFile lines lines lineLen lineLen N/A N/A open tempSortFile r fileHandle N/A N/A l N/A N/A fileHandle readlines N/A N/A N/A l sort N/A firstJob N/A N/A Job wrapJobFn setup tempSortFile N downCheckpoints downCheckpoints memory sortMemory N/A N/A options restart True N/A N/A N/A N/A Toil options toil N/A N/A N/A N/A toil restart N/A N/A N/A self fail N/A N/A N/A JobStoreCreationException N/A N/A N/A N/A options restart False N/A N/A N/A N/A Toil options toil N/A N/A N/A N/A toil start firstJob N/A i 0 N/A N/A N/A FailedJobsException N/A e N/A N/A i N/A e numberOfFailedJobs N/A N/A N/A N/A Toil options toil N/A N/A N/A N/A toil start firstJob N/A N/A N/A self fail N/A N/A N/A JobStoreCreationException N/A N/A N/A N/A options restart True N/A totalTrys 1 N/A N/A i 0 N/A N/A N/A options useExistingOptions N/A N/A N/A random random 0.5 N/A N/A N/A N/A Toil options toil N/A N/A N/A N/A toil restart N/A i 0 N/A N/A N/A FailedJobsException N/A e N/A N/A i N/A e numberOfFailedJobs N/A N/A totalTrys 32 N/A N/A N/A N/A self fail N/A totalTrys 1 N/A N/A N/A N/A Toil options toil N/A N/A N/A N/A toil restart N/A N/A N/A JobException N/A N/A N/A N/A N/A N/A self fail N/A Expected %s to be raised JobException N/A N/A open tempSortFile r fileHandle N/A N/A l2 N/A N/A fileHandle readlines N/A N/A N/A self assertEquals l l2 N/A N/A N/A N/A subprocess check_call N/A N/A resolveEntryPoint toil clean jobStore N/A testAwsSingle N/A N/A self N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _awsJobStore batchSystem singleMachine N/A needs_aws testAwsMesos N/A N/A self N/A N/A N/A N/A N/A self _startMesos N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _awsJobStore batchSystem mesos N/A N/A N/A N/A self _stopMesos N/A needs_aws needs_mesos testFileMesos N/A N/A self N/A N/A N/A N/A N/A self _startMesos N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _getTestJobStorePath batchSystem mesos N/A N/A N/A N/A self _stopMesos N/A needs_mesos testAzureSingle N/A N/A self N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _azureJobStore batchSystem singleMachine N/A needs_azure testAzureMesos N/A N/A self N/A N/A N/A N/A N/A self _startMesos N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _azureJobStore batchSystem mesos N/A N/A N/A N/A self _stopMesos N/A needs_azure needs_mesos testFileSingle N/A N/A self N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _getTestJobStorePath batchSystem singleMachine N/A testFileSingleCheckpoints N/A N/A self N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _getTestJobStorePath batchSystem singleMachine retryCount 2 downCheckpoints True N/A testFileSingle10000 N/A N/A self N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _getTestJobStorePath batchSystem singleMachine lines 10000 N 10000 N/A testFileGridEngine N/A N/A self N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _getTestJobStorePath batchSystem gridengine N/A needs_gridengine testFileParasol N/A N/A self N/A N/A N/A N/A N/A self _startParasol N/A N/A N/A N/A N/A self _toilSort jobStore N/A N/A self _getTestJobStorePath batchSystem parasol N/A N/A N/A N/A self _stopParasol N/A needs_parasol N/A N/A unittest skip skipping until parasol support is less flaky (see github issue #449 N/A testNo 5 testSort N/A N/A self N/A N/A N/A test N/A xrange N/A self testNo N/A N/A tempFile1 N/A N/A N/A os path join N/A self tempDir fileToSort.txt N/A N/A makeFileToSort tempFile1 N/A lines1 N/A N/A self _loadFile tempFile1 N/A N/A N/A lines1 sort N/A N/A sort tempFile1 N/A N/A open tempFile1 r f N/A N/A lines2 N/A N/A f readlines N/A N/A N/A self assertEquals lines1 lines2 N/A testMerge N/A N/A self N/A N/A N/A test N/A xrange N/A self testNo N/A N/A tempFile1 N/A N/A N/A os path join N/A self tempDir fileToSort1.txt N/A tempFile2 N/A N/A N/A os path join N/A self tempDir fileToSort2.txt N/A tempFile3 N/A N/A N/A os path join N/A self tempDir mergedFile.txt N/A N/A makeFileToSort tempFile1 N/A N/A makeFileToSort tempFile2 N/A N/A sort tempFile1 N/A N/A sort tempFile2 N/A N/A open tempFile3 w fileHandle N/A N/A N/A open tempFile1 tempFileHandle1 N/A N/A N/A open tempFile2 tempFileHandle2 N/A N/A N/A merge tempFileHandle1 tempFileHandle2 fileHandle N/A lines1 N/A N/A N/A self _loadFile tempFile1 N/A N/A self _loadFile tempFile2 N/A N/A N/A lines1 sort N/A N/A open tempFile3 r f N/A N/A lines2 N/A N/A f readlines N/A N/A N/A self assertEquals lines1 lines2 N/A testCopySubRangeOfFile N/A N/A self N/A N/A N/A test N/A xrange N/A self testNo N/A N/A tempFile N/A N/A N/A os path join N/A self tempDir fileToSort1.txt N/A outputFile N/A N/A N/A os path join N/A self tempDir outputFileToSort1.txt N/A N/A makeFileToSort tempFile lines 10 lineLen defaultLineLen N/A fileSize N/A N/A N/A os path getsize tempFile N/A N/A fileSize 0 N/A fileStart N/A N/A random choice N/A xrange 0 fileSize N/A fileEnd N/A N/A random choice N/A xrange fileStart fileSize N/A fileHandle N/A open outputFile w N/A N/A copySubRangeOfFile tempFile fileStart fileEnd fileHandle N/A N/A N/A fileHandle close N/A l N/A N/A N/A open outputFile r read N/A l2 N/A N/A N/A N/A open tempFile r read N/A fileStart fileEnd N/A N/A N/A self assertEquals l l2 N/A testGetMidPoint N/A N/A self N/A N/A N/A test N/A xrange N/A self testNo N/A N/A tempFile N/A N/A N/A os path join N/A self tempDir fileToSort.txt N/A N/A makeFileToSort tempFile N/A l N/A N/A N/A open tempFile r read N/A fileSize N/A N/A N/A os path getsize tempFile N/A midPoint N/A getMidPoint tempFile 0 fileSize N/A N/A the mid point is %i of a file of %i bytes N/A midPoint fileSize N/A N/A midPoint fileSize N/A N/A N/A l N/A midPoint N/A N/A midPoint 0 N/A _awsJobStore N/A N/A self N/A N/A N/A N/A aws:us-west-2:sort-test-%s N/A uuid4 N/A _azureJobStore N/A N/A self N/A N/A N/A N/A azure:toiltest:sort-test-%s N/A uuid4 N/A _loadFile N/A N/A self path N/A N/A N/A N/A open path r f N/A N/A N/A N/A f readlines N/A N/A makeFileToSort N/A N/A fileName lines lineLen N/A defaultLines defaultLineLen N/A N/A N/A open fileName w fileHandle N/A N/A _ N/A xrange lines N/A N/A line N/A N/A N/A join N/A N/A N/A random choice actgACTGNXYZ N/A _ N/A xrange N/A lineLen 1 N/A N/A N/A fileHandle write line N/A