Package: tolerant-toposort
Tolerant toposortTolerant toposortextends the PyPi packagetoposortto support disabled nodes within the graphIt takes a (directed) dependency graph, and disabled nodes as input,and returns ordered batches of nodes which are independent of disabled nodesdata={2:{11},9:{11,8},10:{11,3},11:{7,5},8:{7,3},}disabled={5}toposort(data,disabled)[{3,7},{8}]ExamplesSimpleItem  1 depends on Item 2, depends on 3, depends on 4With Item 3 disabled both 2 and 1 are implicitly disabled.However, using tolerant toposort, we find we can still process Item 4data={1:{2},2:{3},3:{4},}disabled={3}result=toposort(data,disabled)[{4}]Less SimpleA more complicated graph with Item 7 disabledAgain, using tolerant toposort, we find we can still process Items 3 and 5, then 10, and then 12:data={2:{2,11},9:{11,8,10},10:{3},11:{7,5},8:{7,3},12:{10},}disabled={7}result=toposort(data,disabled)[{3,5},{10},{12}]Use CaseThe original use case was building packages.The aim was to process(build) as many nodes(packages) as possible in a tree of nodes:Whilst processing, a node might fail to be processedOr the node might be known to be failed prior to processingIf a node was failed then it and its dependants could not be processedFixing, aka re-enabling, nodes took timeProcessing nodes took timeProcessing a node only to find its dependant was failed, took timeOnce a node was processed, it needed no further processingThe main issue was if a node failed ( or was failed prior), then no more nodes could safely be processed, and that switched what should have been concurrent:fixing nodesprocessing nodesinto a repetition of process batches,node fails,fix node,process batches,node fails,fix node,process batches...Requirements for the improvements:To have the best handle on the number of disabled nodes (ie attempt to process as many nodes much as possible)Concurrently be able to:repeatedly process batches, revealing failed nodes as they appear, recalculate batches, continue processingFix failed nodes as they 'appear', removing them from the disabled list as they are supposedly fixedWith tolerant toposort:Processing of all independant ( of disabled ) nodes can be attemptedAs disabled nodes are encountered they can be added to the disabled set, and a revised batch set createdProcessing can then continue until all possible nodes have been attemptedThe maximum set of disabled nodes is returnedTypical Usage FooTypical Usage Toofromtolerant.toposortimporttoposort,CircularDependencyErrorclassProcessException(Exception):def__init__(self,node):super(ProcessException,self).__init__('')self.node=nodedefget_graph():""" build and return your graph """return{2:{2,11},9:{11,8,10},10:{3},11:{7,5},8:{7,3},12:{10},13:{12},14:{2}}defprocess(node):"""perform a once-only process on a node.return the success of the proceess"""print(f"processing{node}")returnnode!=12;defmain():disabled={9}# add any known disabled items at start-upalready_processed=set()# persist this between runs!!graph=get_graph()while(True):print(f"calling toposort")batches=toposort(graph,disabled)ifnotbatches:print(f"batches is empty")breakprocessedAny=Falsetry:forbatchinbatches:fornodeinbatch:ifnodeinalready_processed:print(f"already processed{node}")continueifprocess(node):already_processed.add(node)print(f"processed{node}")processedAny=Trueelse:raiseProcessException(node)# our work is done ( bar enabling any disabled)ifnotprocessedAny:print(f"no processing so finished")breakexceptProcessExceptionaspe:# pe.node can now be concurrently 'enabled'print(f"disabled{pe.node}")disabled.add(pe.node)ifnotdisabled:# we are donepassmain()APIModuletolerant.toposortGenerates successive batches of dependant items which are enabled and do not depend
on disabled itemsBased ontoposortwith these changes:toposortandtoposort_flattentake an optional set of disabled items.
These disabled items, and their dependents, will not be included
in the returned batch(es)FunctionsFunctiontoposortdeftoposort(data,disabled=set())Dependencies are expressed as a dictionary whose keys are items
and whose values are a set of dependent items.Returns a list of sets in topological order. The first set consists of items with no
dependences, each subsequent set consists of items that depend upon
items in the preceeding sets.Argsdata- the dependency graph
dependencies are expressed as a dictionary whose keys are items
and whose values are a set of dependent items.disabled(optional) - Set of items which, with their dependents, should not be
included in the outputReturnsa list of sets in topological order which are not disabled, or depend on
a disabled item.The first set consists of items with no dependences, each subsequent set
consists of items that depend upon items in the preceeding sets.Functiontoposort_flattendeftoposort_flatten(data,sort=True,disabled=set())Returns a single list of dependencies. For any set returned by
toposort(), those items are sorted and appended to the result (just to
make the results deterministic).Argsdata- the dependency graph
dependencies are expressed as a dictionary whose keys are items
and whose values are a set of dependent items.sort(True) - should each batch be sorteddisabled(optional) - Set of items which, with their dependents, should not be
included in the outputClassesClassCircularDependencyErrorclassCircularDependencyError(data)An itemeventuallydepends on itselfNOTE: we tolerate itemsdirectlydepending on themeselvesArgsdata: the list containing  the circular dependencyTestingnose2python3setup.pytestInstallsudopython3setup.pyinstall