Package: dependencynet
dependencynetThis module wraps some data and graph manipulation tools to help analyzing and building representations of directed graphs.This modules make it easier to build graphs having the properties listed belowelements can be categorized as levels and resourceslevels consists of composite keys forming a hierarchy, for instancearea > country > towngroup > artifact > versiondatabase > table > attributeworkflow > step > taskresource is any element related to a level (and only one), for instancemonuments or POI of a towntables used by a task in a workflowresources having the same label might be connected to each other to form new dependencies, for instancefiles in and out of a workflow task  might be connected to each other to form data flowstowns may be connected to form a trip if one has a flight out and the other a flight in with the same labelWhat can I do with this tool ?analyse dependencies in code basesWhich modules are depending of a given module ?Which modules may be inderectly impacted by a change in some module ?analyse of data flows between tasks or microservicesGiven lists of input and output tables of a task, what is the data flow ?Show me the data flow zoomed out at the workflow level and hide out data flow within the workflowStems of this projectsThis project stems from the need of featuring out how a large very messy project is made. I started by trying to list and connect elements manually, then I automated some parts. I ended up with a lot of very messy scripts to extract the most visible dependencies and it helped me to draw some schemas in Powerpoint.This pointed me to the fact that this system was a graph, a network of workflows, treatments, tasks, classes, tables, exported files, external packages, delivery bundles. I could clean up a little bit by representing all these elements as a graph. When this step is done it is easier to select what kind of nodes and links I want to show for a given view or documentation.I started to rework those scripts in Python notebooks as a side project. It took me a while. Graphs are powerful, but power comes with lots of concepts to master. In the Python ecosystem, networkx is the library to go with graphs. It has a lot of graph manipulation features but charts are not so easy, and charts was the main goal. I found ipycytoscape (a port of cytoscape) easy to use with charts. But I lacked some networkx features. Hopefully there are ways to integrate networkx and cytoscape. I also had a lot of pandas code to collect and build data structues required by the graph libraries. Again, I had a lot of code to maintain in order to achieve my goals.The last rework is this library. I factored out functions I found useful in the process of building the analyze tool. This is also a way to learn how to make a module library in Python.Please keep in mind thatthis is a side projectit is not intended to cover all the needs but a provide a specific subset required to easily build, clean up and show directed graphs of connected elementsConceptsThe graph production is done in 4 major stepsModel : build datasource for each type of elements and organize the levels hierarchySchema : describe the Model metadata (levels, resources, connections)GraphModel : represents the Model as a networkx graphGraphViewer : encapsulate the steps to show the cytoscape graphStyleBuilder : generates a convenient cytoscape style from the schemaGraphMLConverter : turns the graph and the style into a GraplML file for YedTODO link to YedExampleTODOHow to installTo use this library please usepip install dependencynetIf you want to build from the sources, please clone the repository and run setupMain dependenciesThis project relies on the following librariespandasnetworksipycytoscapepyyednoxflake8pytestTODO links and description from the moduleThanks to the maintainers and contributors of these modules.Changelog0.2.1Enhancement #22 -  how to build the model or the graph model programmatically #22More scenario tests0.2.0Fix bug #5 - I would like to be able to define any number of key levelsRefactoring of core classes
Tests refactoring
Added integration tests for each sample dataset
Added unit tests for StyleBuilder
Lint and smoke test of notebooks0.1.9Fix bug #8 GraphMl generation fail when nodes already exists0.1.8Fix bug #2 PyYed generation failed if graph use connections0.1.7Fix bug #3 Missing links while using resources connectionsnow support links from one node to many nodes0.1.6minor fixes0.1.5New features:allow to make a copy of a network in order to alter the copyallow to add input/output role to resources and connect output to inputallow to remove nodes having a given categoryallow to aggregate the levelsallow to replace input/output connection with a single nodeallow to fold categories and show links throughget a summary of the graph contentSome refactoring:package datasource.loaders is now datasource.coreminor change of schema interface for input/output connections0.1.3 - 0.1.4Bug fixes:packaging issue (missing dependencies)0.1.2New features:allow to type resource as input / outputinputs are directed toward the node0.1.1New features:helper to build the cytoscape style documenthelper convert the graph into a pyyed file (GraphML)0.1.0New features:Build a networkx/cytoscape networkImprovements of Data Loader:explode a list a column consisting in a items into multiple linesignore nan in resource columns0.0.5New features:Load a 3 levels hierarchy and resources