Package: django-netjsongraph
Reusable django app for collecting and visualizing network topology.Table of Contents:Current featuresProject goalsDeploy it in productionInstall stable version from pypiInstall development versionSetup (integrate in an existing django project)Management Commandsupdate_topologysave_snapshotLoggingStrategiesFETCH strategyRECEIVE strategySettingsNETJSONGRAPH_PARSERSNETJSONGRAPH_SIGNALSNETJSONGRAPH_TIMEOUTNETJSONGRAPH_LINK_EXPIRATIONNETJSONGRAPH_VISUALIZER_CSSNETJSONGRAPH_NODE_EXPIRATIONTOPOLOGY_API_URLCONFTOPOLOGY_API_BASEURLOverriding visualizer templatesExample: overriding the<script>tagExtending django-netjsongraphExtending modelsExtending the adminExtending API viewsAPI URLsExtending Visualizer ViewsVisualizer URLsExtending AppConfigInstalling for developmentContributingChangelogLicenseCurrent featuresnetwork topology collectorsupporting different formats:NetJSON NetworkGraphOLSR (jsoninfo/txtinfo)batman-adv (jsondoc/txtinfo)BMX6 (q6m)CNML 1.0OpenVPNadditional formats can be added byspecifying custom parsersnetwork topology visualizerbased onnetjsongraph.jssimple HTTP APIthat exposes data inNetJSONNetworkGraphformatadmin interfacethat allows to easily manage, audit, visualize and debug topologies and their relative data (nodes, links)receive topologyfrom multiple nodestopology history: allows saving daily snapshots of each topology that can be viewed in the frontendProject goalsmake it easy to visualize network topology data for the formats supported bynetdiffexpose topology data via RESTful resources inNetJSON NetworkGraphformatmake it easy to integrate in larger django projects to improve reusabilitymake it easy to extend its models by providing abstract models (needs improvement in this point)provide ways to customize or replace the visualizer (needs improvement in this point)keep the core very simpleprovide ways to extend the default behaviourencourage new features to be published as extensionsDeploy it in productionAn automated installer is provided by theOpenWISPproject:ansible-openwisp2.Ensure to follow the instructions explained in the following section:Enabling the network topology
module.Install stable version from pypiInstall from pypi:pipinstalldjango-netjsongraphInstall development versionInstall tarball:pipinstallhttps://github.com/openwisp/django-netjsongraph/tarball/masterAlternatively you can install via pip using git:pipinstall-egit+git://github.com/openwisp/django-netjsongraph#egg=django-netjsongraphIf you want to contribute, install your cloned fork:gitclonegit@github.com:<your_fork>/django-netjsongraph.gitcddjango-netjsongraphpythonsetup.pydevelopSetup (integrate in an existing django project)Addrest_frameworkanddjango_netjsongraphtoINSTALLED_APPS:INSTALLED_APPS=[# other apps'rest_framework','openwisp_utils.admin_theme','django_netjsongraph'# ...]Include urls in your urlconf (you can change the prefixes
according to your needs):fromdjango.conf.urlsimportinclude,urlfromdjango_netjsongraph.apiimporturlsasnetjsongraph_apifromdjango_netjsongraph.visualizerimporturlsasnetjsongraph_visualizerurlpatterns=[# your URLs ...url(r'^api/',include(netjsongraph_api)),url(r'',include(netjsongraph_visualizer)),]Create database tables:./manage.py migrateManagement Commandsupdate_topologyAfter topology URLs (URLs exposing the files that the topology of the network) have been
added in the admin, theupdate_topologymanagement command can be used to collect data
and start playing with the network graph:./manage.py update_topologyThe management command accepts a--labelargument that will be used to search in
topology labels, eg:./manage.py update_topology --label mytopologysave_snapshotThesave_snapshotmanagement command can be used to save the topology graph data which
could be used to view the network topology graph sometime in future:./manage.py save_snapshotThe management command accepts a--labelargument that will be used to search in
topology labels, eg:./manage.py save_snapshot --label mytopologyLoggingTheupdate_topologymanagement command will automatically try to log errors.For a good defaultLOGGINGconfiguration refer to thetest settings.StrategiesThere are mainly two ways of collecting topology information:FETCHstrategyRECEIVEstrategyEachTopologyinstance has astrategyfield which can be set to the desired setting.FETCH strategyTopology data will be fetched from a URL.When some links are not detected anymore they will be flagged as “down” straightaway.RECEIVE strategyTopology data is sent directly from one or more nodes of the network.The collector waits to receive data in the payload of a POST HTTP request;
when such a request is received, akeyparameter it’s first checked against
theTopologykey.If the request is authorized the collector proceeds to update the topology.If the data is sent from one node only, it’s highly advised to set theexpiration_timeof theTopologyinstance to0(seconds), this way the
system works just like in theFETCH strategy, with the only difference that
the data is sent by one node instead of fetched by the collector.If the data is sent from multiple nodes, youSHOULDset theexpiration_timeof theTopologyinstance to a value slightly higher than the interval used
by nodes to send the topology, this way links will be flagged as “down” only if
they haven’t been detected for a while. This mechanism allows to visualize the
topology even if the network has been split in several parts, the disadvantage
is that it will take a bit more time to detect links that go offline.SettingsNETJSONGRAPH_PARSERStype:listdefault:[]Additional customnetdiff parsers.NETJSONGRAPH_SIGNALStype:strdefault:NoneString representing python module to import on initialization.Useful for loading django signals or to define custom behaviour.NETJSONGRAPH_TIMEOUTtype:intdefault:8Timeout when fetching topology URLs.NETJSONGRAPH_LINK_EXPIRATIONtype:intdefault:60If a link is down for more days than this number, it will be deleted by theupdate_topologymanagement command.Setting this toFalsewill disable this feature.NETJSONGRAPH_VISUALIZER_CSStype:strdefault:netjsongraph/css/style.cssPath of the visualizer css file. Allows customization of css according to user’s
preferences.NETJSONGRAPH_NODE_EXPIRATIONtype:intdefault:FalseIf a node has not been modified since the days specified and if it has no links,
it will be deleted by theupdate_topologymanagement command. This depends onNETJSONGRAPH_LINK_EXPIRATIONbeing enabled.
ReplaceFalsewith an integer to enable the feature.TOPOLOGY_API_URLCONFtype:stringdefault:NoneUse theurlconfoption to change receive api url to point to
another module, example,myapp.urls.TOPOLOGY_API_BASEURLtype:stringdefault:NoneIf you have a seperate instanse of django-netjsongraph on a
different domain, you can use this option to change the base
of the url, this will enable you to point all the API urls to
your django-netjsongraph API server’s domain,
example value:https://mytopology.myapp.com.Overriding visualizer templatesFollow these steps to override and customise the visualizer’s default templates:create a directory in your django project and put its full path inTEMPLATES['DIRS'],
which can be found in the djangosettings.pyfilecreate a sub directory namednetjsongraphand add all the templates which shall override
the defaultnetjsongraph/*templatescreate a template file with the same name of the template file you want to overrideMore information about the syntax used in django templates can be found in thedjango templates
documentation.Example: overriding the<script>tagHere’s a step by step guide on how to change the javascript options passed tonetjsongraph.js, remember to replace<project_path>with the
absolute filesytem path of your project.Step 1: create a directory in<project_path>/templates/netjsongraphStep 2: open yoursettings.pyand edit theTEMPLATES['DIRS']setting so that it looks
like the following example:# settings.pyTEMPLATES=[{'DIRS':[os.path.join(BASE_DIR,'templates')],# ... all other lines have been omitted for brevity ...}]Step 3: create a new file namednetjsongraph-script.htmlin
the new<project_path>/templates/netjsongraph/directory, eg:<!-- <project_path>/templates/netjsongraph/netjsongraph-script.html --><script>window.__njg_el__=window.__njg_el__||"body";window.__njg_default_url__="{{ graph_url }}";window.loadNetJsonGraph=function(graph){graph=graph||window.__njg_default_url__;d3.select("svg").remove();d3.select(".njg-overlay").remove();d3.select(".njg-metadata").remove();returnd3.netJsonGraph(graph,{el:window.__njg_el__,// customizations of netjsongraph.jslinkClassProperty:"status",defaultStyle:false,labelDy:"-1.4em",circleRadius:8,charge:-100,gravity:0.3,linkDistance:100,linkStrength:0.2,});};window.graph=window.loadNetJsonGraph();window.initTopologyHistory(jQuery);</script>Extending django-netjsongraphdjango-netjsongraphprovides a set of models, admin classes and generic views which can be imported, extended and reused by third party apps.To extenddjango-netjsongraph,you MUST NOTadd it tosettings.INSTALLED_APPS, but you must create your own app (which goes intosettings.INSTALLED_APPS), import the base classes from django-netjsongraph and add your customizations.Extending modelsThis example provides an example of how to extend the base models ofdjango-netjsongraph.# models.py of your custom ``network`` appfromdjango.dbimportmodelsfromdjango_netjsongraph.base.linkimportAbstractLinkfromdjango_netjsongraph.base.nodeimportAbstractNodefromdjango_netjsongraph.base.snapshotimportAbstractSnapshotfromdjango_netjsongraph.base.topologyimportAbstractTopology# the model ``organizations.Organization`` is omitted for brevity# if you are curious to see a real implementation, check out django-organizations# https://github.com/bennylope/django-organizationsclassOrganizationMixin(models.Model):organization=models.ForeignKey('organization.Organization')classMeta:abstract=TrueclassTopology(OrganizationMixin,AbstractTopology):defclean(self):# your own validation logic herepassclassMeta(AbstractTopology.Meta):abstract=FalseclassNode(AbstractNode):topology=models.ForeignKey('Topology')classMeta:abstract=FalseclassLink(AbstractLink):topology=models.ForeignKey('Topology')source=models.ForeignKey('Node',related_name='source_link_set')target=models.ForeignKey('Node',related_name='source_target_set')classMeta:abstract=FalseclassSnapshot(OrgMixin,AbstractSnapshot):topology=models.ForeignKey('topology.Topology',on_delete=models.CASCADE)classMeta(AbstractSnapshot.Meta):abstract=FalseExtending the adminFollowing the above example, you can avoid duplicating the admin code by importing the base admin classes and registering your models with.# admin.py of your appfromdjango.contribimportadminfromdjango_netjsongraph.base.adminimport(AbstractLinkAdmin,AbstractNodeAdmin,AbstractTopologyAdmin)# these are you custom modelsfrom.modelsimportLink,Node,TopologyclassTopologyAdmin(AbstractTopologyAdmin):model=TopologyclassNodeAdmin(AbstractNodeAdmin):model=NodeclassLinkAdmin(AbstractLinkAdmin):model=Linkadmin.site.register(Link,LinkAdmin)admin.site.register(Node,NodeAdmin)admin.site.register(Topology,TopologyAdmin)Extending API viewsIf your use case doesn’t vary much from the base, you may also want to try to reuse the API views:# your app.api.viewsfrom..modelsimportSnapshot,Topologyfromdjango_netjsongraph.api.genericsimport(BaseNetworkCollectionView,BaseNetworkGraphHistoryView,BaseNetworkGraphView,BaseReceiveTopologyView)classNetworkCollectionView(BaseNetworkCollectionView):queryset=Topology.objects.filter(published=True)classNetworkGraphView(BaseNetworkGraphView):queryset=Topology.objects.filter(published=True)classReceiveTopologyView(BaseReceiveTopologyView):model=TopologyclassNetworkGraphHistoryView(BaseNetworkGraphHistoryView):topology_model=Topologysnapshot_model=Snapshotnetwork_collection=NetworkCollectionView.as_view()network_graph=NetworkGraphView.as_view()network_graph_history=NetworkGraphHistoryView.as_view()receive_topology=ReceiveTopologyView.as_view()API URLsIf you are not making drastic changes to the api views, you can avoid duplicating the URL logic by using theget_api_urlsfunction. Put this in your apiurls.py:# your app.api.urlsfromdjango_netjsongraph.utilsimportget_api_urlsfrom.importviewsurlpatterns=get_api_urls(views)Extending Visualizer ViewsIf your use case doesn’t vary much from the base, you may also want to try to reuse the Visualizer views:# your app.visualizer.viewsfrom..modelsimportTopologyfrom.genericsimportBaseTopologyDetailView,BaseTopologyListViewclassTopologyListView(BaseTopologyListView):topology_model=TopologyclassTopologyDetailView(BaseTopologyDetailView):topology_model=Topologytopology_list=TopologyListView.as_view()topology_detail=TopologyDetailView.as_view()Visualizer URLsIf you are not making any drastic changes to visualizer views, you can avoid duplicating the URL logic by usingget_visualizer_urlsfunction. Put this in your visualizerurls.py# your app.visualizer.urlsfromdjango_netjsongraph.utilsimportget_visualizer_urlsfrom.importviewsurlpatterns=get_visualizer_urls(views)Extending AppConfigYou may want to reuse theAppConfigclass ofdjango-netjsongraphtoo:fromdjango_netjsongraph.appsimportDjangoNetjsongraphConfigclassMyOwnConfig(DjangoNetjsongraphConfig):name='yourapp'label='yourapp'Installing for developmentInstall sqlite:sudoapt-getinstallsqlite3libsqlite3-devInstall your forked repo:gitclonegit://github.com/<your_fork>/django-netjsongraphcddjango-netjsongraph/pythonsetup.pydevelopInstall test requirements:pipinstall-rrequirements-test.txtCreate database:cdtests/./manage.pymigrate./manage.pycreatesuperuserLaunch development server:./manage.pyrunserverYou can access the visualizer athttp://127.0.0.1:8000/and the admin interface athttp://127.0.0.1:8000/admin/.Run tests with:./runtests.pyContributingFirst off, thanks for taking the time to read these guidelines.Trying to follow these guidelines is important in order to minimize waste and
avoid misunderstandings.Ensure your changes meet theProject GoalsIf you found a bug please send a failing test with a patchIf you want to add a new feature, announce your intentions in theissue trackerFork this repo and install it by following the instructions inInstalling for developmentFollowPEP8, Style Guide for Python CodeWrite codeWrite tests for your codeEnsure all tests passEnsure test coverage is not under 90%Document your changesSend pull requestChangelogSeeCHANGES.LicenseSeeLICENSE.This projects bundles third-party javascript libraries in its source code:D3.js (BSD-3-Clause)