Package: netdiff
Netdiff is a simple abstraction layer for parsing network topology data of
open source dynamic routing protocols or any other networking software which
has knowledge about the topology of a network.Its goal is to allow applications likeopenwisp-network-topologyto collect, visualize and monitor network topology data without having to deal
with the details of each networking software from which the data is retrieved.Features:parse different formatsdetect changes in two topologiesreturn consistent NetJSON outputuses the popularnetworkxlibrary under the hoodTable of Contents:Install stable version from pypiInstall development versionBasic Usage ExampleParsersInitialization argumentsInitialization examplesNetJSON outputExceptionsConversionExceptionParserErrorNetJsonErrorTopologyRetrievalErrorSpecialized featuresKnown IssuesRunning testsContributingSupportChangelogLicenseInstall stable version from pypiInstall from pypi:pipinstallnetdiffInstall development versionInstall tarball:pipinstallhttps://github.com/openwisp/netdiff/tarball/masterAlternatively you can install via pip using git:pipinstall-egit+git://github.com/openwisp/netdiff#egg=netdiffIf you want to contribute, install your cloned fork:gitclonegit@github.com:<your_fork>/netdiff.gitcdnetdiffpythonsetup.pydevelopBasic Usage ExampleCalculate diff of an OLSR 0.6.x topology:fromnetdiffimportOlsrParserfromnetdiffimportdiffold=OlsrParser(file='./stored-olsr.json')new=OlsrParser(url='http://127.0.0.1:9090')diff(old,new)In alternative, you may also use the subtraction operator:fromnetdiffimportOlsrParserfromnetdiffimportdiffold=OlsrParser(file='./stored-olsr.json')new=OlsrParser(url='http://127.0.0.1:9090')old-newThe output will be an ordered dictionary with three keys:addedremovedchangedEach key will contain a dict compatible with theNetJSON NetworkGraphformat
representing respectively:the nodes and links that have been added to the topologythe nodes and links that have been removed from the topologythe nodes and links that are present in both topologies but their attributes have changedIf no changes are present, keys will containNone.So if betweenoldandnewthere are no changes, the result will be:{"added":None"removed":None,"changed":None}While if there are changes, the result will look like:{"added":{"type":"NetworkGraph","protocol":"OLSR","version":"0.6.6","revision":"5031a799fcbe17f61d57e387bc3806de","metric":"ETX","nodes":[{"id":"10.150.0.7","label":"Node A","local_addresses":[],"properties":{}},{"id":"10.150.0.6","label":"Node B","local_addresses":["10.56.2.1"],"properties":{"hostname":"nodeb.lan"}}],"links":[{"source":"10.150.0.3","target":"10.150.0.7","cost":1.50390625,"cost_text","","properties":{}},{"source":"10.150.0.3","target":"10.150.0.6","cost":1.0"cost_text","","properties":{}}]},"removed":{"type":"NetworkGraph","protocol":"OLSR","version":"0.6.6","revision":"5031a799fcbe17f61d57e387bc3806de","metric":"ETX","nodes":[{"id":"10.150.0.8","label":"Node C","local_addresses":[],"properties":{}}],"links":[{"source":"10.150.0.7","target":"10.150.0.8","cost":1.0"cost_text","","properties":{}}]},"changed":{"type":"NetworkGraph","protocol":"OLSR","version":"0.6.6","revision":"5031a799fcbe17f61d57e387bc3806de","metric":"ETX","nodes":[],"links":[{"source":"10.150.0.3","target":"10.150.0.2","cost":1.0"cost_text","","properties":{}}]}}ParsersParsers are classes that extendnetdiff.base.BaseParserand implement aparsemethod
which is in charge of converting a python data structure intonetworkx.Graphobject and return the result.Parsers also have ajsonmethod which returns validNetJSON output.The available parsers are:netdiff.OlsrParser: parser for theolsrd jsoninfo pluginor the oldertxtinfo pluginnetdiff.BatmanParser: parser for thebatman-advanced alfred tool(supports also the legacy txtinfo format inherited from olsrd)netdiff.Bmx6Parser: parser for the BMX6b6m toolnetdiff.CnmlParser: parser forCNML 0.1netdiff.NetJsonParser: parser for theNetJSON NetworkGraphformatnetdiff.OpenvpnParser: parser for theOpenVPN status fileInitialization argumentsData can be supplied in 3 different ways, in the following order of precedence:data:dictorstrrepresenting the topology/graphurl: URL to fetch data fromfile: file path to retrieve data fromOther available arguments:timeout: integer representing timeout in seconds for HTTP or telnet requests, defaults toNoneverify: boolean indicating to therequest library whether to do SSL certificate
verification or notdirected: boolean that enables the use of a directed graph (networkx.DiGraph), defaults toFalseInitialization examplesLocal file example:fromnetdiffimportBatmanParserBatmanParser(file='./my-stored-topology.json')HTTP example:fromnetdiffimportNetJsonParserurl='https://raw.githubusercontent.com/interop-dev/netjson/master/examples/network-graph.json'NetJsonParser(url=url)Telnet example withtimeout:fromnetdiffimportOlsrParserOlsrParser(url='telnet://127.0.1',timeout=5)HTTPS example with self-signed SSL certificate usingverify=False:fromnetdiffimportNetJsonParserOlsrParser(url='https://myserver.mydomain.com/topology.json',verify=False)NetJSON outputNetdiff parsers can return a validNetJSON NetworkGraphobject:fromnetdiffimportOlsrParserolsr=OlsrParser(url='telnet://127.0.0.1:9090')# will return a dictolsr.json(dict=True)# will return a JSON formatted stringprint(olsr.json(indent=4))Output:{"type":"NetworkGraph","protocol":"OLSR","version":"0.6.6","revision":"5031a799fcbe17f61d57e387bc3806de","metric":"ETX","nodes":[{"id":"10.150.0.3"},{"id":"10.150.0.2"},{"id":"10.150.0.4"}],"links":[{"source":"10.150.0.3","target":"10.150.0.2","cost":2.4},{"source":"10.150.0.3","target":"10.150.0.4","cost":1.0}]}ExceptionsAll the exceptions are subclasses ofnetdiff.exceptions.NetdiffException.ConversionExceptionnetdiff.exceptions.ConversionExceptionRaised when netdiff can’t recognize the format passed to the parser.Not necessarily an error, should be caught and managed in order to support additional formats.The data which was retrieved from network/storage can be accessed via the “data” attribute, eg:defto_python(self,data):try:returnsuper().to_python(data)exceptConversionExceptionase:returnself._txtinfo_to_jsoninfo(e.data)ParserErrornetdiff.exceptions.ParserErrorRaised when the format is recognized but the data is invalid.NetJsonErrornetdiff.exceptions.NetJsonErrorRaised when thejsonmethod ofnetdiff.parsers.BaseParserdoes not have enough data
to be compliant with theNetJSON NetworkGraphspecification.TopologyRetrievalErrornetdiff.exceptions.TopologyRetrievalErrorRaised when it is not possible to retrieve the topology data
(eg: the URL might be temporary unreachable).Specialized featuresOpenVPNBy default, the OpenVPN parser uses the common name to identify a client,
this was chosen because if the public IP address is used, the same client
will not be recognized if it connects with a different IP address
(very probable since many ISPs use dynamic public IP addresses).This does not work when the vpn server configuration allows different clients
to use the same common name (which is generally not recommended anyway).If you need to support legacy systems which are configured with the OpenVPNduplicate-cnfeature enabled, you can passduplicate_cn=Trueduring
the initialization ofOpenvpnParser.
This will change the behavior of the parser so that each client is identified
by their common name and IP address (and additionally the port used if there
are multiple clients with same common name and IP).Known IssuesConnectionError: BadStatusLineIf you get a similar error when performing a request to thejsoninfo pluginofolsrd(version 0.6 to 0.9) chances are high that http headers are disabled.To fix it turn on http headers in your olsrd configuration file, eg:LoadPlugin "olsrd_jsoninfo.so.0.0"
{
    PlParam "httpheaders" "yes"   # add this line
    PlParam "Port" "9090"
    PlParam "accept" "0.0.0.0"
}Running testsInstall your forked repo:gitclonegit://github.com/<your_fork>/netdiffcdnetdiff/pythonsetup.pydevelopInstall test requirements:pipinstall-rrequirements-test.txtRun tests with:./runtests.py./run-qa-checksAlternatively, you can use thenose2command (which has a ton of available options):nose2nose2tests.test_olsr# run only olsr related testsnose2tests/test_olsr.py# variant form of the previous commandnose2tests.test_olsr:TestOlsrParser# variant form of the previous commandnose2tests.test_olsr:TestOlsrParser.test_parse# run specific testSee test coverage with:coveragerun--source=netdiffruntests.py&&coveragereportContributingPlease refer to theOpenWISP contributing guidelines.SupportSeeOpenWISP Support Channels.ChangelogSeeCHANGES.LicenseSeeLICENSE.