Hey everyone! This is a follow-up article on a local Cisco Russia DevNet Marathon online event I attended in May 2020. It was a series of educational webinars on network automation followed by daily challenges based on the discussed topics.
On a final day, the participants were challenged to automate a topology analysis and visualization of an arbitrary network segment and, optionally, track and visualize the changes.
 The task was definitely not trivial and not widely covered in public blog posts. In this article, I would like to break down my own solution that finally took first place and describe the selected toolset and considerations.

Let's get started.  The final task description was the following: A summary: IP-addresses and credentials as an input, a visualized topology as an output (and a great space for experiments and options somewhere in between).

 I also denoted some additional personal considerations to follow while choosing the solution toolset: It is always a good idea to check whether the wheel is already invented before reinventing it by yourself. I have made some research on available existing solutions for network visualization. Not surprisingly, no solution could fit all the requirements out of the box. Most such solutions were built into much larger (and typically far not free) enterprise-grade network monitoring and analysis systems which would significantly reduce reusability and customization potential. I would divide an abstract network visualization task into the following layers and steps:
 Let's focus on each of them while keeping our requirement in mind: Network Equipment
The initial task requires us to support IOS and IOS-XE.
Real-world networks tend to be much more heterogeneous. Let's try to consider this. Topology Data Sources
The task statement suggests us to use LLDP (Link Layer Discovery Protocol) protocol. This is an L2 (OSI Link Layer) protocol described in IEEE 802.1AB standard. It is widely supported by modern network platforms (including IOS and IOS-XE) and operating systems (including Linux and Window), so it suits our case.
The topology data can also be enriched by various outputs from the network devices, such as routing and switching tables, routing protocols data, and so on. Let's just mark it for future improvements. Data Access Protocols and Standards
The most modern platforms usually support shiny and chrome NETCONF, REST APIs, RESTCONF with YANG models and data structures. The existence of legacy equipment and platforms will usually force us to revert to SSH, Telnet, and good ol' CLI. Protocol- and Vendor-Specific Drivers or Plugins
The core logic of the solution will be written in Python because of two primary reasons: Python has a pretty comprehensive set of handy modules and libraries for network automation and this is a programming language I am most experienced in.
API-driven network automation in Python can be done using the requests module or some specialized modules.
Bare SSH/Telnet access to network equipment from Python commonly relies on netmiko, paramiko, or scrapli modules. They let you emulate the standard CLI: sending some text commands to the session and expecting back the text output of the more or less predictable readability level and formatting.
There are also several high-level Python frameworks allowing for additional features on top of the tools I mentioned above. The two most useful of them in our case are NAPALM and Nornir. NAPALM provides vendor-neutral GETTERs for getting structured data from the network devices. Nornir implements many useful abstractions and multithreading out of the box.
As for SNMP, let's leave it for network monitoring purposes. Unstructured Data -> Data Normalization Toolset -> Structured Data
Data gathering with API usually allows you to get structured output straight away. A text output you get from network equipment CLI is natively inapplicable for further machine processing. A traditional way to extract the data from the CLI output in Python is re module and regular expressions. Modern approaches are TextFSM framework developed by Google and brand new TTP (Template Text Parser) developed by dmulyalin. Both tools perform data parsing with more usable templates in comparison to regular expressions.
NAPALM module mentioned above performs unstructured data normalization internally for supported GETTERs and returns the structured output. This may make things some easier in our case. Data Processing and Analysis -> Topology Representation in Python Datastructures
Once we get the structured topology data pieces from all our devices, all we need to do is to bring it to common representation, analyze and assemble a final puzzle. Topology Representation in Visualization Engine Format
Depending on visualization engine selection, you may need to transform a final topology data format according to what the tool supports as an input. Visualization Engine
This point was the most not obvious for me and I had no prior experience in such development. Google search and discussions in DevNet Marathon telegram channel with colleagues introduced me to several Python (pygraphviz, matplotlib, networkx) and JavaScript (JS D3.js, vis.js.) frameworks. Then I found JavaScript+HTML5 NeXt UI Toolkit I once bookmarked while digging through Cisco DevNet labs before. This is a specialized network visualization toolkit developed by Cisco. It has many features and decent documentation. Visualized Topology
Our final goal. A view may vary from a simple static image or an HTML document to something more advanced and interactive. Here is a summary of the most common tools we have got:


 Based on the requirements above I have selected the following tools for my target solution: I have already used NAPALM and Nornir successfully for network audits and data gathering from hundreds of network devices before. Default NAPALM GETTERs support LLDP on Cisco IOS/IOS-XE, IOS-XR, NX-OS, Juniper JunOS, and Arista EOS.
Furthermore, the logic separation discussed above would allow us to add more data sources and network connectors without affecting the whole codebase.
Next UI was a thing to familiarize with and figure out how it works on the run. However, the examples looked promising. I used Cisco Modeling Labs as a test lab. This is a new version of the VIRL network emulator. Cisco DevNet Sandbox allows using it for free for a limited time window. You just have to register and proceed with a reservation which is a matter of a few mouse clicks (and a few more to connect to the lab using AnyConnect VPN once you receive the reservation details back to your email). In the old days we would have to use a production network a bare metal homelab or to have fun with GNS3.

 Lab topology on the CML web interface looks as follows (we should get the similar picture as result):


It consists of Cisco devices of any kind: IOS (edge-sw01), IOSXE (internet-rtr01, distr-rtr01, distr-rtr02), NXOS (dist-sw01, dist-sw02), IOSXR (core-rtr01, core-rtr02), ASA (edge-firewall01). LLDP is enabled on all of them. SSH access is available on IOS, IOSXE, and NXOS nodes. Nornir is an open-source Python framework. It is available on PyPI for Python 3.6.2 and above. Nornir has a dozen of dependencies, including NAPALM and netmiko. It is recommended to use Python virtual environments (venv to isolate the dependencies. My local development environment used Nornir 2.4.0 with Python 3.7 on MacOS 10.15. This should work on Linux and Windows as well. Nornir installation is straightforward: Important: Nornir has had some massive changes in the 3.X release. Some of them were not backward compatible with 2.X versions. Nornir related configurations and code are relevant to 2.X versions. Nornir supports various inventory plugins. They all provide a convenient way to structure and operate your network devices' information programmatically. For this solution, the standard SimpleInventory plugin is sufficient.
General Nornir settings are listed in a set of YAML files. Configuration file names can be arbitrary but you should point Nornir to their exact names during initialization from Python.
 nornir_config.yaml: A sample Nornir primary configuration file you can see above contains references to two more YAML files: hosts file and groups file. These files define the SimpleInventory plugin configuration. The Hosts file contains a list of our network devices (hosts) with their attributes. Groups file contains a list of groups and their attributes. An individual host can be included in one or more groups. The host inherits the attributes of all groups it belongs to. Hosts and groups file names and locations can be arbitrary as well. inventory/hosts_devnet_sb_cml.yml has the following structure: Just two hosts are being displayed for brevity. Both hosts have IP-address, platform attributes. dist-sw01 has transport type assigned specifically. For internet-rtr01, transport type will be chosen based on platform type (it is SSH for IOS by default) by Nornir. Both hosts belong to the 'devnet-cml-lab' group. groups.yml will define all group settings for them: Group attributes above contain access credentials and enable secret for Cisco devices. These attributes will be inherited by all group members.
Important: Never store credentials (and any sensitive data) in your production environment in clear text configs like this. This simple config is used for demonstrational and lab purposes only.
Those are all general Nornir configuration steps. All we need to do now is to initialize it from a Python code. For local usage and testing, it is enough to download the NeXt UI source code from GitHub. Let's put the sources into ./next_sources in our project root directory. Our progress upon download completion: The main logic will be written in a Python script named generate_topology.py. Once our Nornir config is ready, it can be initialized in Python as simply as: That's it. Nornir is ready to work.
napalm_get imported above allows us to use NAPALM straight from Nornir.
 LLDP-enabled devices exchange periodic LLDP messages consisting of TLV-fields with their direct neighbors. LLDP messages are not normally relayed.
Mandatory TLV fields: Chassis ID, Port ID, Time-to-Live.
Optional TLV fields: System Name and Description; Port Name and Description; VLAN Name; IP Management address; System Capabilities (switching, routing, etc.), and more.
As the examined topology segment is under our control, let's consider System Name and Port Name TLV fields required and advertisable internally.
It does not cause significant security risks but allows us to identify multi-chassis devices with a shared control plane (e.g. stacked switches) and device interconnections uniquely.

In this case, a topology analysis task as a whole can be reduced to the analysis of the neighborship data received on each device. This allows us to identify unique devices and their interconnections (i.e. Vertices and Edges of the topology Graph).
By the way, OSPF LSA exchange and analysis work in a very similar way. Visualizing routing protocol data may also be a good use case (I'd recommend to check out the Topolograph service released in October 2020 by @Vadims06). But let's focus on LLDP for now.

 In our lab environment, all edge, core, and distribution layer devices should see their direct LLDP neighbors. internet-rtr01 is isolated from the rest of the network so it should not have any LLDP neighbors.
 Here is a manual " show lldp neighbors" output from dist-rtr01: Five neighbors. Looks good.
The same output from core-rtr02: Four neighbors. That's correct as well.
Please note that the output contains incomplete hostnames in a Device ID column in both cases.
CLI automation always comes along with such issues.
In our given case the workaround is to use a detailed output format.
As an example: We are going to gather the LLDP data from the devices running IOS (edge-sw01), IOSXE (internet-rtr01, distr-rtr01, distr-rtr02), and NXOS (dist-sw01, dist-sw02).
IOS-XR-based core routers (core-rtr01, core-rtr02) will be intentionally restricted from management access.
Thereby the following scenarios will be covered: NAPALM GETTERs to use: Let's wrap the data gathering task into a Nornir Task function.
This is one of useful method of grouping actions on the individual hosts. Now we can run the Task and save the result into variable for a further processing.
Default execution scope is all devices. You may also use simple and complex inventory filters to limit the execution scope to individual hosts or groups. get_host_data_result variable contains a get_host_data task execution results for each target device. Every host result object has failed method returning a boolean value. False means no errors occured during task execution on a specific host.
The global task result is iterable as a dictionalry object: Looks expectedly.
 Some complete result outputs for the reference: Result object is a dictionary with a keys matching selected GETTER names: 'facts' and 'lldp_neighbors_detail'.
Dictionary values contain a structured data processed and returned by NAPALM.
Let's compare a neighbor sets: Five neighbors for dist-rtr01 and four neighbors for dist-sw01 which is exactly what we saw in CLI outputs before.
The rest of the data is also valid.
 For ease of processing let's split LLDP and facts data into separate entities.
Any devices may be also present in multiple outputs. To distinguish between them, it is necessary to use some unique node identifiers. Let's choose it in the following descending order: LLDP relies on the first two steps too. Then we should extract a list of all neighbors known by devices and build based on that: To ensure unambiguous identification of links, we will store them in a following format:
((source_device_id, source_port_name), (destination_device_id, destination_port_name))
 It is also necessary to keep in mind that: To ensure data consistency, we will translate the port names into a full format for the analysis stage. At the same time, let's implement a name shortening function to provide better visual experience during visualization.
Automatic icon selection can be implemented based on device capabilities advertised in LLDP. Let's extract them into a separate {"hostname": "primary_capability"} dictionary.
The same code wise: Topology visualization logic will be implemented in next_app.js script based on Next UI.
Let's start with the basics: The topology data structure will be stored in a topologyData variable. Let's move it into a separate topology.js file. The format details will be discussed below.
 A final visualization result will be displayed in a local HTML form with attached JS components: We have already written required data gathering result handlers and normalized it using Python data structures.
Let's apply this: General NeXt UI topology representation looks as follows: As you can see, this is a JSON object which can be mapped to a Python data structure of the following format: {'nodes': [], 'links': []}. 
We will bring all our data together into that.
In addition, let's take device model into account for node icon selection to handle missing LLDP capabilities case.
Node objects will also be populated with some extended attributes derived from GET_FACTS (model, S/N, etc) to enrich the topology view. Then we should just write this Python topology dictionary into the topology.js file. A standard  json  module will work for this perfectly providing a readable and formatted output: Now let's finally run main.html and see our visualization Hello World:

Looks correct. All known nodes and links between them are displayed.
The nodes are draggable in any direction. On mouse click on nodes and links, a Next UI tooltip menu appears. It contains all the attributes we previously passed into node and link topology objects in Python:

Not so bad. There is also room for improvement. We will return to this some later on. Let's implement a solution for the second part of the task for now. A bonus task is to detect and visualize topology changes.
To get it done, some additions will be required: Diff HTML-form will look as follows: All we need to read and write topology cache files: Topology diff analysis steps: Extract the node and link attributes for comparison from the current and cached topology dictionaries.

Node format:
(node object with all attributes, (hostname,))
Link format:
(link object with all attributes, (source_hostnme, source_port), (dest_hostname, dest_port))
Both node and link format allows further extension. Compare the extracted node and link objects. Link format permutations should be considered.

Diff result for nodes and links will be written into two dictionaries in the following format: Merge current and cached topology with the diff data.
The resulting topology will be written to diff_merged_topology dictionary.
Deleted node and link objects will be extended with the is_dead attribute. For a better visual experience, deleted node icons will be customized (Next UI changes for that will be discussed below).
New node and link objects will be extended with the is_new attribute.
 Let's code: get_topology_diff implements comparison of two arbitrary topology dictionaries of a valid format
This allows us to implement a topology cache versioning in the future.
Let's also implement a console diff print function: Finally, let's summarize the code pieces we have written above into a dedicated main() function.
Here are a fairly self-documenting code and my personal answer to a question "why not Ansible": To begin with, let's restrict the access to dist-rtr01 and run the script. Resulting topology:


Then let's recover the access to dist-rtr02, restrict the access to edge-sw01, and execute the script again.
The previous version becomes cached. The current topology looks as follows:
 A console output on the last run: Everything looks correct. The output matches the changes.
To visualize the diff topology properly, we will make some adjustments in next_app.js below.
 Most of the improvements below were made based on the examples in the Next UI documentation and tutorials. In order to add the interface labels, let's extend the standard nx.graphic.Topology.Link class: A customized link class can now be listed in the topo topology object properties.
Let's also highlight the links on the diff topology. New links will be green, deleted links will be red and dashed. Next UI already contains a broad set of default icons for network devices.
However, you are free to add custom icons based on your requirements. For deleted nodes, we need something special.
To add a new icon, you should put an image into a directory accessible by Next UI and initialize it in topo object. We have actually done everything we need. Let's open diff_page.html and see how the changes we have done before look like:


A topology view speaks for itself, doesn't it? By default, a node tooltip contains excessive information like internal node id and coordinates.
In NeXt UI, it is possible to customize it for better readability and usability.
Let's include the following information for now: In order to implement this, let's extend a standard nx.ui.Component class and build a simle HTML form inside of it: Now the custom class version can be listed in topo topology object properties: This is what we see on mouse click on the node after that:
 As discussed before, the default Next UI data processor is 'force'. It relies on the best effort algorithm spreading the Nodes so they would be as distant from each other as possible.
 This logic produces a proper layout even for complex hierarchical network topologies but the topology layers may not be oriented as desired. For sure, you can drag them manually afterward. However, this is not our way.

Fortunately, there are some built-in tools to work with layers in Next UI.
Let's use a new numeric layerSortPreference attribute inside nodes in our Next UI application.
The logic defining this value can be implemented outside the visualization application on the topology object generation stage in this case. Next UI would just order the layers in a way we tell it to. This is a more scalable approach.
 Let's add some functions to be able to switch between layouts: Map these function to button elements on main.html and diff_page.html: Let's improve the generate_topology.py script and add some additional attributes to Nornir hosts file to implement an automatic node hierarchy calculation.
The script will define an ordered list of human-friendly layer names and make a conversion into numeric values: A numeric layer sort order for the layer will be defined by its relative position in NX_LAYER_SORT_ORDER.
Important: NeXt UI interprets 0(null) as undefined. Valid layer indexes start with 1.
 The device layer will be based on its role attribute inside the Nornir Hosts Inventory file.
data field allows us to specify a list of attributes with arbitrary names: Any attribute inside host data can then be called in Python as a dict key as follows: To reflect the changes, let's update our Python code. A new nr_role node attribute will be appended along with others into global_facts inside the normalize_result function: Then we should read this attribute during node object generation inside the generate_topology_json function: And now we can control the chaos align the layers horizontally and vertically on button click. This is how it looks like:  The final project structure looks as follows: First of all, thank you for reading. I hope you enjoyed it. In this article, I tried to reproduce and document the solution creation stages and my considerations behind them. A full source code is available on my GitHub page. The solution took first place on the Marathon based on attendees' and organizers' votes. And equally important, it has a potential for reuse and scalability(spoiler: I developed a Netbox plugin reusing the core code from this project). How do you like the solution? What could be improved? How would you solve this task?
Please feel free to share your own experience and thoughts. Trust me. I'm an Engineer.