Everything I see in this area tends to be "here's how to configure ansible to log into a couple of switches and do a show run".

Whoopy do.

This is a refreshing blog, at least confiming my suspicions that nobody doing network automation actually says anything about it.

My team (of 2) run an internal mixed-vendor network of about 150 switches and routers. New sites are deployed from templates, that's easy enough, although it's not "automation". Testing and rollback is pretty tricky when you're talking about physical bits of hardware.

I’ve written our own internal stuff to manage part of the network - managing firewall/nat rules on fortigates for example, and obviously theres a great deal of useful open tools out there to report the current state of the network (rancid, librenms, portmappers), but to actually make atomic attributed changes is tricky and relies on the person say changing an interface from vlan 301 to vlan351 to record the reason in the change log (description says what should be on the end and a jira reference for more about it)

Currently I’m trying to automate vlan creation on our larger sites. We tend to have a standard dual pair distribution (not stacked - that’s a single point of failure I’ve seen take us out time and time again) vrrp based site with say 30 access switches hanging off.

Creating a new vlan involves
1) adding to ipam (vlan and the ip range, then adding the individual hosts)

2) creating the vlans on the core switch

3) adding the vlans to the interface between the two cores

4) adding the l3 interface (including setting the right priority for which core I want to be the normal master), setting spanning tree and pim priorities to match

5) configuring dhcp

6) adding vlans to specific trunk ports to the access switches

7) adding vlans to the access switches and their uplinks

And then often

8) adding vlans to vm trunk interfaces

9) creating networks on vms

It’s a cesspit of manual errors, and takes an age, almost as bad as punching holes in the firewalls.

Automation for automating bgp adjacencies? Meh. There’s very little time saving. For setting vlans on access ports on agile deployments? Again we have web based tools for that to let end users do it, but that's not a pr/test/commit/apply/test/rollback workflow.

Currently my CI process pulls the show run from the Ciscos and works out what needs to happen to perform the steps, if any, then put out the instructions as an artifact. I guess next step would be a manual push of the artefact from the CI, and then testing that the IPs actually are resolvable, maybe create a test VM machine on the network (in the case the vlan is pushed to a VM host)

Trouble is the testing and rollback process is extremely tricky, especially with Cisco IOS based kit. Network kit can’t be easily or cheaply virtualised (I tried using terraform to deploy a small copy of our network onto an AWS bare-metal system, but the $5/hour you need for bare metal to run gns3). We can't afford to have an identical network setup, so any test system would not scale to the size where automation would tease out problems.

I'd love to see how other people do it, but when I talk to the department which looks after our external networking they just ssh into switches and make changes and hope for the best. Online as I say it's all people with ansible running "show version" and then saying "the rest is left as an exercise to the reader". Vendors want to sell proprietary management systems which don't seem to be the kinds of things you can drive from a gitops workflow. Maybe some SDN overlay network can be automated in some fashion, but you're basically learning a new CLI rather than implementing a test-based automation system.