Automation forNetworks and more
Christoff Visser – IIJ Lab
christoff@iij.ad.jp1

Introduction• Crash course for commonly used tools for network automation• Not an in-depth tutorial• Give you an idea how you could use automation in your day to day• Small version of tutorial hosted by myself and Anurag Bhatia at APNIC 56• Resources will be available in Gitlab• Slides• Tutorials• Instructions• Link: https://bit.ly/iijlab_automate_techtrend2023
2

Agenda• Docker + Containers• Ansible• Gitlab + CI/CD• ChatGPT3

Automation as a tool4Automation as a tool5Automation as a tool6

Why do we automate?
• Increase productivity• Reduce cost
• Minimise errors• Increased capability

7Why do we should you automate?
• Increase productivity• Reduce cost
• Minimise errors

Spend less time on
boring and tedious tasks• Increased capability

8Event driven automation
Process

Observe

Change9Docker and containers

10Typical software deployment workflow

Download
package

Installdependencies

Configuredatabase

Configurepackage

Deploy!

11

Challenges with traditional software
deployment1. Takes time to go through documentation, install package andmaintain it
2. Time consuming process to transfer software to a different server
3. Prone to errors and mistakes
4. “Works on my system”
5. Dependency conflicts
12

Introducing
containers

13

Virtual Machines vs Containers

14

Container based software deployment

Download
image

Configure

Deploy!

15

Building a Container - ansible

Choose OS

Installdependencies

Install package

Configurepackage

Deploy!

16

Running docker ad-hoc17

Docker ad-hoc challenges• Gets complicated the more arguments are passed• Hard to remember all previously used arguments• Easy to misconfigure when running multiple ad-hoc containers

18

docker-compose.yml• Single yaml text file for multiple containers• Easier to read and includes all instructions for all containers• Simplify creating/attaching volumes to bind to• Ensure you’re exposing only what you need to• Simplify upgrading and maintaining containers
19Docker compose - smokeping

20

Notes on backup• Docker containers are reproducible. No need to backup• User data is stored using volumes or bind mounts• Only these need to be backed up• Popular tools like Restic Duplicati (can be run as dockercontainer)• Always encrypt data before storing it on the cloud21

Some containers to play with
• RIPE Atlas - https://hub.docker.com/r/jamesits/ripe-atlas• HTML 5 speedtest - https://hub.docker.com/r/adolfintel/speedtest• iperf3 - https://hub.docker.com/r/networkstatic/iperf3• Nextcloud - https://hub.docker.com/_/nextcloud• Docker-speedtest-grafana - https://github.com/frdmn/docker-speedtest-grafana• Kerberos - https://doc.kerberos.io/opensource/installation#docker• Nginx Proxy Manager - https://nginxproxymanager.com/• Linux-server.io - Many great images actively maintained by the open source community
22

For more information:• Gitlab repo:• More in detailed information• Tutorials and labs to follow• Guide for minimising the size ofdocker images• Link to docker section

23

24

Automation• Execute a task without active human interaction• Commonly done with scripting (bash, python, etc.)• Focus typically is single task and not overall workflow

25Orchestration• Execute a workflow without human interactions• Requires a platform to achieve complex workflow executions• Focus workflow involving multiple tasks, with scheduling,different devices/services etc.
• Combination of automation
26

Possible tasks to automate/template• Basic configuration - hostname, SNMP, NTP, AAA, DNS resolvers
etc• Configuration of firewall• Configuration of BGP session• Local account management• Configuration of interface• Configuration of backup• And anything else you do regularly
27Orchestration tool options• Ansible• Saltstack• Puppet• Chef• And more…
28Introduction to Ansible• Popular open source tool for Orchestration• Agentless i.e. target devices do not need to run any software• Python based• Supports various networking devices – Cisco, Juniper, Mikrotik,Huawei, VyOS, Ubnt, Dell and more
• Supports almost every OS – Linux, MacOS, Windows via WSL• Most are “idempotent”: Only do something when a change isrequired• By default comes with CLI & not web UI (Web UI can be added via
Ansible AWX or Ansible semaphore)29

Ansible Key Concepts

Image source:
https://geekflare.com/ansible-basics/
30

Writing a playbook• Written in yaml format (.yml). It’s easy to read format for humans aswell as computers• Yaml can be written in any text editor but there’s risk of breaking
yaml syntax and thus a basic code editor like VSCode is a goodoption• One can write playbook on locally installed VSCode on Linux/OSX/Windows & save it on remote server using SSH within VSCode• Use a Ansible extension in VSCode to ensure it throws error if syntaxis broken
31Sample playbook - simple--- hosts: routerstasks:- name: Setup hostname of Routervyos.vyos.vyos_system:
host_name: "{{ inventory_hostname }}"32

Sample playbook - more complex

33

Sample playbook – even more complex

34

For more information• Gitlab repo:• More in detailed information• Tutorials and labs to follow• Link to Ansible Section

35Gitlab and CI/CD

36

Introducing git

37

Introducing git

38

Distributed Workflows

Image source: https://www.git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows

39

Where to store the repository?
• GitHub, Gitlab, Bitbucket, AWS CodeCommit• Gitlab• Self-host your own instance• Some more freedom with the CI/CD

40

What is CI/CD• Continuous Integration (CI)• Continuous Delivery (CD)• Continuous Deployment (CD)41

Where does the code run• Code typically runs inside a docker container as a job• One can use available popular containers like alpine, ubuntu,centos, or application specific containers• One can also build own container using base image of any of
the available containers if installing multiple packages

42

And where does the job run?
• Typically a runner - can be shared runner offered by popular hosted Git
providers like Gitlab, Github etc and also dedicated runners which you canhost on your machine (desktop/server)• Runner can be a program installed & running on machine or simply adocker image with special permissions• One can have multiple runners configured in a project & use them as
needed across various tasks. E.g task 1 on runner on server1, task 2 onrunner on server2 etc• Good idea to have basic understanding of docker ecosystem to makeefficient use of CI/CD
43

Key Objective• Make use of extremely low code, fast to deploy tool like Ansible
to automate or semi-automate repetitive tasks• Trigger Ansible as a docker container running Ansible on runnerof your choice• Trigger (Ansible + Docker) via CI/CD pipelines

44

Stage & Jobs
• Config is divided in stages• Each stage can have one or more jobs which run in parallel (by default)• Stages run sequentially• Any job can have dependency on any other job if neededTypical design of pipeline• Build containers• Deploy containers• Compile code

• Deploy code
• Test code incontainers• Deploy toproduction• Interact withproduction system• e.g., Revert toprevious state
47

Sample .gitlab-ci.yml

48

Final Workflow result

49For more information• Gitlab repo:• More in detailed information• Introduction to git slides• Tutorials and labs to follow• Link to CI/CD section50ChatGPT

58

The 5 stages of grief

59

The 5 7 stages of grief ChatGPT

https://twitter.com/solomania/status/1625458520811339777

60

Benefits vs Risks

61

Benefits vs Risks• Automate mundane things• Text generation• Virtual writing assistant• Generate summaries• Write, debug and explain code

62Benefits vs Risks• Automate mundane things• Text generation• Virtual writing assistant• Generate summaries• Write, debug and explain code
• Good for explaining andintroductions to topics• Misinformation anddisinformation (Hallucinating)• Privacy concerns• Outputs can be “generic”• Biased• Predictive not Smart• General not single purpose• Not good for complex projects
63Benefits vs Risks• Automate mundane things• Text generation• Virtual writing assistant• Generate summaries• Write, debug and explain code
• Good for explaining andintroductions to topics• Misinformation anddisinformation (Hallucinating)• Privacy concerns• Outputs can be “generic”• Biased• Predictive not Smart• General not single purpose• Not good for complex projects
64Benefits vs Risks• Automate mundane things• Text generation• Virtual writing assistant• Generate summaries• Write, debug and explain code
• Good for explaining andintroductions to topics• Misinformation anddisinformation (Hallucinating)• Privacy concerns• Outputs can be “generic”• Biased• Predictive not Smart• General not single purpose• Not good for complex projects
65Benefits vs Risks• Automate mundane things• Text generation• Virtual writing assistant• Generate summaries• Write, debug and explain code
• Good for explaining andintroductions to topics• Misinformation anddisinformation (Hallucinating)• Privacy concerns• Outputs can be “generic”• Biased• Predictive not Smart• General not single purpose• Not good for complex projects
66Benefits vs Risks• Automate mundane things• Text generation• Virtual writing assistant• Generate summaries• Write, debug and explain code
• Good for explaining andintroductions to topics• Misinformation anddisinformation (Hallucinating)• Privacy concerns• Outputs can be “generic”• Biased• Predictive not Smart• General not single purpose• Not good for complex projects
67

How to not use ChatGPT• Open ChatGPT• ????
• Profit!!!

68How to use ChatGPT?

69

70

71

Conclusion

72

Conclusion - backup• Automation is a tool – not a crutch• Can be very powerful• But with great power…

• Docker containers• Self contained applications• Portable and lightweight• Ansible• Clean up your scripts to be more readable• Easy to backup using git• Git + CI/CD• Add accountability• Automate your workflow

73

Questions?
• Gitlab public wiki
• Slides• Examples• Further resources• Link: https://bit.ly/iijlab_automate_techtrend2023• Join us at APNIC 56!
• https://academy.apnic.net/en/events?id=a0B2e000000eANMEA2• christoff@iij.ad.jp74