trents_blog/docs/posts/ansible-kvm-router-lab-part-5.md

---
title: "Ansible KVM Router Lab Part 5"
date: 2021-10-17
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 30
---
date: 2021-10-17

## Introduction
This is Part 5 of a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.

[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}
is an overview.

In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I break down the script 
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.

In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and
[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.

In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.

In this post I explain how I use Ansible to finish constructing the lab.

In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.

## Setup Ansible
* Configure ansible host file
```cfg
# ~/.ansible.cfg
[defaults]
inventory = ~/router-lab/ansible/hosts.yml
```
* Setup bashrc
```bash
# ~/.bashrc
export LIBVIRT_DEFAULT_URI="qemu+ssh://<user>@<server>/system"

alias ansible-pb=anspb
anspb() {
  ANS_DIR=~/router-lab/ansible/playbooks;
  echo Changing to "${ANS_DIR}" and executing: ansible-playbook "${@}"
  (cd $ANS_DIR || exit ; ansible-playbook "${@}")
}
```
* install apps
```bash
apt install ansible ansible-lint
```

## Run Ansible
```shell
ansible-pb build_out_routers.yml -K
```
or if you want to first update all the clients
```shell
ansible-pb update_and_build.yml -K
```
## Ansible Tasks
This is an explaination of the tasks in the Ansible Playbook.
Playbooks are executed from top to bottom.
### Install `dnsmasq`, `iptables-persistent`
This task is only run against the first and second lab clients as
they are the routers.
### Install `traceroute`
Traceroute is parsed in a later task to confirm that traffic is
following the correct route.
(Also incidentally installs `needrestart` and `screen`.)
### Backup `/etc/network/interfaces`
This is a simple bash command that tests if `/etc/network/interfaces.bak`
exists, and if not creates it.
### Update Network Config
This task updates `/etc/network/interfaces` in all the lab clients
to describe the network interfaces needed to connect to each other.

For instance, here is the new `/etc/network/interfaces` file for _dnettwo_.
```cfg
# /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp1s0
iface enp1s0 inet dhcp

# The primary network interface
allow-hotplug enp7s0
iface enp7s0 inet dhcp

auto enp8s0
iface enp8s0 inet static
        address 10.4.4.1
        network 10.4.4.0
        netmask 255.255.255.0
        broadcast 10.4.4.255
```
### Backup `/etc/dnsmasq.conf`
This is a simple bash command that tests if `/etc/dnsmasq.conf.bak`
exists, and if not creates it. (only applies to the two router clients)
### Configure `dnsmasq`
This task copies the templates for `/etc/dnsmasq.conf` to each of
the two router clients.

`dnsmasq` is used to provide _DHCP_ (and name resolution).
For instance, here is the new `/etc/dnsmasq.conf` for _dnetone_.
```cfg
# /etc/dnsmasq.conf
dhcp-range=10.5.5.50,10.5.5.150
listen-address=127.0.0.1, 10.5.5.1
```
### Configure Network _ifup_
This applies to all the lab clients except for the first one,
changes the default route. A bash script is copied from
template to `/etc/network/if-up.d/ifup-script`.

For instance here is `ifup-script` for _dnetthree_.
```bash
#!/bin/bash
# /etc/network/if-up.d/ifup-script

default_dev="$(ip route | head -1 | awk '{print $5}')"
echo "${default_dev}"

if [ "${default_dev}" == "enp1s0" ]
then
  ip route del default via 10.55.44.1 dev enp1s0
fi

if [ "${default_dev}" != "enp7s0" ]
then
  ip route add default via 10.4.4.1 dev enp7s0
fi
```
### Restart Network and `dnsmasq`
This is sequential:

1. _enp7s0_ is restarted on _dnet_
2. `dnsmasq` is restarted on _dnetone_, offering service on _enp7s0_
3. _enp7s0_ and _enp8s0_ are restarted on _dnettwo_, thus soliciting dhcp service on _enp7s0_, and triggering `/etc/network/if-up.d/ifup-script`
4. `dnsmasq` is restarted on _dnettwo_, offering service on _enp8s0_
5. _enp7s0_ is restarted on _dnetthree_, _dnetfour_, and _dnetfive_, thus soliciting dhcp service on _enp7s0_, and triggering `/etc/network/if-up.d/ifup-script`

### Backup `/etc/sysctl.conf`
This is a simple bash command that tests if `/etc/sysctl.conf.bak`
exists, and if not creates it. (only applies to the two router clients)

### Enable _ipv4 forwarding_
This is a simple bash command that uncomments the option for _ipv4 forwarding_
in `/etc/sysctl.conf`, applies only to the two routers.
```cfg
# /etc/sysctl.conf
...
# this
#net.ipv4.ip_forward=1
...
# becomes this
net.ipv4.ip_forward=1
...
```

### Start _ipv4 forwarding_
This simple bash command starts _ipv4 forwarding_, applies only
to the two routers.
```bash
bash -c "sysctl -w net.ipv4.ip_forward=1"
```

### Configure `iptables` _workaround_
This applies only to the two router clients.
From `iptables`'s point of view, the ansible connection isn't a RELATED INPUT
connection, thus it is necessary to bring up a firewall in a two-step
process that involves first ACCEPTING RELATED OUTPUT connections in a workaround.

From ansible template, the following is copied to `/dev/shm/iptables_workaround`
```iptables
# /dev/shm/iptables_workaround
*filter
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]

-A INPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATED
-A OUTPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATED

COMMIT
```

### Apply `iptables` _workaround_
This applies only to the two router clients.
The following command is dispatched to apply the above _iptables_workaround_:
```bash
bash -c "iptables-restore < /dev/shm/iptables_workaround"
```

### Configure `iptables`
This applies only to the two router clients.

From ansible template the following is copied to `/etc/iptables/rules.v4` on _dnetone_.
```iptables
*nat
-A POSTROUTING -o enp1s0 -j MASQUERADE
COMMIT

*filter
-A INPUT -i lo -j ACCEPT
# allow ssh, so that we do not lock ourselves
-A INPUT -i enp1s0 -p tcp -m tcp --dport 22 -j ACCEPT
# allow incoming traffic to the outgoing connections,
# et al for clients from the private network
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# prohibit everything else incoming
-A INPUT -i enp1s0 -j DROP
COMMIT
```

From ansible template the following is copied to `/etc/iptables/rules.v4` on _dnettwo_.
```iptables
*nat
-A POSTROUTING -o enp7s0 -j MASQUERADE
COMMIT

*filter
-A INPUT -i lo -j ACCEPT
# allow ssh, so that we do not lock ourselves
-A INPUT -i enp7s0 -p tcp -m tcp --dport 22 -j ACCEPT
# allow incoming traffic to the outgoing connections,
# et al for clients from the private network
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# prohibit everything else incoming
-A INPUT -i enp7s0 -j DROP
COMMIT
```

### Apply `iptables` firewall
This applies only to the two router clients.
The following command is dispatched to apply the above from `/etc/iptables/rules.v4`:
```bash
bash -c "iptables-restore < /etc/iptables/rules.v4"
```

### `traceroute` test
The following script is dispatched to _dnettwo_:
```bash
#!/bin/bash

RESULT="$(traceroute 8.8.8.8)"

FIRST_HOP="$(echo "${RESULT}" | head -2 | tail -1 | awk '{print $2}')"

if [ "${FIRST_HOP}" == "10.5.5.1" ]
then
  exit 0
else
  exit 1
fi
```
The following script is dispatched to _dnetthree_, _dnetfour_, and _dnetfive_:
```bash
#!/bin/bash

RESULT="$(traceroute 8.8.8.8)"

FIRST_HOP="$(echo "${RESULT}" | head -2 | tail -1 | awk '{print $2}')"

if [ "${FIRST_HOP}" != "10.4.4.1" ]
then
  exit 1
fi

SECOND_HOP="$(echo "${RESULT}" | head -3 | tail -1 | awk '{print $2}')"

if [ "${SECOND_HOP}" == "10.5.5.1" ]
then
  exit 0
else
  exit 1
fi
```

## To Be Continued
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.