trent
/
trents_blog
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

add ansible-kvm-router-lab

This commit is contained in:
Trent Palmer 2021-10-18 03:26:11 -07:00
parent ff573d8b18
commit e56519937a
7 changed files with 880 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

124
docs/posts/ansible-kvm-router-lab-part-1.md Normal file
View File

@ -0,0 +1,124 @@
---
title: "Ansible KVM Router Lab Part 1"
date: 2021-10-16
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 26
---
date: 2021-10-16
## Introduction
This is a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.
This achieves the ability to quickly set up a router lab for the
purposes of experimenting with iptables, or whatever else you
want to use for routing or firewalls.
This is also, for myself, an opportunity to learn ansible.
In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I break down the script
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.
In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and [shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"},
I explain the ansible playbook tasks used to finish building the lab.
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.
## Networking
I begin by setting up a new network in libvirt, which will serve
as an _out-of-band_ network for connecting to the lab virtual machines.
This is covered in a
[previous blog post](/posts/add-kvm-network-with-virsh/){target="_blank"}.
## Overview
The lab consists of seven virtual machines.
I begin by creating a _base_ Debian 11 virtual machine called `dnet` by connecting to
my physical server using `virt-manager`.
After creating a _base_ virtual machine, the next step is to create
a clone from which to work. I call this machine `dcon`.
The client clones consist of 5 virtual machines named
`dnetone` through `dnetfive`. Once set up, all five virtual machines
are reachable through the _out-of-band_ network.
But there are also two bridge networks connecting the client clones
to each other. The first and second clones are connected to each other on
the **upper** bridge network, with the first clone acting as a router for the
second. The second, third, fourth, and fifth clones are connected to each
other on the **lower** bridge network, with the second clone acting
as a router for the third, fourth, and fifth clones. Traffic from the
second clone will go through the first clone to reach the internet, and
traffic from the third, fourth, and fifth clones will go through
the second clone and then through the first clone to reach the internet.
DHCP is handled by dnsmasq on the first clone and the second clone.
## Resources
For ansible I used the
[ansible documentation](https://docs.ansible.com/ansible/latest/index.html){target="_blank"}.
This
[blog post](https://www.brianlinkletter.com/2019/02/build-a-network-emulator-using-libvirt/){target="_blank"}
by Brian Linkletter is also really helpful.
## Control Node Setup
* Create a control node by cloning the _base_ virtual machine.
```shell
virt-clone --original dnet --name dcon --auto-clone
```
* Configure ansible host file
```cfg
# ~/.ansible.cfg
[defaults]
inventory = ~/router-lab/ansible/hosts.yml
```
* Setup bashrc
```bash
# ~/.bashrc
export LIBVIRT_DEFAULT_URI="qemu+ssh://<user>@<server>/system"
alias ansible-pb=anspb
anspb() {
ANS_DIR=~/router-lab/ansible/playbooks;
echo Changing to "${ANS_DIR}" and executing: ansible-playbook "${@}"
(cd $ANS_DIR || exit ; ansible-playbook "${@}")
}
```
* configure Vim or similar for editing bash and python
* install apps
```bash
apt install ansible ansible-lint libvirt-clients
apt install --no-install-recommends virtinst
```
The control node needs root ssh access to the _base_ virtual machine so
that it will have root ssh access to the clones.
## To Be Continued
In the next blog post,
[Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I begin breaking down the bash scripts which build out the lab, beginning with
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.

105
docs/posts/ansible-kvm-router-lab-part-2.md Normal file
View File

@ -0,0 +1,105 @@
---
title: "Ansible KVM Router Lab Part 2"
date: 2021-10-16
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 27
---
date: 2021-10-16
## Introduction
This is Part 2 of a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.
[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}
is an overview.
In this post I begin breaking down the bash scripts which build the router
lab, beginning with
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.
In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and [shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"},
I explain the ansible playbook tasks used to finish building the lab.
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.
## `build_vms.bash`
### `check_uid "0"`
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash)
begins by making sure that it is run as the _root_ user. This is because root is required to
ssh into the clones to change their hostnames, machine-ids, and host-ssh-keys. You can call
with `sudo bash build_vms.bash`.
For this same reason, `~/.ssh/known_hosts` is useless so it is deleted (and then rebuilt).
### `function build_vms()`
Next,
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}
calls `build_vms`, which loops over the array of MACHINES, which is an
array that holds that names of the lab clients, passing each name in turn
to `create_vm`.
`create_vm` creates the virtual machine if it does not already exist, using `virt-clone`, and
then calls `start_vm` to start it.
`start_vm` is exported from
[env.bash](https://github.com/TrentSPalmer/router-lab/blob/master/env.bash.example){target="_blank"},
and per parsing the output of `virsh list --inactive`, starts the virtual machine if it is not running.
### `function set_hostnames()`
Next,
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}
calls `set_hostnames`, which simultaneously calls `set_hostname` on the entire MACHINES array. `set_hostname`
in turn waits for the virtual machine to be fully booted, then updates the files `/etc/hostname` and
`/etc/hosts`, and then reboots the virtual machine to apply the new hostname.
### `function confirm_hostnames()`
`confirm_hostnames` simultaneously calls `confirm_hostname` against the entire MACHINES array.
`confirm_hostname` waits for the virtual machine to be fully booted, then confirms the correct
_hostname_ in `/etc/hostname`.
### `function confirm_hostnames_in_hosts()`
`confirm_hostnames_in_hosts` works almost exactly the same as `confirm_hostnames`, but this time
the file `/etc/hosts` on the virtual machine is grepped for the proper _hostname_, and corrected
if necessary.
### `function reset_hosts_ssh_keys()`
`reset_hosts_ssh_keys` simultaneously calls `reset_host_ssh_keys` against the MACHINES array,
which in turn compares the host_ssh_key of the virtual machine against the _bas3_ virtual
machine, and if necessary deletes `/etc/ssh/ssh_host_*`, generates new host_ssh_keys, restarts
`sshd` on the virtual machine, removes `~/.ssh/known_hosts`, and then reruns itself in
order to confirm the new host_ssh_keys.
### `function reset_machine_ids()`
`reset_machine_ids` simultaneously calls `reset_machine_id` against the entire MACHINES array,
which in turn checks the _machine-id_ of the virtual machine to make sure that it is different
than the _machine-id_ of the _base_ virtual machine, and if necessary deletes
`/etc/machine-id` and `/var/lib/dbus/machine-id` and recreates them.
## To Be Continued
In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and
[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.

104
docs/posts/ansible-kvm-router-lab-part-3.md Normal file
View File

@ -0,0 +1,104 @@
---
title: "Ansible KVM Router Lab Part 3"
date: 2021-10-16
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 28
---
date: 2021-10-16
## Introduction
This is Part 3 of a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.
[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}
is an overview.
In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I break down the script
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.
In this post I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and
[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"},
I explain the ansible playbook tasks used to finish building the lab.
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.
## `define_bridge_networks.bash`
The router-lab has two bridge networks, in addition to the initial out-of-band
network which is used to contact the virtual machines directly.
### `check_uid "${USER_UID}"`
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash define_bridge_networks.bash`.
### `function define_bridge_networks()`
`define_bridge_networks` calls `define_bridge_network` twice, once for each of the
upper bridge and the lower bridge. `define_bridge_network` parses the output of
`virsh net-list --all` to determine if the network is defined yet. If not,
`virsh net-define vm_router_lab_lower_bridge.xml` or
`virsh net-define vm_router_lab_upper_bridge.xml` are invoked as necessary.
`define_bridge_network` then recursively calls itself for confirmation.
Links for
[vm_router_lab_upper_bridge.xml](https://github.com/TrentSPalmer/router-lab/blob/master/vm_router_lab_upper_bridge.xml){target="_blank"}
and
[vm_router_lab_lower_bridge.xml](https://github.com/TrentSPalmer/router-lab/blob/master/vm_router_lab_lower_bridge.xml){target="_blank"}.
### `function start_bridge_networks()`
`start_bridge_networks` calls `start_bridge_network` twice, once for each of the
upper and the lower bridge. `start_bridge_network` in turn parses the output of
`virsh net-info vm_router_lab_upper_bridge` and/or `virsh net-info vm_router_lab_lower_bridge`
to determine if the cooresponding network is running, and if not invokes
`virsh net-start vm_router_lab_upper_bridge` or `virsh net-start vm_router_lab_lower_bridge`,
and then recursively calls itself again for confirmation.
### `function autostart_bridge_networks()`
`autostart_bridge_networks` is nearly identical to `start_bridge_networks`, but
`virsh net-autostart vm_router_lab_upper_bridge` or `virsh net-autostart vm_router_lab_lower_bridge`,
are invoked in order to mark the cooresponding network to autostart.
## `shutdown_vms.bash`
After creating the upper and lower bridge networks, it is necessary to shut down
the lab clients before connecting the lab clients to the bridge networks. This is
because network interfaces must be _permanently_ added to the lab client definitions.
### `check_uid "${USER_UID}"`
[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash shutdown_vms.bash`.
### `function shutdown_vms()`
`shutdown_vms` simultaneously calls `shutdown_vm` on the entire MACHINES array.
`shutdown_vm` in turn parses the output of `virsh list --state-running` to determine if
the virtual machine is running, and if so invokes `virsh shutdown <vm>`. `shutdown_vm`
then recursively calls itself to confirm that the virtual machine is indeed shut down.
## To Be Continued
In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and
[rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.

107
docs/posts/ansible-kvm-router-lab-part-4.md Normal file
View File

@ -0,0 +1,107 @@
---
title: "Ansible KVM Router Lab Part 4"
date: 2021-10-17
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 29
---
date: 2021-10-17
## Introduction
This is Part 4 of a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.
[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}
is an overview.
In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I break down the script
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.
In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and [shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.
In this post I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and
[rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"},
I explain the ansible playbook tasks used to finish building the lab.
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.
## `connect_vms_to_bridges.bash`
Aside from the _out-of-band_ network which can be used to contact the lab clients
directly, the lab clients are connected to each other using two bridge networks.
As explained in
[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"},
lab clients one and two are connected to the upper bridge, and lab clients two, three, four,
and five are connected to the lower bridge with the first client acting as a router
for the second client, and the second client acting as a client for the third, fourth, and
fifth clients.
### `check_uid "${USER_UID}"`
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash connect_vms_to_bridges.bash`.
### `function connect_upper_bridge()`
`connect_upper_bridge` calls `connect_vm_to_bridge` against the first lab client and the upper bridge,
and again against the second lab client and the upper bridge.
### `function connect_lower_bridge()`
`connect_lower_bridge` calls `connect_vm_to_bridge` against the second lab client and the lower bridge,
against the third lab client and the lower bridge, against the fourth lab client and the lower bridge,
and against the fifth lab client and the lower bridge.
### `function connect_vm_to_bridge()`
`connect_vm_to_bridge` parses the output of `virsh dominfo` against the intended lab client to verify that it
is shutdown. Then, if the intended lab client is shutdown, `connect_vm_to_bridge` parses the output of
`virsh domiflist` to find out if the intended new interface is yet defined, and if not invokes
`virsh attach-interface <vm> --type network --source <bridge network>`. Finally, `connect_vm_to_bridge`
recursively calls itself for verification.
## `start_vms.bash`
After defining the new network interfaces for all the lab clients, you can boot them.
### `check_uid "${USER_UID}"`
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash start_vms.bash`.
### `function start_vms()`
`start_vms` calls `start_vm` against the entire MACHINES array, simultaneously.
`start_vm` is exported from
[env.bash](https://github.com/TrentSPalmer/router-lab/blob/master/env.bash.example){target="_blank"},
and per parsing the output of `virsh list --inactive`, starts the virtual machine if it is not running.
## `rebuild_known_hosts.bash`
You will need to have a valid list of known_hosts in order for ansible to connect to the lab
clients.
The script deletes `~/.ssh/known_hosts` and then initiates an ssh connection to all the router lab clients
in order to repopulate `~/.ssh/known_hosts`.
### `check_uid "${USER_UID}"`
[rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash rebuild_known_hosts`.
## To Be Continued
In the next blog post,
[Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"},
I explain the ansible playbook tasks used to finish building the lab.

311
docs/posts/ansible-kvm-router-lab-part-5.md Normal file
View File

@ -0,0 +1,311 @@
---
title: "Ansible KVM Router Lab Part 5"
date: 2021-10-17
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 30
---
date: 2021-10-17
## Introduction
This is Part 5 of a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.
[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}
is an overview.
In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I break down the script
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.
In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and
[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.
In this post I explain how I use Ansible to finish constructing the lab.
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.
## Setup Ansible
* Configure ansible host file
```cfg
# ~/.ansible.cfg
[defaults]
inventory = ~/router-lab/ansible/hosts.yml
```
* Setup bashrc
```bash
# ~/.bashrc
export LIBVIRT_DEFAULT_URI="qemu+ssh://<user>@<server>/system"
alias ansible-pb=anspb
anspb() {
ANS_DIR=~/router-lab/ansible/playbooks;
echo Changing to "${ANS_DIR}" and executing: ansible-playbook "${@}"
(cd $ANS_DIR || exit ; ansible-playbook "${@}")
}
```
* install apps
```bash
apt install ansible ansible-lint
```
## Run Ansible
```shell
ansible-pb build_out_routers.yml -K
```
or if you want to first update all the clients
```shell
ansible-pb update_and_build.yml -K
```
## Ansible Tasks
This is an explaination of the tasks in the Ansible Playbook.
Playbooks are executed from top to bottom.
### Install `dnsmasq`, `iptables-persistent`
This task is only run against the first and second lab clients as
they are the routers.
### Install `traceroute`
Traceroute is parsed in a later task to confirm that traffic is
following the correct route.
(Also incidentally installs `needrestart` and `screen`.)
### Backup `/etc/network/interfaces`
This is a simple bash command that tests if `/etc/network/interfaces.bak`
exists, and if not creates it.
### Update Network Config
This task updates `/etc/network/interfaces` in all the lab clients
to describe the network interfaces needed to connect to each other.
For instance, here is the new `/etc/network/interfaces` file for _dnettwo_.
```cfg
# /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
allow-hotplug enp1s0
iface enp1s0 inet dhcp
# The primary network interface
allow-hotplug enp7s0
iface enp7s0 inet dhcp
auto enp8s0
iface enp8s0 inet static
address 10.4.4.1
network 10.4.4.0
netmask 255.255.255.0
broadcast 10.4.4.255
```
### Backup `/etc/dnsmasq.conf`
This is a simple bash command that tests if `/etc/dnsmasq.conf.bak`
exists, and if not creates it. (only applies to the two router clients)
### Configure `dnsmasq`
This task copies the templates for `/etc/dnsmasq.conf` to each of
the two router clients.
`dnsmasq` is used to provide _DHCP_ (and name resolution).
For instance, here is the new `/etc/dnsmasq.conf` for _dnetone_.
```cfg
# /etc/dnsmasq.conf
dhcp-range=10.5.5.50,10.5.5.150
listen-address=127.0.0.1, 10.5.5.1
```
### Configure Network _ifup_
This applies to all the lab clients except for the first one,
changes the default route. A bash script is copied from
template to `/etc/network/if-up.d/ifup-script`.
For instance here is `ifup-script` for _dnetthree_.
```bash
#!/bin/bash
# /etc/network/if-up.d/ifup-script
default_dev="$(ip route | head -1 | awk '{print $5}')"
echo "${default_dev}"
if [ "${default_dev}" == "enp1s0" ]
then
ip route del default via 10.55.44.1 dev enp1s0
fi
if [ "${default_dev}" != "enp7s0" ]
then
ip route add default via 10.4.4.1 dev enp7s0
fi
```
### Restart Network and `dnsmasq`
This is sequential:
1. _enp7s0_ is restarted on _dnet_
2. `dnsmasq` is restarted on _dnetone_, offering service on _enp7s0_
3. _enp7s0_ and _enp8s0_ are restarted on _dnettwo_, thus soliciting dhcp service on _enp7s0_, and triggering `/etc/network/if-up.d/ifup-script`
4. `dnsmasq` is restarted on _dnettwo_, offering service on _enp8s0_
5. _enp7s0_ is restarted on _dnetthree_, _dnetfour_, and _dnetfive_, thus soliciting dhcp service on _enp7s0_, and triggering `/etc/network/if-up.d/ifup-script`
### Backup `/etc/sysctl.conf`
This is a simple bash command that tests if `/etc/sysctl.conf.bak`
exists, and if not creates it. (only applies to the two router clients)
### Enable _ipv4 forwarding_
This is a simple bash command that uncomments the option for _ipv4 forwarding_
in `/etc/sysctl.conf`, applies only to the two routers.
```cfg
# /etc/sysctl.conf
...
# this
#net.ipv4.ip_forward=1
...
# becomes this
net.ipv4.ip_forward=1
...
```
### Start _ipv4 forwarding_
This simple bash command starts _ipv4 forwarding_, applies only
to the two routers.
```bash
bash -c "sysctl -w net.ipv4.ip_forward=1"
```
### Configure `iptables` _workaround_
This applies only to the two router clients.
From `iptables`'s point of view, the ansible connection isn't a RELATED INPUT
connection, thus it is necessary to bring up a firewall in a two-step
process that involves first ACCEPTING RELATED OUTPUT connections in a workaround.
From ansible template, the following is copied to `/dev/shm/iptables_workaround`
```iptables
# /dev/shm/iptables_workaround
*filter
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
-A INPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATED
-A OUTPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATED
COMMIT
```
### Apply `iptables` _workaround_
This applies only to the two router clients.
The following command is dispatched to apply the above _iptables_workaround_:
```bash
bash -c "iptables-restore < /dev/shm/iptables_workaround"
```
### Configure `iptables`
This applies only to the two router clients.
From ansible template the following is copied to `/etc/iptables/rules.v4` on _dnetone_.
```iptables
*nat
-A POSTROUTING -o enp1s0 -j MASQUERADE
COMMIT
*filter
-A INPUT -i lo -j ACCEPT
# allow ssh, so that we do not lock ourselves
-A INPUT -i enp1s0 -p tcp -m tcp --dport 22 -j ACCEPT
# allow incoming traffic to the outgoing connections,
# et al for clients from the private network
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# prohibit everything else incoming
-A INPUT -i enp1s0 -j DROP
COMMIT
```
From ansible template the following is copied to `/etc/iptables/rules.v4` on _dnettwo_.
```iptables
*nat
-A POSTROUTING -o enp7s0 -j MASQUERADE
COMMIT
*filter
-A INPUT -i lo -j ACCEPT
# allow ssh, so that we do not lock ourselves
-A INPUT -i enp7s0 -p tcp -m tcp --dport 22 -j ACCEPT
# allow incoming traffic to the outgoing connections,
# et al for clients from the private network
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# prohibit everything else incoming
-A INPUT -i enp7s0 -j DROP
COMMIT
```
### Apply `iptables` firewall
This applies only to the two router clients.
The following command is dispatched to apply the above from `/etc/iptables/rules.v4`:
```bash
bash -c "iptables-restore < /etc/iptables/rules.v4"
```
### `traceroute` test
The following script is dispatched to _dnettwo_:
```bash
#!/bin/bash
RESULT="$(traceroute 8.8.8.8)"
FIRST_HOP="$(echo "${RESULT}" | head -2 | tail -1 | awk '{print $2}')"
if [ "${FIRST_HOP}" == "10.5.5.1" ]
then
exit 0
else
exit 1
fi
```
The following script is dispatched to _dnetthree_, _dnetfour_, and _dnetfive_:
```bash
#!/bin/bash
RESULT="$(traceroute 8.8.8.8)"
FIRST_HOP="$(echo "${RESULT}" | head -2 | tail -1 | awk '{print $2}')"
if [ "${FIRST_HOP}" != "10.4.4.1" ]
then
exit 1
fi
SECOND_HOP="$(echo "${RESULT}" | head -3 | tail -1 | awk '{print $2}')"
if [ "${SECOND_HOP}" == "10.5.5.1" ]
then
exit 0
else
exit 1
fi
```
## To Be Continued
In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"},
I explain
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}
which are used to destroy the lab.

117
docs/posts/ansible-kvm-router-lab-part-6.md Normal file
View File

@ -0,0 +1,117 @@
---
title: "Ansible KVM Router Lab Part 6"
date: 2021-10-17
draft: false
tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"]
authors: ["trent"]
post: 31
---
date: 2021-10-17
## Introduction
This is Part 6 of a multi-part series of blog posts for building a
[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"}
automatically using a series of bash scripts and ansible.
[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}
is an overview.
In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"},
I break down the script
[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}.
In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"},
I explain
[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"}
and
[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"},
I explain
[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"},
[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"},
and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"}
scripts which are used to construct the lab.
In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"},
I explain the ansible playbook tasks used to finish building the lab.
In this post I explain how I use
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"},
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"},
and
[remove_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"},
to destroy the lab.
## `shutdown_vms.bash`
I explain `shutdown_vms.bash` in
[Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/#shutdown_vmsbash){target="_blank"}.
## `disconnect_vms_from_bridges.bash`
### `check_uid "${USER_UID}"`
[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash disconnect_vms_from_bridges.bash`.
### `function detach_vms()`
`detach_vms` loops over the MACHINES array, passing each name twice to `detach_vm`,
once for the upper bridge network, and again for the lower bridge network.
### `function detach_vm()`
`detach_vm` invokes `virsh dominfo` against the given virtual machine, and parses the output
to decide if the machine is running or not. If the given virtual machine is running,
`detach_vm` calls `detach_running_vm` against the given virtual machine and the given network.
If the given virtual machine is not running, `detach_vm` calls `detach_shut_off_vm` against the
given virtual machine and against the given network.
### `function detach_running_vm()`
`detach_running_vm` invokes `virsh domiflist` against the given virtual machine and greps that
for the given network to decide whether or not the given virtual machine is attached to the
given network. If the given virtual machine is attached to the given network, `detach_running_vm`
once again similarly invokes `virsh domiflist`, but this time parsing the mac of the attached interface.
`detach_running_vm` then invokes `virsh detach-interface` against parsed mac, and then recursively
calls itself for the purpose of verification.
### `function detach_shut_off_vm()`
`detach_shut_off_vm` is almost identical to `detach_running_vm`, but the options for the
invocation of `virsh detach-interface` are adjusted to be appropriate for a virtual machine
which is not running.
## `undefine_and_remove_vms.bash`
### `check_uid "${USER_UID}"`
[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash undefine_and_remove_vms.bash`.
### `function destroy_vms()`
`destroy_vms` simultaneously passes the entire MACHINES array to `destroy_vm`, which parses the output of
`virsh list --all` to find out if the virtual machine exists, and if it does invokes the command
`virsh undefine` with the `--remove-all-storage` option, against the virtual machine.
## `remove_bridge_networks.bash`
### `check_uid "${USER_UID}"`
[remove_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash)
begins by making sure that it is run as a _non-privileged_ user. You can call
the script with `bash remove_bridge_networks.bash`.
### `function disable_autostart_bridge_networks()`
`disable_autostart_bridge_networks` passes each of the upper and lower bridge network names to `disable_autostart_bridge_network`,
which parses the output of `virsh net-info` to find out if the network has autostart enabled, and if it is,
invokes `virsh net-autostart` with the `--disable` option to disable autostart for the given network, and then recursively calls itself
for the purpose of verification.
### `function stop_bridge_networks()`
`stop_bridge_networks` passes each of the upper and lower bridge network names to `stop_bridge_network`,
which parses the output of `virsh net-info` in order to find out if the given network is running, and it if is,
invokes `virsh net-destroy` against the given network to stop it, and then recursively calls itself for the purpose of
verification.
### `function undefine_bridge_networks()`
`undefine_bridge_networks` passes each of the upper and lower bridge network names to `undefine_bridge_network`,
which parses the output of `virsh net-list --all` in order to find out if the given network is defined, and if it is,
invokes `virsh net-undefine` against the given network to undefine it, and then recursively calls itself for the purpose of
verification.

12
mkdocs.yml
View File

@ -24,6 +24,12 @@ markdown_extensions:
nav:
- Home:
- Home: index.md
- "Ansible KVM Router Lab Part 6": posts/ansible-kvm-router-lab-part-6.md
- "Ansible KVM Router Lab Part 5": posts/ansible-kvm-router-lab-part-5.md
- "Ansible KVM Router Lab Part 4": posts/ansible-kvm-router-lab-part-4.md
- "Ansible KVM Router Lab Part 3": posts/ansible-kvm-router-lab-part-3.md
- "Ansible KVM Router Lab Part 2": posts/ansible-kvm-router-lab-part-2.md
- "Ansible KVM Router Lab Part 1": posts/ansible-kvm-router-lab-part-1.md
- "Add KVM Network With Virsh": posts/add-kvm-network-with-virsh.md
- "KVM On Arch": posts/kvm-on-arch.md
- "RaspberryPi LTE-Failover Router With DNS Caching": posts/raspberrypi-lte-failover-router-with-dns-caching.md
@ -63,6 +69,12 @@ nav:
- FreeCodeCampChallenges: https://trentspalmer.github.io/fcc-challenges/
- DeviceLayout: https://trentpalmer.work/6a57bbe24d8244289610bf57533d6c6f/
- Posts:
- "Ansible KVM Router Lab Part 6": posts/ansible-kvm-router-lab-part-6.md
- "Ansible KVM Router Lab Part 5": posts/ansible-kvm-router-lab-part-5.md
- "Ansible KVM Router Lab Part 4": posts/ansible-kvm-router-lab-part-4.md
- "Ansible KVM Router Lab Part 3": posts/ansible-kvm-router-lab-part-3.md
- "Ansible KVM Router Lab Part 2": posts/ansible-kvm-router-lab-part-2.md
- "Ansible KVM Router Lab Part 1": posts/ansible-kvm-router-lab-part-1.md
- "Add KVM Network With Virsh": posts/add-kvm-network-with-virsh.md
- "KVM On Arch": posts/kvm-on-arch.md
- "RaspberryPi LTE-Failover Router With DNS Caching": posts/raspberrypi-lte-failover-router-with-dns-caching.md