From e56519937a90a80b4e41020ae8d6e04091cda0f9 Mon Sep 17 00:00:00 2001 From: Trent Palmer Date: Mon, 18 Oct 2021 03:26:11 -0700 Subject: [PATCH] add ansible-kvm-router-lab --- docs/posts/ansible-kvm-router-lab-part-1.md | 124 ++++++++ docs/posts/ansible-kvm-router-lab-part-2.md | 105 +++++++ docs/posts/ansible-kvm-router-lab-part-3.md | 104 +++++++ docs/posts/ansible-kvm-router-lab-part-4.md | 107 +++++++ docs/posts/ansible-kvm-router-lab-part-5.md | 311 ++++++++++++++++++++ docs/posts/ansible-kvm-router-lab-part-6.md | 117 ++++++++ mkdocs.yml | 12 + 7 files changed, 880 insertions(+) create mode 100644 docs/posts/ansible-kvm-router-lab-part-1.md create mode 100644 docs/posts/ansible-kvm-router-lab-part-2.md create mode 100644 docs/posts/ansible-kvm-router-lab-part-3.md create mode 100644 docs/posts/ansible-kvm-router-lab-part-4.md create mode 100644 docs/posts/ansible-kvm-router-lab-part-5.md create mode 100644 docs/posts/ansible-kvm-router-lab-part-6.md diff --git a/docs/posts/ansible-kvm-router-lab-part-1.md b/docs/posts/ansible-kvm-router-lab-part-1.md new file mode 100644 index 0000000..2b0aac3 --- /dev/null +++ b/docs/posts/ansible-kvm-router-lab-part-1.md @@ -0,0 +1,124 @@ +--- +title: "Ansible KVM Router Lab Part 1" +date: 2021-10-16 +draft: false +tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"] +authors: ["trent"] +post: 26 +--- +date: 2021-10-16 + +## Introduction +This is a multi-part series of blog posts for building a +[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"} +automatically using a series of bash scripts and ansible. + +This achieves the ability to quickly set up a router lab for the +purposes of experimenting with iptables, or whatever else you +want to use for routing or firewalls. + +This is also, for myself, an opportunity to learn ansible. + +In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"}, +I break down the script +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. + +In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"}, +I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and [shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"}, +I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"}, +I explain the ansible playbook tasks used to finish building the lab. + +In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"}, +I explain +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"} +which are used to destroy the lab. + +## Networking +I begin by setting up a new network in libvirt, which will serve +as an _out-of-band_ network for connecting to the lab virtual machines. +This is covered in a +[previous blog post](/posts/add-kvm-network-with-virsh/){target="_blank"}. + +## Overview +The lab consists of seven virtual machines. + +I begin by creating a _base_ Debian 11 virtual machine called `dnet` by connecting to +my physical server using `virt-manager`. + +After creating a _base_ virtual machine, the next step is to create +a clone from which to work. I call this machine `dcon`. + +The client clones consist of 5 virtual machines named +`dnetone` through `dnetfive`. Once set up, all five virtual machines +are reachable through the _out-of-band_ network. + +But there are also two bridge networks connecting the client clones +to each other. The first and second clones are connected to each other on +the **upper** bridge network, with the first clone acting as a router for the +second. The second, third, fourth, and fifth clones are connected to each +other on the **lower** bridge network, with the second clone acting +as a router for the third, fourth, and fifth clones. Traffic from the +second clone will go through the first clone to reach the internet, and +traffic from the third, fourth, and fifth clones will go through +the second clone and then through the first clone to reach the internet. + +DHCP is handled by dnsmasq on the first clone and the second clone. + +## Resources +For ansible I used the +[ansible documentation](https://docs.ansible.com/ansible/latest/index.html){target="_blank"}. + +This +[blog post](https://www.brianlinkletter.com/2019/02/build-a-network-emulator-using-libvirt/){target="_blank"} +by Brian Linkletter is also really helpful. + +## Control Node Setup +* Create a control node by cloning the _base_ virtual machine. +```shell +virt-clone --original dnet --name dcon --auto-clone +``` +* Configure ansible host file +```cfg +# ~/.ansible.cfg +[defaults] +inventory = ~/router-lab/ansible/hosts.yml +``` +* Setup bashrc +```bash +# ~/.bashrc +export LIBVIRT_DEFAULT_URI="qemu+ssh://@/system" + +alias ansible-pb=anspb +anspb() { + ANS_DIR=~/router-lab/ansible/playbooks; + echo Changing to "${ANS_DIR}" and executing: ansible-playbook "${@}" + (cd $ANS_DIR || exit ; ansible-playbook "${@}") +} +``` +* configure Vim or similar for editing bash and python +* install apps +```bash +apt install ansible ansible-lint libvirt-clients +apt install --no-install-recommends virtinst +``` +The control node needs root ssh access to the _base_ virtual machine so +that it will have root ssh access to the clones. + +## To Be Continued +In the next blog post, +[Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"}, +I begin breaking down the bash scripts which build out the lab, beginning with +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. diff --git a/docs/posts/ansible-kvm-router-lab-part-2.md b/docs/posts/ansible-kvm-router-lab-part-2.md new file mode 100644 index 0000000..21aad41 --- /dev/null +++ b/docs/posts/ansible-kvm-router-lab-part-2.md @@ -0,0 +1,105 @@ +--- +title: "Ansible KVM Router Lab Part 2" +date: 2021-10-16 +draft: false +tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"] +authors: ["trent"] +post: 27 +--- +date: 2021-10-16 + +## Introduction +This is Part 2 of a multi-part series of blog posts for building a +[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"} +automatically using a series of bash scripts and ansible. + +[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"} +is an overview. + +In this post I begin breaking down the bash scripts which build the router +lab, beginning with +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. + +In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"}, +I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and [shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"}, +I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"}, +I explain the ansible playbook tasks used to finish building the lab. + +In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"}, +I explain +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"} +which are used to destroy the lab. + +## `build_vms.bash` +### `check_uid "0"` +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash) +begins by making sure that it is run as the _root_ user. This is because root is required to +ssh into the clones to change their hostnames, machine-ids, and host-ssh-keys. You can call +with `sudo bash build_vms.bash`. + +For this same reason, `~/.ssh/known_hosts` is useless so it is deleted (and then rebuilt). + +### `function build_vms()` +Next, +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"} +calls `build_vms`, which loops over the array of MACHINES, which is an +array that holds that names of the lab clients, passing each name in turn +to `create_vm`. + +`create_vm` creates the virtual machine if it does not already exist, using `virt-clone`, and +then calls `start_vm` to start it. + +`start_vm` is exported from +[env.bash](https://github.com/TrentSPalmer/router-lab/blob/master/env.bash.example){target="_blank"}, +and per parsing the output of `virsh list --inactive`, starts the virtual machine if it is not running. + +### `function set_hostnames()` +Next, +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"} +calls `set_hostnames`, which simultaneously calls `set_hostname` on the entire MACHINES array. `set_hostname` +in turn waits for the virtual machine to be fully booted, then updates the files `/etc/hostname` and +`/etc/hosts`, and then reboots the virtual machine to apply the new hostname. + +### `function confirm_hostnames()` +`confirm_hostnames` simultaneously calls `confirm_hostname` against the entire MACHINES array. +`confirm_hostname` waits for the virtual machine to be fully booted, then confirms the correct +_hostname_ in `/etc/hostname`. + +### `function confirm_hostnames_in_hosts()` +`confirm_hostnames_in_hosts` works almost exactly the same as `confirm_hostnames`, but this time +the file `/etc/hosts` on the virtual machine is grepped for the proper _hostname_, and corrected +if necessary. + +### `function reset_hosts_ssh_keys()` +`reset_hosts_ssh_keys` simultaneously calls `reset_host_ssh_keys` against the MACHINES array, +which in turn compares the host_ssh_key of the virtual machine against the _bas3_ virtual +machine, and if necessary deletes `/etc/ssh/ssh_host_*`, generates new host_ssh_keys, restarts +`sshd` on the virtual machine, removes `~/.ssh/known_hosts`, and then reruns itself in +order to confirm the new host_ssh_keys. + +### `function reset_machine_ids()` +`reset_machine_ids` simultaneously calls `reset_machine_id` against the entire MACHINES array, +which in turn checks the _machine-id_ of the virtual machine to make sure that it is different +than the _machine-id_ of the _base_ virtual machine, and if necessary deletes +`/etc/machine-id` and `/var/lib/dbus/machine-id` and recreates them. + +## To Be Continued +In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"}, +I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and +[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. diff --git a/docs/posts/ansible-kvm-router-lab-part-3.md b/docs/posts/ansible-kvm-router-lab-part-3.md new file mode 100644 index 0000000..1012e6a --- /dev/null +++ b/docs/posts/ansible-kvm-router-lab-part-3.md @@ -0,0 +1,104 @@ +--- +title: "Ansible KVM Router Lab Part 3" +date: 2021-10-16 +draft: false +tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"] +authors: ["trent"] +post: 28 +--- +date: 2021-10-16 + +## Introduction +This is Part 3 of a multi-part series of blog posts for building a +[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"} +automatically using a series of bash scripts and ansible. + +[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"} +is an overview. + +In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"}, +I break down the script +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. + +In this post I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and +[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"}, +I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"}, +I explain the ansible playbook tasks used to finish building the lab. + +In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"}, +I explain +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"} +which are used to destroy the lab. + +## `define_bridge_networks.bash` +The router-lab has two bridge networks, in addition to the initial out-of-band +network which is used to contact the virtual machines directly. + +### `check_uid "${USER_UID}"` +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash define_bridge_networks.bash`. + +### `function define_bridge_networks()` +`define_bridge_networks` calls `define_bridge_network` twice, once for each of the +upper bridge and the lower bridge. `define_bridge_network` parses the output of +`virsh net-list --all` to determine if the network is defined yet. If not, +`virsh net-define vm_router_lab_lower_bridge.xml` or +`virsh net-define vm_router_lab_upper_bridge.xml` are invoked as necessary. +`define_bridge_network` then recursively calls itself for confirmation. + +Links for +[vm_router_lab_upper_bridge.xml](https://github.com/TrentSPalmer/router-lab/blob/master/vm_router_lab_upper_bridge.xml){target="_blank"} +and +[vm_router_lab_lower_bridge.xml](https://github.com/TrentSPalmer/router-lab/blob/master/vm_router_lab_lower_bridge.xml){target="_blank"}. + +### `function start_bridge_networks()` +`start_bridge_networks` calls `start_bridge_network` twice, once for each of the +upper and the lower bridge. `start_bridge_network` in turn parses the output of +`virsh net-info vm_router_lab_upper_bridge` and/or `virsh net-info vm_router_lab_lower_bridge` +to determine if the cooresponding network is running, and if not invokes +`virsh net-start vm_router_lab_upper_bridge` or `virsh net-start vm_router_lab_lower_bridge`, +and then recursively calls itself again for confirmation. + +### `function autostart_bridge_networks()` +`autostart_bridge_networks` is nearly identical to `start_bridge_networks`, but +`virsh net-autostart vm_router_lab_upper_bridge` or `virsh net-autostart vm_router_lab_lower_bridge`, +are invoked in order to mark the cooresponding network to autostart. + +## `shutdown_vms.bash` +After creating the upper and lower bridge networks, it is necessary to shut down +the lab clients before connecting the lab clients to the bridge networks. This is +because network interfaces must be _permanently_ added to the lab client definitions. + +### `check_uid "${USER_UID}"` +[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash shutdown_vms.bash`. + +### `function shutdown_vms()` +`shutdown_vms` simultaneously calls `shutdown_vm` on the entire MACHINES array. +`shutdown_vm` in turn parses the output of `virsh list --state-running` to determine if +the virtual machine is running, and if so invokes `virsh shutdown `. `shutdown_vm` +then recursively calls itself to confirm that the virtual machine is indeed shut down. + +## To Be Continued +In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"}, +I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and +[rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. diff --git a/docs/posts/ansible-kvm-router-lab-part-4.md b/docs/posts/ansible-kvm-router-lab-part-4.md new file mode 100644 index 0000000..2ef7861 --- /dev/null +++ b/docs/posts/ansible-kvm-router-lab-part-4.md @@ -0,0 +1,107 @@ +--- +title: "Ansible KVM Router Lab Part 4" +date: 2021-10-17 +draft: false +tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"] +authors: ["trent"] +post: 29 +--- +date: 2021-10-17 + +## Introduction +This is Part 4 of a multi-part series of blog posts for building a +[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"} +automatically using a series of bash scripts and ansible. + +[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"} +is an overview. + +In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"}, +I break down the script +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. + +In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"}, +I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and [shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. + +In this post I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and +[rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"}, +I explain the ansible playbook tasks used to finish building the lab. + +In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"}, +I explain +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"} +which are used to destroy the lab. + +## `connect_vms_to_bridges.bash` +Aside from the _out-of-band_ network which can be used to contact the lab clients +directly, the lab clients are connected to each other using two bridge networks. + +As explained in +[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"}, +lab clients one and two are connected to the upper bridge, and lab clients two, three, four, +and five are connected to the lower bridge with the first client acting as a router +for the second client, and the second client acting as a client for the third, fourth, and +fifth clients. + +### `check_uid "${USER_UID}"` +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash connect_vms_to_bridges.bash`. + +### `function connect_upper_bridge()` +`connect_upper_bridge` calls `connect_vm_to_bridge` against the first lab client and the upper bridge, +and again against the second lab client and the upper bridge. + +### `function connect_lower_bridge()` +`connect_lower_bridge` calls `connect_vm_to_bridge` against the second lab client and the lower bridge, +against the third lab client and the lower bridge, against the fourth lab client and the lower bridge, +and against the fifth lab client and the lower bridge. + +### `function connect_vm_to_bridge()` +`connect_vm_to_bridge` parses the output of `virsh dominfo` against the intended lab client to verify that it +is shutdown. Then, if the intended lab client is shutdown, `connect_vm_to_bridge` parses the output of +`virsh domiflist` to find out if the intended new interface is yet defined, and if not invokes +`virsh attach-interface --type network --source `. Finally, `connect_vm_to_bridge` +recursively calls itself for verification. + +## `start_vms.bash` +After defining the new network interfaces for all the lab clients, you can boot them. + +### `check_uid "${USER_UID}"` +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash start_vms.bash`. + +### `function start_vms()` +`start_vms` calls `start_vm` against the entire MACHINES array, simultaneously. +`start_vm` is exported from +[env.bash](https://github.com/TrentSPalmer/router-lab/blob/master/env.bash.example){target="_blank"}, +and per parsing the output of `virsh list --inactive`, starts the virtual machine if it is not running. + +## `rebuild_known_hosts.bash` +You will need to have a valid list of known_hosts in order for ansible to connect to the lab +clients. + +The script deletes `~/.ssh/known_hosts` and then initiates an ssh connection to all the router lab clients +in order to repopulate `~/.ssh/known_hosts`. + +### `check_uid "${USER_UID}"` +[rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash rebuild_known_hosts`. + +## To Be Continued +In the next blog post, +[Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"}, +I explain the ansible playbook tasks used to finish building the lab. diff --git a/docs/posts/ansible-kvm-router-lab-part-5.md b/docs/posts/ansible-kvm-router-lab-part-5.md new file mode 100644 index 0000000..d376e3d --- /dev/null +++ b/docs/posts/ansible-kvm-router-lab-part-5.md @@ -0,0 +1,311 @@ +--- +title: "Ansible KVM Router Lab Part 5" +date: 2021-10-17 +draft: false +tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"] +authors: ["trent"] +post: 30 +--- +date: 2021-10-17 + +## Introduction +This is Part 5 of a multi-part series of blog posts for building a +[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"} +automatically using a series of bash scripts and ansible. + +[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"} +is an overview. + +In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"}, +I break down the script +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. + +In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"}, +I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and +[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"}, +I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. + +In this post I explain how I use Ansible to finish constructing the lab. + +In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"}, +I explain +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"} +which are used to destroy the lab. + +## Setup Ansible +* Configure ansible host file +```cfg +# ~/.ansible.cfg +[defaults] +inventory = ~/router-lab/ansible/hosts.yml +``` +* Setup bashrc +```bash +# ~/.bashrc +export LIBVIRT_DEFAULT_URI="qemu+ssh://@/system" + +alias ansible-pb=anspb +anspb() { + ANS_DIR=~/router-lab/ansible/playbooks; + echo Changing to "${ANS_DIR}" and executing: ansible-playbook "${@}" + (cd $ANS_DIR || exit ; ansible-playbook "${@}") +} +``` +* install apps +```bash +apt install ansible ansible-lint +``` + +## Run Ansible +```shell +ansible-pb build_out_routers.yml -K +``` +or if you want to first update all the clients +```shell +ansible-pb update_and_build.yml -K +``` +## Ansible Tasks +This is an explaination of the tasks in the Ansible Playbook. +Playbooks are executed from top to bottom. +### Install `dnsmasq`, `iptables-persistent` +This task is only run against the first and second lab clients as +they are the routers. +### Install `traceroute` +Traceroute is parsed in a later task to confirm that traffic is +following the correct route. +(Also incidentally installs `needrestart` and `screen`.) +### Backup `/etc/network/interfaces` +This is a simple bash command that tests if `/etc/network/interfaces.bak` +exists, and if not creates it. +### Update Network Config +This task updates `/etc/network/interfaces` in all the lab clients +to describe the network interfaces needed to connect to each other. + +For instance, here is the new `/etc/network/interfaces` file for _dnettwo_. +```cfg +# /etc/network/interfaces +# This file describes the network interfaces available on your system +# and how to activate them. For more information, see interfaces(5). + +source /etc/network/interfaces.d/* + +# The loopback network interface +auto lo +iface lo inet loopback + +# The primary network interface +allow-hotplug enp1s0 +iface enp1s0 inet dhcp + +# The primary network interface +allow-hotplug enp7s0 +iface enp7s0 inet dhcp + +auto enp8s0 +iface enp8s0 inet static + address 10.4.4.1 + network 10.4.4.0 + netmask 255.255.255.0 + broadcast 10.4.4.255 +``` +### Backup `/etc/dnsmasq.conf` +This is a simple bash command that tests if `/etc/dnsmasq.conf.bak` +exists, and if not creates it. (only applies to the two router clients) +### Configure `dnsmasq` +This task copies the templates for `/etc/dnsmasq.conf` to each of +the two router clients. + +`dnsmasq` is used to provide _DHCP_ (and name resolution). +For instance, here is the new `/etc/dnsmasq.conf` for _dnetone_. +```cfg +# /etc/dnsmasq.conf +dhcp-range=10.5.5.50,10.5.5.150 +listen-address=127.0.0.1, 10.5.5.1 +``` +### Configure Network _ifup_ +This applies to all the lab clients except for the first one, +changes the default route. A bash script is copied from +template to `/etc/network/if-up.d/ifup-script`. + +For instance here is `ifup-script` for _dnetthree_. +```bash +#!/bin/bash +# /etc/network/if-up.d/ifup-script + +default_dev="$(ip route | head -1 | awk '{print $5}')" +echo "${default_dev}" + +if [ "${default_dev}" == "enp1s0" ] +then + ip route del default via 10.55.44.1 dev enp1s0 +fi + +if [ "${default_dev}" != "enp7s0" ] +then + ip route add default via 10.4.4.1 dev enp7s0 +fi +``` +### Restart Network and `dnsmasq` +This is sequential: + +1. _enp7s0_ is restarted on _dnet_ +2. `dnsmasq` is restarted on _dnetone_, offering service on _enp7s0_ +3. _enp7s0_ and _enp8s0_ are restarted on _dnettwo_, thus soliciting dhcp service on _enp7s0_, and triggering `/etc/network/if-up.d/ifup-script` +4. `dnsmasq` is restarted on _dnettwo_, offering service on _enp8s0_ +5. _enp7s0_ is restarted on _dnetthree_, _dnetfour_, and _dnetfive_, thus soliciting dhcp service on _enp7s0_, and triggering `/etc/network/if-up.d/ifup-script` + +### Backup `/etc/sysctl.conf` +This is a simple bash command that tests if `/etc/sysctl.conf.bak` +exists, and if not creates it. (only applies to the two router clients) + +### Enable _ipv4 forwarding_ +This is a simple bash command that uncomments the option for _ipv4 forwarding_ +in `/etc/sysctl.conf`, applies only to the two routers. +```cfg +# /etc/sysctl.conf +... +# this +#net.ipv4.ip_forward=1 +... +# becomes this +net.ipv4.ip_forward=1 +... +``` + +### Start _ipv4 forwarding_ +This simple bash command starts _ipv4 forwarding_, applies only +to the two routers. +```bash +bash -c "sysctl -w net.ipv4.ip_forward=1" +``` + +### Configure `iptables` _workaround_ +This applies only to the two router clients. +From `iptables`'s point of view, the ansible connection isn't a RELATED INPUT +connection, thus it is necessary to bring up a firewall in a two-step +process that involves first ACCEPTING RELATED OUTPUT connections in a workaround. + +From ansible template, the following is copied to `/dev/shm/iptables_workaround` +```iptables +# /dev/shm/iptables_workaround +*filter +:INPUT ACCEPT [0:0] +:OUTPUT ACCEPT [0:0] +:FORWARD ACCEPT [0:0] + +-A INPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATED +-A OUTPUT -j ACCEPT -m conntrack --ctstate ESTABLISHED,RELATED + +COMMIT +``` + +### Apply `iptables` _workaround_ +This applies only to the two router clients. +The following command is dispatched to apply the above _iptables_workaround_: +```bash +bash -c "iptables-restore < /dev/shm/iptables_workaround" +``` + +### Configure `iptables` +This applies only to the two router clients. + +From ansible template the following is copied to `/etc/iptables/rules.v4` on _dnetone_. +```iptables +*nat +-A POSTROUTING -o enp1s0 -j MASQUERADE +COMMIT + +*filter +-A INPUT -i lo -j ACCEPT +# allow ssh, so that we do not lock ourselves +-A INPUT -i enp1s0 -p tcp -m tcp --dport 22 -j ACCEPT +# allow incoming traffic to the outgoing connections, +# et al for clients from the private network +-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT +# prohibit everything else incoming +-A INPUT -i enp1s0 -j DROP +COMMIT +``` + +From ansible template the following is copied to `/etc/iptables/rules.v4` on _dnettwo_. +```iptables +*nat +-A POSTROUTING -o enp7s0 -j MASQUERADE +COMMIT + +*filter +-A INPUT -i lo -j ACCEPT +# allow ssh, so that we do not lock ourselves +-A INPUT -i enp7s0 -p tcp -m tcp --dport 22 -j ACCEPT +# allow incoming traffic to the outgoing connections, +# et al for clients from the private network +-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT +# prohibit everything else incoming +-A INPUT -i enp7s0 -j DROP +COMMIT +``` + +### Apply `iptables` firewall +This applies only to the two router clients. +The following command is dispatched to apply the above from `/etc/iptables/rules.v4`: +```bash +bash -c "iptables-restore < /etc/iptables/rules.v4" +``` + +### `traceroute` test +The following script is dispatched to _dnettwo_: +```bash +#!/bin/bash + +RESULT="$(traceroute 8.8.8.8)" + +FIRST_HOP="$(echo "${RESULT}" | head -2 | tail -1 | awk '{print $2}')" + +if [ "${FIRST_HOP}" == "10.5.5.1" ] +then + exit 0 +else + exit 1 +fi +``` +The following script is dispatched to _dnetthree_, _dnetfour_, and _dnetfive_: +```bash +#!/bin/bash + +RESULT="$(traceroute 8.8.8.8)" + +FIRST_HOP="$(echo "${RESULT}" | head -2 | tail -1 | awk '{print $2}')" + +if [ "${FIRST_HOP}" != "10.4.4.1" ] +then + exit 1 +fi + +SECOND_HOP="$(echo "${RESULT}" | head -3 | tail -1 | awk '{print $2}')" + +if [ "${SECOND_HOP}" == "10.5.5.1" ] +then + exit 0 +else + exit 1 +fi +``` + +## To Be Continued +In [Ansible KVM Router Lab Part 6](/posts/ansible-kvm-router-lab-part-6/){target="_blank"}, +I explain +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and [remove_bridge_networks](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"} +which are used to destroy the lab. diff --git a/docs/posts/ansible-kvm-router-lab-part-6.md b/docs/posts/ansible-kvm-router-lab-part-6.md new file mode 100644 index 0000000..4b3d064 --- /dev/null +++ b/docs/posts/ansible-kvm-router-lab-part-6.md @@ -0,0 +1,117 @@ +--- +title: "Ansible KVM Router Lab Part 6" +date: 2021-10-17 +draft: false +tags: ["linux", "kvm", "libvirt", "virsh", "ansible", "bash"] +authors: ["trent"] +post: 31 +--- +date: 2021-10-17 + +## Introduction +This is Part 6 of a multi-part series of blog posts for building a +[router lab](https://github.com/TrentSPalmer/router-lab){target="_blank"} +automatically using a series of bash scripts and ansible. + +[Ansible KVM Router Lab Part 1](/posts/ansible-kvm-router-lab-part-1/){target="_blank"} +is an overview. + +In [Ansible KVM Router Lab Part 2](/posts/ansible-kvm-router-lab-part-2/){target="_blank"}, +I break down the script +[build_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/build_vms.bash){target="_blank"}. + +In [Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/){target="_blank"}, +I explain +[define_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/define_bridge_networks.bash){target="_blank"} +and +[shutdown_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/shutdown_vms.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 4](/posts/ansible-kvm-router-lab-part-4/){target="_blank"}, +I explain +[connect_vms_to_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/connect_vms_to_bridges.bash){target="_blank"}, +[start_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/start_vms.bash){target="_blank"}, +and [rebuild_known_hosts.bash](https://github.com/TrentSPalmer/router-lab/blob/master/rebuild_known_hosts.bash){target="_blank"} +scripts which are used to construct the lab. + +In [Ansible KVM Router Lab Part 5](/posts/ansible-kvm-router-lab-part-5/){target="_blank"}, +I explain the ansible playbook tasks used to finish building the lab. + +In this post I explain how I use +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash){target="_blank"}, +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash){target="_blank"}, +and +[remove_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash){target="_blank"}, +to destroy the lab. + +## `shutdown_vms.bash` +I explain `shutdown_vms.bash` in +[Ansible KVM Router Lab Part 3](/posts/ansible-kvm-router-lab-part-3/#shutdown_vmsbash){target="_blank"}. + +## `disconnect_vms_from_bridges.bash` + +### `check_uid "${USER_UID}"` +[disconnect_vms_from_bridges.bash](https://github.com/TrentSPalmer/router-lab/blob/master/disconnect_vms_from_bridges.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash disconnect_vms_from_bridges.bash`. + +### `function detach_vms()` +`detach_vms` loops over the MACHINES array, passing each name twice to `detach_vm`, +once for the upper bridge network, and again for the lower bridge network. + +### `function detach_vm()` +`detach_vm` invokes `virsh dominfo` against the given virtual machine, and parses the output +to decide if the machine is running or not. If the given virtual machine is running, +`detach_vm` calls `detach_running_vm` against the given virtual machine and the given network. +If the given virtual machine is not running, `detach_vm` calls `detach_shut_off_vm` against the +given virtual machine and against the given network. + +### `function detach_running_vm()` +`detach_running_vm` invokes `virsh domiflist` against the given virtual machine and greps that +for the given network to decide whether or not the given virtual machine is attached to the +given network. If the given virtual machine is attached to the given network, `detach_running_vm` +once again similarly invokes `virsh domiflist`, but this time parsing the mac of the attached interface. +`detach_running_vm` then invokes `virsh detach-interface` against parsed mac, and then recursively +calls itself for the purpose of verification. + +### `function detach_shut_off_vm()` +`detach_shut_off_vm` is almost identical to `detach_running_vm`, but the options for the +invocation of `virsh detach-interface` are adjusted to be appropriate for a virtual machine +which is not running. + +## `undefine_and_remove_vms.bash` + +### `check_uid "${USER_UID}"` +[undefine_and_remove_vms.bash](https://github.com/TrentSPalmer/router-lab/blob/master/undefine_and_remove_vms.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash undefine_and_remove_vms.bash`. + +### `function destroy_vms()` +`destroy_vms` simultaneously passes the entire MACHINES array to `destroy_vm`, which parses the output of +`virsh list --all` to find out if the virtual machine exists, and if it does invokes the command +`virsh undefine` with the `--remove-all-storage` option, against the virtual machine. + +## `remove_bridge_networks.bash` + +### `check_uid "${USER_UID}"` +[remove_bridge_networks.bash](https://github.com/TrentSPalmer/router-lab/blob/master/remove_bridge_networks.bash) +begins by making sure that it is run as a _non-privileged_ user. You can call +the script with `bash remove_bridge_networks.bash`. + +### `function disable_autostart_bridge_networks()` +`disable_autostart_bridge_networks` passes each of the upper and lower bridge network names to `disable_autostart_bridge_network`, +which parses the output of `virsh net-info` to find out if the network has autostart enabled, and if it is, +invokes `virsh net-autostart` with the `--disable` option to disable autostart for the given network, and then recursively calls itself +for the purpose of verification. + +### `function stop_bridge_networks()` +`stop_bridge_networks` passes each of the upper and lower bridge network names to `stop_bridge_network`, +which parses the output of `virsh net-info` in order to find out if the given network is running, and it if is, +invokes `virsh net-destroy` against the given network to stop it, and then recursively calls itself for the purpose of +verification. + +### `function undefine_bridge_networks()` +`undefine_bridge_networks` passes each of the upper and lower bridge network names to `undefine_bridge_network`, +which parses the output of `virsh net-list --all` in order to find out if the given network is defined, and if it is, +invokes `virsh net-undefine` against the given network to undefine it, and then recursively calls itself for the purpose of +verification. diff --git a/mkdocs.yml b/mkdocs.yml index dd7142a..489f021 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,6 +24,12 @@ markdown_extensions: nav: - Home: - Home: index.md + - "Ansible KVM Router Lab Part 6": posts/ansible-kvm-router-lab-part-6.md + - "Ansible KVM Router Lab Part 5": posts/ansible-kvm-router-lab-part-5.md + - "Ansible KVM Router Lab Part 4": posts/ansible-kvm-router-lab-part-4.md + - "Ansible KVM Router Lab Part 3": posts/ansible-kvm-router-lab-part-3.md + - "Ansible KVM Router Lab Part 2": posts/ansible-kvm-router-lab-part-2.md + - "Ansible KVM Router Lab Part 1": posts/ansible-kvm-router-lab-part-1.md - "Add KVM Network With Virsh": posts/add-kvm-network-with-virsh.md - "KVM On Arch": posts/kvm-on-arch.md - "RaspberryPi LTE-Failover Router With DNS Caching": posts/raspberrypi-lte-failover-router-with-dns-caching.md @@ -63,6 +69,12 @@ nav: - FreeCodeCampChallenges: https://trentspalmer.github.io/fcc-challenges/ - DeviceLayout: https://trentpalmer.work/6a57bbe24d8244289610bf57533d6c6f/ - Posts: + - "Ansible KVM Router Lab Part 6": posts/ansible-kvm-router-lab-part-6.md + - "Ansible KVM Router Lab Part 5": posts/ansible-kvm-router-lab-part-5.md + - "Ansible KVM Router Lab Part 4": posts/ansible-kvm-router-lab-part-4.md + - "Ansible KVM Router Lab Part 3": posts/ansible-kvm-router-lab-part-3.md + - "Ansible KVM Router Lab Part 2": posts/ansible-kvm-router-lab-part-2.md + - "Ansible KVM Router Lab Part 1": posts/ansible-kvm-router-lab-part-1.md - "Add KVM Network With Virsh": posts/add-kvm-network-with-virsh.md - "KVM On Arch": posts/kvm-on-arch.md - "RaspberryPi LTE-Failover Router With DNS Caching": posts/raspberrypi-lte-failover-router-with-dns-caching.md