Automated provisioning - The Foreman in 2022
I've got a need (okay, more of a want) to be able to deploy various operating systems in my home lab easily without too much fuss, to be able to try out various operating systems and distributions and so on more easily and with less hand-holding. Having an easy way of deploying machines means you're more likely to do it more often and more readily, and it slots in nicely with tools like Ansible, Terraform and the like to encourage you to keep things patched and up-to-date and documented and reproducible and all the various virtues.
Enter Foreman. A PXE (and others) provisioning tool that makes deploying machines easy. In theory.
I've dabbled with Foreman in the past, and I'm curious as to how it holds up in 2022, especially in a home lab environment that isn't using Puppet. I want to use it as a fairly lightweight tool for deploying VMs, and not have it interfere too badly with the rest of the environment. To summarise my past experiences - I like Foreman quite a lot, and had some success with it.
So, the goal of this post: Set up a Foreman server, minus the built-in Puppet ENC, in Proxmox, and have it boot and install machines automatically. I'd like at least Ubuntu, Debian and CentOS, but it would be nice if I could also get k3sos, Rocky Linux and a few exotics too.
Getting started
The first port of call is the Foreman quick start guide, the latest version of which is https://theforeman.org/manuals/3.2/quickstart_guide.html as of this writing - and as you might guess, it's for Foreman 3.2. This tells us the various operating systems we can run Foreman on, and other useful information about it - in my case, I'm going to go with Debian 11, which is often my distribution of choice at the moment. (Sidenote: At the moment I'm partial to Ubuntu, Arch, Manjaro, CentOS, Rocky Linux - all in various roles at various times, mostly sticking with Ubuntu and Debian on servers).
Peeking at the hardware requirements, a good starting point for my uses is a VM with 4 vCPUs, 4GiB of RAM and 10GiB of disk space.
Opening Proxmox, I created a VM called "foreman01", with Ceph-backed storage and the appropriate hardware specs, then went to the task of installing Debian 11, the specifics of which I don't need to go into.
Sidenote: One option I've used in the past with Foreman is "Katello", which can be thought of as a "distribution" of Foreman with added bundled features, integrated into one nice whole system. The division between Foreman and Katello is a bit blurry, and Katello is largely documented and useful for CentOS and Red Hat shops (but it can be used for Debian too!), so I'll be sticking with plain Foreman today. To fill in the gaps of what Katello would have offered, I'll likely setup apt-cacher-ng at some point, and manage packages and groups of machines via Ansible and Ansible AWX.
Once foreman01 has been created and Debian 11 installed, I'll give the machine a static DHCP lease with an IP address of 10.1.1.22, leaving open the possibility for changes down the road.
Manually specifying the DHCP lease in Kea's configuration for now, I restarted kea02 and waited for Stork (future blog post!) to show it as healthy and happy.
Then restarted kea01, checked Stork again, and rebooted the new machine - and poked it until it picked up its assigned IP address (hint - dhclient -r && dhclient -v ).
The Foreman installer
Time to install Foreman! We'll select Debian 11 from the dropdown...
and essentially just copy-and-paste the instructions.
wings@foreman01:~$ sudo apt-get -y install ca-certificates
cd /tmp && wget https://apt.puppet.com/puppet7-release-bullseye.deb
sudo apt-get install /tmp/puppet7-release-bullseye.deb
[sudo] password for wings:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ca-certificates is already the newest version (20210119).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
--2022-04-24 09:15:16-- https://apt.puppet.com/puppet7-release-bullseye.deb
Resolving apt.puppet.com (apt.puppet.com)... 13.32.127.78, 13.32.127.12, 13.32.127.60, ...
Connecting to apt.puppet.com (apt.puppet.com)|13.32.127.78|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11752 (11K) [application/x-debian-package]
Saving to: ‘puppet7-release-bullseye.deb’
puppet7-release-bullseye.de 100%[=========================================>] 11.48K --.-KB/s in 0.001s
2022-04-24 09:15:17 (17.0 MB/s) - ‘puppet7-release-bullseye.deb’ saved [11752/11752]
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'puppet7-release' instead of '/tmp/puppet7-release-bullseye.deb'
The following NEW packages will be installed:
puppet7-release
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/11.8 kB of archives.
After this operation, 29.7 kB of additional disk space will be used.
Get:1 /tmp/puppet7-release-bullseye.deb puppet7-release all 7.0.0-6bullseye [11.8 kB]
Selecting previously unselected package puppet7-release.
(Reading database ... 32823 files and directories currently installed.)
Preparing to unpack .../puppet7-release-bullseye.deb ...
Unpacking puppet7-release (7.0.0-6bullseye) ...
Setting up puppet7-release (7.0.0-6bullseye) ...
wings@foreman01:/tmp$ sudo wget https://deb.theforeman.org/foreman.asc -O /etc/apt/trusted.gpg.d/foreman.asc
echo "deb http://deb.theforeman.org/ bullseye 3.2" | sudo tee /etc/apt/sources.list.d/foreman.list
echo "deb http://deb.theforeman.org/ plugins 3.2" | sudo tee -a /etc/apt/sources.list.d/foreman.list
--2022-04-24 09:16:48-- https://deb.theforeman.org/foreman.asc
Resolving deb.theforeman.org (deb.theforeman.org)... 151.101.106.49, 2a04:4e42:19::561
Connecting to deb.theforeman.org (deb.theforeman.org)|151.101.106.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2495 (2.4K) [text/plain]
Saving to: ‘/etc/apt/trusted.gpg.d/foreman.asc’
/etc/apt/trusted.gpg.d/fore 100%[=========================================>] 2.44K --.-KB/s in 0s
2022-04-24 09:16:49 (15.6 MB/s) - ‘/etc/apt/trusted.gpg.d/foreman.asc’ saved [2495/2495]
deb http://deb.theforeman.org/ bullseye 3.2
deb http://deb.theforeman.org/ plugins 3.2
wings@foreman01:/tmp$ sudo apt-get update && sudo apt-get -y install foreman-installer
Hit:1 http://deb.debian.org/debian bullseye InRelease
Hit:2 http://deb.debian.org/debian bullseye-updates InRelease
Get:3 http://deb.theforeman.org bullseye InRelease [10.9 kB]
Get:4 http://apt.puppetlabs.com bullseye InRelease [69.8 kB]
Get:5 http://deb.theforeman.org plugins InRelease [49.3 kB]
Hit:6 http://security.debian.org/debian-security bullseye-security InRelease
Get:7 http://deb.theforeman.org bullseye/3.2 amd64 Packages [26.6 kB]
Get:8 http://apt.puppetlabs.com bullseye/puppet7 all Packages [3,378 B]
Get:9 http://apt.puppetlabs.com bullseye/puppet7 amd64 Packages [59.2 kB]
Get:10 http://deb.theforeman.org plugins/3.2 amd64 Packages [17.2 kB]
Fetched 236 kB in 2s (95.6 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
curl fonts-lato javascript-common libcurl4 libjs-jquery libruby2.7 libyaml-0-2 puppet-agent rake ruby
ruby-ansi ruby-clamp ruby-hashie ruby-highline ruby-kafo ruby-kafo-parsers ruby-kafo-wizards
ruby-little-plugger ruby-logging ruby-minitest ruby-multi-json ruby-net-telnet ruby-oj ruby-power-assert
ruby-powerbar ruby-rubygems ruby-test-unit ruby-xmlrpc ruby2.7 rubygems-integration unzip zip
Suggested packages:
apache2 | lighttpd | httpd ri ruby-dev ruby-activesupport bundler
The following NEW packages will be installed:
curl fonts-lato foreman-installer javascript-common libcurl4 libjs-jquery libruby2.7 libyaml-0-2
puppet-agent rake ruby ruby-ansi ruby-clamp ruby-hashie ruby-highline ruby-kafo ruby-kafo-parsers
ruby-kafo-wizards ruby-little-plugger ruby-logging ruby-minitest ruby-multi-json ruby-net-telnet ruby-oj
ruby-power-assert ruby-powerbar ruby-rubygems ruby-test-unit ruby-xmlrpc ruby2.7 rubygems-integration
unzip zip
0 upgraded, 33 newly installed, 0 to remove and 0 not upgraded.
Need to get 31.2 MB of archives.
After this operation, 175 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 fonts-lato all 2.0-2.1 [2,696 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 libcurl4 amd64 7.74.0-1.3+deb11u1 [341 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 curl amd64 7.74.0-1.3+deb11u1 [267 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 rubygems-integration all 1.18 [6,704 B]
Get:5 http://deb.debian.org/debian bullseye/main amd64 rake all 13.0.3-1 [84.7 kB]
Get:6 http://deb.debian.org/debian bullseye/main amd64 ruby-minitest all 5.13.0-1 [57.3 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 ruby-net-telnet all 0.1.1-2 [12.5 kB]
Get:8 http://deb.debian.org/debian bullseye/main amd64 ruby-power-assert all 1.1.7-2 [11.5 kB]
Get:9 http://deb.debian.org/debian bullseye/main amd64 ruby-test-unit all 3.3.9-1 [86.1 kB]
Get:10 http://deb.debian.org/debian bullseye/main amd64 ruby-xmlrpc all 0.3.0-2 [23.7 kB]
Get:11 http://deb.debian.org/debian bullseye/main amd64 libyaml-0-2 amd64 0.2.2-1 [49.6 kB]
Get:12 http://deb.debian.org/debian bullseye/main amd64 libruby2.7 amd64 2.7.4-1+deb11u1 [4,202 kB]
Get:13 http://deb.theforeman.org bullseye/3.2 amd64 ruby-kafo-parsers all 1.2.1-1 [7,060 B]
Get:14 http://apt.puppetlabs.com bullseye/puppet7 amd64 puppet-agent amd64 7.16.0-1bullseye [20.1 MB]
Get:15 http://deb.debian.org/debian bullseye/main amd64 ruby2.7 amd64 2.7.4-1+deb11u1 [747 kB]
Get:16 http://deb.theforeman.org bullseye/3.2 amd64 ruby-kafo-wizards all 0.0.1-1 [8,160 B]
Get:17 http://deb.debian.org/debian bullseye/main amd64 ruby-rubygems all 3.2.5-2 [281 kB]
Get:18 http://deb.debian.org/debian bullseye/main amd64 ruby amd64 1:2.7+2 [11.7 kB]
Get:19 http://deb.theforeman.org bullseye/3.2 amd64 ruby-kafo all 6.4.0-1 [41.3 kB]
Get:20 http://deb.theforeman.org bullseye/3.2 amd64 foreman-installer all 3.2.0-1 [1,070 kB]
Get:21 http://deb.debian.org/debian bullseye/main amd64 ruby-clamp all 1.1.1-1 [18.1 kB]
Get:22 http://deb.debian.org/debian bullseye/main amd64 ruby-highline all 2.0.3-2 [61.7 kB]
Get:23 http://deb.debian.org/debian bullseye/main amd64 ruby-little-plugger all 1.1.4-1 [7,208 B]
Get:24 http://deb.debian.org/debian bullseye/main amd64 ruby-multi-json all 1.14.1-1 [21.0 kB]
Get:25 http://deb.debian.org/debian bullseye/main amd64 ruby-logging all 2.2.2-1 [59.0 kB]
Get:26 http://deb.debian.org/debian bullseye/main amd64 ruby-ansi all 1.5.0-1.1 [36.0 kB]
Get:27 http://deb.debian.org/debian bullseye/main amd64 ruby-hashie all 3.5.5-2 [39.3 kB]
Get:28 http://deb.debian.org/debian bullseye/main amd64 ruby-powerbar all 1.0.18-1 [7,482 B]
Get:29 http://deb.debian.org/debian bullseye/main amd64 javascript-common all 11+nmu1 [6,260 B]
Get:30 http://deb.debian.org/debian bullseye/main amd64 libjs-jquery all 3.5.1+dfsg+~3.5.5-7 [315 kB]
Get:31 http://deb.debian.org/debian bullseye/main amd64 ruby-oj amd64 3.11.0-1 [126 kB]
Get:32 http://deb.debian.org/debian bullseye/main amd64 unzip amd64 6.0-26 [171 kB]
Get:33 http://deb.debian.org/debian bullseye/main amd64 zip amd64 3.0-12 [232 kB]
Fetched 31.2 MB in 4s (8,186 kB/s)
Extracting templates from packages: 100%
Selecting previously unselected package fonts-lato.
(Reading database ... 32828 files and directories currently installed.)
Preparing to unpack .../00-fonts-lato_2.0-2.1_all.deb ...
Unpacking fonts-lato (2.0-2.1) ...
Selecting previously unselected package libcurl4:amd64.
Preparing to unpack .../01-libcurl4_7.74.0-1.3+deb11u1_amd64.deb ...
Unpacking libcurl4:amd64 (7.74.0-1.3+deb11u1) ...
Selecting previously unselected package curl.
Preparing to unpack .../02-curl_7.74.0-1.3+deb11u1_amd64.deb ...
Unpacking curl (7.74.0-1.3+deb11u1) ...
Selecting previously unselected package rubygems-integration.
Preparing to unpack .../03-rubygems-integration_1.18_all.deb ...
Unpacking rubygems-integration (1.18) ...
Selecting previously unselected package rake.
Preparing to unpack .../04-rake_13.0.3-1_all.deb ...
Unpacking rake (13.0.3-1) ...
Selecting previously unselected package ruby-minitest.
Preparing to unpack .../05-ruby-minitest_5.13.0-1_all.deb ...
Unpacking ruby-minitest (5.13.0-1) ...
Selecting previously unselected package ruby-net-telnet.
Preparing to unpack .../06-ruby-net-telnet_0.1.1-2_all.deb ...
Unpacking ruby-net-telnet (0.1.1-2) ...
Selecting previously unselected package ruby-power-assert.
Preparing to unpack .../07-ruby-power-assert_1.1.7-2_all.deb ...
Unpacking ruby-power-assert (1.1.7-2) ...
Selecting previously unselected package ruby-test-unit.
Preparing to unpack .../08-ruby-test-unit_3.3.9-1_all.deb ...
Unpacking ruby-test-unit (3.3.9-1) ...
Selecting previously unselected package ruby-xmlrpc.
Preparing to unpack .../09-ruby-xmlrpc_0.3.0-2_all.deb ...
Unpacking ruby-xmlrpc (0.3.0-2) ...
Selecting previously unselected package libyaml-0-2:amd64.
Preparing to unpack .../10-libyaml-0-2_0.2.2-1_amd64.deb ...
Unpacking libyaml-0-2:amd64 (0.2.2-1) ...
Selecting previously unselected package libruby2.7:amd64.
Preparing to unpack .../11-libruby2.7_2.7.4-1+deb11u1_amd64.deb ...
Unpacking libruby2.7:amd64 (2.7.4-1+deb11u1) ...
Selecting previously unselected package ruby2.7.
Preparing to unpack .../12-ruby2.7_2.7.4-1+deb11u1_amd64.deb ...
Unpacking ruby2.7 (2.7.4-1+deb11u1) ...
Selecting previously unselected package ruby-rubygems.
Preparing to unpack .../13-ruby-rubygems_3.2.5-2_all.deb ...
Unpacking ruby-rubygems (3.2.5-2) ...
Selecting previously unselected package ruby.
Preparing to unpack .../14-ruby_1%3a2.7+2_amd64.deb ...
Unpacking ruby (1:2.7+2) ...
Selecting previously unselected package ruby-clamp.
Preparing to unpack .../15-ruby-clamp_1.1.1-1_all.deb ...
Unpacking ruby-clamp (1.1.1-1) ...
Selecting previously unselected package ruby-highline.
Preparing to unpack .../16-ruby-highline_2.0.3-2_all.deb ...
Unpacking ruby-highline (2.0.3-2) ...
Selecting previously unselected package ruby-kafo-parsers.
Preparing to unpack .../17-ruby-kafo-parsers_1.2.1-1_all.deb ...
Unpacking ruby-kafo-parsers (1.2.1-1) ...
Selecting previously unselected package ruby-kafo-wizards.
Preparing to unpack .../18-ruby-kafo-wizards_0.0.1-1_all.deb ...
Unpacking ruby-kafo-wizards (0.0.1-1) ...
Selecting previously unselected package ruby-little-plugger.
Preparing to unpack .../19-ruby-little-plugger_1.1.4-1_all.deb ...
Unpacking ruby-little-plugger (1.1.4-1) ...
Selecting previously unselected package ruby-multi-json.
Preparing to unpack .../20-ruby-multi-json_1.14.1-1_all.deb ...
Unpacking ruby-multi-json (1.14.1-1) ...
Selecting previously unselected package ruby-logging.
Preparing to unpack .../21-ruby-logging_2.2.2-1_all.deb ...
Unpacking ruby-logging (2.2.2-1) ...
Selecting previously unselected package ruby-ansi.
Preparing to unpack .../22-ruby-ansi_1.5.0-1.1_all.deb ...
Unpacking ruby-ansi (1.5.0-1.1) ...
Selecting previously unselected package ruby-hashie.
Preparing to unpack .../23-ruby-hashie_3.5.5-2_all.deb ...
Unpacking ruby-hashie (3.5.5-2) ...
Selecting previously unselected package ruby-powerbar.
Preparing to unpack .../24-ruby-powerbar_1.0.18-1_all.deb ...
Unpacking ruby-powerbar (1.0.18-1) ...
Selecting previously unselected package ruby-kafo.
Preparing to unpack .../25-ruby-kafo_6.4.0-1_all.deb ...
Unpacking ruby-kafo (6.4.0-1) ...
Selecting previously unselected package puppet-agent.
Preparing to unpack .../26-puppet-agent_7.16.0-1bullseye_amd64.deb ...
Unpacking puppet-agent (7.16.0-1bullseye) ...
Selecting previously unselected package foreman-installer.
Preparing to unpack .../27-foreman-installer_3.2.0-1_all.deb ...
Unpacking foreman-installer (3.2.0-1) ...
Selecting previously unselected package javascript-common.
Preparing to unpack .../28-javascript-common_11+nmu1_all.deb ...
Unpacking javascript-common (11+nmu1) ...
Selecting previously unselected package libjs-jquery.
Preparing to unpack .../29-libjs-jquery_3.5.1+dfsg+~3.5.5-7_all.deb ...
Unpacking libjs-jquery (3.5.1+dfsg+~3.5.5-7) ...
Selecting previously unselected package ruby-oj:amd64.
Preparing to unpack .../30-ruby-oj_3.11.0-1_amd64.deb ...
Unpacking ruby-oj:amd64 (3.11.0-1) ...
Selecting previously unselected package unzip.
Preparing to unpack .../31-unzip_6.0-26_amd64.deb ...
Unpacking unzip (6.0-26) ...
Selecting previously unselected package zip.
Preparing to unpack .../32-zip_3.0-12_amd64.deb ...
Unpacking zip (3.0-12) ...
Setting up javascript-common (11+nmu1) ...
Setting up fonts-lato (2.0-2.1) ...
Setting up ruby-highline (2.0.3-2) ...
Setting up ruby-power-assert (1.1.7-2) ...
Setting up libyaml-0-2:amd64 (0.2.2-1) ...
Setting up unzip (6.0-26) ...
Setting up rubygems-integration (1.18) ...
Setting up ruby-minitest (5.13.0-1) ...
Setting up zip (3.0-12) ...
Setting up ruby-test-unit (3.3.9-1) ...
Setting up ruby-net-telnet (0.1.1-2) ...
Setting up libcurl4:amd64 (7.74.0-1.3+deb11u1) ...
Setting up curl (7.74.0-1.3+deb11u1) ...
Setting up libjs-jquery (3.5.1+dfsg+~3.5.5-7) ...
Setting up ruby-xmlrpc (0.3.0-2) ...
Setting up puppet-agent (7.16.0-1bullseye) ...
Created symlink /etc/systemd/system/multi-user.target.wants/pxp-agent.service → /lib/systemd/system/pxp-agent.service.
Created symlink /etc/systemd/system/multi-user.target.wants/puppet.service → /lib/systemd/system/puppet.service.
Removed /etc/systemd/system/multi-user.target.wants/pxp-agent.service.
Setting up ruby-rubygems (3.2.5-2) ...
Setting up rake (13.0.3-1) ...
Setting up libruby2.7:amd64 (2.7.4-1+deb11u1) ...
Setting up ruby2.7 (2.7.4-1+deb11u1) ...
Setting up ruby (1:2.7+2) ...
Setting up ruby-kafo-parsers (1.2.1-1) ...
Setting up ruby-little-plugger (1.1.4-1) ...
Setting up ruby-ansi (1.5.0-1.1) ...
Setting up ruby-clamp (1.1.1-1) ...
Setting up ruby-multi-json (1.14.1-1) ...
Setting up ruby-hashie (3.5.5-2) ...
Setting up ruby-powerbar (1.0.18-1) ...
Setting up ruby-kafo-wizards (0.0.1-1) ...
Setting up ruby-oj:amd64 (3.11.0-1) ...
Setting up ruby-logging (2.2.2-1) ...
Setting up ruby-kafo (6.4.0-1) ...
Setting up foreman-installer (3.2.0-1) ...
Processing triggers for mailcap (3.69) ...
Processing triggers for libc-bin (2.31-13+deb11u3) ...
Processing triggers for man-db (2.9.4-2) ...
As advised, I ran ping $(hostname -f) and found that my new host didn't quite qualify yet;
wings@foreman01:/tmp$ ping $(hostname -f)
PING foreman01.windowpa.in (127.0.1.1) 56(84) bytes of data.
64 bytes from foreman01.windowpa.in (127.0.1.1): icmp_seq=1 ttl=64 time=0.019 ms
A quick trip to /etc/hosts and I removed the hostname-specific line.
However, this led to further issues:
wings@foreman01:/tmp$ hostname -f
hostname: Name or service not known
Oh no! Now it doesn't even know who it is. Time for a quick trip to our friendly DNS service...
Let's try again...
wings@foreman01:/tmp$ ping $(hostname -f)
PING foreman01.windowpa.in (10.1.1.22) 56(84) bytes of data.
64 bytes from foreman01.windowpa.in (10.1.1.22): icmp_seq=1 ttl=64 time=0.012 ms
With the installer ready to go, we just need to hunt down the flags to avoid installing the Puppet server and ENC 😁 Fortunately, the flags are well documented these days as easy to find installer options.
We can have our first naive attempt now:
sudo foreman-installer \
--puppet-server=false
And I'm off to make some tea...
to come back to a hung terminal. Trying again in screen this time.
wings@foreman01:~$ sudo foreman-installer --puppet-server=false -l info
[sudo] password for wings:
2022-04-24 09:48:39 [INFO ] [pre_migrations] Executing hooks in group pre_migrations
2022-04-24 09:48:39 [INFO ] [pre_migrations] All hooks in group pre_migrations finished
2022-04-24 09:48:39 [INFO ] [boot] Executing hooks in group boot
2022-04-24 09:48:39 [INFO ] [boot] All hooks in group boot finished
2022-04-24 09:48:39 [NOTICE] [root] Loading installer configuration. This will take some time.
2022-04-24 09:48:39 [INFO ] [init] Executing hooks in group init
2022-04-24 09:48:39 [INFO ] [init] All hooks in group init finished
2022-04-24 09:48:39 [INFO ] [root] Loading default values from puppet modules...
2022-04-24 09:48:46 [INFO ] [root] ... finished loading default values from puppet modules.
2022-04-24 09:48:46 [INFO ] [pre_values] Executing hooks in group pre_values
2022-04-24 09:48:46 [INFO ] [pre_values] All hooks in group pre_values finished
2022-04-24 09:48:46 [NOTICE] [root] Running installer with log based terminal output at level INFO.
2022-04-24 09:48:46 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2022-04-24 09:48:47 [INFO ] [pre_validations] Executing hooks in group pre_validations
2022-04-24 09:48:50 [INFO ] [pre_validations] All hooks in group pre_validations finished
2022-04-24 09:48:50 [INFO ] [root] Running validation checks.
2022-04-24 09:48:50 [INFO ] [pre_commit] Executing hooks in group pre_commit
2022-04-24 09:48:50 [INFO ] [pre_commit] All hooks in group pre_commit finished
2022-04-24 09:48:50 [INFO ] [pre] Executing hooks in group pre
2022-04-24 09:48:50 [INFO ] [pre] All hooks in group pre finished
2022-04-24 09:48:50 [NOTICE] [configure] Starting system configuration.
2022-04-24 09:48:57 [INFO ] [configure] Compiled catalog for foreman01.windowpa.in in environment production in 2.66 seconds
2022-04-24 09:48:59 [ERROR ] [configure] Could not set groups on user[foreman]: Execution of '/sbin/usermod -G puppet foreman' returned 6: usermod: group 'puppet' does not exist
2022-04-24 09:48:59 [ERROR ] [configure] /Stage[main]/Foreman::Config/User[foreman]/groups: change from to 'puppet' failed: Could not set groups on user[foreman]: Execution of '/sbin/usermod -G puppet foreman' returned 6: usermod: group 'puppet' does not exist
2022-04-24 09:48:59 [ERROR ] [configure] Could not set groups on user[foreman-proxy]: Execution of '/sbin/usermod -G puppet foreman-proxy' returned 6: usermod: group 'puppet' does not exist
2022-04-24 09:48:59 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Config/User[foreman-proxy]/groups: change from to 'puppet' failed: Could not set groups on user[foreman-proxy]: Execution of '/sbin/usermod -G puppet foreman-proxy' returned 6: usermod: group 'puppet' does not exist
2022-04-24 09:48:59 [NOTICE] [configure] 250 configuration steps out of 1162 steps complete.
2022-04-24 09:48:59 [NOTICE] [configure] 500 configuration steps out of 1164 steps complete.
2022-04-24 09:48:59 [NOTICE] [configure] 750 configuration steps out of 1272 steps complete.
2022-04-24 09:49:00 [NOTICE] [configure] 1000 configuration steps out of 1273 steps complete.
2022-04-24 09:49:00 [INFO ] [configure] /Service[foreman-proxy]: Dependency User[foreman-proxy] has failures: true
2022-04-24 09:49:00 [INFO ] [configure] /Stage[main]/Foreman::Database::Postgresql/Postgresql::Server::Db[foreman]/Postgresql::Server::Role[foreman]/Postgresql_psql[CREATE ROLE foreman ENCRYPTED PASSWORD ****]: Dependency User[foreman] has failures: true
2022-04-24 09:49:05 [NOTICE] [configure] 1250 configuration steps out of 1273 steps complete.
2022-04-24 09:49:07 [INFO ] [configure] Applied catalog in 9.10 seconds
2022-04-24 09:49:17 [NOTICE] [configure] System configuration has finished.
2022-04-24 09:49:17 [INFO ] [post] Executing hooks in group post
There were errors detected during install.
Please address the errors and re-run the installer to ensure the system is properly configured.
Failing to do so is likely to result in broken functionality.
The full log is at /var/log/foreman-installer/foreman.log
2022-04-24 09:49:17 [INFO ] [post] All hooks in group post finishedWell, shoot. That isn't what I wanted, and possibly indicates a half-finished installation. At this point it's wise to blow away the VM completely and start fresh, but we'll try and clean it ourselves and installing again... (Do as I say, not as I do!)
It goes without saying that the following commands are dangerous, and you are by no means supposed to try this at home. You've been warned...
sudo apt remove --purge -yyy foreman postgresql-* bind9-host redis-server
sudo apt autoremove -y
sudo rm -rf /etc/redis/ /var/lib/postgresql/ /var/lib/foreman-proxy/ /var/lib/redis/ /etc/foreman /etc/foreman-proxy
sudo userdel foreman-proxy
sudo userdel foreman
sudo userdel redis
sudo userdel postgresWith that out of the way, we'll give it one last attempt before trying a reinstalled, clean VM.
This time, run the installer in screen: sudo foreman-installer --puppet-server=false
Time for breakfast! To be continued...
After a brief interlude, I was able to get the same errors as before:
At which point I discovered that while it's possible to install Foreman without Puppet when you're installing Katello, it's not currently possible to install plain Foreman without installing a Puppet server. This actually isn't as big a deal as it sounds! It just means that you'll need to instruct Foreman not to install the Puppet agent on machines it deploys, and ignore the Puppet functionality when using Foreman. Drat.
A very relevant Foreman community thread:
lzap
Perhaps one day I'll do this again and write that blog post for ekohl 😉
Time to try again, this time explicitly enabling the Puppet server: sudo foreman-installer --puppet-server=true...
wings@foreman01:~$ sudo foreman-installer --puppet-server=true
2022-04-24 10:10:13 [NOTICE] [root] Loading installer configuration. This will take some time.
2022-04-24 10:10:18 [NOTICE] [root] Running installer with log based terminal output at level NOTICE.
2022-04-24 10:10:18 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2022-04-24 10:10:22 [NOTICE] [configure] Starting system configuration.
2022-04-24 10:10:32 [NOTICE] [configure] 250 configuration steps out of 1338 steps complete.
2022-04-24 10:12:05 [NOTICE] [configure] 500 configuration steps out of 1340 steps complete.
2022-04-24 10:12:13 [NOTICE] [configure] 750 configuration steps out of 1448 steps complete.
2022-04-24 10:12:14 [NOTICE] [configure] 1000 configuration steps out of 1449 steps complete.
2022-04-24 10:12:14 [ERROR ] [configure] Error executing SQL; psql returned pid 42589 exit 1: 'ERROR: invalid locale name: "en_US.utf8"
2022-04-24 10:12:14 [ERROR ] [configure] '
2022-04-24 10:12:14 [ERROR ] [configure] /Stage[main]/Foreman::Database::Postgresql/Postgresql::Server::Db[foreman]/Postgresql::Server::Database[foreman]/Postgresql_psql[CREATE DATABASE "foreman"]/command: change from 'notrun' to 'CREATE DATABASE "foreman" WITH TEMPLATE = "template0" ENCODING = \'utf8\' LC_COLLATE = \'en_US.utf8\' LC_CTYPE = \'en_US.utf8\' ' failed: Error executing SQL; psql returned pid 42589 exit 1: 'ERROR: invalid locale name: "en_US.utf8"
2022-04-24 10:12:14 [ERROR ] [configure] '
2022-04-24 10:12:14 [ERROR ] [configure] /Stage[main]/Foreman::Database::Postgresql/Postgresql::Server::Db[foreman]/Postgresql::Server::Database[foreman]/Postgresql_psql[CREATE DATABASE "foreman"]: Failed to call refresh: Error executing SQL; psql returned pid 42607 exit 1: 'ERROR: invalid locale name: "en_US.utf8"
2022-04-24 10:12:14 [ERROR ] [configure] '
2022-04-24 10:12:14 [ERROR ] [configure] /Stage[main]/Foreman::Database::Postgresql/Postgresql::Server::Db[foreman]/Postgresql::Server::Database[foreman]/Postgresql_psql[CREATE DATABASE "foreman"]: Error executing SQL; psql returned pid 42607 exit 1: 'ERROR: invalid locale name: "en_US.utf8"
2022-04-24 10:12:14 [ERROR ] [configure] '
2022-04-24 10:12:20 [NOTICE] [configure] 1250 configuration steps out of 1449 steps complete.
2022-04-24 10:12:45 [NOTICE] [configure] System configuration has finished.
There were errors detected during install.
Please address the errors and re-run the installer to ensure the system is properly configured.
Failing to do so is likely to result in broken functionality.
The full log is at /var/log/foreman-installer/foreman.logOkay, things are really broken now. Let's blow everything away again (see above) and try a fresh install, then a fresh machine if that doesn't work.
Time to run the installer again. This time - no flags at all. We're going stock Foreman! Just the way God intended.
Boom! Same errors. Looks like https://projects.theforeman.org/issues/13496 bit us... Time to add the en_US.UTF-8 locale (mine was en_AU.UTF-8 only by default)...
sudo dpkg-reconfigure locales
and reboot... then try again. In fairness to the Foreman project, expecting en_US.UTF-8 to be available is pretty reasonable, and it is documented, just not on the Quickstart guide.
Success! We have Foreman!
Finally, we can login and look around.
Now it's really time for a break. Until next time...