Differences

This shows you the differences between two versions of the page.

--- notes:linux_automatic_network_install [2024/05/03 14:20] – Nikita Kipriyanov
+++ notes:linux_automatic_network_install [2024/05/15 09:53] (current) – [Bootloader configuration] Nikita Kipriyanov
@@ Line 1: / Line 1: @@
 ====== Linux Automatic Network Install ======
-General outline:
+Purpose: to install Linux automatically unattended, without any manual user input during the process.
+This can be used in further automation using tool like Ansible. It should create a VM and obtain its MAC address from the virtualization environment, or reach the BMC and obtain a MAC address of the physical server. Then, knowing this info, it can generate all the boot configuration and installation scripts and wait for the OS to be installed. At last, it can customize it further in the next play. This facilitates a complete server provision with a single tool using a single configuration file in a single run.
+General outline of the technology:
   * PXE boot server with support for "legacy" BIOS and EFI boot; most VMs use "legacy" and most recent equipment uses EFI
   * HTTP server for EL (it doesn't support downloading a kickstart file from TFTP server, what a shame)
   * A set of boot images and minimal autoinstall files for Debian (at least, Bookworm) and EL (at least, Oracle Linux 8)
+Possible optional additions:
+  * A proxy server to perform installation and updates in restricted environments
+  * Set of repositories to store packages needed by installers, beneficial for mass installs. The set of "popular" packages could be even retrieved from the proxy server logs
 ===== Boot server configuration =====
-==== dnsmasq ====
+We can use ISC DHCPD + BIND + tftpd-hpa, or dnsmasq on the server side, and GRUB or PXELinux/SysLinux as downloadable Linux boot loader. GRUB is beneficial because it is a single piece of software that supports both “legacy” and EFI boot with a single set of configuration files; however, the prior development of this topic was focused on primarily PXELinux. The latter doesn’t support EFI boot, which can be supported by SysLinux (a sibling program), but there is a caveat: it doesn’t have a convenient way to make per-system dedicated boot configuration. Dnsmasq’s feature can be used to fill this gap. Overall, GRUB looks more promising.
+==== PXE bootloader ====
+All (more or less) dynamic data is to be stored in ''/srv/tftp''. The structure of the directory:
+<tabbox GRUB>
+GRUB manual is available here: [[https://www.gnu.org/software/grub/manual/grub/]]
+<code>
+srv/
+┠─ dhcp-hostsfile.cfg
+┗━ tftp/
+   ┠─ grub.0 → grub/i386-pc/core.0
+   ┠─ grub.efi → grub/x86_64-efi/core.efi
+   ┣━ grub/
+   ┃  ┣━ fonts/
+   ┃  ┃  ┗━ ...
+   ┃  ┣━ locale/
+   ┃  ┃  ┗━ ...
+   ┃  ┣━ i386-pc/
+   ┃  ┃  ┠─ core.0
+   ┃  ┃  ┗━ ...
+   ┃  ┣━ x86_64-efi
+   ┃  ┃  ┠─ core.efi
+   ┃  ┃  ┗━ ...
+   ┃  ┠─ localboot
+   ┃  ┠─ menu
+   ┃  ┠─ debian-bookworm
+   ┃  ┠─ ol8u9
+   ┃  ┠─ grub.cfg → menu
+   ┃  ┠─ docker-vm → localboot
+   ┃  ┠─ test-pve → debian-bookworm
+   ┃  ┠─ test-lvm → ol8u9
+   ┃  ┠─ grub.cfg-01-bc-24-11-51-cf-fc → docker-vm
+   ┃  ┠─ grub.cfg-01-bc-24-11-4b-ae-dd → test-pve
+   ┃  ┖─ grub.cfg-01-bc-24-11-28-81-76 → test-lvm
+   ┣━ debian-bookworm/
+   ┃  ┠─ initrd.gz
+   ┃  ┠─ linux
+   ┃  ┖─ preseed-noswap.cfg
+   ┣━ ol8u9/
+   ┃  ┣━ images/
+   ┃  ┃  ┖─ install.img
+   ┃  ┠─ initrd.img
+   ┃  ┠─ ks.cfg
+   ┃  ┖─ vmlinuz
+   ┗━ gentoo/
+      ┠─ gentoo
+      ┖─ initramfs.igz
+</code>
+The most logic lies within ''/srv/tftp/grub'' directory. There are several boot configurations:
+  * ''localboot'' — exit a bootloader, causing firmware to try next configured boot source
+  * ''menu'' — present a human-friendly menu, with a timeout to do same thing as ''localboot''
+  * ''debian-bookworm'' — start an unattended installation of a Debian 12 release
+  * ''ol8u9'' — start an unattended installation of a Oracle Linux 8 release
+The rest items are symlinks to these files:
+  * ''grub.cfg'' — will be used by GRUB if no other configuration file was matched; here it symlinks to the ''menu'', but can be diverted anywhere else
+  * ''test-lvm'', ''test-pve'' and ''docker-vm'' are VMs or HW machines which are provisioned using this system, and in particular, ''test-lvm'' is being provisioned with OL8, ''test-pve'' is being provisioned with Debian and the ''docker-vm'' is already installed, so it is instructed to boot locally
+  * ''grub.cfg-01-xx-xx-xx-xx-xx-xx'' are symlinks named after the respective system's MAC address. If there are several NICs, we create several files, for each MAC. GRUB will first try such a file and load it if it exists, and if it doesn't, it will load ''grub.cfg''. This way, we precisely target each system to divert its boot path independently of each other.
+This approach is new to me, but it can support both "legacy" and EFI clients, and what's even more appelaing, with a single set of configuration files.
+To generate the contents of that directory (including GRUB images themselves) a Gentoo system was used, where GRUB was emerged with ''%%GRUB_PLATFORMS="i386 x86_64-efi"%%'', with the following commands:
+<code>
+grub-mknetdir --net-directory=/srv/tftp --subdir=/grub -d /usr/lib/grub/i386-pc
+grub-mknetdir --net-directory=/srv/tftp --subdir=/grub -d /usr/lib/grub/x86_64-efi
+</code>
+Then the ''/srv/tftp/grub'' directory was simply archived and transferred to the boot server.
+<tabbox PXELinux>
+<code>
+srv/
+┠─ dhcp-hostsfile.cfg
+┗━ tftp/
+   ┠─ pxelinux.0
+   ┠─ ldlinux.c32
+   ┣━ pxelinux.cfg/
+   ┃  ┠─ harddisk
+   ┃  ┠─ menu
+   ┃  ┠─ debian-bookworm
+   ┃  ┠─ ol8u9
+   ┃  ┠─ default → menu
+   ┃  ┠─ docker-vm → harddisk
+   ┃  ┠─ test-pve → debian-bookworm
+   ┃  ┠─ test-lvm → ol8u9
+   ┃  ┠─ 01-bc-24-11-51-cf-fc → docker-vm
+   ┃  ┠─ 01-bc-24-11-4b-ae-dd → test-pve
+   ┃  ┖─ 01-bc-24-11-28-81-76 → test-lvm
+   ┣━ debian-bookworm/
+   ┃  ┠─ initrd.gz
+   ┃  ┠─ linux
+   ┃  ┖─ preseed-noswap.cfg
+   ┣━ ol8u9/
+   ┃  ┣━ images/
+   ┃  ┃  ┖─ install.img
+   ┃  ┠─ initrd.img
+   ┃  ┠─ ks.cfg
+   ┃  ┖─ vmlinuz
+   ┗━ gentoo/
+      ┠─ gentoo
+      ┖─ initramfs.igz
+</code>
+The most logic is within ''/srv/tftp/pxelinux.cfg'' directory. There are several boot configurations:
+  * ''harddisk'' — boot from local hard disk
+  * ''menu'' — present a human-friendly menu, with a timeout to do the same thing as ''harddisk''
+  * ''debian-bookworm'' — start an unattended installation of a Debian 12 release
+  * ''ol8u9'' — start an unattended installation of a Oracle Linux 8 release
+The rest items are symlinks to these files:
+  * ''default'' — will be used by PXELinux if no other configuration file was matched; here it symlinks to the ''menu'', but can be diverted anywhere else
+  * ''test-lvm'', ''test-pve'' and ''docker-vm'' are VMs or HW machines which are provisioned using this system, and in particular, ''test-lvm'' is being provisioned with OL8, ''test-pve'' is being provisioned with Debian and the ''docker-vm'' is already installed, so it is instructed to boot locally
+  * ''01-xx-xx-xx-xx-xx-xx'' are symlinks named after the respective system's MAC address. If there are several NICs, we create several files, for each MAC. PXELinux will first try such a file and load it if it exists, and if it doesn't, it will load ''default''. This way, we precisely target each system to divert its boot path independently of each other.
+This is well tested setup. The main downside is that PXELinux doesn't support EFI boot, and we need to use some another bootloader which is capable of targeting computers precisely and manage its configuration separately.
+</tabbox>
+Boot menu can contain other entries if needed, for example, it can boot [[https://wiki.gentoo.org/wiki/Installation_alternatives#Diskless_install_using_PXE_from_the_LiveCD|Gentoo LiveCD converted for PXE boot]]. It defaults to booting from hard disk after timeout.
+We need some more work in preseed/kickstart/whatever autoinstall partitioning schemes, to support both "legacy" and EFI partitioning and boot settings, or develop another generation of autoinstall files for EFI and select which one to present in a bootloader.
+==== DNS, DHCP and TFTP servers ====
+<tabbox dnsmasq>
 The default Debian's ''dnsmasq'' configuration can use the base configuration //and// the drop directory. We opt to use the latter.
 The most configuration is ''/etc/dnsmasq.d/base.conf'':
 <file - /etc/dnsmasq.d/base.conf>
 no-dhcp-interface = tun0
+# upstream DNS
 server = 10.226.130.130
 server = 10.226.130.131
 no-resolv
-domain = test.ucom.am, 172.31.1.0/24,   local
+domain = auto.example.org, 172.31.1.0/24,   local
-domain = test.ucom.am, 172.31.255.0/24, local
+domain = auto.example.org, 172.31.255.0/24, local
 domain-needed
 expand-hosts
-# no dynamic allocation for DHCP service on this subnet; static allocation via dhcp-host will still be possible
+# no dynamic allocation for DHCP service on these subnets; static allocation via dhcp-host and dhcp-hostsfile will still be possible
 dhcp-range = 192.168.205.224, static
 dhcp-range = 172.31.255.0,    static
@@ Line 42: / Line 184: @@
 dhcp-match = set:http, option:client-arch, 16
-dhcp-boot = tag:bios, pxelinux.0
+dhcp-boot = tag:bios, grub.0
-dhcp-boot = tag:uefi, bootx64.efi
+dhcp-boot = tag:uefi, grub.efi
-#pxe-prompt = "Select a system to boot"
-#pxe-prompt = "Press F8 for menu.", 60
-#pxe-service = x86PC, "Boot from local disk"
-#pxe-service = x86PC, "Run pxelinux", pxelinux
 log-queries
@@ Line 54: / Line 191: @@
 </file>
-It references the ''/srv/dhcp-hostsfile.cfg'' where we can match certain MACs to IP addresses and names. This is mostly convenient for networks defined as "static DHCP ranges" above, since this way only hosts listed in the hostsfile will be provided a service. Also we can put other hosts there, for example, VMs, to fix IP addresses. Example contents is:
+It references the ''/srv/dhcp-hostsfile.cfg'' where we can match certain MACs to IP addresses and names. This is convenient for networks defined as "static DHCP ranges" above, since this way only hosts listed in the hostsfile will be provided a service. Also we can put other hosts there, for example, VMs, to pin IP addresses to somewhere outside the dynamic allocation range. Example contents is:
 <file - /srv/dhcp-hostsfile.cfg>
 BC:24:11:51:CF:FC, 172.31.1.5,  docker-vm
@@ Line 72: / Line 209: @@
 </file>
-I put auxiliary DNS records into a separate file too:
+Auxiliary DNS records are put into yet another separate file:
-<file /etc/dnsmasq.d/dns-auxiliary.conf ->
+<file - /etc/dnsmasq.d/dns-auxiliary.conf>
 cname = boot.test.ucom.am, vmgw.test.ucom.am
 </file>
-==== Mini HTTPD ====
+<tabbox ISC DHCPD, ISC BIND, tftpd-hpa>
-Needed for EL variants to be able to run installer from the network. We only set ''data_dir=/srv/tftp'' in ''/etc/mini-httpd.conf''; this directory is already served by TFTP; we can use a separate directory, but this setup feels simpler.
+  * [[https://www.hpe.com/psnow/resources/ebooks/a00112595en_us_v2/s_config_pxe_uefi.html|Configuring PXE servers for UEFI-based clients in a Linux environment]]
+</tabbox>
+==== HTTP server ====
+Needed for EL variants to be able to run installer from the network. I went as low as using ''mini_httpd'', the simplest I found. Only need to set ''data_dir=/srv/tftp'' in ''/etc/mini-httpd.conf''.
+This directory is already served via TFTP; we can use a separate directory, but this setup feels simpler. If it will be decided to serve private local repositories, which is infeasible to share over TFTP, it might be worth splitting the HTTP-only and TFTP-only parts.
 ===== Debian: preseed ====
+==== Network installer files ====
 Debian conveniently maintains all the required bits for the netboot as an archive distributed through its mirrors, available, for example, [[http://ftp.debian.org/debian/dists/stable/main/installer-amd64/current/images/netboot/netboot.tar.gz|here]]. From it, we need only two files (which can be also downloaded separately):
@@ Line 89: / Line 238: @@
 ==== Bootloader configuration ====
+<tabbox GRUB>
+<file - /srv/tftp/grub/debian-bookworm>
+set timeout=1
+menuentry 'Debian Bookworm PXE installer' {
+    linux debian-bookworm/linux priority=critical debian-installer/language=en debian-installer/country=AM debian-installer/locale=ru_RU.UTF-8 keyboard-configuration/xkb-keymap=ru netcfg/get_hostname=install keyboard-configuration/optionscode=grp:caps_lock_toggle,grp_led:scroll preseed/url=tftp://boot.test.ucom.am/debian-bookworm/preseed-noswap.cfg
+    initrd debian-bookworm/initrd.gz
+}
+</file>
+<tabbox PXELinux>
 <file - /srv/tftp/pxelinux.cfg/debian-bookworm>
 default debian-bookworm-auto
@@ Line 96: / Line 260: @@
     kernel debian-bookworm/linux
     append priority=critical initrd=debian-bookworm/initrd.gz debian-installer/language=en debian-installer/country=AM debian-installer/locale=ru_RU.UTF-8 keyboard-configuration/xkb-keymap=ru netcfg/get_hostname=install keyboard-configuration/optionscode=grp:caps_lock_toggle,grp_led:scroll preseed/url=tftp://boot.test.ucom.am/debian-bookworm/preseed-noswap.cfg
 </file>
+</tabbox>
 ==== Example preseed file ====
 Works for Bullseye too (it was actually originally developed for Bullseye, but worked with Bookworm like a charm)!
+<tabbox BIOS>
 <file - /srv/tftp/debian-bookworm/preseed-noswap.cfg>
@@ Line 226: / Line 394: @@
   in-target chmod 0600 /root/.ssh/authorized_keys;
 </file>
+<tabbox UEFI>
+</tabbox>
 ==== Additional documentation ====
@@ Line 238: / Line 410: @@
 Need to consider which else packages may be omitted.
-==== A directory tree ====
+==== Network installer files ====
 For Oracle Linux, the best is to obtain a "Boot UEK" ISO image, approx. 1G of size (obviously, the more "enterpriseish" is the system, the less efficient it is).
@@ Line 250: / Line 422: @@
 ==== Bootloader configuration ====
+<tabbox GRUB>
+<file - /srv/tftp/grub/ol8u9>
+set timeout=1
+menuentry 'Oracle Linux 8 PXE installer' {
+    linux ol8u9/vmlinuz inst.repo=http://vmgw/ol8u9 inst.ks.sendmac inst.ks=http://vmgw/ol8u9/ks.cfg
+    initrd ol8u9/initrd.img
+}
+</file>
+<tabbox PXELinux>
 <file - /srv/tftp/pxelinux.cfg/ol8u9>
 default ol8u9
@@ Line 259: / Line 445: @@
 </file>
-==== A complete working kickstart file ====
+</tabbox>
+==== Example kickstart file ====
+<tabbox BIOS>
 <file - /srv/tftp/ol8u9/ks.cfg>
 #platform=x86, AMD64, or Intel EM64T
@@ Line 366: / Line 557: @@
 %end
 </file>
-==== More documentation ====
+<tabbox UEFI>
+</tabbox>
+==== Additional documentation ====
   * https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/boot_options_for_rhel_installer/index
   * https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/performing_an_advanced_rhel_8_installation/kickstart-commands-and-options-reference_installing-rhel-as-an-experienced-user