Differences

This shows you the differences between two versions of the page.

--- notes:proxmox [2024/04/02 13:09] – bootloader and time sync Nikita Kipriyanov
+++ notes:proxmox [2024/06/26 06:33] (current) – [ECC error notifications] Nikita Kipriyanov
@@ Line 1: / Line 1: @@
 ====== Proxmox virtualization cluster ======
+  * LXC images collection: http://download.proxmox.com/images/system/
 ===== Storage =====
 ===== Network structure =====
@@ Line 6: / Line 8: @@
 ==== Bootloader ====
-At least when installation was done using serial console, the Proxmox installer configures system in a very useful manner, so that bootloader and the kernel appear both on the serial and the VGA console. This allows later convenient access to the system via IPMI Serial-over-LAN. To replicate this setup for Debian setups, one could use the following settings:
+=== Console redirection, VSP, IPMI SoL ===
+At least when installation was done using serial console, the Proxmox installer configures system in a very useful manner, so that bootloader and the kernel appear both on the serial and the VGA console:
 <file - /etc/default/grub.d/installer.cfg>
@@ Line 14: / Line 18: @@
 GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX    console=ttyS0,115200"
 </file>
+This allows later convenient access to the system via IPMI Serial-over-LAN. The inconvenient part of this that it diverts kernel boot messages (''kmsg'') away from VGA console. It's possible to show them both on VGA and TTY, for which we create yet another file:
+<file - /etc/default/grub.d/vgaconsole.cfg>
+GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX console=tty0"
+</file>
+I also don't like the default "quiet" option for boot messsages, so I override it:
+<file /etc/default/grub.d/verbose.cfg>
+GRUB_CMDLINE_LINUX_DEFAULT=""
+</file>
+In case of installation over Debian, where Proxmox installer did not run, all this setup needs to be replicated:
+<file - /etc/default/grub.d/console.cfg>
+GRUB_CMDLINE_LINUX_DEFAULT=""
+GRUB_CMDLINE_LINUX="console=tty0 console=ttyS1,57600n8"
+GRUB_TERMINAL="serial console"
+GRUB_SERIAL_COMMAND="serial --speed=57600 --unit=1 --word=8 --parity=no --stop=1"
+</file>
+Other COM port is used with another speed here, just for illustration. Notice also, instead of separate ''GRUB_TERMINAL_INPUT'' and ''GRUB_TERMINAL_OUTPUT'' I use a single setting for both.
+==== Fast reboots with kexec ====
+FIXME this needs more work
+  * https://forum.proxmox.com/threads/tip-fast-reboots-with-kexec.35624/
+  * https://forum.proxmox.com/threads/proxmox-7-fast-reboot-with-kexec.93422/
 ==== Time sync ====
@@ Line 23: / Line 55: @@
 chronyc reload sources
 </code>
+==== ECC error notifications ====
+Install `rasdaemon` utility to receive reports from hardware via EDAC interface and get them into logs.
+See [[https://www.setphaserstostun.org/posts/monitoring-ecc-memory-on-linux-with-rasdaemon/]]
 ==== Backup ====
@@ Line 34: / Line 72: @@
 ''0'' means "use half of available cores". If you know the number of the cores in the target system, you can use other, more tailored settings here. We set ''pigz'' too, just for the case somebody changes to gzip.
+**Note:** it's possible to [[https://forum.proxmox.com/threads/reading-blob-files-qemu-server-conf-blob.95551/|read blob files]] directly on the server, either with `proxmox-backup-debug` or "by hand":
+<code>
+proxmox-backup-debug inspect file /path/to/blob --decode -
+dd if=/path/to/blob bs=1 skip=12 | zstdcat
+</code>
 === Node backup ===
-Hosts may need backup, too. Theoretically we need "thin" backups as barely needed for the recovery, as explained here: https://pve.proxmox.com/wiki/Proxmox_VE_4.x_Cluster#Re-installing_a_cluster_node . Yet, I still find it useful to do just full host backup. It will not be very large (in my experience, 2 GB at most), it will be strongly deduplicated (between backups of same node and between nodes — they're similar), so why bother?
+Hosts may need backup, too. Theoretically we need "thin" backups as barely needed for the recovery, as explained here: https://pve.proxmox.com/wiki/Proxmox_VE_4.x_Cluster#Re-installing_a_cluster_node . Yet, I still find it useful to do just full host backup. It will not be very large (in my experience, around 5 GB), it will be strongly deduplicated (between backups of same node and between nodes — they're similar), so why bother?
 It is useful to create a simple shell script and run it, say, mounthly:
@@ Line 45: / Line 89: @@
 export PBS_REPOSITORY=<token name>@<server>:<datastore>
 export PBS_PASSWORD=<token secret>
+NS=<namespace>
+NOTES=$(hostname -f)
+TMP=$(mktemp -d -p /dev/shm)
 if mountpoint -q /boot/efi
 then
     # for modern UEFI boot
-    proxmox-backup-client backup --ns <namespace> root.pxar:/ pve.pxar:/etc/pve esp.pxar:/boot/efi
+    proxmox-backup-client backup --ns ${NS} root.pxar:/ pve.pxar:/etc/pve exp.pxar:/boot/efi 2>&1 | tee ${TMP}/client.log
 else
     # for legacy BIOS boot
-    proxmox-backup-client backup --ns <namespace> root.pxar:/ pve.pxar:/etc/pve sda1.img:/dev/sda1
+    proxmox-backup-client backup --ns ${NS} root.pxar:/ pve.pxar:/etc/pve sda1.img:/dev/sda1 2>&1 | tee ${TMP}/client.log
 fi
+SNAPSHOT=$(grep "Starting backup:" ${TMP}/client.log | cut -d':' -f 3-)
+proxmox-backup-client snapshot upload-log --ns ${NS} ${SNAPSHOT} ${TMP}/client.log
+rm -rf ${TMP}
+proxmox-backup-client snapshot notes update --ns ${NS} ${SNAPSHOT} ${NOTES}
 </file>