This might just be the most exciting thing to happen to Linux since sliced bread (or maybe even since sliced cheese).
So what’s kdump and fadump? Well, they’re both tools that allow you to perform kernel dumps in case of a system crash or hang. But here’s the kicker with kdump, you can do it without having to reboot your entire system!
And if that wasn’t enough, fadump takes things up a notch by allowing you to perform kernel dumps while running in full memory mode (firmware assisted debugging). This means you don’t have to sacrifice any resources or performance just to get those precious kernel dumps.
Now, I know what some of you might be thinking “But how do we configure these magical tools on an IBM Z system?” Well, my friend, that’s where the real fun begins!
Before anything else, let’s make sure your kernel supports kdump and fadump. You can check this by running:
# This script checks if the kernel supports kdump and fadump on an IBM Z system.
# First, we use the grep command to search for the configuration option "CONFIG_KDUMP" in the /boot/config-$(uname -r) file.
# The $(uname -r) command returns the current kernel version, which is then used in the file path.
# The -r flag in grep indicates that we want to search for a regular expression.
# The output of this command should include "CONFIG_KDUMP=y" if the kernel supports kdump.
grep -r CONFIG_KDUMP /boot/config-$(uname -r)
# Next, we use the grep command again to search for the configuration option "CONFIG_FADUMP" in the same file.
# The output of this command should include "CONFIG_FADUMP=y" if the kernel supports fadump.
grep -r CONFIG_FADUMP /boot/config-$(uname -r)
If you see “y” for both kdump and fadump, then congratulations! You’re already halfway there.
Next, let’s create a new partition for our dump files:
# Create a new partition table on the specified device using the GPT format
sudo parted /dev/sdb mklabel gpt
# Create a new primary partition on the specified device with the ext4 file system, starting at 100MiB and ending at 2GiB
sudo parted /dev/sdb mkpart primary ext4 100MiB 2GiB
# Format the newly created partition with the ext4 file system
sudo mkfs.ext4 /dev/sdb2
# Mount the newly created partition to the specified mount point
sudo mount /dev/sdb2 /mnt/dump
Now, let’s configure kdump and fadump:
# This script is used to configure kdump and fadump by editing the /etc/sysctl.d/kdump-fadump.conf file.
# The first line uses the "echo" command to add the line "vm.panic_on_oops=1" to the /etc/sysctl.d/kdump-fadump.conf file.
echo "vm.panic_on_oops=1" >> /etc/sysctl.d/kdump-fadump.conf
# The second line uses the "echo" command to add the line "vm.crashkernel=2G:/mnt/dump" to the /etc/sysctl.d/kdump-fadump.conf file.
echo "vm.crashkernel=2G:/mnt/dump" >> /etc/sysctl.d/kdump-fadump.conf
# The third line uses the "sysctl" command to apply the changes made in the /etc/sysctl.d/kdump-fadump.conf file.
sysctl -p /etc/sysctl.d/kdump-fadump.conf
And finally, let’s create a new initramfs image that includes kdump and fadump:
# Edit /etc/initramfs-tools/modules
# Add "kdump" and "fadump" to the modules file
echo "kdump" >> /etc/initramfs-tools/modules
echo "fadump" >> /etc/initramfs-tools/modules
# Regenerate initramfs image
# Use sudo to run the command as root
sudo update-initramfs -u
And that’s it! You should now be able to test kdump and fadump by running:
# This script is used to test the functionality of kdump and fadump by triggering a kernel panic and checking for dump files in the specified directory.
# Trigger a kernel panic by writing "oops" to the /proc/sysrq-trigger file.
echo "oops" > /proc/sysrq-trigger
# Check for dump files in the /mnt/dump directory.
ls -lh /mnt/dump
# The output of the ls command should show the total size of the files in the directory, as well as the permissions, owner, and date of creation for each file.
# The first line of the output should be "total 16M", indicating that the total size of the files in the directory is 16 megabytes.
# The second line shows the permissions, owner, and date of creation for the "kdump_fadump" directory.
# The third line shows the permissions, owner, and date of creation for the "vmcore-3.10.0-927-gse2eba" file.
# The "vmcore" file is the actual dump file that contains information about the kernel panic.
And there you have it, Configuring kdump and fadump on an IBM Z system is as easy as pie (or maybe even easier)!
So go ahead, give it a try, and let us know how it goes. And if you’re feeling adventurous, why not try out some of the other cool tools that come with Linux on an IBM Z system? Who knows what kind of magic you might discover!