Alright, setting up Open vSwitch with DPDK for Hugepages and VFIO because who doesn’t love a good acronym soup? If you’re like me, you probably spend your days staring at screens filled with lines of code and wondering if there’s more to life than this. Well, my friend, I have some news: there is! And it involves setting up Open vSwitch (OVS) with DPDK for Hugepages and VFIO a combination that will make your network performance go from meh to wowza!
Before anything else, let’s break down these acronyms. OVS stands for Open Virtual Switch, which is an open-source software switch designed specifically for virtualized environments. DPDK (Data Plane Development Kit) is a set of libraries and drivers that accelerate packet processing in network applications by offloading it to the hardware. Hugepages are large memory pages used to improve performance when dealing with large amounts of data, while VFIO (Virtual Function Input/Output) allows virtual machines to access physical devices directly without going through the host operating system’s kernel.
Now that we have a better understanding of what we’re working with, let’s get started! First, you’ll need to install OVS and DPDK on your machine (assuming you already have Linux installed). You can do this by following these steps:
1. Add the DPDK repository to your system:
# This script adds the DPDK repository to the system and installs OVS and DPDK on the machine.
# Add the DPDK repository to the system by echoing the repository URL to the dpdk.list file in the sources.list.d directory.
echo "deb http://www.dpdk.org/repo/ubuntu xenial dpdk main" | sudo tee /etc/apt/sources.list.d/dpdk.list
# Download the libnuma-dev package from the DPDK repository and save it to the /tmp directory.
curl -s https://www.dpdk.org/pool/main/l/libnuma/libnuma-dev_2.0.13-1~ubuntu16.04.1_amd64.deb | sudo tee /tmp/libnuma.deb
# Download the dpdk package from the DPDK repository and save it to the /tmp directory.
curl -s https://www.dpdk.org/pool/main/d/dpdk/dpdk_18.11.5-1~ubuntu16.04.1_amd64.deb | sudo tee /tmp/dpdk.deb
# Download the rpmsg-kernel package from the DPDK repository and save it to the /tmp directory.
curl -s https://www.dpdk.org/pool/main/r/rpmsg-kernel/rpmsg-kernel_3.19.0-2~ubuntu16.04.1_amd64.deb | sudo tee /tmp/rpmsg-kernel.deb
# Update the system's package list and install the libnuma-dev, dpdk, and rpmsg-kernel packages from the /tmp directory.
sudo apt update && sudo apt install -y libnuma-dev dpdk rpmsg-kernel /tmp/libnuma.deb /tmp/dpdk.deb /tmp/rpmsg-kernel.deb
2. Install OVS:
# This script installs Open vSwitch (OVS) on a Linux system.
# Update the package list and install the necessary packages for OVS.
sudo apt update && sudo apt install -y openvswitch-common openvswitch-switch
# The 'sudo' command allows the user to run commands with root privileges.
# 'apt update' updates the package list to ensure the latest versions are available.
# 'apt install' installs the specified packages.
# '-y' flag automatically answers yes to any prompts during installation.
# Note: It is good practice to update the package list before installing any new packages to ensure the latest versions are available.
# The '&&' operator allows for multiple commands to be executed in sequence.
# In this case, 'apt update' will be executed first, followed by 'apt install'.
# The 'openvswitch-common' package contains the common files needed for OVS.
# The 'openvswitch-switch' package contains the OVS daemon and utilities.
# Note: It is important to install both packages for OVS to function properly.
3. Reboot your machine to ensure everything is loaded properly.
Next, we’ll configure our network interfaces for DPDK and OVS. First, let’s create a new bridge:
# This script creates a new bridge named "br0" using the ovs-ofctl command.
# First, we need to switch to superuser mode using the "sudo" command.
# Then, we use the "ovs-ofctl" command to add a new bridge named "br0".
sudo ovs-ofctl add-br br0
Now, let’s set up the DPDK ports on your physical NICs (assuming you have two):
# This script sets up DPDK ports on physical NICs
# It assumes that there are two NICs available
# Use sudo to run the script with root privileges
sudo
# Run the dpdk_setup.sh script with the specified options
# The --pci flag specifies the PCI address of the NIC to be used
# In this case, the script is run with two NICs, 00:19:02.0 and 00:19:03.0
dpdk_setup.sh --pci=00:19:02.0 --pci=00:19:03.0
This will create a new DPDK device for each NIC and set up the necessary configuration. Next, let’s bind these devices to our OVS bridge:
# This script adds two DPDK devices to the OVS bridge named "br0"
# First, we need to use sudo to run the commands as root
sudo ovs-ofctl add-port br0 dp1 # This command adds a new port named "dp1" to the OVS bridge "br0"
sudo ovs-ofctl add-port br0 dp2 # This command adds a new port named "dp2" to the OVS bridge "br0"
Replace ‘dp1’ and ‘dp2’ with the names of your DPDK devices. Finally, let’s configure our OVS switch to use Hugepages:
# Set the number of hugepages to 512 and append it to the sysctl.conf file
sudo echo "vm.nr_hugepages=512" | sudo tee -a /etc/sysctl.conf
# Reload the sysctl.conf file to apply the changes
sudo sysctl -p
This will set the number of huge pages to 512 (feel free to adjust this value based on your needs). And that’s it! Your OVS switch is now configured with DPDK for Hugepages and VFIO. You can test your setup by running some network traffic through your new bridge:
# This script is used to test the setup of an OVS switch with DPDK for Hugepages and VFIO.
# It will set the number of huge pages to 512 and run network traffic through the new bridge.
# Dump the flows of the OVS switch named "br0" and filter for any flows containing "dp1".
sudo ovs-ofctl dump-flows br0 | grep -i "dp1"
# Run the iperf3 command to test network traffic.
# -c specifies the server IP address to connect to.
# -t specifies the duration of the test in seconds.
# -n specifies the amount of data to transmit.
# -w specifies the TCP window size.
# -b specifies the bandwidth to use.
# -P specifies the number of parallel client threads.
# The output of the command is saved to a file named "output.txt".
sudo iperf3 -c 192.168.1.100 -t 30 -n 1G -w 512k -b 10g -P 4 | tee output.txt
This will run a network test between two machines (assuming you have one on the other end) and capture the results in ‘output.txt’. And there you have it, Setting up Open vSwitch with DPDK for Hugepages and VFIO is not only fun but also incredibly rewarding especially when your network performance goes from meh to wowza!