SR-IOV Enablement
This page describes the procedure required in order to set up SR-IOV.

Overview

Many hardware vendors ship their products with the SR-IOV feature disabled. On such platforms, the feature must be enabled prior to installing the Weka system. This enablement applies to both the server BIOS and the NIC. If already enabled, it is recommend to verify the current state before proceeding with the installation of the Weka system.
This section assumes that NIC drivers have been installed and loaded successfully. If this is not the case, complete the procedure described in NIC Driver Installation and then continue as described below.

SR-IOV Enablement in the Server BIOS

Refer to the screenshots below to enable the SR-IOV support in the server BIOS.
Note: The following screenshots are vendor-specific and provided as a courtesy. Depending on the vendor, the same settings may appear differently or be located in other places. Therefore, always refer to your hardware platform and NIC vendor documentation for the latest information and updates.
Reboot Server and Force it to Enter the BIOS Setup
Locate the PCIe Configuration and Drill Down
Locate SR-IOV Support and Drill Down
Enable SR-IOV Support
Save and Exit

SR-IOV Enablement in the Mellanox NICs

While it is possible to change the SR-IOV configuration through the NIC BIOS, Mellanox OFED offers command line tools that allow for the convenient modification and validation of SR-IOV settings, as described below:
Step 1: Run Mellanox Software Tools (mst).
1
# mst start
Copied!
Step 2: Identify the device node for PCIe configuration access to the connected NIC device to be used with the Weka system.
1
# ibdev2netdev
2
mlx5_0 port 1 ==> enp24s0 (Up)
3
mlx5_1 port 1 ==> ib0 (Down)
4
mlx5_2 port 1 ==> ib1 (Down)
5
mlx5_3 port 1 ==> ib2 (Down)
6
mlx5_4 port 1 ==> ib3 (Down)
Copied!
Using the output received from the above, ascertain the following:
  • The host is equipped with 5 Mellanox ports.
  • Only one of the ports (the one marked Up) has connectivity to the switch.
  • The connected port name is enp24s0. The Mellanox notation of the NIC is mlx5_0.
Step 3: Using the Mellanox device notation, find the device node that can be used for PCIe configuration access of the NIC.
1
# mst status -v | grep mlx5_0
2
ConnectX4(rev:0) /dev/mst/mt4115_pciconf0 18:00.0 mlx5_0 net-enp24s0 0
Copied!
Step 4: Using the PCIe access device node, check the current SR-IOV setting on the NIC.
1
# mlxconfig -d /dev/mst/mt4115_pciconf0 q | grep -e SRIOV_EN -e VFS
2
NUM_OF_VFS 0
3
SRIOV_EN False(0)
Copied!
Step 5: Modify the SR-IOV settings. In the following example, the SR-IOV is enabled and the number of Virtual Functions (VFs) is set to 16.
1
# mlxconfig -y -d /dev/mst/mt4115_pciconf0 set SRIOV_EN=1 NUM_OF_VFS=16
Copied!
Step 6: Reboot the host.
Step 7: On completion of the server reboot, validate the SR-IOV settings.
1
# mst start && mlxconfig -d /dev/mst/mt4115_pciconf0 q | grep -e SRIOV_EN -e VFS
2
Starting MST (Mellanox Software Tools) driver set
3
Loading MST PCI module - Success
4
Loading MST PCI configuration module - Success
5
Create devices
6
-W- Missing "lsusb" command, skipping MTUSB devices detection
7
Unloading MST PCI module (unused) - Success
8
NUM_OF_VFS 16
9
SRIOV_EN True(1)
Copied!
This concludes the SR-IOV enablement procedure.
Last modified 1mo ago