Commands in this section configure hardware settings for DPDK devices.
- dataplane dpdk dev <pci-id> (crypto|crypto-vf)
- dataplane dpdk dev (<pci-id>|<vmbus-uuid>) network [name <name>] [num-rx-queues [<rq>]] [num-tx-queues [<tq>]] [num-rx-desc [<rd>]] [num-tx-desc [<td>]] [tso (off|on)] [devargs <name>=<value>]
Configures a manually approved list of network interface PCI devices or Hyper-V/Azure VMBUS device UUIDs and their options. Typically the dataplane will automatically attempt to use eligible interfaces, but this command overrides that behavior by explicitly listing devices which will be used by the dataplane.
See Setup NICs in Dataplane for more information and examples for adding devices in this manner.
Adding devices in this way is not compatible with
dataplane dpdk blacklist, but when devices are listed manually via
dataplane dpdk devdenying in that way is unnecessary.
- name <name>
Sets a custom name for a network device in TNSR instead of the automatically generated name (
<Link Speed><Bus Location>). For example device
0000:06:00.0can have a custom name of
WANinstead of the default
GigabitEthernet6/0/0. Used for convenience and to make interface names self-documenting.
See Customizing Interface Names for additional details including limitations on names.
- num-rx-queues [<rq>] num-tx-queues [<tq>]
Receive and transmit queue sizes for this device.
- num-rx-desc [<rd>] [num-tx-desc [<td>]
Receive and transmit descriptor sizes for this device. Certain network cards, such as Fortville models, may need the descriptors set to
2048to avoid dropping packets at high loads.
- tso (on|off)
TCP segmentation offload (TSO). When enabled on hardware which supports TSO, packet data is offloaded to hardware in large quantities and the hardware handles segmentation into MTU-sized chunks rather than performing segmentation in software. This results in improved throughput as shifting the per-packet processing to hardware reduces the burden on the network stack. Disabled by default.
The default values for these configuration options can be set by
dataplane dpdk dev default network <options>. These default values are used by the dataplane when an interface does not have a specific value set. The
nameoption must be unique for each interface and thus does not support a default value.
- devargs <name>=<value>
Configures a device argument name and value pair with those components separated by
=. Device arguments enable or control optional features on a device. For example,
dataplane dpdk dev 0000:06:00.0 network devargs disable_source_pruning=1.
A single command can only set one name and value pair. However, it is possible to set multiple device arguments by running the command multiple times each one with a different device argument name and value pair.
The combined length of a each
name=valuepair must be 128 bytes or less.
Use extreme caution when forming these entries. Due to the large variation in drivers and acceptable parameters, TNSR cannot validate
An invalid or incompatible
devargentry will cause the dataplane to not attach to the affected interfaces correctly, rendering the interface inoperable until the
devargentry is corrected.
Each driver supports a different set of arguments. Look in the DPDK NIC Drivers Documentation for information on the arguments supported by a specific poll mode driver (PMD).
A few examples of DPDK documentation pages for PMDs with configuration options are:
- dataplane dpdk dev all-inactive-network
Iterates over all network devices on the system and adds individual configuration entries for any interface not in use by the host OS. This results in a set of configuration entries for DPDK network devices which is the same as if all of the interfaces had been defined manually. This behavior is equivalent to the default whitelisting in previous versions of TNSR.
This command only acts on interfaces on the system when the command is executed, and the results of the command are added to the configuration, not this directive. It does not automatically update the configuration if interfaces are added or removed at a later time.
- dataplane dpdk blacklist <vendor-id>:<device-id>
Prevents the dataplane from automatically attaching to any device which matches a specific PCI vendor and device identifier. Useful for preventing the dataplane from attaching to hardware devices which are known to be incompatible.
Listing devices in this way is not compatible with
dataplane dpdk dev.
- dataplane dpdk blacklist (<pci-id>|<vmbus-uuid>)
Similar to the previous form, but explicitly prevents the dataplane from attaching to a specific PCI device or Hyper-V/Azure VMBUS device UUID.
Listing devices in this way is not compatible with
dataplane dpdk dev.
- dataplane dpdk decimal-interface-names
Disabled by default. When set, interface names automatically generated by the dataplane will use decimal values for bus location values rather than hexadecimal values. Linux uses decimal values when forming interface names (e.g.
enp0s20f1), so administrators may find using decimal values more familiar.
For example, device ID
enp0s20f1in the host OS) would normally be
GigabitEthernet0/14/1since the value
14in the bus slot is in hexadecimal. With
decimal-interface-namesset, the name would be
- dataplane dpdk iova-mode (pa|va)
Manually configures the IO Virtual Addresses (IOVA) mode used by DPDK when performing hardware IO from user space. Hardware must use IO addresses, but it cannot utilize user space virtual addresses directly. These IO addresses can be either physical addresses (PA) or virtual addresses (VA). No matter which mode is set, these are abstracted to TNSR as IOVA addresses so it does not need to use them directly.
In most cases the default IOVA mode selected by DPDK is optimal.
For more detail on IOVA, consult the DPDK documentation.
Physical Address mode. IOVA addresses used by DPDK correspond to physical addresses, and both physical and virtual memory layouts match. This mode is safest from the perspective of the hardware, and is the mode chosen by default. Most hardware supports PA mode at a minimum.
The primary downside of PA mode is that memory fragmentation in physical space must also be reflected in virtual memory space.
Virtual Address mode. IOVA addresses do not follow the layout of physical memory; Physical memory is changed to match the virtual memory instead. Because virtual memory appears as one continuous segment, large memory allocations are more likely to succeed.
The primary downside of VA mode is that it relies on kernel support and the availability of IOMMU.
- dataplane dpdk log-level (alert|critical|debug|emergency|error|info|notice|warning)
Sets the log level for messages generated by DPDK. The default log level is
- dataplane dpdk lro
Enables Large Receive Offload (LRO) on compatible interfaces. When LRO is enabled, incoming connection streams are buffered in hardware until they can be reassembled and processed in large batches rather than processing each packet as it arrives individually. This can result in improved throughput as shifting the per-packet processing to hardware reduces the burden on the network stack. Disabled by default.
While this can improve performance in certain cases it also alters the incoming packets which may be undesirable in routing roles.
- dataplane dpdk no-multi-seg
Disables multi-segment buffers for network devices. Can improve performance, but disables jumbo MTU support.
Required for Mellanox devices.
This option is not currently compatible with Intel X552 10G network interfaces. When enabled on incompatible hardware this option can lead to instability such as dataplane crashes while under load.
- dataplane dpdk no-pci
Disables scanning of the PCI bus for interface candidates when the dataplane starts. By default, interfaces which are administratively down in the host OS can be selected for use by the dataplane.
- dataplane dpdk no-tx-checksum-offload
Disables transmit checksum offloading of TCP/UDP for network devices.
- dataplane dpdk outer-checksum-offload
Enable hardware checksum offload for tunnel packets. Requires
- dataplane dpdk tcp-udp-checksum
Enables receive checksum offloading of TCP/UDP for network devices. Disabled by default.
- dataplane dpdk telemetry
Enables the telemetry thread in DPDK to collect performance statistics. Disabled by default as it consumes resources and can decrease performance.
- dataplane dpdk uio-driver [<driver-name>]
Configures the UIO driver for interfaces.
For more information on the driver choices, see Interface Drivers.
The interface driver controls how the dataplane communicates with interfaces, either hardware or virtual functions. Certain types of interfaces may only be compatible with certain drivers, either because of the hardware or how they are utilized. For example, if the interface is directly attached to bare metal hardware or via hardware passthrough of virtual functions in a guest VM. When interfaces are compatible with more than one driver, certain drivers may offer increased performance or features that make it a better choice for a given workload or environment.
Dataplane interface drivers currently fall into two categories:
The procedure for changing drivers is covered in Interface Driver Management.
Mellanox devices use RDMA and not UIO, so changing the driver may not have any effect on their behavior. If a Mellanox device does not appear automatically, TNSR may not support that device.
The VFIO PCI driver (
vfio-pci) is a safer alternative to UIO and, in theory,
more widely compatible. It employs techniques to communicate with hardware from
userspace safely within defined boundaries. The VFIO driver framework is
device-agnostic so in theory can work on most interfaces without compatibility
concerns. VFIO support is built into the kernel and does not require loading an
This is the current default driver and the driver recommended by DPDK.
vfio-pci driver has compatibility issues with certain QAT devices,
including DH895x, C3xxx, and C62x devices. Though there is a way to
bypass the compatibility check and let it work,
the current best practice for users with QAT devices is to continue using the
vfio-pci driver is active, TNSR automatically configures
the driver with
noiommu mode for compatibility with QAT and other
functions. Some may consider
noiommu mode unsafe as it provides the user
full access to a DMA capable device without the security of I/O management.
Take this into consideration when choosing the
The IGB UIO driver (
igb_uio) handles hardware or virtual interfaces. It
supports a wide variety of hardware, including various Intel Ethernet
controllers. While not as safe as
vfio-pci, it can perform faster in certain
environments and workloads. TNSR must load an additional kernel module for this
driver when it is active.
Some devices, such as ENA and VMXNET3, have trouble setting up interrupts
igb_uio driver. For these devices, use the
The generic UIO PCI driver (
uio_pci_generic) can work with PCI hardware
interfaces which support legacy interrupts. This driver does not support virtual
Ethernet 700 Series Network Adapters based on the Intel Ethernet Controller
X710/XL710/XXV710 and Intel Ethernet Connection X722 are not compatible with
this driver. For these devices, use the