Kernel Configuration¶

Kernel Command Line Arguments¶

TNSR is capable of managing boot-time kernel command line arguments so that they do not need to be maintained by hand.

Kernel command line arguments encompass a wide range of purposes such as enabling alternate consoles, isolating CPUs, changing hugepage size and allocation, blacklisting drivers/modules, and increasing logging and debug information. There are far too many different arguments to document them all and why they are useful, so refer to documentation on the Linux kernel for additional information.

When TNSR manages these entries, TNSR handles updating the kernel command line arguments in the boot configuration, there no need for the user to make any other changes outside of the TNSR CLI.

Automatic IOMMU Configuration¶

TNSR can automatically manage the kernel command line arguments for IOMMU, rather than having to specify them in manual arguments.

There are two parameters, an Intel-specific enable/disable option and a general option.

To configure the Intel-specific IOMMU argument, use:

tnsr(config)# system kernel arguments auto intel_iommu (off|on)

The available options on that command are:

off:: Explicitly disable Intel IOMMU.
on:: Explicitly enable Intel IOMMU.

To configure the general IOMMU argument, use:

tnsr(config)# system kernel arguments auto iommu (nopt|off|on|pt)

The available options on that command are:

nopt:: Disables IOMMU pass through (uses DMA translation).
off:: Explicitly disable IOMMU.
on:: Explicitly enable Intel IOMMU.
pt:: Enables IOMMU pass through (no DMA translation).

Automatic CPU Isolation¶

When configuring CPU workers for TNSR via corelist-workers (CPU Workers and Affinity), it can be beneficial to also have the kernel treat those CPU cores as isolated CPUs which are untouched by the OS scheduler. This ensures that the CPU cores in question are always free to operate only for TNSR.

This automatic mode configures CPU isolation for the same cores defined in dataplane cpu corelist-workers so the user does not have to manually maintain the same core list in multiple places:

tnsr(config)# system kernel arguments auto isolcpus
Changes to kernel command line arguments will take effect after system is rebooted.

Manual¶

Arbitrary kernel command line arguments can be entered using the manual method. These custom manual directives are added to the end of the existing kernel command line.

Note

The default kernel command line on current installations includes definitions for both video and serial consoles. The kernel uses the first matching directive per console type (one video, one serial). When specifying manual parameters, existing parameters of that type are replaced. For example, adding a manual console= parameter will replace all existing console= parameters. In this case, specify both the new and existing parameters to keep to ensure they are all present. See Enabling the Serial Console for an example.

To manually define a list of isolated CPUs:

tnsr(config)# system kernel arguments manual isolcpus=1-3
Changes to kernel command line arguments will take effect after system is rebooted.

To enable IOMMU:

tnsr(config)# system kernel arguments manual intel_iommu=on iommu=pt
Changes to kernel command line arguments will take effect after system is rebooted.

Checking the Current Kernel Command Line¶

To see the current arguments passed to the kernel at boot time, check the kernel message buffer and/or boot log. For example:

$ sudo dmesg | grep Command
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-5.15.0-58-generic
root=/dev/mapper/ubuntu--vg-ubuntu--lv ro console=ttyS0,115200n8 console=tty0

Kernel Modules¶

VFIO noiommu¶

By default the vfio-pci interface driver runs with IOMMU disabled for increased compatibility with hardware and for performance reasons. However, certain use cases, such as those involving QAT, may require enabling IOMMU.

Warning

Changing this setting may introduce a performance penalty, prevent certain hardware from working, and potentially raise security concerns. Read the information under vfio-pci for details.

Warning

Enabling or disabling IOMMU for the VFIO PMD requires restarting the dataplane which will disrupt network traffic.

Use the following command to enable IOMMU for the VFIO PMD:

tnsr(config)# no system kernel modules vfio noiommu
tnsr(config)# service dataplane restart

To disable IOMMU for the VFIO PMD (default):

tnsr(config)# system kernel modules vfio noiommu
tnsr(config)# service dataplane restart