Buffers

The commands in this section control the amount of memory pre-allocated by the dataplane for buffers.

Buffers per NUMA

Systems with multiple CPU sockets and Non-uniform memory access (NUMA) capabilities may need specific tuning to ensure that enough buffer space is available for the number of separate NUMA nodes. The number of NUMA nodes is typically the number of populated CPU sockets. Specifically, the scenarios which require tuning typically involve a large number of interfaces combined with multiple CPU worker threads.

Note

This refers to separate hardware CPUs, not a single CPU with multiple cores.

The dataplane buffers buffers-per-numa <buffers-per-numa> command allocates the given number of buffers for each CPU socket (e.g. 16384).

Default Data Size

The dataplane buffers default-data-size <default-data-size> controls the default size of each buffer, in bytes (e.g. 2048).

API Segment

Commands in this section configure memory allocation for the dataplane API segment, which is the binary API used internally between TNSR, VPP, and various other parts of the system.

dataplane api-segment api-size <mem-size>:

The amount of memory allocated for the API region. Default value is 16M. This value must be less than the global-size.

dataplane api-segment global-size <mem-size>:

The amount of memory shared across all dataplane instances, packet buffers, and other similar global areas. Default value is 64M.

dataplane api-segment api-pvt-heap-size <mem-size>:

The amount of memory allocated to the private memory heap for the API. Default value is 128K.

dataplane api-segment global-pvt-heap-size <mem-size>:

The amount of memory allocated to the private memory heap for the global VM. Default value is 128K.

Memory

Commands in this section configure memory allocation for the dataplane.

dataplane memory main-heap-size heap-size [<size>]:

Defines the amount of memory to be allocated for the main memory heap. This includes ACL data, ACL hash data (e.g. reflect sessions), NAT data, and the dataplane FIB (IPv4 and IPv6), among other uses.

For more information, see Memory Usage and Tuning.

Warning

If this value is undersized and the main heap memory is exhausted, the dataplane may crash.

Note

When tuning this value, also consider increasing the Statistics Segment heap-size.

Warning

Increasing this beyond the default allocated huge pages size of 2GB will also require increasing that huge page allocation to match. See sysctl vm nr_hugepages under Host Memory Management Configuration for details.

dataplane memory main-heap-page-size (default|4k|2m|1g):

Defines the memory page size for the main heap. Memory is allocated is chunks of this size. Larger values will allocate larger amount of memory faster, but are not as granular as smaller values.

Tip

The current best practice is to use 2m for the page size to avoid delays in memory allocation.

dataplane ip6 hash-buckets [<size>]:

Defines the number of IPv6 forwarding table hash buckets. The default is 65536.