Tip
This is the documentation for the 20.10 version. Looking for the documentation of the latest version? Have a look here.
CPU Workers and Affinity¶
The dataplane has a variety of commands to fine-tune how it uses available CPU resources on the host. These commands control CPU cores TNSR will use, both the number of cores and specific cores.
See also
Cores defined here may also be pinned to interface receive (RX)
queues, provided that cores are defined using either the corelist-workers
or coremask-workers
methods. See Interface Configuration Options for
details.
Default Worker Behavior¶
The host operating system and daemons reserve core 0
by default,
unless another core is reserved elsewhere and core 0
is reassigned. For
example, if skip-cores
is set to a value other than 0
, and core 0
is
assigned to another task, then the host OS will use one of the other available
skipped cores.
Note
Core 0
could be used as a last resort by automatic assignment
(e.g. workers
with skip-cores
or default main-core
) if no other
cores are available.
The dataplane reserves core 1
by default. Workers may not be assigned to
core 1
unless the dataplane is moved to a different core using main-core
as detailed in the next section. An exception to this is a single core system in
which case, the main-core
would be 0
since it is the only available
core.
The dataplane does not use any additional cores by default until configured
using one of the available methods (workers
, corelist-workers
,
coremask-workers
). Given that two cores are reserved for the host OS and
dataplane, the amount of cores available for use by workers is always 2
less
than the total number of cores.
Worker Configuration¶
- dataplane cpu corelist-workers <first> [- <last>]:
Defines a specific list of CPU cores to be used by the dataplane. The command supports adding single cores to the list at a time, or ranges of cores. Run the command multiple times with different core numbers or ranges to define the full list of cores to utilize. When removing items with
no
, the command accepts a specific core to remove from the list.- dataplane cpu coremask-workers <mask>:
Similar to
corelist-workers
, but the cores are defined as a hexadecimal mask instead of a list. For example,0x0000000000C0000C
- dataplane cpu main-core <n>:
Assigns the main dataplane process to a specific CPU core. The default core is
1
, and core0
should not be used as it is reserved for the operating system.- dataplane cpu scheduler-policy (batch|fifo|idle|other|rr):
Defines a specific scheduler policy for worker thread processor usage allocation
- batch:
Scheduling batch processes. Uses dynamic priorities based on
nice
values in the host OS, but always gives the thread a small scheduling penalty so that other processes take precedence.- fifo:
First in-first out scheduling. Will preempt other types of threads and threads with a lower priority.
- idle:
Scheduling very low priority jobs.
- other:
Default Linux time-sharing scheduling. Uses dynamic priorities based on
nice
values in the host OS, similar tobatch
but without the built-in penalty.- rr:
Round-robin scheduling. Similar to
fifo
but each thread is time-limited
- dataplane cpu scheduler-priority <n>:
For the
fifo
andrr
scheduler policies, this number sets the priority of processes for the dataplane. It can be any number between 1 (low) and 99 (high).- dataplane cpu skip-cores <n>:
Defines the number of cores to skip when creating additional worker threads, in the range of
0
to the highest available core number. The first<n>
cores will not be used by worker threads.Note
This does not affect the core used by the main thread, which is set by
dataplane cpu main-core <n>
. A skipped core can, however, be used by the host OS instead of core0
.Warning
This option is incompatible with interface RX queue core pinning. To utilize interface RX queue core pinning, define a list of cores using either
corelist-workers
orcoremask-workers
instead.- dataplane cpu workers <n>:
Defines the number of worker threads to create for the dataplane. The placement of workers on CPU cores is handled automatically but can be influenced by
skip-cores
.Note
The number of worker threads is in addition to the main process. For example, with a worker count of
4
, the dataplane will use one main process with four worker threads, for a total of five threads.The maximum value for
workers
is2
less than the total number of available cores. This is because one core is reserved for the host, and one for the dataplane itself.Warning
This option is incompatible with interface RX queue core pinning. To utilize interface RX queue core pinning, define a list of cores using either
corelist-workers
orcoremask-workers
instead.
Note
The workers
, corelist-workers
and coremask-workers
methods
are all mutually exclusive. Only one of these methods of defining workers is
allowed in the configuration at any time.
Worker Example¶
This example sets four additional worker threads, and instructs the dataplane to skip one core when assigning worker threads to cores:
tnsr(config)# dataplane cpu workers 4
tnsr(config)# dataplane cpu skip-cores 1
tnsr(config)# service dataplane restart
Worker Status¶
The show dataplane cpu threads command displays the current dataplane process list, including the core usage and process IDs. This output corresponds to the example above:
tnsr(config)# show dataplane cpu threads
ID Name Type PID LCore Core Socket
-- -------- ------- ---- ----- ---- ------
0 vpp_main 2330 1 0 0
1 vpp_wk_0 workers 2346 2 2 0
2 vpp_wk_1 workers 2347 3 3 0
3 vpp_wk_2 workers 2348 4 4 0
4 vpp_wk_3 workers 2349 5 5 0
The output includes the following columns:
- id:
Dataplane thread ID.
- name:
Name of the dataplane process.
- type:
The type of thread, which will be blank for the main process.
- pid:
The host OS process ID for each thread.
- LCore:
The logical core used by the process.
- Core:
The CPU core used by the process.
- Socket:
The CPU socket associated with the core used by the process.