Tip

This is the documentation for the 23.11 version. Looking for the documentation of the latest version? Have a look here.

TNSR 23.06 Release Notes¶

About the TNSR 23.06 Release¶

This is a regularly scheduled TNSR software release including new features and bug fixes.

General¶

The default interface driver for new installations has changed from igb_uio to vfio-pci. This does not affect upgrades.

Warning

The vfio-pci driver has compatibility issues with certain QAT devices, including DH895x, C3xxx, and C62x devices. Though there is a way to bypass the compatibility check and let it work, the best practice for users with QAT devices is to continue using the igb_uio driver.
Under certain conditions interfaces may not be available in the dataplane during the first boot after completing the installation process. Should this happen, restart the dataplane via service dataplane restart from config mode to activate the interfaces.

Note

Users who follow the documentation/best practices recommendations (Dataplane Interfaces) and manually list interfaces during the initial configuration in the TNSR CLI will not be affected as that process already involves restarting the dataplane at the end.
This version includes support for importing host OS networking settings from the installer or cloud-init from Netplan into TNSR. This includes interface addresses and static routes (e.g. default gateway configuration).

Warning

Currently TNSR does not import DNS configuration entries from Netplan.

For most users this will not be a problem as such entries are not necessary when using DHCP to obtain DNS information or if host OS DNS settings are configured via the TNSR CLI as suggested in the Zero-to-Ping: Getting Started document.

Users who configured interface IP address and DNS settings statically in the installer/cloud-init or via Netplan may not have functional DNS in the host OS after upgrading. To work around this, add DNS settings for the host namespace as described in System DNS Resolution Behavior. Add these settings before upgrading for a smoother transition.
This version corrects a problem with PKI certificate generation for entries including multiple Subject Alternative Name (SAN) values. If a certificate includes multiple SAN entries it should be regenerated after upgrading as the previous format was incorrect. Corrected certificates will work correctly in the RESTCONF daemon.

Changes¶

Changes in TNSR software version 23.06

Bridge¶

Fixed: Bridge domain ARP entries cannot be removed via CLI [2380]
Fixed: Bridge domain mac-age value cannot be removed via CLI [2381]

CLI¶

Fixed: CLI autocompletion suggests BGP neighbors not within the same VRF [9316]
Added: show running-config command as shortcut for show configuration running cli [9758]
Added: write command as shortcut for configuration copy running startup [9759]
Fixed: show host interface command allows repeated use of identical parameters [9969]
Added: Output of service restconf status command obscures information [10260]
Fixed: ping command requires an unspecified order for its options [10301]
Fixed: Incorrect description of interval parameter for ping command [10367]
Fixed: traceroute command requires an unspecified order for its options [10415]
Fixed: ls -l and ls commands behave identically [10452]
Fixed: CLI auto completion does not work for some Bridge Domain commands [10544]

Dataplane¶

Fixed: Interrupt rx-mode does not function on some hardware [9039]
Changed: Set vfio-pci as default uio-driver for VPP DPDK plugin [10058]
Fixed: VPP crashes while initializing VMXNET3 interfaces using default configuration [10064]
Added: Update VPP to stable/2302 (DPDK 22.07) [10366]
Added: Add VPP startup options for vhost-user tuning [10647]

Host¶

Changed: Parse host static routes configured in netplan configurations generated by installer/cloud-init [9949]
Fixed: Unable to delete IPv6 host route entries [10621]
Fixed: Unable to configure from option in config-host-route-ip6 mode [10946]

IPsec¶

Fixed: Adding multiple IPsec tunnels with remote FQDN destinations fails if DNS resolution does not work for more than one FQDN [10358]

Interfaces¶

Added: Support for vhost-user interfaces [747]
Fixed: Unable to set IPv6 link-local address on an interface [2394]
Fixed: Unable to create subinterface with dot1q any tag [2652]
Added: Command to view IPv6 Router Advertisements state [9658]
Fixed: Interfaces disappear at boot until dataplane is restarted with vfio-pci driver [10280]
Fixed: IPv6 neighbor discovery can use incorrect MAC address on subinterfaces of bond interface [11013]

Operating System¶

Changed: Update Ubuntu to 22.04.2 [10076]
Changed: Disable kernel accept_ra setting in dataplane namespace [10138]
Changed: Remove unused optional kernels from ISO installer [10194]

Neighbor / ARP / NDP¶

TNSR may send ARP requests for non-subnet addresses [10972]

PKI¶

Fixed: PKI certificate entries do not include Key Usage/Extended Key Usage properties and may be rejected for some purposes when SANs are present [10018]

Packaging¶

Added: Generate TNSR deb packages for Debian 11 (“bullseye”) [10500]

RESTCONF¶

Fixed: RESTCONF daemon exits when certain clients fail to validate the server certificate [10112]
Added: Shortcut command to simplify creation of certificates for use with RESTCONF [11008]

Routing¶

Fixed: TNSR does not prevent removing extended and large community lists referred by route maps [9499]
Changed: Adjust CLI help text and YANG description fields related to VRF [10569]

SNMP / IPFIX / Prometheus¶

Fixed: SNMP subagent crashes when no data received for an interface from VPP [10828]

Tunnel Protocols¶

Fixed: Changes to an existing VXLAN tunnel configuration do not apply until the dataplane is restarted [1778]
Fixed: Changing crypto asynchronous dispatch-mode greatly increases the latency between IPsec tunnel IP addresses [10030]
Added: WireGuard support for chained buffers [10070]
Added: New dataplane startup.conf options outer-checksum-offload and lro [10656]

Known Issues¶

Known Issues in TNSR software version 23.06

ACLs¶

show acl pretty-print formatting is misaligned in some cases [10564]
Attempting to remove an in-use ACL produces an ambiguous error message [11066]

BFD¶

IPv6 session is not restored when virtual direct link gets disabled/enabled [4916]
TNSR cannot commit configuration candidate database loaded from a file if it contains a BFD session for an interface that does not exist [7150]
BFD configuration inconsistently displayed [9425]
No ping response from peer when BFD session is down [9447]
IPv6 BFD sessions are intolerant of dataplane restart [9475]

Bridge¶

Bridging fails with virtual interfaces as members [7762]
Bridge domain is not removed in VPP when deleted via RESTCONF [10831]
TNSR does not retransmit ARP replies if arp entry option is enabled in a bridge domain [10880]
Bridge domain shg and bvi options cannot be removed alone without bridge domain in interface configuration [10926]
Bridge domains behave incorrectly when restarting dataplane [11012]

CLI¶

Deleting the startup configuration database does not fully remove the active configuration [3723]
Specifying interface to traceroute requires root privileges [5376]
Input validation of unbound message cache slabs value does not work as expected [5472]
CLI and RESTCONF behavior are different for no bgp default ipv4-unicast [6303]
RIP information does not contain a legend for kernel routes [7230]
show configuration history version-diff does not autocomplete full command [10477]
CLI shows incorrect routing table attached to an interface in cloud environments [10589]
VRRP prints empty interface definitions in show config running cli output [11072]

Counters¶

Contradictory output of detailed counters on bond interface in ‘broadcast’ mode [8351]

DHCP Server¶

CLI offers to delete mandatory variable in DHCP server subnet configuration [5240]
DHCP4 Kea config-file output shows VPP TAP interface names in its configuration instead of TNSR interface names [5264]
Unable to setup a custom DHCP option with certain data types in the record [5299]

DNS¶

show system output does not contain DNS resolver parameters [5397]
Unbound fails to start with a number of values set to zero [10448]

Dataplane¶

Cannot create rx-queues for interfaces on KVM and VirtualBox [3674]
Static routes with an interface as the next hop using resolve-via-attached appear to break dataplane ARP [5259]
TNSR on AWS does not pass traffic when using the igb_uio or uio_pci_generic driver [7015]
IPv6 Neighbor Discovery starts to fail until Linux neighbor cache is cleared [9135]
SEGV in VPP [9312]
VPP crash from process node scheduling and expiration issues [9339]
VPP hangs resulting in SNMP segfault [9665]
VPP does not start with the expected uio-driver in certain cases [10373]
Dataplane fails to start up after system reboot if it is configured to use number of huge pages that exceeds the default number [10848]
Interfaces are not attached to driver after first boot post-install [11042]

General¶

Non-root users cannot access the FRR log file [4826]
Unable to specify TNSR interface as a source in ping and traceroute commands via REST [5605]
Startup entry is not created in configuration history log [7400]
Cannot commit a candidate configuration database if a tap interface is present [7458]
Incorrect error message is shown when removing ABF policy attached to an interface [9530]
system-ping call via REST does not return any data if it is called with timeout flag and no response from the server [10608]

Host¶

Cannot configure the default gateway for host namespace via TNSR CLI [3702]
VRF interface for a custom route table persists in the operating system after restarting services [4866]
dns-resolver configured for host namespace remains in system after removing from TNSR [7830]
dns-resolver configuration values for host namespace remain in resolv.conf after restarting TNSR [7975]
Unable to show two identical host routes in TNSR [10752]
Some host route options configured in TNSR are not applied correctly by the Linux network subsystem [10827]
Some types of host static routes are not displayed by show host route command [10905]
Option scope for IPv6 host static routes does not apply in the Linux network subsystem [11011]
DNS issues can occur with netplan configurations containing static interface addresses [11017]

IPsec¶

IPsec daemon does not support using non-default VRF entries [7266]
Cannot disable IPsec dpd-interval option [8012]
Cannot configure IPsec with manual key type [8396]
Error when creating IPsec tunnel via RESTCONF with tunnel-enable set [8432]
IPsec tunnel without a child SA does not appear in IPsec state data [8433]
TNSR allows unsupported IPsec encryption algorithms to be configured [10503]
IPsec tunnel with initially unresolvable FQDN destination does not pass traffic after remote address gets resolved if there is another IPsec tunnel using the same source [10798]

Installation¶

TNSR installer fails if interfaces are configured with IP addresses but have no Internet connectivity [7807]

Interfaces¶

VLAN subinterfaces do not work with virtio network drivers on KVM [2189]
Invalid routes remain in table when next-hop IP address is no longer directly connected [3161]
Reassembly timeout is not working when full IP reassembly is configured [3269]
Shallow virtual reassembly cannot be disabled when it is implicitly enabled by other features [3361]
Second fragment of a packet is not virtually reassembled when max-reassemblies is set to 1 [3384]
Unable to delete a MAC address explicitly set for the TNSR side of a TAP interface [4433]
Netgate 1541 link speed auto-negotiation incorrect with direct connected interfaces [5323]
Errors indicate TNSR is attempting to assign a MAC address to IPsec ipipX interfaces [6285]
L3 packets can be sent from bridged interfaces [6975]
Unable to setup DPDK uio_pci_generic driver on Netgate 1541 [6981]
TAP instance tcpdump method only captures received packets [7137]
Unable to delete a non-existent multicast-interface from VXLAN tunnel configuration [7278]
Pings between IPIP interfaces become intermittent when BGP is applied to them [7392]
Interface IP address is shown in IPv4 route table instead of associated subnet [7511]
Setting a new MTU value does not affect the MRU for IPv6 packets [8245]
Unable to delete link MTU from an interface when default MTU is set less than 1280 [8837]
Evaluate presence of interface configuration items for loopback interfaces [9380]
Link state of a bond interface does not follow the link state of the underlying interfaces [10093]
Reinstantiation of an interface does not automatically re-create subinterfaces [10725]
Adjacencies for subinterfaces are not updated when the MAC address of the parent interface changes [10726]
show interface tap does not print IPv4 and IPv6 gateway information [10849]
Intel I226-V interfaces can periodically stop working in VPP [10857]
show interface <name> subif command does not produce any output [10879]

LLDP¶

no lldp enable command shows CLI error [10925]
LLDP interface configuration parameters cannot be removed via CLI [10982]
TNSR sends incorrect LLDP management address if only lldp port-name is configured on an interface [11047]
TNSR continues sending LLDP frames after lldp port-name is removed from an interface using RESTCONF [11048]
LLDP router configuration cannot be removed [11049]

Memif¶

Unable to connect to memif interface using default socket [4448]

NACM¶

It is possible to remove an NACM group used in a rule list [10115]
NACM rule paths created via RESTCONF are not validated and can lead to broken configuration databases [10116]

NAT¶

Twice-NAT does not work with output-feature/postrouting NAT [1023]
1:1 NAT drops packets with ttl=2 from inbound interface [2849]
Full IP reassembly does not work with MAP [3386]
MAP-T adds bogus zeroes when translating short IPv4 to IPv6 [3460]
NAT pool route table option only available when specifying a range [3628]
Packets larger than 2034 bytes are dropped when performing IPv4 to IPv6 MAP translation [3742]
MAP-T domain usage causes IPv6 traffic class value to always be copied from IPv4 ToS value [3774]
TCP MSS value is not applied to IPv4 packets when IPv6 to IPv4 decapsulation is performed on MAP-E BR [3783]
MAP does not relay IPv6 ICMP error messages to IPv4 [3809]
NAT static mappings for ICMP do not work [4373]
NAT static mappings for TCP/UDP protocol on any port result in translation for port 0 instead [4384]
NAT static mappings assume external port 0 when port is omitted [4432]
Packets not destined to a NAT pool are dropped when NAT simple mode is configured with out2in-dpo option [4927]
Full IPv4 reassembly doesn’t work with NAT endpoint-independent mode [5476]
Cannot increase NAT Sessions per thread past ~1e6 [6550]
Dataplane SIGSEGV crash and backtrace when exceeding NAT session limit [6551]
Expired NAT sessions become active again when increasing the timeout value [7090]
NAT sessions do not expire in endpoint-independent mode [7098]
Cannot commit a clean candidate configuration database if NAT static mapping is configured [7286]
Unable to establish NAT hairpin connection [8014]
NAT in endpoint-dependent mode drops packets when it cannot identify the correct worker thread [8262]
Routing through NAT in EI mode does not work if NAT outside interface is IPIP or GRE [8333]
VPP can return incomplete session data for a user when NAT forwarding is enabled with multiple worker threads [9510]
Traffic from TNSR itself sourced from inside NAT interface does not get NAT applied when egressing via NAT outside interface [9706]

NTP¶

NTP does not properly handle IPv6 restrictions [4626]
Delay in CLI display of NTP configuration when NTP has noquery set [6818]
Interfaces in the TNSR NTP configuration are not validated when generating the NTP daemon configuration [7153]

Neighbor / ARP / NDP¶

Packet loss during ARP transactions [2868]
The MAC address of a static IPv6 neighbor cannot be changed [4454]

PKI¶

PKCS#12 archives are not generated correctly when the ca-name is not specified [10320]

RESTCONF¶

Adding a user via RESTCONF requires a password even when providing an ssh key [2875]
RESTCONF “pretty-printed” JSON contains incorrect indentation [3521]
OSPF interfaces are not validated when configured via RESTCONF [3528]
Cannot change GRE tunnel type to or from ERSPAN via RESTCONF [4353]
Response of /restconf/data/ and /restconf/data/netgate-interface:interfaces-state/ does not include any of *-table [5399]
RESTCONF allows configuring dataplane options for non-existent devices [5748]
RESTCONF route-state response does not contain actual state data [7115]
RESTCONF dataplane service does not work on interfaces in a non-default VRF [7265]
History version count does not match the count of REST configuration requests if they are sent without a delay [7440]
Unable to clear trace filters over RESTCONF [9476]
RESTCONF does not validate payload body to prevent invalid arguments in certain cases [10413]
RESTCONF does not work with IPv6 sockets after TNSR reboot [10729]
Newlines are removed from PKI certificate and key data when importing via RESTCONF [10794]

Routing¶

Changing default metric for OSPF server does not result in update on other routers [2586]
OSPF RIB is not updated when the ABR type is changed between standard and shortcut [2699]
BGP updates for new prefixes ignore the advertisement-interval value and are sent every 60 seconds [2757]
ttl-security hops value can be set when ebgp-multihop is already configured [2832]
extended-nexthop capability isn’t being negotiated between IPv6 BGP peers [2850]
Unable to verify received prefix-list entries via CLI when using ORF capability [2864]
BGP network backdoor feature isn’t working without service restart [2873]
BGP next-hop attribute aren’t being sent unmodified to the eBGP peer when route-server-client option is configured [2940]
Unable to verify dynamic BGP peer information from TNSR CLI [3044]
Unable to delete OSPF3 config for an interface [3481]
TNSR does not prevent creating static routes for directly connected networks [3813]
OSPF conditional default route injection does not work [3846]
Unable to verify received routes when high number of routes received via BGP [3918]
TNSR allows OSPF network type for a loopback interface, which is rejected by FRR [4800]
Reverting to the startup configuration doesn’t restore packet forwarding for BGP over IPsec prefixes [5321]
RIP route-map-filter option does not filter routes [5910]
Unable to disable IPv4 AF without BGP service restart [6393]
BGP failover logs “Failed to delete neighbor” error from linux-cp [6400]
OSPF virtual-link authentication does not work [6601]
Unable to remove OSPF virtual-link configuration [6962]
OSPF can announce interfaces from other VRFs on initial configuration [7002]
Cannot add a static recursive route [7010]
VPP crashes on applying custom VRF to loopback interface used in OSPF [7056]
Creating route-map, prefix-list, or access-list entries takes longer than expected [7068]
Cannot disable logging of adjacency changes for OSPF6 if detail option is set [7097]
Routes that exactly overlap an interface link route are accepted by CLI but are problematic [7101]
OSPF neighbor adjacency is established in wrong VRF in VirtualBox [7144]
Interfaces in the TNSR RIP configuration are not validated when generating the FRR RIP daemon configuration [7155]
Interfaces in TNSR route-map entries are not validated when generating the FRR daemon configurations [7156]
Interfaces in the TNSR OSPF configuration are not validated when generating the FRR OSPF daemon configuration [7177]
Interfaces in the TNSR BGP configuration are not validated when generating the FRR BGP daemon configuration [7218]
Dynamic routing protocols lose static routes after link they resolve through goes down and then comes up [7357]
OSPF logging for some options does not work if logging level is set explicitly [7411]
BGP debug option updates in <peer> does not filter messages for selected peer [7476]
BGP session does not become active after interface goes down and recovers [7501]
OSPF6 continues to redistribute connected/kernel routes resolved via interface with linkdown status [7624]
BGP address family neighbor option maximum-prefix restart does not work correctly [7709]
Malfunction of BGP process after entering maximum-prefix restart without the basic maximum-prefix limit command [7748]
OSPF6 does not advertise loopback address to another area if the loopback is configured first [7757]
Routes remain in table after interface with VRRP configured is marked down until dataplane is restarted [7790]
OSPF stops working after configuring mtu-ignore option on an interface [8085]
Routes do not match by route-map if match criteria is set to ip next-hop ... [8148]
Output of show conf differs for route-map [8375]
Route map source-protocol match condition matches routes from any source [8381]
redistribute table configuration in RIP/OSPF does not affect route redistribution [8390]
Cannot change distance for one BGP prefix [8690]
Forwarding address from OSPF6 LSA5 is not installed as the next hop for the route [8732]
BGP bestpath med missing-as-worst command does not function correctly [8805]
OSPFv3 repeatedly drops connection on AWS when redistribution is configured [8822]
Route Map with IPv6 Access List does not filter redistributed OSPF6 routes [8857]
Route-Map set src option does not function correctly [9045]
show route displays no routes for a VRF until it is placed on an interface [9073]
FRR cannot connect to RPKI cache server if a route to it does not exist in default VRF [9146]
The redistribute kernel and import vrf BGP options do not work at the same time if the static route is redistributed with an output interface in a third-party VRF [9147]
Applying a subsequent route map with import vrf cancels a previous applied route map [9156]
A route map applied to the import vrf option using a prefix list does not work correctly [9235]
Changing BGP as-number in default VRF leads to the termination of the import of routes to another VRF [9244]
Cannot change an interface to a new VRF when BGP is configured to import the current VRF [9259]
Changing an interface VRF does not stop importing routes from the previous VRF [9298]
RPKI expire-interval option does not get put into the FRR running configuration after restarting BGP/dataplane [9331]
Route maps with match rpki * conditions do not get re-applied when RPKI status of routes changes [9439]
set community command disappears from FRR configuration without warning after setting an invalid community [9508]
Suppression of specific routes when applied to an aggregated route of a route map containing set aggregator as <asn> ip address <ipv4-address> command [9547]
Deprecation warning from FRR OSPF6 for interface area syntax [9783]
BGP soft-reconfiguration inbound option does not work for IPv6 peers [10086]
BGP selects incorrect path to a network when changing bestpath rules [10210]
zebra causes out-of-memory error on AWS when restarting TNSR after receiving 1.5-2 million prefixes via BGP [10273]
FRR fails to reload configuration if set as-path prepend values are incorrectly enclosed in quotes [10309]
OSPF6 conditional default route injection does not work correctly [10311]
BGP route-reflector-client option does not work on neighbor configurations using IP addresses instead of peer groups [10356]
BGP does not select the best path for a route after updating the router-id of a neighbor when bestpath compare-routerid is enabled [10391]
Cannot remove BGP unsuppress-map option by route-map name for IPv6 neighbor [10409]
OSPFv3 default-information originate options do not stack when configured separately [10478]
OSPFv2 metric-type 2 option explicitly set for default-information originate does not get placed into the FRR configuration [10479]
Unexpected delay in distribution of route information between OSPF database and RIB during propagation of OSPF default route [10721]

SNMP / IPFIX / Prometheus¶

Prometheus filters with non-alphanumeric characters can cause HTTP requests to fail [5467]
Prometheus filters containing spaces cannot be removed [5470]
SNMP does not work on interfaces in a non-default VRF [7261]

SPAN¶

Span config disappears/appears when repeatedly restarting dataplane [6526]
Incorrect error message when requesting SPAN info from a missing interface [7209]
SPAN mirroring can not be disabled [7560]
SPAN does not work correctly for outbound packets on VLAN subinterface [7801]

Static Routes¶

Static route description is not showing up in show commands or REST state data [5478]
Static route overwrites kernel route in the operating system routing table [7215]
Transit traffic goes to an interface with inactive link when there is another (active) path [8041]

Tunnel Protocols¶

TNSR IPv6 interface address does not appear in traceroute when next-hop is IPsec tunnel interface [5178]
VxLAN with multicast destination does not pass traffic [6491]
GRE interface configuration remains in running config after changing GRE tunnel ID [7050]
Configuring option route-table in a WireGuard peer does not affect next-hop lookup of the endpoint address [8070]
VPP processes packets received on disabled tunnel interfaces [8111]
WireGuard tunnel interfaces still function with a tunnel next-hops entry having an incorrect next-hop-address [8256]
Tunnel next-hop entries do not function in non-default VRFs [8653]
Incorrect WireGuard tunnel next-hop after roaming [8764]
IPIP interface loses attached ACLs when DNS resolution of the remote endpoint changes [10171]
IPIP interface loses TCP MSS setting when DNS resolution of the remote endpoint changes [10312]
IPv6 VxLAN does not pass traffic if it is configured over IPv6 IPsec [10592]
Lower than expected throughput over VXLAN interfaces terminated on a loopback BVI [10643]

Updates¶

Router upgraded to 22.10-2 will not start without an IKE prf entry [9368]
Clixon hangs until restarted after upgrading TNSR to 23.06 via TNSR CLI package upgrade command [11039]

VRRP¶

VRRP accept-mode may cause invalid ARP requests, leading to loss of connectivity during failover [9881]

clixon¶

log_upgrade does not print cxobj paths correctly in tnsr-upgrade.log [4747]
clixon_backend exhausts memory while displaying high amount of routes [5226]
Configuration upgrade does not run when loading configuration via history [6968]
Unable to set up a password that starts and finishes with a double quotation mark [7571]
Unable to set up a password that contains a backslash symbol [7572]