Tip
This is the documentation for the 21.07 version. Looking for the documentation of the latest version? Have a look here.
Updating TNSR¶
Warning
TNSR Home+Lab installations cannot be updated. Reinstall with TNSR Business or install a new version of TNSR Home+Lab. The operating system may be updated, but not TNSR.
With a signed client certificate from Netgate in place, TNSR has access to the
Netgate software repositories which contain important updates to TNSR. These
updates can be retrieved using the package
command in the TNSR CLI, or
dnf
in the host OS shell.
Note
Updating TNSR will also update the operating system. Even when there are no TNSR updates available, it is a good practice to periodically perform an update to obtain important operating system updates such as security vulnerability mitigations.
See also
Most of this document covers in-place updates. For information on updating by redeploying/reinstalling, see Upgrading by Redeploying TNSR.
Pre-Upgrade Tasks¶
Before updating TNSR, perform the following tasks:
Read through the Netgate TNSR Releases release notes for the new version to identify relevant changes in behavior which may require special actions before or after the upgrade
Make sure the signed certificate is in place (Install the certificate)
Make sure the TNSR instance has working Internet connectivity
Have installation media ready for the new version of TNSR software. Problems during an in-place update may require reinstallation of TNSR software.
Take a backup of the running and startup configurations, plus other important files such as the signed certificate and keys (Configuration Backups)
If TNSR is running as a virtual machine, take a snapshot
Tip
Though it is optional, best practices for updating include a pre-upgrade reboot. This reboot ensures that the hardware and installation are functional before attempting the upgrade. This can help identify potential hardware and other issues, such as storage failures, so they do not present themselves unexpectedly during the upgrade.
Updating to TNSR 20.08 from Previous Versions¶
TNSR 20.08 is based on CentOS 8, while previous versions of TNSR software were based on CentOS 7.x. Due to this major operating system change, updating to TNSR 20.08 requires special handling.
Warning
While Netgate has tested common update scenarios, updating in-place from a CentOS 7 base to CentOS 8 base may not work in all installations.
Installing TNSR 20.08 directly and then restoring the TNSR configuration data is a safer approach. However, that method requires physical access or equivalent out-of-band access and is potentially more time consuming. See Upgrading by Redeploying TNSR for details.
Before starting, update the current installation of TNSR software to version 20.02 as described in Updates via the TNSR CLI. Upgrading from previous versions of TNSR software has not been tested.
The following shell script will upgrade TNSR software installations in place from CentOS 7 to CentOS 8 and TNSR software version 20.08. Download the script to the router, inspect and verify its contents, then execute the script from the console (serial, video, IPMI/OOB access).
Warning
The script must not be run using an SSH session connected through TNSR, as TNSR services will be stopped and started during the upgrade, breaking connectivity between the client and TNSR.
1#!/bin/sh
2
3_fail() {
4 echo "Upgrade process has failed! Please contact support"
5}
6
7YUM="yum -y"
8DNF="dnf -y"
9CENTOS_RPM_SERVER="http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages"
10CENTOS_VERSION="8.2-2.2004.0.1"
11
12if [ "$(rpm -q kernel | wc -l)" = "1" ]; then
13 echo "Only one kernel found, upgrading current system"
14 ${YUM} upgrade
15 echo "Reboot your system and run this script again"
16 exit 0
17fi
18
19echo "This script will upgrade your TNSR instance to a new major"
20echo "version based on CentOS 8. This operation is irreversible."
21echo "Make sure you backup your configuration files before start."
22echo ""
23read -p "Are you ready to proceed? (y/N) " -e -n 1 _proceed
24
25if [ "${_proceed}" != "y" -a "${_proceed}" != "Y" ]; then
26 echo "Aborting..."
27 exit 0
28fi
29
30# Install items that are part of CentOS 8 Core group so they will be upgraded
31# at distro-sync step
32${YUM} install epel-release dnf tnsr-release NetworkManager \
33 NetworkManager-team NetworkManager-tui dnf-plugins-core \
34 rng-tools sssd-common sssd-kcm libibverbs
35
36systemctl disable NetworkManager >/dev/null 2>&1 || true
37
38# Adjust dnf.conf
39if ! grep -q 'clean_requirements_on_remove=False' /etc/dnf/dnf.conf; then
40 sed -i -e '/clean_requirements_on_remove/d' /etc/dnf/dnf.conf
41 echo 'clean_requirements_on_remove=False' >> /etc/dnf/dnf.conf
42fi
43
44# At this point tnsr/tnsr-release packages will be removed due to yum-utils
45# dependency
46rpm -e --nodeps yum-utils
47${DNF} remove yum yum-metadata-parser '*rpmconf*' redhat-rpm-config
48
49mv -f /etc/yum /etc/yum.backup_tnsr
50
51# Remove unnecessary kernels
52TO_BE_REMOVED=$(dnf repoquery --installonly --latest-limit=-2 -q)
53if [ -n "${TO_BE_REMOVED}" ]; then
54 ${DNF} remove ${TO_BE_REMOVED}
55fi
56
57trap '' HUP INT TERM
58trap _fail EXIT
59
60set -e
61
62${DNF} --disablerepo=* install \
63 ${CENTOS_RPM_SERVER}/centos-repos-${CENTOS_VERSION}.el8.x86_64.rpm \
64 ${CENTOS_RPM_SERVER}/centos-release-${CENTOS_VERSION}.el8.x86_64.rpm \
65 ${CENTOS_RPM_SERVER}/centos-gpg-keys-${CENTOS_VERSION}.el8.noarch.rpm
66
67${DNF} --disablerepo=* upgrade \
68 https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
69
70# Cleanup all local files
71dnf -y clean all
72
73# Remove CentOS 7 kernel used to boot
74rpm -e --nodeps $(rpm -q kernel-$(uname -r))
75
76# Remove conflicting packages
77rpm -e --nodeps sysvinit-tools
78
79${DNF} --releasever=8 makecache
80
81# Enable nginx 1.16 module
82${DNF} --releasever=8 module enable nginx:1.16
83
84# Cache all packages that will be used to upgrade
85${DNF} --releasever=8 --allowerasing --setopt=deltarpm=false --best \
86 --downloadonly distro-sync
87${DNF} --releasever=8 --allowerasing --setopt=deltarpm=false --best \
88 --downloadonly groupinstall Core
89${DNF} --releasever=8 --allowerasing --setopt=deltarpm=false --best \
90 --downloadonly install tnsr-release libibverbs
91
92# Cleanup old dpdk stuff
93rpm -e --nodeps dpdk
94rm -rf /var/lib/dkms/dpdk
95
96# Upgrade CentOS
97${DNF} --releasever=8 --allowerasing --setopt=deltarpm=false --best \
98 -C distro-sync
99
100# Enable legacy network service
101systemctl disable NetworkManager >/dev/null 2>&1 || true
102systemctl enable network
103
104trap "-" HUP INT TERM EXIT
105
106echo "Upgrade is complete! Reboot your system!"
When the script completes, reboot the router and monitor the console for errors.
Warning
In certain cases, the first reboot after upgrading to CentOS 8 may
also require an extra dataplane restart to ensure interfaces have full
connectivity. This can be performed from the shell with sudo tnsrctl
restart
or from the TNSR CLI with config
then service dataplane
restart
. Alternately, performing another reboot will restart into the
expected state.
If the upgrade failed, or it is otherwise not running and passing traffic properly after the upgrade, access the console and check the state of the system from there. Before contacting support, review TNSR 20.08 Release Notes for relevant changes in behavior which may require manual adjustments. For assistance, please contact Netgate TAC.
Updating after TNSR 20.08¶
The following methods are not a viable upgrade procedures to reach TNSR 20.08 from earlier versions. They may be used to upgrade from TNSR 20.08 to later versions as well as to obtain regular updates from CentOS.
AppStream Repository Priority¶
Before proceeding with the upgrade, check the CentOS AppStream repository
priority. It should be set to a priority
value of 100
to ensure that
appropriate TNSR packages are used instead of CentOS versions with the same
name.
Tip
The commands below are performed from a host shell, but may also be run from
the TNSR cli using the host shell
command.
First, look to see if the priority is already set:
$ grep priority /etc/yum.repos.d/CentOS-Linux-AppStream.repo
priority=100
In the above example, the priority is already set.
Note
If the grep
command reports a No such file or directory
error, try
again with /etc/yum.repos.d/CentOS-AppStream.repo
. If that succeeds, use
that path in the following command as well.
If the grep
command returns no output, add the priority as follows:
$ sudo bash -c 'echo "priority=100" >> /etc/yum.repos.d/CentOS-Linux-AppStream.repo'
After running that command, check again with grep
or inspect the contents
with cat
or similar commands to ensure the priority
line is present with
the correct value.
Updates via the TNSR CLI¶
The easiest way to update TNSR is from within the TNSR CLI itself.
tnsr# package upgrade
That command will download and apply all available updates. Afterward, exit the CLI and start it again.
Note
There will be no output from this command until the process completely finishes, which may take a few minutes for larger updates.
Updating via the shell¶
TNSR can also be updated from the command line using the host OS package
management commands, in this case, dnf
:
$ sudo dnf -y module disable BaseOS
$ sudo dnf clean expire-cache
$ sudo dnf -y upgrade
Update Script¶
The following shell script may be used to keep TNSR and CentOS updated. In addition to the updates it also makes a local backup before performing the update.
1#!/bin/sh
2
3# Time to make the backups
4mkdir -p ~/tnsr-config-backup
5sudo cp -p /var/tnsr/running_db ~/tnsr-config-backup/running_db-`date +%Y%m%d%H%M%S`.xml
6sudo cp -p /var/tnsr/startup_db ~/tnsr-config-backup/startup_db-`date +%Y%m%d%H%M%S`.xml
7
8# Update all RPMs
9sudo dnf -y module disable BaseOS
10sudo dnf clean expire-cache
11sudo dnf -y upgrade
12
13# Restart TNSR services to ensure a clean and consistent run
14sudo tnsrctl restart
Post-Upgrade Reboot¶
TNSR upgrades include kernel updates, driver updates, and other operating system component updates. As such, the best practice is to reboot after upgrading these to ensure the device is running the proper kernel and is using a consistent set of updated system components.
The reboot procedure is covered in Rebooting the Router.
Updating the Configuration Database¶
Automatic Configuration Update¶
When upgrading from TNSR 20.02 to a later version, TNSR has its own automatic configuration upgrade procedures which accommodate changes made to the configuration database structure between versions.
Warning
The TNSR configuration upgrade only alters the running configuration database and not the startup database. After starting TNSR the first time post-upgrade, validate the running configuration. If the running configuration is OK, copy it to the startup configuration:
tnsr# config
tnsr(config)# configuration copy running startup
Configuration Update for Previous Versions of TNSR¶
When upgrading from a version older than 20.02, or restoring a configuration backup from before 20.02, additional steps may be necessary to update the configuration database.
If TNSR fails to start due to a change in the configuration, use the included
configuration database migration utility, /usr/bin/tnsr-db-update
, to check
for and correct potential problems with older configuration files.
Warning
TNSR must be stopped before attempting to run this script, which
modifies the running database at /var/tnsr/running_db
.
The /usr/bin/tnsr-db-update
script must be run as root either directly or
via sudo
:
$ sudo /usr/bin/tnsr-db-update
The utility will make a backup of the configuration before making alterations.
This backup is placed in
/var/lib/tnsr/db-backups/tnsr-<version>/running_db-<timestamp>
.
If TNSR still will not start after running this utility, then the configuration database may contain errors which were not able to be corrected by the script.
Once all errors are corrected and TNSR starts successfully, copy the running configuration to the startup configuration.
Manual Configuration Update¶
Any errors which could not be corrected by the automatic script must be corrected by hand. Alternately, the configuration databases may be erased and recreated from scratch.
To attempt manual corrections, check the system logs after attempting to start
TNSR for information about which configuration entries are causing the failure.
View the logs with sudo systemctl status clixon-backend.service
, sudo
journalctl -xelu clixon-backend.service
and sudo journalctl -xe
.
A log entry for a configuration problem could look like the following example:
clixon_backend: startup_failsafe: 297: Database error: Startup failed and no
Failsafe database found, exiting
clixon_backend: <rpc-reply><rpc-error>
<error-type>application</error-type>
<error-tag>unknown-element</error-tag><error-info>
<bad-element>someinvalidtag</bad-element></error-info>
<error-severity>error</error-severity>
<error-message>namespace is: urn:ietf:params:xml:ns:netconf:base:1.0</error-message>
</rpc-error></rpc-reply>
To correct such problems, edit the configuration in /var/tnsr/running_db
(e.g. sudo vi /var/tnsr/running_db
), erase or adjust the offending tag or
configuration section, copy the repaired configuration to
/var/tnsr/startup_db
, and attempt to start TNSR again. Repeat until no
errors are reported and TNSR starts normally.
To erase the configuration database, remove its files from /var/tnsr
:
$ sudo rm /var/tnsr/*_db
After removing the configuration and starting TNSR, the TNSR configuration will need to be created again manually from scratch using the CLI or RESTCONF. Open the contents of a configuration backup in a text editor to use as a guide.
Additional Reboot / Update Verification¶
After performing a TNSR update and updating the configuration, administrators may wish to perform a reboot of the router to ensure it starts up correctly with the expected configuration.
This practice ensurs that the router performs as expected at startup during an upgrade maintenance window.
Update Troubleshooting¶
If the TNSR CLI method does not work, use the shell method instead.
If either method prints an error referring to a broken package database, recover it as follows:
$ mkdir -p ~/tmp/
$ sudo mv /var/lib/rpm/__db* ~/tmp/
$ sudo rpm --rebuilddb
$ sudo dnf clean all
Upgrading by Redeploying TNSR¶
Rather than performing an in-place update of a TNSR installation, administrators may instead choose to deploy a fresh instance of TNSR using the new version. This practice is typical of environments such as cloud providers or virtual machines, but may be performed for ISO installations and others as well.
In those cases, follow this general procedure:
Take a backup of the configuration
Deploy a new instance of TNSR using the installation instructions for the chosen platform
Restore the configuration
Update the configuration (Updating the Configuration Database)
See Configuration Backups for details on saving and restoring configuration backups, and review Updating the Configuration Database for important information about updating the configuration for a new version of TNSR.