Current versions of pfSense® software include kernels built with the option
ROUTE_MPATH which enables multi-path routing.
This allows the routing table to contain multiple entries to the same destination, which allows for weight-based balancing of traffic including Equal-cost multi-path routing (ECMP) if all gateways for a destination are weighted the same.
Currently multi-path routing can only be utilized by the dynamic routing daemon package FRR. The base system GUI does not currently support managing multiple routes to the same destination, but support is planned for future releases.
Multi-Path Routing Behavior¶
Packets may only take alternate paths when they are different in some way. When there are multiple paths to a destination the operating system computes outbound flow hashing for connections to decide which path to use for a packet. This concept is similar to the LAGG Hash Algorithm. The hashing takes the 5-tuple connection property set into account: “(protocol, source address, destination address, source port, destination port)”.
For example, testing with ICMP only from one host to another with no variation may never see flows take a second path. In contrast, testing repeated TCP connections could take multiple paths if the source port is randomized. The best way to test is by using multiple sources and destinations passing through the firewall and not from the firewall itself.
View Nexthop Data¶
The first thing to check is the routing table to ensure that there are multiple routes to the same destination:
: netstat -rWn4 Routing tables Internet: Destination Gateway Flags Nhop# Mtu Netif Expire [...] 10.5.0.0/16 198.51.100.5 UGS 0 1500 ix3 10.5.0.0/16 203.0.113.5 UGS 0 1500 ix2
The route table in the status output of the FRR package will also show
multiple entries for the same destination with different
Next, there are two items to check to verify that multi-path routing is taking effect: Nexthop data and Nexthop group data.
Check the nexthop data with
netstat -4onW and/or
: netstat -4onW Nexthop data Internet: Idx Type IFA Gateway Flags Use Mtu Netif Addrif Refcnt Prepend [...] 32 v4/gw 198.51.100.17 198.51.100.5 GS 0 1500 ix3 1 33 v4/gw 203.0.113.17 203.0.113.5 GS 0 1500 ix2 1
Check nexthop group data with
netstat -4OnW and/or
: netstat -4OnW Nexthop groups data Internet: GrpIdx NhIdx Weight Slots Gateway Netif Refcnt 34 ------- ------- ------- ----------------- --------- 2 32 1 1 198.51.100.5 ix3 33 1 1 203.0.113.5 ix2
The outputs of those commands should show both gateways and indicate that they belong to the same “group”.