Troubleshooting Asymmetric Routing¶
Asymmetric routing happens when traffic between two nodes takes a different path in each direction (e.g. A->B->C, C->D->A). This can pose a problem for TCP which has strict state tracking but often does not affect “stateless” protocols such as ICMP or UDP.
Common Scenario¶
What happens in most cases is this:
Client sends a TCP SYN packet which arrives at the firewall
The firewall creates a state table entry
The firewall sends an ICMP redirect to the client letting it know to reach the target server through an alternate gateway on the same interface
The target server sends a TCP SYN+ACK packet to the client by a different path that does not pass through the firewall
The client sends its ACK and later responses to the server using the other gateway and they are not visible to the firewall
After 30 seconds the firewall removes its state table entry as the firewall did not see the client and server fully establish a connection
Some time later the route the client learned via ICMP redirect expires and the client sends another packet back to the firewall
Since this packet is not starting a new connection the firewall drops the packet
The client gets disconnected as it no longer knows how to reach the server
Automatic Fix¶
The Bypass firewall rules for traffic on the same interface option located under System > Advanced on the Firewall & NAT tab activates rules for traffic to/from the static route networks which are much more permissive when it comes to creating states for TCP traffic and allowing it to pass. The rules allow any TCP packets regardless of their flags to create a state, and also utilize “Sloppy” state tracking which performs a less strict state match.
Manual Fix¶
The same rules may be created manually by adding one on the affected interface tab (e.g. LAN), and a second rule on the Floating tab using the same interface (LAN again) to match the traffic in the out direction. The rule must be set for a protocol of TCP, under TCP flags check Any Flags, and use a State Type of Sloppy.
The options for TCP flags and State Type can be found in the Advanced Options when editing a firewall rule.
Alternate Causes¶
On occasion these issues can be caused by other factors that lead to asymmetric
routing, such as issues with route-to or reply-to, both having to do
with gateways on interface settings.
Defining gateways under System > Routing does not cause this, but rather these problems can come up when the gateway is improperly configured on the interface pages, Interfaces > WAN, Interfaces > LAN, and so on.
See also
Gateway set when it should not be set¶
If a gateway is set on an internal interface, such as LAN, it can cause
problematic behavior. Setting a gateway on an internal interface will tag
outbound rule on that interfacewith route-to, and inbound rules with
reply-to which will cause the firewall to forward packets to the defined
gateway rather than following their natural path. For WANs this is typically a
good thing! For LANs it is not. Among other ill effects, it can lead to a loop
of sorts where packets bounce between the firewall and the defined gateway,
eventually being blocked or dropped when their TTL expires.
Gateway not set when it should be set¶
A gateway should usually be set on a WAN or other external-type interface
settings (MPLS, IP VPN, etc.) In these cases the reply-to and route-to
behavior is desired and likely required. If it is missing the packets may be
blocked or dropped as they attempt to leave the wrong interface. A packet could
enter via the alternate WAN, but the reply would leave by the default gateway.
Similar to the effect seen when improperly using an Interface Group for WAN interfaces.