Troubleshooting IPsec Traffic

Tunnel establishes but no traffic passes

The first place to look if a tunnel comes up but will not pass traffic is the IPsec firewall rules tab. If Site A cannot reach Site B, check the Site B firewall log and rules. Conversely, if Site B cannot contact Site A, check the Site A firewall log and rules.

Inspect the firewall logs at Status > System Logs, on the Firewall tab. Check for log entries indicating traffic is blocked involving the subnets used in the IPsec tunnel. Also check for traffic on the WAN interface used by the tunnel for the protocol ESP or UDP port 4500 both of which could be used to carry encapsulated IPsec traffic. If there are log entries which match either case, continue on to check the rules. If there are no log entries indicating blocked packets, revisit the section on IPsec routing considerations in Client Routing and Gateway Considerations.

Blocked packets on the IPsec or enc0 interface indicate that the tunnel itself has established but traffic is being blocked by firewall rules. Blocked packets on the LAN or other internal interface may indicate that an additional rule may be needed on that interface ruleset to allow traffic from the internal subnet out to the remote end of the IPsec tunnel. Blocked packets on WAN type interfaces would prevent a tunnel from establishing. Typically this only happens when the automatic VPN rules are disabled. Adding a rule to allow the ESP protocol along with UDP ports 500 and 4500 from that remote IP address will allow the tunnel to establish and pass traffic. In the case of mobile tunnels, allow traffic from any source to connect to those ports. See IPsec and firewall rules for more details.

Rules for the IPsec interface can be found under Firewall > Rules, on the IPsec tab. Common mistakes include setting a rule to only allow TCP traffic, which means things like ICMP ping and DNS would not work across the tunnel. See Firewall for more information on how to properly create and troubleshoot firewall rules.

In some cases it is possible that a setting mismatch can also cause traffic to fail passing the tunnel. In one instance, a subnet defined on a third-party firewall was 192.0.2.1/24, and on the firewall running pfSense® software it was 192.0.2.0/24. The tunnel established, but traffic would not pass until the subnet was corrected.

Routing issues are another possibility. Running a traceroute (tracert on Windows) to an IP address on the opposite side of the tunnel can help track down these types of problems. Repeat the test from both sides of the tunnel. Check Client Routing and Gateway Considerations for more information. When using traceroute , traffic which enters and leaves the IPsec tunnel will seem to be missing interim hops. This is normal and part of how IPsec works. Traffic which does not properly enter an IPsec tunnel will appear to leave the WAN interface and route outward across the Internet, which would point to either a routing issue such as the firewall running pfSense software not being the gateway (as in Client Routing and Gateway Considerations), policy routing rules matching the traffic, an incorrectly specified remote subnet on the tunnel definition, a tunnel which has been disabled.

Some hosts work but not all

If traffic between some hosts over the VPN functions properly, but some hosts do not, this is commonly one of four things:

Missing, incorrect or ignored default gateway:

If the device does not have a default gateway or has one pointing to something other than the firewall running pfSense software, it does not know how to properly get back to the remote network on the VPN (see Client Routing and Gateway Considerations).

Some devices, even with a default gateway specified, do not use that gateway. This has been seen on various embedded devices, including IP cameras and some printers. There isn’t anything that can be done about that other than getting the software on the device fixed. This can be verified by running a packet capture on the inside interface of the firewall connected to the network containing the device. Troubleshooting with tcpdump is covered in Using tcpdump on the command line, and an IPsec-specific example can be found in IPsec tunnel will not connect.

If traffic is observed leaving the inside interface of the firewall, but no replies return, the device is not properly routing its reply traffic or could potentially be blocking it via a local client firewall.

Incorrect subnet mask:

If the subnet in use on one end is 10.0.0.0/24 and the other is 10.254.0.0/24, and a host has an incorrect subnet mask of 255.0.0.0 or /8, it will never be able to communicate across the VPN because it thinks the remote VPN subnet is part of the local network and hence routing will not function properly. The system with the broken configuration will attempt to contact the remote system via ARP instead of using the gateway.

Host firewall:

If there is a firewall on the target host, it may not be allowing the connections. Check for things like Windows Firewall, iptables, or similar utilities that may be preventing the traffic from being processed by the host.

Firewall rules on pfSense software:

Ensure the rules on both ends allow the desired network traffic.

Connection hangs

IPsec does not gracefully handle fragmented packets. Many of these issues have been resolved over the years, but there may be lingering problems and edge cases. If hangs or packet loss are seen only when using specific protocols (SMB, RDP, etc.), MSS clamping for the VPN may be necessary. MSS clamping can be activated under Firewall & NAT. A good starting point for MSS clamping is 1400. If that works slowly increase the MSS value until the breaking point is hit, then back off a little from there.

Disappearing traffic

If IPsec traffic arrives but never appears on the IPsec interface (enc0), check for conflicting routes/interface IP addresses. For example, if an IPsec tunnel is configured with a remote network of 192.0.2.0/24 and there is a local OpenVPN server with a tunnel network of 192.0.2.0/24 then the ESP traffic may arrive, strongSwan may process the packets, but they never show up on enc0 as arriving to the OS for delivery.

Resolve the duplicate interface/route and the traffic will begin to flow.