State Synchronization (pfsync) Overview

pfSense® software uses pfsync to synchronize firewall state table data between cluster nodes. Changes to the state table on the primary are sent to the secondary nodes over the Sync interface, and vice versa. When State Synchronization is active and properly configured, all nodes have knowledge of every connection flowing through the cluster. If the primary node fails, the secondary node will take over and most clients will not notice the transition since both nodes knew about the connection beforehand.

See also

State Synchronization Settings (pfsync)

State synchronization with pfsync uses multicast by default, though an IP address can be defined to force unicast updates. This is ideal for environments with only two firewalls where multicast traffic is unnecessary and may not function properly. Any active interface can be used for sending pfsync updates, however utilizing a dedicated interface is the best practice for security and performance.

Warning

pfsync does not support any method of authentication. If the interface is set to anything other than an isolated segment it is possible for a user with access to the network on that interface to manipulate the state table. For example, they could insert states into the state table.

In low throughput environments that aren’t security paranoid, use of the LAN interface for this purpose may be acceptable. Bandwidth required for this state synchronization will vary significantly from one environment to another, but could be as high as 10% of the throughput traversing the firewall depending on the rate of state insertions and deletions.

Failover can still operate without state synchronization, but it will not be seamless. Without state synchronization, if a node fails and another takes over, user connections are dropped. Users may immediately reconnect through the other node, but they would be disrupted during the transition. Depending on the usage in a particular environment, this may go unnoticed or it could be a significant, but brief, outage.

When state synchronization is in use, State Synchronization settings must be enabled on all nodes participating in state synchronization, including secondary node(s), or state synchronization will not function properly.

pfsync and Firewall Rules

Traffic for pfsync must be explicitly passed on the Sync interface. The rule must pass the pfsync protocol from a source of the Sync network to any destination. A rule passing all traffic of any protocol would also allow the required traffic, but a more specific rule is more secure.

pfsync and Physical Interfaces

States contain information about the interface to which they are bound. Whether or not this impacts pfsync depends on the default State Policy for the node, which can be “Interface Bound States” or “Floating States” (Firewall State Policy).

If the default state policy is Floating States and no rules are set to use Interface Bound States, then there is no conflict and state synchronization will work even if the hardware on the nodes is different.

If the default policy is set to Interface Bound States, or any rules are set to use Interface Bound States, then there may be a potential conflict with High Availability nodes which have different hardware.

If the interfaces are not both physically identical and assigned in the same order on both nodes then the states will not properly sync, for example if WAN is ix0 on one node and igb0 on the other.

While having identical hardware is always the best practice, mismatched hardware can still function with Interface Bound States by using LAGG interfaces to abstract the assignments. LAGGs can work around this since the states would be bound to the laggX interface on each node rather than the underlying physical interface. For example, lagg0 on primary contains ix0, lagg0 on secondary contains igb0, but the states are on lagg0 for both so sync will function.

pfsync and Upgrades

Normally pfSense software allows HA firewall upgrades without network disruption. Unfortunately, this isn’t always the case with upgrades as the pfsync protocol can change to accommodate additional functionality. Always check the upgrade guide linked in all release announcements before upgrading to see if there are any special considerations for CARP users.