State Synchronization (pfsync) Overview¶
pfsync handles synchronization of the firewall state table between cluster nodes. Changes to the state table on the primary are sent to the secondary nodes over the Sync interface, and vice versa. When State Synchronization is active and properly configured all nodes will have knowledge of each connection flowing through the cluster. If the master node fails, the backup node will take over and clients will not notice the transition since both nodes knew about the connection beforehand.
State Synchronization with pfsync uses multicast by default, though an IP address can be defined to force unicast updates. This is ideal for environments with only two firewalls where multicast traffic is unnecessary and may not function properly. Any active interface can be used for sending pfsync updates, however utilizing a dedicated interface is the best practice for security and performance.
Warning
pfsync does not support any method of authentication. If the interface is set to anything other than an isolated segment it is possible for a user with access to the network on that interface to manipulate the state table. For example, they could insert states into the state table.
In low throughput environments that aren’t security paranoid, use of the LAN interface for this purpose may be acceptable. Bandwidth required for this state synchronization will vary significantly from one environment to another, but could be as high as 10% of the throughput traversing the firewall depending on the rate of state insertions and deletions in a network.
Failover can still operate without State Synchronization, but it will not be seamless. Without State Synchronization if a node fails and another takes over, user connections would be dropped. Users may immediately reconnect through the other node, but they would be disrupted during the transition. Depending on the usage in a particular environment, this may go unnoticed or it could be a significant, but brief, outage.
When State Synchronization is in use, State Synchronization settings must be enabled on all nodes participating in state synchronization, including secondary node(s), or State Synchronization will not function properly.
pfsync and Firewall Rules¶
Traffic for pfsync must be explicitly passed on the Sync interface. The rule must pass the pfsync protocol from a source of the Sync network to any destination. A rule passing all traffic of any protocol would also allow the required traffic, but a more specific rule is more secure.
pfsync and Physical Interfaces¶
Note
This is no longer the case on pfSense Plus software version 22.01 and later and pfSense CE software version 2.6.0 and later. On these versions, the states are no longer bound to interfaces in the way described in this section.
On some versions of pfSense® software states are bound to specific operating system Interfaces. For example, if WAN is em0, then a state on WAN would be tied to em0. If the cluster nodes have identical hardware and interface assignments then this works as expected. In cases when different hardware is used, this can be a problem. If WAN on one node is em0 but on another node it is igb0, the states will not match and they will not be treated the same.
It is always preferable to have identical hardware, but in cases where this is impractical there is a workaround: Adding interfaces to a LAGG will abstract the actual underlying physical interface so in the above example, WAN would be lagg0 on both and states would be bound to lagg0, even though lagg0 on one node contains em0 and it contains igb0 on the other node.
pfsync and Upgrades¶
Normally pfSense software allows HA firewall upgrades without network disruption. Unfortunately, this isn’t always the case with upgrades as the pfsync protocol can change to accommodate additional functionality. Always check the upgrade guide linked in all release announcements before upgrading to see if there are any special considerations for CARP users.