LAGG (Link Aggregation)
Link aggregation is handled by
lagg(4) type interfaces (LAGG) on pfSense.
LAGG combines multiple physical interfaces together as one logical interface.
There are several ways this can work, either for gaining extra bandwidth,
redundancy, or some combination of the two.
To create or manage LAGG interfaces:
Navigate to Interfaces > (assign), LAGGs tab
Click Add to create a new LAGG, or click to
edit an existing instance.
Complete the settings as follows:
- Parent Interfaces
This list contains all currently unassigned interfaces,
and members of the current LAGG interface when editing an existing instance.
To add interfaces to this LAGG, select one or more interfaces in this list.
An interface may only be added to a LAGG group if it is not
assigned. If an interface is not present in the list, it is likely
already assigned as an interface.
- LAGG Protocol
There are currently six different operating modes for LAGG
interfaces: LACP, Failover, Load Balance, Round Robin, and None.
The most commonly used LAGG protocol. This mode supports IEEE 802.3ad
Link Aggregation Control Protocol (LACP) and the Marker Protocol. In LACP
mode, negotiation is performed with the switch – which must also support
LACP – to form a group of ports that are all active at the same time.
This is knowns as a Link Aggregation Group, or LAG. The speed and MTU of
each port in a LAG must be identical and the ports must also run at full-
duplex. If link is lost to a port on the LAG, the LAG continues to
function but at reduced capacity. In this way, an LACP LAGG bundle can
gain both redundancy and increased bandwidth.
Traffic is balanced between all ports on the LAG, however, for
communication between two single hosts it will only use one single port at
a time because the client will only talk to one MAC address at a time. For
multiple connections through multiple devices, this limitation effectively
becomes irrelevant. The limitation is also not relevant for failover.
In addition to configuring this option on pfSense, the switch must enable
LACP on these ports or have the ports bundled into a LAG group. Both sides
must agree on the configuration in order for it to work properly.
When using the Failover LAGG protocol traffic will only be sent
on the primary interface of the group. If the primary interface fails,
then traffic will use the next available interface. The primary interface
is the first interface selected in the list, and will continue in order
until it reaches the end of the selected interfaces.
By default, traffic may only be received on the active
interface. Create a system tunable for
net.link.lagg.failover_rx_all with a value of
1 to allow
traffic to be received on every member interface.
- Load Balance
Load Balance mode accepts inbound traffic on any port of the
LAGG group and balances outgoing traffic on any active ports in the LAGG
group. It is a static setup that does not monitor the link state nor does it
negotiate with the switch. Outbound traffic is load balanced based on all
active ports in the LAGG using a hash computed using several factors, such
as the source and destination IP address, MAC address, and VLAN tag.
- Round Robin
This mode accepts inbound traffic on any port of the LAGG
group and sends outbound traffic using a round robin scheduling
algorithm. Typically this means that traffic will be sent out in sequence,
using each interface in the group in turn.
This mode disables traffic on the LAGG interface without disabling the
interface itself. The OS will still believe the interface is up and usable,
but no traffic will be sent or received on the group.
A short note about the purpose of this LAGG instance.
After creating a LAGG interface, it works like any other physical interface.
Assign the lagg interface under Interfaces > (assign) and give it an IP
address, or build other things on top of it such as VLANs.
LAGG and Traffic Shaping
Due to limitations in FreeBSD,
lagg(4) does not support
altq(4) so it is
not possible to use the traffic shaper on LAGG interfaces directly.
altq(4) and VLANs can be used on top of LAGG interfaces,
so using VLANs can work around the problem. As an alternate workaround, Limiters
can control bandwidth usage on LAGG interfaces.
Using a LAGG does not necessarily guarantee full throughput equal to the sum of
all interfaces. In particular, a single flow will not exceed the throughput of a
LAGG member interface. Traffic on a LAGG is hashed in such a way that flows
between two hosts, such as pfSense and an upstream gateway, would only use a
single link since the flow is between a single MAC address on each side.
In networks where there are many hosts communicating with different MAC
addresses, the usage can approach the sum of all interfaces in the LAGG.