Mellanox ConnectX-5 Firmware Requirements¶
Mellanox ConnectX-5 network interface cards (mlx5) in the MT27800 family are
currently shipping with firmware revision
16.26.1040 which is not compatible
The incompatible firmware can be identified by errors in the log (e.g.
vppctl show errors), such as:
net_mlx5: probe of PCI device [PCI ID] aborted after encountering an error: Operation not supported
Interface WAN error -12: Unknown error -12 net_mlx5: Failed to query QP using DevX net_mlx5: Fail to query port 0 Tx queue 0 QP TIS transport domain net_mlx5: port 0 Tx queue allocation failed: Cannot allocate memory Device with port_id=0 already stopped
16.24.1000 is compatible with TNSR, and can be manually
downgraded using the following procedure.
First, identify the first PCI ID of the card in question. This can be found by
looking in the boot logs, the output of a utility such as
lspci, or similar
methods. The ID will take the form of
xx:yy.z, for example
ID will be used in the following set of commands.
Next, download and decompress the appropriate firmware:
$ curl -LO http://www.mellanox.com/downloads/firmware/fw-ConnectX5-rel-16_24_1000-MCX516A-CCA_Ax-UEFI-14.17.11-FlexBoot-3.5.603.bin.zip $ unzip fw-ConnectX5-rel-16_24_1000-MCX516A-CCA_Ax-UEFI-14.17.11-FlexBoot-3.5.603.bin.zip
Now install the Mellanox firmware tool and perform the firmware downgrade:
$ sudo dnf install mstflint $ sudo mstconfig q > mst.log $ sudo mstfwmanager -d <PCI ID> -u -f -i fw-ConnectX5-rel-16_24_1000-MCX516A-CCA_Ax-UEFI-14.17.11-FlexBoot-3.5.603.bin
<PCI ID> in the last command with the first PCI ID of the card.
With the appropriate firmware loaded on the card, it will no longer produce errors and it will be usable by TNSR.
Reboot after performing the firmware downgrade to ensure the card is fully reinitialized with the appropriate firmware.