This is an old revision of the document!
FreeNAS 11.2-U5 and 10Gbe
This is just a note as I see a lot of posts out there saying use Chelsio cards for FreeNAS as they are well supported. Well I tried this and yes the card was recognized but the DAC cable was not.
Aug 9 18:45:00 ale cxgbc0: <Chelsio T310, 1 port> mem 0xfb881000-0xfb881fff,0xfb000000-0xfb7fffff,0xfb880000-0xfb880fff irq 42 at device 0.0 on pci10 Aug 9 18:45:00 ale kernel: PHY 0 i2c read of dev.addr a0.0 timed out Aug 9 18:45:00 ale kernel: PHY 0 i2c read of dev.addr a0.0 timed out Aug 9 18:45:00 ale cxgbc0: using MSI-X interrupts (9 vectors) Aug 9 18:45:00 ale kernel: found old FW minor version(7.10), driver compiled for version 7.11 Aug 9 18:45:00 ale kernel: found old FW minor version(7.10), driver compiled for version 7.11 Aug 9 18:45:00 ale cxgbc0: firmware needs to be updated to version 7.11.0 Aug 9 18:45:00 ale cxgb0: <Port 0 10GBASE-R> on cxgbc0 Aug 9 18:45:00 ale cxgb0: Using defaults for TSO: 65518/35/2048 Aug 9 18:45:00 ale cxgb0: Ethernet address: 00:07:43:05:fe:ae Aug 9 18:45:00 ale cxgbc0: Firmware Version 7.10.0
Firmware update - I don't think this was necessary as the driver auto updates we just need a reboot
cxgbtool cxgb0 loadfw t3fw-7.11.0.bin Reboot
Next time around the firmware has been updated
Aug 9 18:55:44 ale cxgbc0: using MSI-X interrupts (9 vectors) Aug 9 18:55:44 ale cxgb0: <Port 0 10GBASE-R> on cxgbc0 Aug 9 18:55:44 ale cxgb0: Using defaults for TSO: 65518/35/2048 Aug 9 18:55:44 ale cxgb0: Ethernet address: 00:07:43:05:fe:ae Aug 9 18:55:44 ale cxgbc0: Firmware Version 7.11.0
The drive is plumbed into the kernel but the link won't come up to ACTIVE.
I was using SPF+ CISCO Passive Twinax cable part 74752-9519 connected to Mikrotik CRS305 switch but the link would not activate. I switched to a different 10Gbe card and presto.
MNPA19-XTR | Mellanox EN ConnectX-2 1-Port 10GbE Network Adapter
Aug 22 19:52:55 ale mlx4_core0: <mlx4_core> mem 0xfaa00000-0xfaafffff,0xf5000000-0xf57fffff irq 42 at device 0.0 on pci10 Aug 22 19:52:55 ale kernel: mlx4_core: Mellanox ConnectX core driver v3.4.1 (October 2017) Aug 22 19:52:55 ale kernel: mlx4_core: Initializing mlx4_core Aug 22 19:52:55 ale mlx4_core0: Unable to determine PCI device chain minimum BW Aug 22 19:52:55 ale kernel: mlx4_en mlx4_core0: Activating port:1 Aug 22 19:52:55 ale mlxen0: Ethernet address: 00:02:c9:51:35:34 Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlxen0: link state changed to DOWN Aug 22 19:52:55 ale kernel: mlxen0: link state changed to DOWN Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Initializing port Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Initializing port
ifconfig output for the device of course at this point it's not configure with an IP but the status is “ACTIVE” and the switch can see it too.
root@ale[~]# ifconfig mlxen0 mlxen0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:02:c9:51:35:34 hwaddr 00:02:c9:51:35:34 nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect (autoselect <full-duplex,rxpause,txpause>) status: active root@ale[~]#