blog:freenas_11.2-u5_and_10gbe [DBZoo]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== FreeNAS 11.2-U5 and 10Gbe ======

This is just a note as I see a lot of posts out there saying use Chelsio cards for FreeNAS as they are well supported.  Well I tried this and yes the card was recognized but the DAC cable was not. 
<code>
Aug  9 18:45:00 ale cxgbc0: <Chelsio T310, 1 port> mem 0xfb881000-0xfb881fff,0xfb000000-0xfb7fffff,0xfb880000-0xfb880fff irq
 42 at device 0.0 on pci10
Aug  9 18:45:00 ale kernel: PHY 0 i2c read of dev.addr a0.0 timed out
Aug  9 18:45:00 ale kernel: PHY 0 i2c read of dev.addr a0.0 timed out
Aug  9 18:45:00 ale cxgbc0: using MSI-X interrupts (9 vectors)
Aug  9 18:45:00 ale kernel: found old FW minor version(7.10), driver compiled for version 7.11
Aug  9 18:45:00 ale kernel: found old FW minor version(7.10), driver compiled for version 7.11
Aug  9 18:45:00 ale cxgbc0: firmware needs to be updated to version 7.11.0
Aug  9 18:45:00 ale cxgb0: <Port 0 10GBASE-R> on cxgbc0
Aug  9 18:45:00 ale cxgb0: Using defaults for TSO: 65518/35/2048
Aug  9 18:45:00 ale cxgb0: Ethernet address: 00:07:43:05:fe:ae
Aug  9 18:45:00 ale cxgbc0: Firmware Version 7.10.0
</code>
Firmware update - I don't think this was necessary as the driver auto updates we just need a reboot
<code>
cxgbtool cxgb0 loadfw t3fw-7.11.0.bin
Reboot
</code>
Next time around the firmware has been updated
<code>
Aug  9 18:55:44 ale cxgbc0: using MSI-X interrupts (9 vectors)
Aug  9 18:55:44 ale cxgb0: <Port 0 10GBASE-R> on cxgbc0
Aug  9 18:55:44 ale cxgb0: Using defaults for TSO: 65518/35/2048
Aug  9 18:55:44 ale cxgb0: Ethernet address: 00:07:43:05:fe:ae
Aug  9 18:55:44 ale cxgbc0: Firmware Version 7.11.0
</code>
The drive is plumbed into the kernel but the link won't come up to ACTIVE.

I was using SPF+ CISCO Passive Twinax cable [[https://www.molex.com/molex/products/datasheet.jsp?part=active/0747521101_CABLE_ASSEMBLIES.xml&channel=Products|part 74752-9519]] connected to [[https://mikrotik.com/product/crs305_1g_4s_in|Mikrotik CRS305]] switch but the link would not activate.  I switched to a different 10Gbe card and presto.

MNPA19-XTR | Mellanox EN ConnectX-2 1-Port 10GbE Network Adapter
<code>
Aug 22 19:52:55 ale mlx4_core0: <mlx4_core> mem 0xfaa00000-0xfaafffff,0xf5000000-0xf57fffff irq 42 at device 0.0 on pci10
Aug 22 19:52:55 ale kernel: mlx4_core: Mellanox ConnectX core driver v3.4.1 (October 2017)
Aug 22 19:52:55 ale kernel: mlx4_core: Initializing mlx4_core
Aug 22 19:52:55 ale mlx4_core0: Unable to determine PCI device chain minimum BW
Aug 22 19:52:55 ale kernel: mlx4_en mlx4_core0: Activating port:1
Aug 22 19:52:55 ale mlxen0: Ethernet address: 00:02:c9:51:35:34
Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 12 TX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 12 TX rings
Aug 22 19:52:55 ale kernel: mlxen0: link state changed to DOWN
Aug 22 19:52:55 ale kernel: mlxen0: link state changed to DOWN
Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 12 TX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 12 TX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 8 RX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 8 RX rings
Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Initializing port
Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Initializing port
</code>
ifconfig output for the device. Its alive and happy.
<code>
mlxen0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu   1500
        options=ad00b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN  _HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6>
        ether 00:02:c9:51:35:34
        hwaddr 00:02:c9:51:35:34
        inet 192.168.1.22 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
        status: active
root@ale[~]#
</code>

{{tag>10Gbe FreeNAS}}

~~LINKBACK~~