Huge traffic for WeMo devices
The Wemo device on my network was generating a lot of traffic and I wanted to know why. Others have asked the same question
This is our Wemo. We'll need the mac address so we can capture traffic to/from this device. My equipment is all UniFi; Router EdgeRouter X, Access point AC Lite.
SSH to the Unifi WiFi point and run a tcpdump for the MAC address of the WEMO device.
tcpdump -w /tmp/wemo.pcap ether host 24:f5:a2:f4:b0:8f
Address 192.168.1.1 is my router
It's performing a UPNP enumeration. Given there are no device opening UPNP rule we will disable the service
$ show upnp2 rules Firewall pin holes pkts bytes target prot opt in out source destination NAT port forwards pkts bytes target prot opt in out source destination pkts bytes target prot opt in out source destination $ configure # disable service upnp2 # commit # save
Other devices on my network such as a printer are also running a UPNP listening. IP ending in 101 is the printer. The WeMo is constantly asking this device for its endpoints via an HTTP request that returns XML (yet more traffic).
Go into the Printers Web Administrative interface and disable the UPNP protocol, we don't want a UPNP service in the printer anyway.
This reduced the amount of Wifi traffic the WeMo is generating as there are no devices that will communicate UPNP with it.
Importing Plex jail failure
I was relocating my jails from one storage pool to another. Shout to digimoot for some good documentation.
When I went to import the Plex jail it failed.
root@ale[~]# iocage import plex -p /mnt/u02/images Importing dataset: plex Importing dataset: plex/root cannot receive: failed to read from stream zsh: killed iocage import plex -p /mnt/u02/images
The problem is the plex jail is 64Gb before exporting 48Gb exported. The database for it should probably be separate from the Jail.
This was the first post that I ran across on this issue
Which led me to discover FreeNAS 11 has a defect. It tries to uncompress the jail root into memory before sending it to ZFS RECV. That's right, we load the 64Gb file into memory (BANG).
This bug still exists in FreeNAS-11.3-U5: /usr/local/lib/python3.7/site-packages/iocage_lib/ioc_image.py
if compression_algo == 'zip':
data = f.open(name).read()
else:
data = f.extractfile(member).read()
recv.stdin.write(data)
recv.communicate()
This results in running out of swap and the import being terminated. Python3.7 is the IOCAGE command running “iocage import plex”
root@ale[/var/log]# fgrep 'out of swap space' messages Aug 17 07:29:21 ale kernel: pid 67038 (python3.7), jid 0, uid 0, was killed: out of swap space Aug 17 07:29:21 ale kernel: pid 67038 (python3.7), jid 0, uid 0, was killed: out of swap space swap_pager_getswapspace(2): failed swap_pager_getswapspace(2): failed swap_pager_getswapspace(32): failed
Code was fixed in TrueNAS 12 - https://github.com/iocage/iocage/blob/master/iocage_lib/ioc_image.py#L288
chunk_size = 10 * 1024 * 1024
with (f.open(name) if compression_algo == 'zip' else f.extractfile(member)) as file:
data = file.read(chunk_size)
while data is not None and len(data) > 0:
recv.stdin.write(data)
data = file.read(chunk_size)
recv.communicate()
It has not been backported FreeNAS 11, we will do that ourselves so we can continue to use the “iocage import” process without hacking about with workaround commands.
That worked well.
root@ale[~]# iocage import plex -p /mnt/u02/images Importing dataset: plex Importing dataset: plex/root Imported: plex
Backing up a FreeNas jails
A simple script that backups up each JAIL in turn, sorted alphabetically by its name. An export requires the jail to be shutdown.
#!/bin/sh # Backup each JAIL # Will restart the jail if required iocage list -h -s name | cut -f 2,3 | while read jail state; do test $state = "up" && iocage stop $jail iocage export $jail test $state = "up" && iocage start $jail done
The images will be in /mnt/[pool]/iocage/images
Installing the Moin2rst plugin
I was trying to install this plugin into my MoinMoin 1.9 wiki → https://github.com/dwf/moin2rst
The installation instruction are incorrect
wiki/data/plugin/action/RenderAsRestructuredtext.py wiki/data/plugin/formatter/text_x-rst.py
Errors and changes to make this work with MoinMoin 1.9
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/MoinMoin/support/werkzeug/wsgi.py", line 766, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 264, in __call__
response = run(context)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 89, in run
response = dispatch(request, context, action_name)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 137, in dispatch
response = handle_action(context, pagename, action_name)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 203, in handle_action
handler(context.page.page_name, context)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/action/__init__.py", line 277, in do_format
do_show(pagename, request, count_hit=0, cacheable=0, mimetype=u'text/plain')
File "/usr/local/lib/python2.7/site-packages/MoinMoin/action/__init__.py", line 267, in do_show
content_only=content_only,
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1165, in send_page
Formatter = wikiutil.searchAndImportPlugin(request.cfg, "formatter", self.output_mimetype)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wikiutil.py", line 1194, in searchAndImportPlugin
raise PluginMissingError("Plugin not found! (%r %r %r)" % (type, name, what))
MoinMoin.wikiutil.PluginMissingError: Plugin not found! ('formatter' 'text/x-rst' 'Formatter')
A change in wikiutil.py#Mime means the formatter plug in is not being found. The file we are looking for won't be what is supplied with the GIT repo.
modname = mimetype.replace("/", "_").replace("-", "_").replace(".", "_")
Rename the file from text_x-rst.py TO text_x_rst.py
-rw-r--r-- 1 root www 34417 Jan 21 17:11 text_x_rst.py
Next problem
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/MoinMoin/support/werkzeug/wsgi.py", line 766, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 264, in __call__
response = run(context)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 89, in run
response = dispatch(request, context, action_name)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 137, in dispatch
response = handle_action(context, pagename, action_name)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 203, in handle_action
handler(context.page.page_name, context)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/action/__init__.py", line 277, in do_format
do_show(pagename, request, count_hit=0, cacheable=0, mimetype=u'text/plain')
File "/usr/local/lib/python2.7/site-packages/MoinMoin/action/__init__.py", line 267, in do_show
content_only=content_only,
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1332, in send_page
start_line=pi['lines'])
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1422, in send_page_content
self.format(parser)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1443, in format
parser.format(self.formatter)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1553, in format
formatted_line = self.scan(line, inhibit_p=inhibit_p)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1363, in scan
result.append(self.replace(match, inhibit_p))
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1407, in replace
result.append(replace_func(hit, match.groupdict()))
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1330, in _macro_repl
return self.formatter.macro(self.macro, macro_name, macro_args, markup=groups.get('macro'))
TypeError: macro() got an unexpected keyword argument 'markup'
text_x_rst.py: Change the definition of this call:
def macro(self, macroObj, name, argString):
to:
def macro(self, macroObj, name, argString, **kwargs):
Next problem
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/MoinMoin/support/werkzeug/wsgi.py", line 766, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 264, in __call__
response = run(context)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 89, in run
response = dispatch(request, context, action_name)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 137, in dispatch
response = handle_action(context, pagename, action_name)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/wsgiapp.py", line 203, in handle_action
handler(context.page.page_name, context)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/action/__init__.py", line 277, in do_format
do_show(pagename, request, count_hit=0, cacheable=0, mimetype=u'text/plain')
File "/usr/local/lib/python2.7/site-packages/MoinMoin/action/__init__.py", line 267, in do_show
content_only=content_only,
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1332, in send_page
start_line=pi['lines'])
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1422, in send_page_content
self.format(parser)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/Page.py", line 1443, in format
parser.format(self.formatter)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1553, in format
formatted_line = self.scan(line, inhibit_p=inhibit_p)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1363, in scan
result.append(self.replace(match, inhibit_p))
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 1407, in replace
result.append(replace_func(hit, match.groupdict()))
File "/usr/local/lib/python2.7/site-packages/MoinMoin/parser/text_moin_wiki.py", line 876, in _link_repl
self._link_description(desc, target, page_name_and_anchor) +
File "/usr/local/www/wiki/data/plugin/formatter/text_x_rst.py", line 488, in pagelink
url = self.request.normalizePagename(pagename)
File "/usr/local/lib/python2.7/site-packages/MoinMoin/web/contexts.py", line 224, in __getattr__
return super(HTTPContext, self).__getattribute__(name)
AttributeError: 'AllContext' object has no attribute 'normalizePagename'
Change these two calls in the text_x_rst.py file
def pagelink...
url = self.request.normalizePagename(pagename)
urlPath = url.split("/")
thisPath = self.request.normalizePagename(self.page.page_name).split("/")
TO
url = wikiutil.normalize_pagename(pagename, self.request.cfg)
urlPath = url.split("/")
thisPath = wikiutil.normalize_pagename(self.page.page_name, self.request.cfg).split("/")
After all this and getting it working I decided it was just easier to use docbook !
FreeNAS 11.2-U5 and 10Gbe
This is just a note as I see a lot of posts out there saying use Chelsio cards for FreeNAS as they are well supported. Well I tried this and yes the card was recognized but the DAC cable was not.
Aug 9 18:45:00 ale cxgbc0: <Chelsio T310, 1 port> mem 0xfb881000-0xfb881fff,0xfb000000-0xfb7fffff,0xfb880000-0xfb880fff irq 42 at device 0.0 on pci10 Aug 9 18:45:00 ale kernel: PHY 0 i2c read of dev.addr a0.0 timed out Aug 9 18:45:00 ale kernel: PHY 0 i2c read of dev.addr a0.0 timed out Aug 9 18:45:00 ale cxgbc0: using MSI-X interrupts (9 vectors) Aug 9 18:45:00 ale kernel: found old FW minor version(7.10), driver compiled for version 7.11 Aug 9 18:45:00 ale kernel: found old FW minor version(7.10), driver compiled for version 7.11 Aug 9 18:45:00 ale cxgbc0: firmware needs to be updated to version 7.11.0 Aug 9 18:45:00 ale cxgb0: <Port 0 10GBASE-R> on cxgbc0 Aug 9 18:45:00 ale cxgb0: Using defaults for TSO: 65518/35/2048 Aug 9 18:45:00 ale cxgb0: Ethernet address: 00:07:43:05:fe:ae Aug 9 18:45:00 ale cxgbc0: Firmware Version 7.10.0
Firmware update - I don't think this was necessary as the driver auto updates we just need a reboot
cxgbtool cxgb0 loadfw t3fw-7.11.0.bin Reboot
Next time around the firmware has been updated
Aug 9 18:55:44 ale cxgbc0: using MSI-X interrupts (9 vectors) Aug 9 18:55:44 ale cxgb0: <Port 0 10GBASE-R> on cxgbc0 Aug 9 18:55:44 ale cxgb0: Using defaults for TSO: 65518/35/2048 Aug 9 18:55:44 ale cxgb0: Ethernet address: 00:07:43:05:fe:ae Aug 9 18:55:44 ale cxgbc0: Firmware Version 7.11.0
The drive is plumbed into the kernel but the link won't come up to ACTIVE.
I was using SPF+ CISCO Passive Twinax cable part 74752-9519 connected to Mikrotik CRS305 switch but the link would not activate. I switched to a different 10Gbe card and presto.
MNPA19-XTR | Mellanox EN ConnectX-2 1-Port 10GbE Network Adapter
Aug 22 19:52:55 ale mlx4_core0: <mlx4_core> mem 0xfaa00000-0xfaafffff,0xf5000000-0xf57fffff irq 42 at device 0.0 on pci10 Aug 22 19:52:55 ale kernel: mlx4_core: Mellanox ConnectX core driver v3.4.1 (October 2017) Aug 22 19:52:55 ale kernel: mlx4_core: Initializing mlx4_core Aug 22 19:52:55 ale mlx4_core0: Unable to determine PCI device chain minimum BW Aug 22 19:52:55 ale kernel: mlx4_en mlx4_core0: Activating port:1 Aug 22 19:52:55 ale mlxen0: Ethernet address: 00:02:c9:51:35:34 Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlxen0: link state changed to DOWN Aug 22 19:52:55 ale kernel: mlxen0: link state changed to DOWN Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 12 TX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Using 8 RX rings Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Initializing port Aug 22 19:52:55 ale kernel: mlx4_en: mlxen0: Initializing port
ifconfig output for the device. Its alive and happy.
mlxen0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ad00b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN _HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6>
ether 00:02:c9:51:35:34
hwaddr 00:02:c9:51:35:34
inet 192.168.1.22 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
status: active
root@ale[~]#