Setting up Opengrok in a FreeNAS 11.2 iocage jail

Create storage dataset's for GROK

Create a jail for GROK

Setup jail mount points for src and data. You need to stop the jail to add mount points and then restart afterwards.

  • /mnt/u03/opengrok/src → /var/opengrok/src
  • /mnt/u03/opengrok/data → /var/opengrok/data

Find the JAIL and open a shell

root@ale[~]# jls
   JID  IP Address      Hostname                      Path
     1                  plex                          /mnt/u02/iocage/jails/plex/root
     2                  bind                          /mnt/u02/iocage/jails/bind/root
     5                  beer                          /mnt/u02/iocage/jails/beer/root
     6                  grok                          /mnt/u02/iocage/jails/grok/root
root@ale[~]# jexec 6 /bin/sh

Install opengrok and git. To generate the history caches from GIT based repositories we need the git command line tool.

# pkg install -y opengrok git

# pkg info opengrok
opengrok-1.0
Name           : opengrok
Version        : 1.0
Installed on   : Tue May 14 09:04:03 2019 EDT
Origin         : devel/opengrok
Architecture   : FreeBSD:11:*
Prefix         : /usr/local
Categories     : devel java
Licenses       : APACHE20, CDDL
Maintainer     : ports@FreeBSD.org
WWW            : http://opengrok.github.io/OpenGrok/
Comment        : Fast and powerful code search and cross-reference engine
Options        :
        DOCS           : on
        RESIN3         : off
        TOMCAT6        : off
        TOMCAT7        : off
        TOMCAT8        : on
        TOMCAT85       : off
Annotations    :
        repo_type      : binary
        repository     : FreeBSD
Flat size      : 20.5MiB
Description    :
OpenGrok is a fast source code search and cross reference engine.
It helps you search, cross-reference and navigate your source tree.  It can
understand various program file formats and version control histories like
Mercurial, Git, SCCS, RCS, CVS, Subversion, Teamware, ClearCase, Perforce
and Bazaar.  In other words it lets you grok (profoundly understand) the
open source, hence the name OpenGrok.  It is written in Java.

WWW: http://opengrok.github.io/OpenGrok/

This installs tomcat8 we need to autostart this.

# ls /usr/local/etc/rc.d
tomcat8

Enable for auto start /etc/rc.conf

echo tomcat8_enable="YES" >>/etc/rc.conf

and start it

service tomcat8 start

This should fire up with an error. That's OK as we have not indexed anything the error will disappear after indexing.

Make a directory

mkdir /var/opengrok/etc

Create a helper script and run it.

cat <<EOF >~/opengrok-index
opengrok -c /usr/local/bin/exctags -s /var/opengrok/src -d /var/opengrok/data -H -P -S -G -W /var/opengrok/etc/configuration.xml
service tomcat8 restart
EOF
chmod a+x ~/opengrok-index
~/opengrok-index

The -U parameter only takes host:port and it assumes the WEBAPP is called /source which means the indexer can't tell the running opengrok to reload. So restart tomcat as part of the index run. A minor wart.

Making opengrok the default webapp and use port 80

Moving tomcat from port 8080 to port 80 Edit /usr/local/apache-tomcat-8.0/conf/server.xml file and replace 8080 with 80

    <Connector port="80" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />

Make opengrok the default web application by adding a <Context> tag inside <Host>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <Context path="" docBase="/usr/local/apache-tomcat-8.0/webapps/opengrok">
                <WatchedResource>WEB-INF/web.xml</WatchedResource>
        </Context>
2019/06/25 00:00 · 0 Linkbacks

Setting up MoinMoin in a FreeNAS 11.2 iocage jail

Create a jail and attach the storage into the JAIL at this location: /usr/local/www/wiki/data

Login to the jail and install moin this version will do nicely.

# pkg search moin
moinmoin-1.9.10                Easy to use, full-featured and extensible wiki software package

Install MoinMoin and run the recommend steps installing from a package.

# pkg install -y moinmoin

Setup for WSGI

export MOINSCRIPT="moin.wsgi"
export MOINDIR="/usr/local/share/moin"
export MOINDEST="/usr/local/www/wiki"
export CGIUSER="www"
export CGIGROUP="www"
 
mkdir -p ${MOINDEST}/data
mkdir -p ${MOINDEST}/underlay
cp -R ${MOINDIR}/data ${MOINDEST}
cp -R ${MOINDIR}/underlay ${MOINDEST}
chmod -R u+rw,go-ws ${MOINDEST}/data
install -m 0555 ${MOINDIR}/config/wikiconfig.py ${MOINDEST}
test -z "${MOINSCRIPT}" || \
        install -m 0555 ${MOINDIR}/server/${MOINSCRIPT} ${MOINDEST}
chown -R ${CGIUSER}:${CGIGROUP} ${MOINDEST}

Based very closely around * https://wiki.freebsd.org/Ports/www/moinmoin

Install nginx and uwsgi server

pkg install -y nginx uwsgi

Replace /usr/local/etc/nginx/nginx.conf with

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;

    keepalive_timeout  65;

    server {
            listen 80;
            server_name wiki.local;

            location / {
                    uwsgi_pass unix:/var/run/moin.sock;
                    include uwsgi_params;
            }

            location ~ ^/moin_static[0-9]+/(.*) {                    
                    alias  /usr/local/lib/python2.7/site-packages/MoinMoin/web/static/htdocs/$1;

            }
    }
}

NOTE: Its important to access the server by a DNS name in the nginx.conf file that resolves to the jail IP. Not doing this will result in the NGINX server not being able to correctly lookup the static content.

Append to /etc/rc.conf

uwsgi_enable="YES"
uwsgi_configfile="/usr/local/www/wiki/uwsgi.ini"
uwsgi_uid="www"
uwsgi_gid="www"

nginx_enable="YES"

Create /usr/local/www/wiki/uwsgi.ini

[uwsgi]
socket = /var/run/moin.sock
chmod-socket = 660
 
chdir = /usr/local/www/wiki
wsgi-file = moin.wsgi
 
master
workers = 3
max-requests = 200
harakiri = 30
die-on-term

Allow anybody to edit anything and for them to create their own account.

Add the acl_rights_default entries and uncomment page_front_page and actions_superuser.

/usr/local/www/wiki/wikiconfig.py

class Config(multiconfig.DefaultConfig):
 
    # Critical setup  ---------------------------------------------------
 
    # Directory containing THIS wikiconfig:
    wikiconfig_dir = os.path.abspath(os.path.dirname(__file__))
 
    acl_rights_default = u"All:read,write,delete,revert,admin"
    page_front_page = u'FrontPage'
 
    actions_superuser = multiconfig.DefaultConfig.actions_superuser[:]
    actions_superuser.remove('newaccount')

Start it up

service uwsgi start
service ngnix start

Speeding up searching by enabling xapian

pkg install xapian-core py27-xapian

/usr/local/www/wiki/wikiconfig.py

    xapian_search = True
    xapian_stemming = True

Restart

service uwsgi restart

You can check to see if its enabled by navigating to SystemInfo

2019/06/24 23:50 · 0 Linkbacks

Chrome randomly shows pink purple letters

Chrome updated itself to 69.0.3497.100 (Official Build) (64-bit) and ever since then I have experienced text that has random pink, purple and blue highlights.

chrome-random-pink-letters.jpg

This was corrected by entering chrome:⁄⁄settings into the chrome address area and then searching for hardware acceleration and disabling it.

chrome-disable-hwaccel.jpg

2018/10/11 18:28 · 0 Linkbacks

CCB request completed with an error

FreeNAS server was randomly crashing:

(da0:umass-sim0:0:0:0): Retrying command
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 00 24 26 9d 00 00 10 00
(da0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(da0:umass-sim0:0:0:0): Error 5, Retries exhausted
root@freenas:/mnt/data #

The USB drive is causing this.

root@wine:~ # usbconfig
ugen0.1: <Intel EHCI root HUB> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen1.1: <Intel EHCI root HUB> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen1.2: <vendor 0x8087 product 0x0024> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.2: <vendor 0x8087 product 0x0024> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.3: <SanDisk Cruzer Blade> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (224mA)

root@wine:~ # usbconfig -u 0 -a 3 dump_device_desc
ugen0.3: <SanDisk Cruzer Blade> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (224mA)

  bLength = 0x0012
  bDescriptorType = 0x0001
  bcdUSB = 0x0210
  bDeviceClass = 0x0000  <Probed by interface class>
  bDeviceSubClass = 0x0000
  bDeviceProtocol = 0x0000
  bMaxPacketSize0 = 0x0040
  idVendor = 0x0781
  idProduct = 0x5567
  bcdDevice = 0x0100
  iManufacturer = 0x0001  <SanDisk>
  iProduct = 0x0002  <Cruzer Blade>
  iSerialNumber = 0x0003  <4C531001561109121142>
  bNumConfigurations = 0x0001

Switched to a different USB stick and the problem went away.

Lesson: Not all USB sticks are created equally.

2018/05/28 14:43 · 0 Linkbacks

PEBS disabled due to CPU errata

I noticed this in /var/log/messages on my freshly installed CentOS 7.5 system.

May 24 09:46:43 localhost kernel: smpboot: CPU0: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz (fam: 06, model: 2a, stepping: 07)
May 24 09:46:43 localhost kernel: Performance Events: PEBS fmt1+, 16-deep LBR, SandyBridge events, full-width counters, Intel PMU driver.
May 24 09:46:43 localhost kernel: core: PEBS disabled due to CPU errata, please upgrade microcode
May 24 09:46:43 localhost kernel: ... version:                3
May 24 09:46:43 localhost kernel: ... bit width:              48
May 24 09:46:43 localhost kernel: ... generic registers:      8
May 24 09:46:43 localhost kernel: ... value mask:             0000ffffffffffff
May 24 09:46:43 localhost kernel: ... max period:             00007fffffffffff
May 24 09:46:43 localhost kernel: ... fixed-purpose events:   3
May 24 09:46:43 localhost kernel: ... event mask:             00000007000000ff

RedHat has this to say about it. ref: https://access.redhat.com/solutions/634443

Root Cause

  • Clovertown and SandyBridge processors have errata regarding PEBS functionality.

Diagnostic Steps

  • Look at /proc/cpuinfo for model number 15, 42 or 45

So I did

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31220 @ 3.10GHz
stepping        : 7
microcode       : 0x29
cpu MHz         : 1599.951
cache size      : 8192 KB

Install a few packages

# yum install microcode_ctl.x86_64
# yum install iucode-tool
# reboot

I see this in /var/log/messages on the reboot

May 26 18:55:50 wine kernel: microcode: microcode updated early to revision 0x2d, date = 2018-02-07

Confirmation the processor has microcode patches applied

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31220 @ 3.10GHz
stepping        : 7
microcode       : 0x2d

The microcode is being updated as the system loads.

2018/05/26 19:04 · 0 Linkbacks

Older entries >>