xAP Plugboard (V2 script engine)
Whilst developing the HAH project I noticed there was an architectural component that would simplify the interactions between xAP compliant devices and daemons. I call this the plug-board.
Consider the following use cases:
- Send a twitter message when the power being monitored with Current Cost goes above 2kW
- If the temperature drops below 10C turn on a heater turning it off again when it reaches 12C
- When a Current Cost/temperature event is seen, log it to pachube
- If Input 1 goes high turn on Relay 1
- If RF 1 is turned on schedule a Google calendar event to turn it off in 1 hour
- Log a Google calendar event whenever Input 2 changes state
- If no change is detected on Input 3 during the last 2 hours, send an SMS message
- If a change is detected on Input 4 AND a named Google calendar event is still active, send an email
- If 'uptime' is > a given number of hours, perform a scheduled reboot
It's possible to cascade rules in the plug-board. It's also possible to code circular dependency which the scripter must be careful to avoid.
In this model we want to decouple the producers from the consumers and use a scripting intermediary, the plugboard, to perform an action. With N producers and N consumers we have N*N ways of plugging these together making a very flexible and expandable system. Consumers can also be thought of as SERVICE providers as they accept an xAP message and perform something useful.
Example of how the script engine is used to interpret a tweet command to turn on a relay.
- User tweets
- This message is picked up by the xap-twitter components and broadcast as an xAP alias command
- The script engine interprets this alias and re-transmits a 'RELAY on' command
- 'Relay on' is seen by the adapter and is processed as a control for the HAH hardware, thus turning the relay on
Examples of daemons that currently conform to this model are:
- xap-twitter
- PRODUCER: monitor a twitter feed and produces a CMD when an action is found
- CONSUMER: for sending a tweet.
- xap-googlecal
- PRODUCER: monitors Google calendar and pushes whatever command has been registered.
- CONSUMER: used as a service to create Google calendar events.
- xap-sms
- PRODUCER: emits an xAP message when an SMS is received
- CONSUMER: accept an xAP message to send an SMS.
- xap-currentcost
- PRODUCER: sends a messages every minute to report on temperature or power levels.
- xap-livebox:
- PRODUCER: generates xAP events on INPUT, 1WIRE or I2C changes
- CONSUMER: accept messages for the RF, RELAYS and I2C sub systems
- xap-pachube:
- CONSUMER: accepts data logging messages.
For simplistic messages, it can can also directly watch for events, manipulate the message, and send these to pachube without the plugboards involvement. This is configurable through the web interface.
If Pachube is used in the CONSUMER model the following would occur:
- xap-currentcost - xmit an xAPBSC.event telling of a power or temperature level.
- xap-plugboard - scripted to watch for any message where the source is CURRENTCOST parse and transform the event to a command and send out a PACHUBE logging message.
- xap-pachube - accepts logging messages and pushes to the pachube web service.
Standard libraries
The HAH is supplied with some standard Lua library offerings.
- luasocket - C - Sockets + modules for SMTP, FTP, HTTP, URL, MIME
- lrexlib - C - POSIX Regular expressions
- penlight - Lua - General purpose library of common patterns - I RECOMMEND YOU REFERENCE THIS
- lfs - C - Lua File System
For more about each of these libraries see their associated reference pages.
The Lua xAP library extension
Lua xAP is an extension library that provides support for the construction of xAP compliant applications. The functions exposed are those used by the xaplib2 library and are extensively used by the 'built in' HAH applications.
Basics
The first thing to note is that using the plugboard generally takes a little personal effort. There are sample scripts available to help get you going, but it is assumed that you can prepare a script, get it onto the HAH and run it. Being familiar with tools such as FTP & vi, together with a basic knowledge of some Unix commands will make your work with the plugboard a lot easier. Familiarity of how xAP works will help too. The extension library has been designed to do much of the 'hard work' for you. Best of all, once you get the basic knowledge under your belt, there is a whole world of possibility for rule based processing and actioning.
There are only two calls that are needed to build an xAP Lua application.
Return Type | Method | Description | |||
---|---|---|---|---|---|
void | xap.init(source,uid) deprecated | Initialise an xAP application, bind with a hub if found and send a heartbeat every minute source - a fully qualified 'vendorid.deviceid.instance' address. |
|||
void | xap.init{vendorid=,deviceid=,instance=,uid=} | Initialize an xAP application, bind with a hub if found and send a heartbeat every minute. vendorid - optional, default 'dbzoo' deviceid - optional, default to hostname or setting from /etc/xap.d/system.ini instance - mandatory, last component of 'vendorid.deviceid.instance' xAP address specification. uid - mandatory, a unique identifier |
|||
void | xap.process() | Enter the xAP processing loop | |||
void | xap.send(msg) | Send RAW data on the well known xAP UDP port 3639 | |||
String | xap.expandShortMsg(msg) | Inject/Replace xap-header information in a message | |||
void | xap.sendShort(msg) | A shortcut for send(expandShortMsg(msg)) | |||
String | xap.buildXapAdddress{vendorid=,deviceid=,instance=} | Build an xAP address target string. vendorid and deviceid will default following the same rules as xap.init{} | |||
String | xap.getDeviceID() | Returns the resolved deviceid, either hostname or /etc/xap.d/system.ini override |
In its most simplistic form a basic application would look like this; It wouldn't do much except send a heartbeat every minute but it's functional.
require("xap") xap.init{instance="simple",uid="FF00DD00"} xap.process()
When building plugboard applets these calls won't need to be made as the plugboard engine will take care of it. They will be needed if writing standalone Lua applications something you'll do whilst in development/testing mode. More on writing Applets later.
Frame
The Frame class provides a container for holding and processing a RAW xAP message. A Filter callback will pass a frame object as the first parameter.
Return Type | Method/Attribute | Description | |||
---|---|---|---|---|---|
Frame | xap.Frame(msg) | Constructor: msg is an xAP message | |||
int | <obj>:getType() | xap message type - xap.MSG_ORDINARY, xap.MSG_HBEAT or xap.MSG_UNKNOWN | |||
string | <obj>:getValue(section,key) | Extract a value from a message by section/key - you may also use the short form <obj>[section].key ie frame[“xap-header”].class | |||
boolean | <obj>:isValue(section,key,value) | Test if a section/key is a value | |||
string | tostring(f) | Reconstruct the xAP message from the supplied Frame (f) |
The following constants can be used when dealing with Frames whether they are global or local.
- xap.FILTER_ANY
- xap.FILTER_ABSENT
These are special keys that can be supplied as the key parameter to getValue() and isValue() to perform wild-carding and negation logic.
Sample usage
A Frame class is constructed like this.
msg = [[ xap-header { target=dbzoo.livebox.Controller:relay.1 class=xAPBSC.cmd } output.state.1 { id=* state=off }]] f = xap.Frame(msg) print(tostring(f)) print(f) -- Implicitly calls the tostring() method as above. -- These assertions are all TRUE, that is the message is *not* displayed. assert(f:getType() == xap.MSG_ORDINARY, "getType failure") assert(f:getValue("xap-header","class") == "xAPBSC.cmd") assert(f:isValue("output.state.1","state","off"), "state is not off") assert(f:isValue("output.state.1","state", xap.FILTER_ANY), "state is absent") assert(f:isValue("output.state.1","missing", xap.FILTER_ABSENT), "missing is present") print(f["xap-header"].class)
A quick note about empty sections: in this message the query section is empty.
xap-header { stuff... } query { }
And the follow tests will all assert to TRUE
assert(f:getValue("query","x") == nil) assert(f:getValue("query",nil) == xap.FILTER_ANY) assert(f:isValue("query",nil, xap.FILTER_ANY))
Filter
A filter allows an inbound xAP message to be acted upon based on the set of filter conditions applied. This is CORE to how a function is called to perform some work when the correct message is seen.
Return Type | Method | Description | |||
---|---|---|---|---|---|
Filter | xap.Filter(filter) | Constructor | |||
void | <obj>:destroy() | Destructor | |||
void | <obj>:add(section,key,value) | A condition matching an inbound xAP message, as many filters as needed can be added to the Filter object for matching. | |||
void | <obj>:delete(section,key,value) | Removing a matching condition from a filter | |||
void | <obj>:callback(function, userdata) | When all the filter conditions are met invoke the function, the function has a single parameter this is the FRAME matched by the filters. The function passed will receive as parameters (frame, userdata) |
Wild-card pattern matching The source and class keys of an xap-header may be used in wild-card filter matching using the add method. Read the xAP protocol definition to find out more about wild-card addressing.
For example:
f:add("xap-header", "source" "dbzoo.livebox.controller:relay.*") f:add("xap-header", "source" "dbzoo.livebox.*:relay.1") f:add("xap-header", "source" "dbzoo.livebox.controller:>") f:add("xap-header", "class" "xapbsc.*")
f:add("xap-header", "target" "dbzoo.livebox.controller:relay.*")
NOT ALLOWED
The constructor can also accept a set of filter patterns directly; these are equivalent.
filter = xap.Filter() filter:add("xap-header","class","xapbsc.event") filter:add("xap-header","source","dbzoo.livebox.>") filter:add("output.state","state","off") filter = xap.Filter {["xap-header"]={ class="xapbsc.event", source="dbzoo.livebox.>" }, ["output.state"]={ state="off" } }
Sample usage
Consider the following inbound xAP message for a heartbeat
xap-hbeat { v=12 hop=1 uid=FFAABB00 class=xap-hbeat.alive source=acme.meteor.home.line1 interval=60 }
We can build a simple standalone heartbeat snooper in Lua with the following code.
require("xap") function snoop(frame, pat) print(string.format(pat, frame["xap-hbeat"].source, frame["xap-hbeat"].class)) end xap.init{instance="test",uid="FF00CC00"} f = xap.Filter() f:add("xap-hbeat","source",xap.FILTER_ANY) f:callback(snoop, "Heartbeat - Source %s Class %s") xap.process()
Timer
A timer object allows a function to be called at a later date. Once created, it will count down from the interval supplied to the constructor until it reaches ZERO. At this point the function will be called and the timer will be reset. It will continue to call the function after every interval, until stopped.
Return Type | Method/Attribute | Description | |||
---|---|---|---|---|---|
Timer | xap.Timer(function, interval, userdata) | Constructor: Callback function and number of interval seconds before it's called. The function, when called, is passed the Timer object. | |||
Timer | <obj>:start(when) | Start the timer. when=true fire immediately, when=false fire on 1st expiry, default: false (can be omitted) | |||
Timer | <obj>:reset() | Reset the time to firing to the start interval | |||
Timer | <obj>:stop() | Stop a timer | |||
int | <obj>.interval | Attribute that is the interval of the timer, as this is mutable it's possible to dynamically adjust when the timer next fires. | |||
int | <obj>.ttl | How long before the timer fires | |||
int | <obj>.userdata | User Datum | |||
void | <obj>:delete() | Stop the timer and free the Timer resources |
Sample usage
This demonstration will fire the tick() function every 2 seconds until 10 seconds have elapsed at which point the timer will stop itself.
require "xap" elapsed = 0 function tick(self, userdata) elapsed = elapsed + self.interval print("Tick "..elapsed) if elapsed > 10 then print(userdata) self:stop() end end xap.Timer(tick, 2, "user data!"):start() xap.process()
Select
The select object allows a program to monitor multiple file descriptors, waiting until one of more of the file descriptors become “ready” for some class of I/O operation (e.g. input possible). A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g. read) without blocking.
Return Type | Method | Description | |||
---|---|---|---|---|---|
Select | xap.Select(function, fd) | Constructor: Function to callback when the file descriptor is ready. | |||
void | <obj>:delete() | Stop listening and clean-up |
Sample usage
As this is a complex class, although simple enough to use, the sample will be a little larger then normal to show fully how this works. The code is split into two pieces; the server and the client. The assert() wrapper just makes sure that the call worked … if not, the code stops at that point.
The SERVER
-- UDP to xAP gateway -- -- Demonstrates listening on a socket and -- rebroadcasting the incoming data as an xAP message require("xap") require("socket") function handleSocket() dgram, ip, port = udp:receivefrom() -- Read data from the UDP socket print("Received '"..dgram .. "' from "..ip..":"..port) -- Retransmit as an xAP payload. msg = string.format([[ xap-header { class=udp.data } body { ip=%s port=%s data=%s }]], ip, port, dgram) xap.sendShort(msg) end -- MAIN -- host = host or "localhost" port = port or 6666 if arg then host = arg[1] or host port = arg[2] or port end xap.init{instance="socket",uid="FF00CC00"} print("Binding to host '" ..host.. "' and port " ..port.. "...") host = socket.dns.toip(host) -- Convert host name to an IP udp = assert(socket.udp()) -- Create a UDP socket assert(udp:setsockname(host, port)) -- on IP:PORT -- Listen on the SOCKET file descriptor and call handleSocket when something is 'ready' xap.Select(handleSocket, udp) ip, port = udp:getsockname() -- Get IP:PORT from the UDP object print("Waiting for packets on " .. ip .. ":" .. port .. "...") xap.process() -- Enter the processing LOOP
the CLIENT
-- UDP to xAP client -- Send data to our gateway local socket = require("socket") host = host or "localhost" port = port or 6666 if arg then host = arg[1] or host port = arg[2] or port end host = socket.dns.toip(host) -- host name to IP udp = assert(socket.udp()) -- Create a UDP socket assert(udp:setpeername(host, port)) -- On IP:PORT print("Using remote host '" ..host.. "' and port " .. port .. "...") print("Enter text to sent to the UDP/xAP gateway, blank line will exit.") while 1 do line = io.read() if not line or line == "" then os.exit() end assert(udp:send(line)) -- Send keyboard input to UDP://host:port end
BSC
This class allows the easy construction of BSC endpoints. By default the following functionality is provided:
- responding to a xAPBSC.query with an xAPBSC.info message
- sending a xAPBSC.info message every 2 mins
- handling a xAPBSC.cmd and sending a xAPBSC.event
Return Type | Method | Description | |||
---|---|---|---|---|---|
Endpoint | Endpoint(table) | Constructor: a container to hold endpoints | |||
nil | <obj>:destroy() | Destructor: remove the endpoint from existence | |||
nil | <obj>:sendEvent() | Send xAPBSC.event for this endpoint | |||
nil | <obj>:sendInfo() | Send xAPBSC.info for this endpoint | |||
nil | <obj>:setText(string) | Populate the “text=” field and mark the “state=on” | |||
nil | <obj>:setDisplayText(string) | Populate the “displaytext=” field and mark the “state=on” | |||
nil | <obj>:setState(arg) | Populate the “state=” field. arg is a BSC state constant | |||
state | decodeState(string) | helper function to decode a string into a BSC state constant, the following strings are correctly handled: 1,0,on,off,yes,no,true,false,toggle |
The following constants can be used when dealing with the state:
- bsc.STATE_ON
- bsc.STATE_OFF
- bsc.STATE_TOGGLE
More about the Endpoint(table) constructor. The key for the table may be one of the following:
name : Append the endpoint name. Completes the triple; vendorid.deviceid.instance(:name)
instance: xAP address for the endpoint. Completes the triple; vendorid.deviceid.(instance)
direction: Has two possible values bsc.INPUT and bsc.OUTPUT. An OUTPUT endpoint will respond to xAPBSC.cmd events, INPUTS will not.
type: BSC supports 3 endpoint types: bsc.STREAM, bsc.LEVEL and bsc.BINARY
cmdCB(Endpoint): This function allows the program to take some action when the endpoint received a directed command. It takes one argument - the endpoint that just fired
infoEventCB(Endpoint, string): This function is called whenever a xAPBSC.info or xAPBSC.event is to be sent out. If you want the action to happen, return true otherwise return false. This allows programmatic control for setting up the displayText or to suppress an event if the change isn't out of range yet. It takes two arguments: the endpoint and the class of the event, being either: bsc.INFO_CLASS which has the value “xAPBSC.info” or bsc.EVENT_CLASS assigned to “xAPBSC.event”.
timeout Allow overriding of the default 120 second timeout for sending an xapBSC.info message.
ttl Time To Live. If a xapBSC.event event is not seen in this many seconds the current value is invalidated and future xapBSC.info message will report a question mark (?). A value of ZERO will disable this functionality. Default 0.
uid Override the UID (last 2 digits) for the created Enpoint. If this is not supplied an internal uid counter will be used. Any time this UID key is supplied it resets the internal counter so subsequent Endpoints that are created automatically start incrementing from this value.
Endpoint attributes that accessible from the Callbacks (all of the above plus)
state The BSC state of the endpoint nominally: on, off or ?
text The current TEXT value: applicable to STREAM and LEVEL types
displaytext An additional parameter that may be include in an EVENT of INFO message, configurable with a infoEventCB callback function.
uid UID key. If nothing is provide one will be automatically assigned based on the order of BSC endpoints created so far.
A standalone example save to a file an run with # lua <filename>
require "xap" require "xap.bsc" function lcdCmd(endpoint) print("LCD action: "..endpoint.text) end function relayCmd(endpoint) print("Relay action: " .. endpoint.name) end function relayInfoEvent(endpoint, clazz) endpoint.displaytext = "Relay is " .. endpoint.state return true -- Return TRUE if the INFO/EVENT should be emitted / FALSE otherwise end xap.init{instance="test",uid="FF0CC000"} -- Form 1 : Constructed using the xAP address from xap.init() -- creates -> dbzoo.livebox.test:lcd, dbzoo.livebox.test:relay.1 etc.. bsc.Endpoint{name="lcd", direction=bsc.OUTPUT, type=bsc.STREAM, cmdCB=lcdCmd} bsc.Endpoint{name="relay.1", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd, infoEventCB=relayInfoEvent} bsc.Endpoint{name="relay.2", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd} -- Form 2 : vendorid and deviceid default, remaining from instance is appended -- creates -> dbzoo.livebox.my:relay.3 bsc.Endpoint{instance="my:relay.3", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd} -- Form 3 : vendorid defaults, override deviceid. -- creates -> dbzoo.test.my:relay.4 bsc.Endpoint{deviceid="test", instance="my:relay.4", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd} -- Form 4 : full override all keys -- creates -> hello.test.my:relay.4 bsc.Endpoint{vendorid="hello", deviceid="test", instance="my:relay.4", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd} -- Form 5 : Shorter way of doing form 4. -- creates -> helloworld.test.my:relay.5 bsc.Endpoint{source="hithere.test.my:relay.5", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd} xap.process()
lcdCMD() and relayCmd() perform some work to generate a message on the LCD or to turn on a Relay - These are the CONTROL function that interface the xAP endpoint to a REAL action. RelayInfoEvent() builds up any additional information that needed to be sent.
A working example. This Applet creates an endpoint that shows how much memory Lua is currently using.
--[[ LUA Garbage memory usage monitor --]] module(...,package.seeall) require("xap") require("xap.bsc") info={ version="1.0", description="LUA GC Usage" } function update(t,e) e:setText( collectgarbage("count")*1024 ) e:sendEvent() end function init() -- Creates: dbzoo.livebox.Plugboard:gc local e = bsc.Endpoint{name="gc", direction=bsc.INPUT, type=bsc.STREAM} xap.Timer(update, 60, e):start() end
BSC Command helper functions
The following functions are used to send an xAPBSC.cmd message.
- bsc.sendState(target, state)
- bsc.sendText(target,text, state) – state is optional, defaults to 'on'
- bsc.sendLevel(target, level, state) – state is optional, defaults to 'on'
- bsc.send{} – takes table of elements. Target will always go into the xap-header, all others will be placed into the body of output.state.1
It's a RAW low level control function that is utilized by the higher level functions: sendText, sendState, sendLevel
STATE can be any of the following: on, off, true, false, 1, 0, toggle (Nominally you would use on and off)
Why are these new API here? Consider this fragment of code to send a BSC state control message.
xap.sendShort(string.format([[xap-header { target=dbzoo.livebox.Controller:ppe.%s class=xAPBSC.cmd } output.state.1 { id=* state=%s }]], ppeTarget, state)) end
This can now be written in a nice compact form.
bsc.sendState("dbzoo.livebox.Controller:ppe."..ppeTarget, state)
Standalone sample demonstrating their usage.
require("xap") require("xap.bsc") xap.init{instance="test",uid="FF00AA00"} bsc.sendState("dbzoo.livebox.Controller:relay.1", "on") bsc.sendState("dbzoo.livebox.Controller:relay.1", "toggle") bsc.sendState("dbzoo.livebox.Controller:relay.1", "off") bsc.send{target="dbzoo.livebox.Controller:relay.1",state="on"} bsc.sendText("dbzoo.livebox.Controller:relay.1","hello") bsc.send{target="dbzoo.livebox.Controller:relay.1",text="hello", state="on"} bsc.send{target="dbzoo.livebox.Controller:relay.1",text="hello", displaytext="hello world", state="on"}
Writing an Applet
Applets are small Lua scripts that are automatically invoked by the plugboard launcher. Their basic structure is
require("xap") module (...,package.seeall) info = {version="1.0", description="What this does"} function init() end
The init() function will be invoked when the applet is loaded by the plugboard.lua script.
To be automatically loaded they must be placed in the /etc/plugboard directory and have the suffix Applet.lua. Example file names:
- hbeatWatchdogApplet.lua
- bindRelaysApplet.lua
As an applet we can run the plugboard engine manually. Running it this way will also display other information not normally seen when running it as a service and is useful for debugging.
# xap-plugboard Loading /etc/plugboard/bindRelaysApplet.lua [ Binding relays 2,3,4 ] Loading /etc/plugboard/hbeatWatchdogApplet.lua [ Process Watchdog ] Running...
Writing xAP Lua applications
When developing a new Applet it may be convenient to start our writing a self contained Lua xAP application using the following structure which is only a small step away from its Applet form.
require("xap") info = {version="1.0", description="What this does"} function init() end xap.init{instance="example",uid="FF00DD00"} init() xap.process()
This allows you to work in another directory without constantly stopping and starting applets already in place when developing something new.
How do you run these code fragments? From the command prompt it's as easy as:
# lua standAloneProgram.lua
More samples
Whilst there is flash space available, some samples will be included with the firmware
cd /etc_ro_fs/plugboard/samples
There is a section of the forum, http://www.homeautomationhub.com/forum dedicated to example scripts, and another area for discussing scripting.