Contents
1. Introduction
The local Message Bus (Mbus) is a light-weight message-oriented coordination protocol for group communication between application components. The Mbus provides automatic location of communication peers, subject based addressing, reliable message transfer and different types of communication schemes. The protocol is layered on top of IP multicast and is specified for IPv4 and IPv6. The IP multicast scope is limited to link-local multicast. The transport protocol mechanisms ware defined in RFC 3259 (http://www.ietf.org/rfc/rfc3259.txt).
pyMbus is python implementation of the Mbus transport layer and parts of the Mbus Guidelines as defined in http://www.mbus.org/drafts/draft-ietf-mmusic-mbus-guidelines-00.txt. The package is completely implemented in python and does require any system dependent modules (hopefully ;-)). It is meant to be used in event based single-thread programs as it is based on [pyNotifier].
The following description implies that RFC 3259 and the Mbus Guidelines have been read. Hopefully it will not be necessary, but definitely helpful.
2. API
The following explanation of the pyMbus API begins with the basics used to create Mbus messages by showing off the usage of the Mbus basic types and the MCommand, MHeader and MMessage classes. Followed by the description of the MEntity class representing an Mbus entity. The classes implementing the Mbus Guidelines are explained after it.
2.1. Mbus Data Types
Messages send via Mbus contain a list of Mbus commands that may have a list of arguments. These arguments can be of any of the pre-defined basic types:
Integer (int): Positive or negative whole-numbered values
Float (float): Positive or negative decimal values
String (str): UTF8 encoded characters
Symbol (MSymbol): UTF8 encoded strings with a limited set of allowed characters. Primarly used for constant value sets like enums
Data (MData): Binary data.
List (MList): a container storing any elements of the basic Mbus types including lists.
Instead of reimplementing all of these types pyMbus tries to reuse as many of the python basic types as possible. Meaning the Mbus types Integer, Float and String are represented by the corresponding basic python types. The Symbol type is based on the python string and ensures that only the allowed characters are used within a Mbus symbol. The Data type is a simple container for any binary data. The container automatically encodes the data before sending to the Mbus session and decodes it after reception. The Mbus list ist based on the python list type and is therefore accessed like such a list (the python sequence type may also be used for an Mbus list).
The following examples depict the usage of the previously described types.
1 import mbus
2
3 i = 5
4 f = 6.7
5 s = "hello world"
6 sym = mbus.MSymbol( 'HELLO_WORLD' )
7 data = mbus.MData( '\007\034\043' )
8 lst = mbus.MList( [ i, 5, f, [ s, sym, data ] ] )
9
10 print 'MSymbol.__str__:', sym
11 print 'MData.__str__:', data
12 print 'MList.__str__:', lst
The output of this example should look similar to the following
MSymbol.__str__: HELLO_WORLD
MData.__str__: <Bxwj>
MList.__str__: (5 5 6.700000 ("hello world" HELLO_WORLD <Bxwj> ) )
As the previous examples show the usage of the Mbus types is straight forward and integrates just wonderful with the basic python types.
2.2. Mbus Messages
The information transported via Mbus are encapsulated in so called Mbus messages. These messages consist of two components: a header and a list of commands. The following explains the usage of the three classes MMessage, MHeader and MCommand to contract and to read Mbus message.
2.2.1. MAddress
Each Mbus entity, meaning each component of application registering at an Mbus session, is has a unique address that identifies it. Such an address is composed of a list of key-value pairs.
1 import mbus
2
3 addr1 = mbus.MAddress( "app:test1 module:address1" )
4 addr2 = mbus.MAddress()
5 addr2[ 'app' ] = 'test2'
6 addr2[ 'module' ] = 'address2'
7
8 print 'Address 1:', addr1
9 print 'Address 2:', addr2
The output of this example is definitely:
Address 1: app:test1 module:address1 Address 2: app:test2 module:address2
2.2.2. MHeader
The Header of a Mbus message contains information like timestamp, sequence number, message type and source and destination address. Most of this information is controlled by the pyMbus package so there is need for a detailed description at this point. If necessary the field of the header are accessable through the MMessage object. The following gives a simple example.
1 import mbus
2
3 header = mbus.MHeader( mbus.MAddress( "app:test module:header-test1" ) )
4 header.source = mbus.MAddress( "app:test module:header-test2" )
5 header.reliable = True
6 print "Header:", header
The following should be seen as the ouput of the above example:
Header: mbus/1.0 3 1128277610 R (app:test module:header-test2) (app:test module:header-test1) ()
2.2.3. MCommand
A command is that what is used to transport information from one Mbus entity to another. It consists of a name and a list of arguments. A Mbus command is managed by the MCommand class as depict in the following example.
1 import mbus
2
3 cmd = mbus.MCommand( 'desktop.configuration.notify' )
4 cmd.args.append( mbus.MSymbol( 'gex/panel/plugin/gxnet/interfaces' ) )
5 cmd.args.append( [ 'eth0', 'eth1', 'tun0' ] )
6 print 'Command:', cmd
The following should match the output of the example above:
Command: desktop.configuration.notify (gex/panel/plugin/gxnet/interfaces ( "eth0" "eth1" "tun0" ) )
2.2.4. MMessage
This class holds a Mbus message and is the most important object for the communication with other Mbus entities. It manages the header and a list of commands.
1 import mbus
2
3 msg = mbus.MMessage( "app:test module:message-test1", hashkey = 'Y2eAxOs7XWLxrby6' )
4 msg.header.source = mbus.MAddress( "app:test module:message-test2" )
5 cmd = mbus.MCommand( 'desktop.configuration.notify' )
6 cmd.args.append( mbus.MSymbol( 'gex/panel/plugin/gxnet/interfaces' ) )
7 cmd.args.append( [ 'eth0', 'eth1', 'tun0' ] )
8 msg.payload.append( cmd )
9
10 print 'Message:\n%s' % msg
This should result in something like the following:
Message:
/ZKBOYGr0HjsWTgU
mbus/1.0 1 1128284981 U (app:test module:message-test2) (app:test module:message-test1) ()
desktop.configuration.notify (gex/panel/plugin/gxnet/interfaces ("eth0" "eth1" "tun0" ) )
2.3. Mbus Entity
This class represents an Mbus entity. It provides several services for sending and receiving Mbus messages and for receiving basic Mbus signal like "new-entity", "lost-entity", "unknown-messages" and "error". What is meant by all this is explained in the following by setting up an Mbus entity.
As always the mbus package must be imported and an object is created.
1 import
2 mbus
3
4 entity = mbus.Entity( "app:test module:entity" )
This new entity has an address and nothing more. To get informed when a new entity has entered the Mbus session the following has to be done:
1 def _new_entity( addr ):
2 print 'new entity', addr
3
4 entity.signal_connect( 'new-entity', _new_entity )
Now the function _new_entity is invoked when a new entity joins the Mbus session. To know when an entity leaves the Mbus session the following was to be done:
1 def _lost_entity( addr ):
2 print 'lost entity', addr
3
4 entity.signal_connect( 'lost-entity', _lost_entity )
The class MEntity provides another two signals: "error" and "unknown-message". The "error" signal is emitted when there are problems on the Mbus transport layer. The "unknown-message" signal informs about the reception of an unknown Mbus message.
1 def _unknown_message( msg ):
2 print 'unknown message', msg
3
4 def _transport_error( error ):
5 print 'transport error', error
6
7 entity.signal_connect( 'unknown-message', _unknown_message )
8 entity.signal_connect( 'error', _transport_error )
For now all Mbus messages received by the Mbus entity are unknown. To invoke a specific function when an Mbus message containing a known command is received the following code does the work:
1 def _desktop_conf_notify( msg ):
2 print 'received', msg.current_command
3
4 entity.addCallback( "desktop.configuration.notify", _desktop_conf_notify )
When this entity receives an Mbus message containing the command with the name 'desktop.comfiguration.notify' the function _desktop_conf_notify is invoked. As an argument the Mbus message with the command is given to the function.
With the above part of the MEntity API the entity is able to register for specific events in the Mbus session and to receive Mbus commands. To be able to send messages the MEntity class provides two methods. The send method can be used to send an previously created MMessage object for a command defined by destination address, command name and arguments. The following example shows both variantes:
1 # send a Mbus message object
2 msg = mbus.Message( mbus.MAddress( "app:test module:doc" ) )
3 cmd = mbus.MCommand( "desktop.configuration.notify" )
4 cmd.args.append( mbus.MCommand( "desktop.configuration.notify" ) )
5 cmd.args.append( [ "eth0", "eth1", "tun0" ] )
6 msg.payload.append( cmd )
7 entity.send( msg )
8
9 # send a command
10 entity.send( mbus.MAddress( "app:test module:doc" ), "desktop.configuration.notify",
11 [ mbus.Symbol( 'gex.panel.plugin.gxnet.interfaces' ), [ "eth0", "eth1", "tun0" ] ] )
In this case both variantes do the same. The other method to send messages to the Mbus session is called sendReliable. The only difference to the previously described method send is that this method sets the message type to reliable before sending it. A reliable message is automatically confirmed by the receiving Mbus entity an the Mbus transport layer. This ensures the reception of the message.
2.4. Mbus Guidelines
The Mbus guidelines internet draft describes services and add-ons on top of the Mbus transport layer, e.g. remote procedure calls (RPCs), events, properties and optional arguments. The pyMbus implementation currently does just provide a small subset of these extensions. The primarily feature supported by pyMbus are the remote procedure calls. This mechanism provides the possibility to invoke a function (Mbus command) on an arbitrary Mbus entity and receiving a complex return value. The return value is not just a boolean value, but any combination of Mbus basic types addtional to an enum value specifying the success of the operation. The following will explain the usage of the Guides class.
An Guides object inherits from the MEntity class and is created as depict in the following code snippet:
1 import mbus
2
3 entity = mbus.Guides( mbus.MAddress( "app:test module:guides" ) )
Such a Guides object provides the same methods for registering for signals and receiving and sending normal Mbus messages as an MEntity. Meaning to register a callback for a normal Mbus command the addCallback function is used. To register for an Mbus RPC command an additional member function addRPCCallback is provided that is used like shown in the following example.
1 import mbus
2
3 def _mbus_rpc_command( rpc ):
4 ret = mbus.RPCReturn( rpc )
5 print 'received an RPC', rpc
6 return ret
7
8 entity.addRPCCallback( 'desktop.configuration.set', _mbus_rpc_command )
If the entity receives the RPC command desktop.configuration.set the function _mbus_rpc_command is invoked. As shown in the example the function should return an object of type mbus.RPCReturn. This object describes the return value send back to the RPC sender. Together with a list of return parameters a few status information are stored in a mbus.RPCReturn. These status information describe the success of the command with three values:
- success: a boolean value set to True if the command was successful otherwise False
- result: an Mbus symbol given possiblity to distinguish different errors
- description: a human understandable description of the result of the function call
The following example demonstration the use of the these status information:
1 import mbus
2
3 def _mbus_rpc_command( rpc ):
4 ret = mbus.RPCReturn( rpc )
5
6 if rpc.arguments[ 0 ] == 'GOOD':
7 # return parameters
8 ret.arguments = [ 'FINE' ]
9 # status information
10 ret.success = True
11 ret.result = 'SUCCESS'
12 ret.description = 'No problems, everyhthing worked fine'
13 else:
14 # return parameters
15 ret.arguments = []
16 # status information
17 ret.success = False
18 ret.result = 'ERROR_NO_GOOD'
19 ret.description = 'Oops, something very bad has happened'
20 return ret
21
22 entity.addRPCCallback( 'desktop.configuration.set', _mbus_rpc_command )
This mechanism provides the possibility to send a detailed status information and arbitrary list of Mbus objects back to the sender of the RPC command. If the receiver of the RPC command may need a little time to calculate the result or needs to request information from another process it should return None from the callback function and store the mbus.RPCReturn object for later use. By default the application of the receiver has two seconds to send the RPC return to the sender. This timeout can be modified by setting the rpcTimeout property of the Guides class (it is specified in milliseconds). To send the mbus.RPCReturn later on, meaning not using the return value of the RPC callback function, the function sendRPCReturn is provided by the Guides class.
To not only be able to receive RPC commands the Guides class provides the function sendRPC. This method provides the possibility to send an RPC command to an arbitrary group of Mbus entities.
1 import mbus
2
3 def _rpc_config_set( ret ):
4 if ret.success:
5 print 'variable is set to the new value'
6
7 return True
8
9 entity.sendRPC( mbus.MAddress( "app:server module:rpc" ), 'desktop.configuration.set',
10 [ '/desktop/gex/panel/type', 'CORNER' ], _rpc_config_set )
The first argument specifies the destination Mbus entities. If the given address matches a single Mbus entity the RPC command is send as a reliable message. The second argument is the name of the RPC command, which is followed by the arguments passed to the RPC command. The last argument is the function to invoke, when the RPC return is received.
2.5. Examples
The documentation contains enough examples, doesn't it?
2.6. Encryption
pyOpenSSL or python-crypto?
3. Configuration
pyMbus reads its configuration from a file that is either located at ~/.mbus or specified by the envrionment variable MBUS. The file is structured like a INI file with just one single group called Mbus. The following documented examples provides the most important keys required for a proper configuration.
[MBUS]
# multicast address for the Mbus communication
ADDRESS=224.255.222.240
# port for the Mbus communication
PORT=47000
# Encryption method and key. This entry is set automatically by
# mbusauth, contained in the mbus application package.
ENCRYPTIONKEY=(NOENCR, )
# Hashn method and key. This entry is set automatically by
# mbusauth, contained in the mbus application package.
HASHKEY=(HMAC-MD5-96, Y2eAxOs7XWLxrby6)
# Scope of Mbus messages (i.e. a symbolic value for TTL of sent
# datagrams). Must be one of {HOSTLOCAL,LINKLOCAL}, aka TTL=0 or
# TTL=1. One some systems, TTL=0 seems not to work as it should
# (WinNT).
#SCOPE=HOSTLOCAL
SCOPE=LINKLOCAL
# If set to yes, an optional Mbus optimization may be used: If there
# is only a single known recipient of a message, this may be sent via
# unicast to minimize traffic on the shared Mbus. Set to 'no' if you
# want to see every message with Mbus sniffing tools.
SEND_UNICAST=no
Additionally to the configuration file it might be necessary to check the multicast routing. To make Mbus work it is required to have a route for multicast traffic. This could be a done by a default route or a specific route for multicast groups.
route add -net 224.0.0.0 netmask 240.0.0.0 <device>
4. Requirements
pyNotifier-0.3.x for pyMbus-0.8.x pyNotifier-0.4.x for pyMbus-0.9.x
5. Download
The latest release (and older version) can be found at Downloads/pymbus as tar archive and debian packages or from the svn repository at svn://rerun.bitkipper.net/pyMbus/(trunk|branches|tags).
6. Installation
If you have downloaded a tar archive the installation follows the normal steps for a python package
# tar xzvf pyMbus-0.8.7.tar.gz # cd pyMbus-0.8.7 # python setup.py install
If you would like to use the bleeding edge from the svn repository you have to do the following
# svn co svn://rerun.bitkipper.net/pyMbus/trunk pyMbus-trunk # cd pyNotifier-trunk # python setup.py install
Any comments or questions can be send to crunchy@bitkipper.net