Database-ZeroMQ Publishing

Building With ZeroMQ DB Plugin

The zeromq plugin is used in place of a database plugin (i.e. MCA parameter db can be a odbc, postgres, print, or zeromq) and will publish data to a ZeroMQ subscriber with specific subscriber keys and the data as a generic JSON object. The only requirement to use the plugin is that the zeromq libraries (version 4.0.4 or newer) are installed. To build Sensys with this functionality you must also install the developers package as well. For example on CentOS 7.x:

% yum install epel-release
% yum install zeromq zeromq-devel

These are the only external requirements to build and use this plugin and only the zeromq package is required to run Sensys.

NOTE: the zeromq plugin has it's priorty lower than the print plugin. You must explicitly ask for the zeromq plugin.

db=zeromq # in openmpi-mca-params.conf file.

or on the command line

--omca db zeromq

Parameters for the ZeroMQ DB Plugin

The ZeroMQ Publisher/Subscriber model makes the Publisher (Sensys) the "server" in networking terms. In order to support multiple configurations a single MCA parameter is required.

The integer parameter is named db_zeromq_bind_port and it will default to port 37001 if not specified using one of the MCA parameter setting methods. There is no significance to this default port choice. The ZeroMQ publisher will bind to all active interfaces on this port.

Using the ZeroMQ DB Plugin to Receive Data

A ZeroMQ subscriber can be built in most high level languages and the C API has MANY language bindings. Some good references are:

There are currently three types of topics published from Sensys.

Topic Name Type of Information
monitoring_inventory Inventory Data
monitoring_data Raw Environmental Data
monitoring_event Event Data

Example of JSON for the "monitoring_data" Topic

{
    "data":
    [
        {"key":"Core1","value":98,"units":"C"},
        {"key":"Core43","value":70,"units":"C"},
        ...
    ]
}

For data that has no units the JSON name-value object for the units may be excluded. This is a format example not a content example, actual data JSON object will be larger and labels will differ. Also preceeding real data there is usually some metadata about the origin of the data but this will still be name-value pair objects in the list. The top level object's name value will always be "data".

Python Subscriber Example

#
#   Simple Subscriber
#   Enter your IP, Port and Topic and it spits out string data.
#   Note Topic must be in ASCII
#

import zmq

#  Socket to talk to server
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://{ip}:{port}")

socket.setsockopt_string(zmq.SUBSCRIBE, "monitoring_data")  # All Topics

while 1:
    string = socket.recv_string()
    print("Recieved %s" % (string))


C# Snippit using NetMQ as the Binding

using NetMQ;
using Net.MQ.Sockets;
// ...
    public void Subscribe(string ip, int port)
    {
        SubscriberSocket socket = new SubscriberSocket();
        socket.Connect($"tcp://{ip}:{port}");
        socket.SubscribeToAnyTopic(); // Could be pecific topics

        Task.Run(()=> {
            while(true)
                try {
                    string topic = socket_.ReceiveFrameString(); // Subscription topic
                    string msg = socket_.ReceiveFrameString(); // Message JSON data
                    IncomingMessage(topic, msg); // Call message handler
                } catch {
                    break; // On exception stop threaded loop.
                }
        });
    }

    public void IncomingMessage(string topic, string message)
    {
        System.Console.WriteLine($"Topic='{topic}'; Message='{message}'");
    }
//...