SAM - Simple Asynchronous Messaging

The aim of the SAM extension is to open up the messaging infrastructure to the PHP scripting environment by providing an API that allows the most common and simple messaging operations to be accomplished with the minimum of fuss. At the same time the API has to allow for the fact that many of these messaging infrastructures are connecting legacy systems and thus it must be possible for someone skilled in messaging technologies to access the low-level details of messages and the associated gorp.

Given the ever increasing popularity of PHP the SAM extension can make deploying solutions for a number of use cases very simple where today they are complex. The use cases range from the simple ability for a web page to post a request to an asynchronous back-end process for later processing, such as requesting a bank statement by mail, to complex AJAX-based solutions that exploit the pub/sub mechanisms available with some messaging systems.

So what is a Messaging and Queueing system?

Fundamentally a messaging and queueing system is based around a set of "queues" that conceptually hold the messages that are being transported between the various systems or applications.

The use of the queuing paradigm allows the sender and the receiver of the message to operate asynchronously. Once the sender has successfully posted the message to the queue it is free to carry on processing and the messaging and queuing software takes responsibility for the message. At some point the receiver will look for messages on the queue and pick up the waiting message.

As the sender and receiver work asynchronously the number of senders and receivers does not need to match. There may be many senders posting messages occasionally but only a few receivers continuously picking up new messages and acting on them.

In an online bank application there may be many thousands of concurrent sessions through which customers occasionally request a printed statement.The statement request is packaged as a message and is posted to a work queue. A small number of statement printing applications then pick up new requests from the queue and print the requested statement. The system may be load balanced by adjusting the number of processes picking up requests from the queue.

Frequently the sender of a message wants to receive some kind of reply. This may be just a status indication to do with the delivery or might be a real response such as details of an insurance quotation or bank account balance. In this case the receiver of the message could post the response on the same queue but usually it will be posted to a queue designated for the purpose. As there may be requests and responses belonging to several senders on the queue messaging systems will usually provide some way for the senders and receivers to identify messages. A common pattern is for the response message to have a "correlation id" property that is set to the message id of the request message. In this case the sender remembers the system assigned id of the message he sends and when requesting the response message specifies it as the correlation id. The receiver on receiving the message notes the id of the message and when posting the response sets the correlation id to the id of the original message. The messaging and queueing system then delivers the correct response to the sender.

All the patterns so far described are usually referred to as "point to point" messaging. In this case messages are explicitly posted to and received from queues. Another common usage of messaging and queuing systems is to support the "publish and subscribe" pattern. In this case the particular queues used are not visible but "topics" are used to identify types of information. This pattern is typically used when the same information is to be delivered to multiple endpoints simultaneously.

Consider a system that provides stock quote information. One or more processes or applications extract information on current stock prices from information sources then "publish" this information under relevant topics in the messaging system. We might have one application extracting information from the London stock exchange and publishing it into the topic "stocks/UK" and another tracking Wall Street and publishing into "stocks/US". Applications that are interested in receiving information "subscribe" to particular topics. To get all stock information they can subscribe to "stocks", to get UK stock information "stocks/UK, etc. When information is published the messaging system matches its current subscriptions against the topic and then passes the information on to those applications that have expressed an interest by subscribing. When an application is no longer interested it can unsubscribe.

Uses of Messaging with PHP

A simple example might be part of an online bank system where customers can ask for a printed bank statement or product leaflets to be mailed to them. The customer, and therefore the application, doesn't require any response other than confirmation that the request has been received. This maps nicely to the messaging pattern where a message is posted to a work queue.

An HTML form is displayed to the customer via the web browser and the customer selects the printed statement option. This results in an HTTP request flowing to the web server which invokes a PHP script. The script interprets the request, retrieves relevant customer details and posts a message to the statement processor queue. Once the message has been received by the messaging system there is nothing more for the sending application to do as responsibility for delivery has been taken by the messaging system. A backend process will receive the message from the queue and process it to print the required statement or distribute the requested leaflets.

An extension to this pattern is when data is sent to a queue and a response is received back, such as is the case perhaps in an insurance quotation system. As before information is input by the user and submitted to the webserver in an HTTP request triggering the execution of a PHP script which takes the input and formats a message which it sends to a processing queue. In this case the message is received by some form of processing capability, perhaps an automated process server or multiple services, that then returns a response containing a quotation.

Commonly the response message will be received by the same PHP script that sent the original request and this will then output the HTML page to the user. In some circumstances though the time it takes to process the request may be unduly long, due to system load, network delays, failures etc. and so having the PHP script wait indefinitely could lead to the system appearing unresponsive to the user. To allow for this the PHP script can opt to wait for only a limited time and then to abandon the receive request and return some kind of interim response to the user then to return and try the request again. In this case the 2nd and possibly subsequent receive requests may well come from a different PHP script. This pattern may be implemented using the AJAX programming style.

The architecture of SAM

The SAM extension is in fact a hybrid framework that is a combination of C code extensions and PHP code. A small C code extension can be built as either part of the PHP core or as a shareable library and provides a simple way of accessing the SAM capabilities. This extension calls the main logic and classes exposed via the API that are defined in PHP. For those that do not wish to have SAM defined as an extension it is possible to simply "require" the SAM PHP code (php_sam.php) after which the API works in exactly the same way.

The support for messaging protocols is provided by separate PHP scripts or other C code PHP extensions. Each implementation provides a small "factory" script which allows SAM to gain access to it. Within the implementation a connection class is defined which supports the SAM API as defined by the SAMConnection class. SAM then selects the appropriate support code based on the protocol requested in a "connect" request and then references the appropriate factory script to create an object of the correct type.

Protocol implementations can provide their underlying messaging support in whatever manner they wish by utilising client libraries or creating and sending packets via metwork APIs. Within the SAM package the MQTT support is coded entirely in PHP and writes to the PHP network interfaces.

In contrast, the XMS support uses a set of C libraries and runtime available freely from IBM (support pack IA94) and is delivered as a PHP extension. The use of this runtime allows SAM to access queues hosted on WebSphere MQSeries, WebSphere Application Server, WebSphere Message Broker, or WebSphere Event Broker systems over various protocols.

More information on XMS is available from the IBM website in particular the article Introducing XMS -- The IBM Message Service API provides a good overview and download links.

The SAM API

The primary aim of the SAM API is to make simple things simple while still allowing complex things to be achievable. To that end the API is kept to the bare minimum which also means the package can be built for PHP4 or PHP5. There are only 2 objects in the API; the SAMMessage object which represents a message to be sent or received, and the SAMConnection object which represents a connection to a Messaging and Queuing server.

SAMMessage

The body is fairly self explanatory and contains the actual message data to be sent or data received. The body may contain any data you wish but is broadly categorised as either UTF8 text or an arbitrary sequence of bytes depending on the setting of the SAM_TYPE header property.

The header property is itself a container for properties. The message header may contain a range of SAM defined message properties, underlying messaging system properties, or user defined properties. Each property has a name and its value is a text string.

    // Create a message
    $msg = new SAMMessage();

    // set the type header property...
    $msg->header->SAM_TYPE = SAM_TEXT;

    // add the message body data
    $msg->body = 'This is a simple text message';

SAMConnection

The following is an example of sending a very simple text message using SAM:

    // Create a connection to a broker
    $conn = new SAMConnection();
    if ($conn->connect(SAM_WMQ, array (SAM_BROKER => 'MyBroker'))) {

        //Create a message
        $msg = new SAMMessage('Simple text message');

        //Send it
        if (!$conn->send('queue://send/test', $msg)) {
            // Oh dear the send failed!
        }


    } else {
      // Oh dear we didn’t successfully connect!
    }

In this example first of all we create a SAMConnection object. We then use the "connect" method on our new SAMConnection object to actually connect to a messaging server.

The first parameter on the connect call defines the protocol to use when talking to the server. The choice of protocol used is largely defined by the type of server being used and in this case the built-in constant SAM_WMQ indicates we will use the IBM WebSphere MQSeries protocol. The second parameter to the call is an array of optional name/value pair parameters which in this case contains the single parameter "SAM_BROKER" which is set to the name of the queue manager to be used on the server, in this case "MyBroker". Other parameters such as the server host name and target port number are defaulted by SAM to common values.

If the connect request fails the method will return a value of FALSE in which case the application will not be able to send or receive messages. If all is well we can then create a new SAMMessage and send it using the "send" method on the SAMConnection object. In this case the parameters are the identity of the queue to send the message to and the message itself. If the send fails then once again we will be returned a value of FALSE.

This example can be extended to show the receipt of a response:

    // Create a connection to a broker
    $conn = new SAMConnection();
    if ($conn->connect(SAM_WMQ, array (SAM_BROKER => 'MyBroker'))) {

        //Create a message
        $msg = new SAMMessage('Simple text message');
        // set a reply queue identity in the message header
        $msg->header->SAM_REPLY_TO = 'queue://receive/test';
        //Send it and capture the correlation id that is returned…
        $correlid = $conn->send('queue://send/test', $msg);
        if (!$correlid)) {
            // Oh dear the send failed!
        } else {
            // receive the answer but only wait for 5 seconds
            $answer = $conn->receive('queue://receive/test',
                            array(SAM_WAIT => 5000, SAM_SELECTOR => $correlId) );
            if (!$answer) {
               // Oh dear the receive failed!
            }
        }

    } else {
      // Oh dear we didn’t successfully connect!
    }

The first differenc in this example is that after creating our text message we set a header property, SAM_REPLY_TO, to indicate the identity of the queue that we will look on for our response. The receiver of the message is expected to read this property and ensure they deliver the response to the right queue.

Also in the case of sending a message and receiving a response we need some way of identifying the response meant for us rather than another script as the response queue may have a number of responses on it. To do this we capture the return value of the send method call as this is the identity that the messaging system gave the message we sent. We can use this as a correlation value by having the process that replies set the correlation id of the reply message to the id of the message it received.

The reply message is received by calling the "receive" method of the SAMConnection object and again naming the queue we wish to read along with an array of optional parameters. The first of the optional parameters simply states that we are only prepared to wait 5000 milliseconds (5 seconds) for the response. If no response is received in this time the receive call will return with an error and no message.

The second optional parameter (SAM_SELECT) indicates we wish to select a particular message from the queue and that the message we are looking for is the message with a correlation id equal to the value in the variable $correlId.

As with the other method os SAMConnection we have used the receive method will return a value of FALSE if it is unable to return a message for any reason.

Summary and further pointers

The source is being made available through the PECL community site (pecl.php.net) under the terms of version 2.0 of the Apache license so that PHP users can build the extension for any version of PHP they wish. Visit the SAM webpage on PECL for the latest available version.

Comments and feedback are welcomed.