SAM - Simple Asynchronous Messaging
Many enterprises both large and small rely on asynchronous messaging
and queueing infrastructures to connect the various pieces of their
business systems. For many the messaging infrastructure is a key
element of their systems and will continue to be so for the foreseeable
future. Currently connecting PHP scripts to this infrastructure is
not simple or convenient.
The aim of the SAM extension is to open up the messaging infrastructure
to the PHP scripting environment by providing an API that allows the
most common and simple messaging operations to be accomplished with
the minimum of fuss. At the same time the API has to allow for the fact
that many of these messaging infrastructures are connecting legacy systems
and thus it must be possible for someone skilled in messaging technologies
to access the low-level details of messages and the associated gorp.
Given the ever increasing popularity of PHP the SAM extension can make
deploying solutions for a number of use cases very simple where today they
are complex. The use cases range from the simple ability for a web page to
post a request to an asynchronous back-end process for later processing,
such as requesting a bank statement by mail, to complex AJAX-based solutions
that exploit the pub/sub mechanisms available with some messaging systems.
So what is a Messaging and Queueing system?
Messaging and queueing systems came about as enterprises identified that
their IT systems were becoming distributed but also needed to be connected.
Prior to the wide availability of messaging and queueing systems connecting
different IT systems together required the application programmers to take
responsibility for making and maintaining the network connections and recovering
from the inevitable failures. Messaging and queueing systems offered relief by
removing the details of the network from the applications and taking responsibility
for ensuring the delivery of messages between them.
Fundamentally a messaging and queueing system is based around a set of "queues" that
conceptually hold the messages that are being transported between the various
systems or applications.
In reality a "queue" probably does not exist in any one place but may be thought of as
existing in the network that interconnects the various systems. Applications post
messages to and retrieve messages from queues through programming interfaces and the
messaging and queuing software makes the details of the transportation of the message between the
endpoints invisible.
The use of the queuing paradigm allows the sender and the receiver of the message to operate
asynchronously. Once the sender has successfully posted the message to the queue it is free
to carry on processing and the messaging and queuing software takes responsibility for the message.
At some point the receiver will look for messages on the queue and pick up the waiting message.
As the sender and receiver work asynchronously the number of senders and receivers does not need to match.
There may be many senders posting messages occasionally but only a few receivers continuously picking
up new messages and acting on them.
In an online bank application there may be many thousands of concurrent
sessions through which customers occasionally request a printed statement.The statement request is
packaged as a message and is posted to a work queue. A small number of statement printing applications then
pick up new requests from the queue and print the requested statement. The system may be load balanced
by adjusting the number of processes picking up requests from the queue.
Frequently the sender of a message wants to receive some kind of reply. This may be just a status indication
to do with the delivery or might be a real response such as details of an insurance quotation or bank
account balance. In this case the receiver of the message could post the response on the same queue but
usually it will be posted to a queue designated for the purpose. As there may be requests and responses
belonging to several senders on the queue messaging systems will usually provide some way for the
senders and receivers to identify messages. A common pattern is for the response message to have a
"correlation id" property that is set to the message id of the request message. In this case the sender
remembers the system assigned id of the message he sends and when requesting the response message specifies
it as the correlation id. The receiver on receiving the message notes the id of the message and when posting
the response sets the correlation id to the id of the original message. The messaging and queueing system
then delivers the correct response to the sender.
All the patterns so far described are usually referred to as "point to point" messaging. In this case messages
are explicitly posted to and received from queues. Another common usage of messaging and queuing systems is
to support the "publish and subscribe" pattern. In this case the particular queues used are not visible but
"topics" are used to identify types of information. This pattern is typically used when the same information
is to be delivered to multiple endpoints simultaneously.
Consider a system that provides stock quote information. One or more processes or applications extract information
on current stock prices from information sources then "publish" this information under relevant topics in the
messaging system. We might have one application extracting information from the London stock exchange and publishing it
into the topic "stocks/UK" and another tracking Wall Street and publishing into "stocks/US". Applications that are
interested in receiving information "subscribe" to particular topics. To get all stock information they can subscribe
to "stocks", to get UK stock information "stocks/UK, etc. When information is published the messaging system
matches its current subscriptions against the topic and then passes the information on to those applications
that have expressed an interest by subscribing. When an application is no longer interested it can unsubscribe.
Uses of Messaging with PHP
There are many potential uses for asynchronous messaging with PHP but in this short article we will only highlight
a couple of potential uses linking web pages with dynamic content to systems using messaging and queuing. These are
likely the most common uses.
A simple example might be part of an online bank system where customers
can ask for a printed bank statement or product leaflets to be mailed to them. The customer, and therefore the
application, doesn't require any response other than confirmation that the request has been received. This maps
nicely to the messaging pattern where a message is posted to a work queue.
An HTML form is displayed to the customer via the web browser and the customer selects the printed statement
option. This results in an HTTP request flowing to the web server which invokes a PHP script. The script interprets
the request, retrieves relevant customer details and posts a message to the statement processor queue. Once the
message has been received by the messaging system there is nothing more for the sending application to do as
responsibility for delivery has been taken by the messaging system. A backend process will receive the message
from the queue and process it to print the required statement or distribute the requested leaflets.
An extension to this pattern is when data is sent to a queue and a response is received back, such as is the
case perhaps in an insurance quotation system. As before information is input by the user and submitted to
the webserver in an HTTP request triggering the execution of a PHP script which takes the input and formats
a message which it sends to a processing queue. In this case the message is received by some form of
processing capability, perhaps an automated process server or multiple services, that then returns a
response containing a quotation.
Commonly the response message will be received by the same PHP script that sent the original request and
this will then output the HTML page to the user. In some circumstances though the time it takes to
process the request may be unduly long, due to system load, network delays, failures etc. and so having
the PHP script wait indefinitely could lead to the system appearing unresponsive to the user. To allow for
this the PHP script can opt to wait for only a limited time and then to abandon the receive request
and return some kind of interim response to the user then to return and try the request again. In this case
the 2nd and possibly subsequent receive requests may well come from a different PHP script. This pattern
may be implemented using the AJAX programming style.
The architecture of SAM
The SAM extension is in fact a hybrid framework that is a combination of C code extensions and PHP code.
A small C code extension can be built as either part of the PHP core or as a shareable library and provides
a simple way of accessing the SAM capabilities. This extension calls the main logic and classes exposed via
the API that are defined in PHP. For those that do not wish to have SAM defined as an extension it is
possible to simply "require" the SAM PHP code (php_sam.php) after which the API works in exactly the same way.
The support for messaging protocols is provided by separate PHP scripts or other C code PHP extensions. Each
implementation provides a small "factory" script which allows SAM to gain access to it. Within the implementation
a connection class is defined which supports the SAM API as defined by the SAMConnection class. SAM then
selects the appropriate support code based on the protocol requested in a "connect" request and then references
the appropriate factory script to create an object of the correct type.
Protocol implementations can provide their underlying messaging support in whatever manner they wish by utilising
client libraries or creating and sending packets via metwork APIs. Within the SAM package the MQTT support
is coded entirely in PHP and writes to the PHP network interfaces.
In contrast, the XMS support uses a set of C libraries and runtime available freely from IBM (support pack IA94)
and is delivered as a PHP extension. The use of this runtime allows SAM to access queues hosted on WebSphere MQSeries, WebSphere Application Server,
WebSphere Message Broker, or WebSphere Event Broker systems over various protocols.
More information on XMS is available from the IBM website in particular the article
Introducing XMS -- The IBM Message Service API
provides a good overview and download links.
The SAM API
The primary aim of the SAM API is to make simple things simple while still allowing complex things to
be achievable. To that end the API is kept to the bare minimum which also means the package can be
built for PHP4 or PHP5. There are only 2 objects in the API; the SAMMessage object which represents
a message to be sent or received, and the SAMConnection object which represents a connection to a
Messaging and Queuing server.
SAMMessage
The SAMMessage object has no methods and is simply used as a container for message data. There are
2 main properties in the object; the body, and the header.
The body is fairly self explanatory and contains the actual message data to be sent or data received.
The body may contain any data you wish but is broadly categorised as either UTF8 text or an
arbitrary sequence of bytes depending on the setting of the SAM_TYPE header property.
The header property is itself a container for properties. The message header may contain a range
of SAM defined message properties, underlying messaging system properties, or user defined
properties. Each property has a name and its value is a text string.
// Create a message
$msg = new SAMMessage();
// set the type header property...
$msg->header->SAM_TYPE = SAM_TEXT;
// add the message body data
$msg->body = 'This is a simple text message';
|
SAMConnection
The SAMConnection object represents the connection to a messaging server of some kind and it is
this object which has all the useful methods for sending and receiving data.
The following is an example of sending a very simple text message using SAM:
// Create a connection to a broker
$conn = new SAMConnection();
if ($conn->connect(SAM_WMQ, array (SAM_BROKER => 'MyBroker'))) {
//Create a message
$msg = new SAMMessage('Simple text message');
//Send it
if (!$conn->send('queue://send/test', $msg)) {
// Oh dear the send failed!
}
} else {
// Oh dear we didn’t successfully connect!
}
|
In this example first of all we create a SAMConnection object. We then use the "connect" method on
our new SAMConnection object to actually connect to a messaging server.
The first parameter on the connect call defines the protocol to use when talking to the server.
The choice of protocol used is largely defined by the type of server being used and in this case
the built-in constant SAM_WMQ indicates we will use the IBM WebSphere MQSeries protocol. The
second parameter to the call is an array of optional name/value pair parameters which in this case
contains the single parameter "SAM_BROKER" which is set to the name of the queue manager to be used
on the server, in this case "MyBroker". Other parameters such as the server host name and target port
number are defaulted by SAM to common values.
If the connect request fails the method will return a value of FALSE in which case the application
will not be able to send or receive messages. If all is well we can then create a new SAMMessage
and send it using the "send" method on the SAMConnection object. In this case the parameters are
the identity of the queue to send the message to and the message itself. If the send fails then
once again we will be returned a value of FALSE.
This example can be extended to show the receipt of a response:
// Create a connection to a broker
$conn = new SAMConnection();
if ($conn->connect(SAM_WMQ, array (SAM_BROKER => 'MyBroker'))) {
//Create a message
$msg = new SAMMessage('Simple text message');
// set a reply queue identity in the message header
$msg->header->SAM_REPLY_TO = 'queue://receive/test';
//Send it and capture the correlation id that is returned…
$correlid = $conn->send('queue://send/test', $msg);
if (!$correlid)) {
// Oh dear the send failed!
} else {
// receive the answer but only wait for 5 seconds
$answer = $conn->receive('queue://receive/test',
array(SAM_WAIT => 5000, SAM_SELECTOR => $correlId) );
if (!$answer) {
// Oh dear the receive failed!
}
}
} else {
// Oh dear we didn’t successfully connect!
}
|
The first differenc in this example is that after creating our text message we
set a header property, SAM_REPLY_TO, to indicate the identity of the queue that
we will look on for our response. The receiver of the message is expected to read
this property and ensure they deliver the response to the right queue.
Also in the case of sending a message and receiving a response we need some way of
identifying the response meant for us rather than another script as the response
queue may have a number of responses on it. To do this we capture the return value
of the send method call as this is the identity that the messaging system gave
the message we sent. We can use this as a correlation value by having the process
that replies set the correlation id of the reply message to the id of the message
it received.
The reply message is received by calling the "receive" method of the SAMConnection
object and again naming the queue we wish to read along with an array of optional
parameters. The first of the optional parameters simply states that we are only
prepared to wait 5000 milliseconds (5 seconds) for the response. If no response
is received in this time the receive call will return with an error and no message.
The second optional parameter (SAM_SELECT) indicates we wish to select a particular
message from the queue and that the message we are looking for is the message with
a correlation id equal to the value in the variable $correlId.
As with the other method os SAMConnection we have used the receive method will return
a value of FALSE if it is unable to return a message for any reason.
Summary and further pointers
The SAM PHP extension is designed to make it easy for PHP script writers who are not
experts in Messaging and Queuing systems to access data and systems which are built
on these common middleware technologies.
The source is being made available through the PECL community site (pecl.php.net)
under the terms of version 2.0 of the Apache license so that PHP users can build
the extension for any version of PHP they wish. Visit the SAM webpage on PECL
for the latest available version.
Comments and feedback are welcomed.