A Sinfonia on Messaging:
Prelude
A complete messaging solution is a three-part invention of business need, architecture, and implementation. In its final form, these three voices blend in harmony, with each one taking a dominant role depending upon which part of the solution one examines.
Neither have I the ability nor skill to seamlessly weave three concepts together while clearly explaining their roles. Therefore, I will separate out the voices from each other and leave it as an exercise for the reader to construct an application and practice the principles involved, thus experiencing well-earned contrapuntal pleasures first-hand.
Introduction
As computing and data exchange systems increased in complexity over the past 30 years, so has the need for improvements -- and where possible, simplifications. Some of these efforts have been focused on decentralization of communications (shared, distributed load) and decoupling of messaging from applications (removing redundancy and increasing delivery speed/throughput). The first steps towards this were made in the 1990s with explorations in "middleware" application universe.
Messaging, as we now refer to it in the industry, arouse from those middleware adventures: out of the business drive to refactor old software as new services to wider, more sophisticated audiences. With many new services replacing a single, monolithic application, formal and well-architected solutions were needed for creating, editing, and deleting shared data.
From Wikipedia:
Message-oriented middleware (MOM) is infrastructure focused on message sending that increases the interoperability, portability, and flexibility of an application by allowing the application to be distributed over multiple heterogeneous platforms. It reduces the complexity of developing applications that span multiple operating systems and network protocols by insulating the application developer from the details of the various operating system and network interfaces.AMQP (Advanced Message Queuing Protocol) is one of these protcols.
In a recent blog post about Ubuntu as a business server, Vaughan-Nichols provides evidence for Ubuntu's and Canonical's commitment to enterprise, saying "... the new [version of] Ubuntu also includes AMQP [...] support. AMQP is an important set of middleware and SOA [...] protocols."
AMQP has demonstrated itself as a compelling protocol for messaging solutions, even to the point of being included in two Linux distributions. The code included in this blog series is txAMQP, an asynchronous Python AMQP library built with Twisted.
Note that last year I had planned to write a blog series on messaging with Twisted and XMPP, but was unable to as a result of time constraints. These days, I'm working with AMQP instead of XMPP, but I still hold some hope that I'll be able to write an anolog for this series from the perspective of XMPP.
The Voice of Business
Dipping into the future, one of the goals for AMQP as developed by a special interest group is the following:
Decentralized, Locally Governed Federated Mesh of AMQP Brokers with standardized Global Addressing. The killer application for AMQP is transacted secure business messages between corporations - e.g. send a banking confirmation message to confirms@bank.com [...]I find this rather exciting due to my interest in ultra large-scale systems; scenarios like the one described above are the seeds for tomorrow's ULS systems :-)
For now, though, let's look at a more immediate use for AMQP: a messaging protocol for shared services between departments in a small store. In this exercise, the voice of business is the primary melody; everything else (architecture and implementation) is done in support of this theme.
An Example
Fhionnlaidh MacGrudder creates hand-made kilts to order. He's got a sales guy who works with movie costume design shops and the like. He's got a web girl who wrote and maintains a custom store front app. He's got a friend who does shipping and billing for him (as well as some other local Glen Orchy artisans). Until now, these three business "groups" associated with the kilt shop have been maintaining their own records, sometimes copying and updating them manually from each others' various export files.
Fhionnlaidh's niece Fíona is programmer, business student, and is dating shipping guy's son. Horrified by the inefficiencies in her uncle's business processes (and tired of her boyfriend's father's complaints), she has proposed the following:
- sales guy will maintain customer contact info offer this as a service to the store and billing
- the web store will dynamically update displayed data when sales guy changes it
- the CRM will dynamically update displayed data when a web store customer updates their info
- MacGrudder will have a new web page he goes to where all pending orders are presented with their full details; changes can be updated by a customer in real-time until MacGrudder has started working on the order
- billing/shipping guy will be notified instantly as soon as MacGrudder marks a kilt order as completed
- Contacts will be maintained in a single data store
- There is zero latency between customer-driven updates and sales guy-driven updates
- Customers have increased post-purchase flexibility with their orders
- Shipping guy can plug into MacGudder's messaging and be notified when packages are ready for pickup
- Everyone has more time for buttered scones and tea (especially shipping guy, who will no longer be making unneeded trips down the glen)
- Contacts will need to be merged into the CRM
- A read/write data service for the contacts will need to be created
- The CRM front-end will need to be upgraded to an AJAX-enabled version
- The web store app will need to be updated to support AJAX
- A new page will be created which displays the status of all orders and allows MacGrudder to change an order from "pending" to "in-progress" to "completed"
- The current "new order" email notification code in the web app will need to be changed so that it uses the same messaging as MacGrudder's status page
- A new service needs to be created for shipping guy so that he can choose to be notified about pending pickups by email or he can check a web page or even make a query directly to the service, thus preventing unnecessary trips to MacGrudder's isolated little shop
- After all the work is done, somone's going to need to order more scones
In the next installment, we'll review the business process (with diagrams!) and the explore the architecture of the system, before and after. Another post will take that architecture and combine it with MacGrudder's already extant infrastructure, reusing as much as possible. With that in place, we will have the opportunity to look at some RabbitMQ basics and some actual txAMQP code.
References
- SOA in Practice
- Message-oriented middleware on Wikipedia
- AMQP on Wikipedia
- The AMQP Spec
- A synchronous Python library for AMQP
- The async Python library for AMQP
- A txAMQP example
- An intro to messaging and py-amqplib
2 comments:
This is an interesting series; dynamic updates are a subject near and dear to my heart. Unfortunately it suffers from a common problem of marketing literature for ESBs: an incomplete description of the problem.
There are two questions that I feel need to be answered to justify your conclusion:
Where are the inefficiencies in the business that Fíona is complaining about? Copying and updating from export files might be perfectly fine. You hint at the problem when referring to the shipping guy's "unneeded trips down the glen", but you don't say why he's making those trips.
Also, what is the advantage of this decentralized system over the simpler solution of all parties using a single, centrally maintained, web-based business software?
As it happens, if I fill in the gaps myself, I believe your conclusions. I just think this could be a very persuasive example with some more explication.
Hey glyph, thanks for your comment.
In answer to your questions, in part, I refer you back to the post:
"This example is not meant to fully justify messaging for businesses, but rather to provide a simple use case for which we can write some simple (and less than robust) code. It is a toy, but a conceptually useful one with a solid, concrete foundation."
For a more complete response, I would ask readers to consider the following:
The problem in illustrating messaging examples is that the need for it has arisen organically over the past decade or so within complex environments with many duplicated resources, redundant solutions, multiplied maintenance burdens, divergent data, etc. Large, diverse systems that are expensive to maintain and whose components have intersecting needs are the ones that benefit most from messaging.
As such, a simple example will necessarily be artificial. However, if one can accept the possibility of the need for messaging, and then work through a simple case (however contrived), then by extrapolation, they can apply this new understanding and first-hand knowledge to real-life situations in complex environments with which they are familiar.
Couple this initial exposure with the related reading materials, and a motivated, interested reader has everything they need to assess their own large environments and begin envisioning the ways in which messaging -- in the long run -- can save them time, effort, and money.
Sadly, this post suffers from more than common marketing problems: it's awkward at best and doesn't pull the story together concisely. I have every intent of returning to it after the next few posts finish up this series to see what I can do to reshape it. That may or may not happen, time will tell.
Post a Comment