One of the technology backbones of my company moving forward is the use of BizTalk to “fan out” messages from our ERP system (SAP). Instead of having hundreds of direct interfaces to SAP by downstream systems who need real-time data, we can use BizTalk to receive a single message from SAP (XI), and distribute it to all interested parties. One issue brought up recently was “ordered delivery” and ensuring that if a downstream system needs messages delivered in the order sent by SAP, that BizTalk honor it. So, I set out to test a variety of scenarios using BizTalk’s built-in ordered delivery capabilities.
Setup: I exposed a BizTalk schema as a web service to simulate our publishing receiver from SAP. A simple Windows Form is sending messages to this web service, and stamping each one with a sequence number. I then built a couple of web services to act as “subscribers” of these SAP data events. These web services publish the data to a database table, thus letting me see the exact order of messages delivered. Each SOAP send port has a subscription based on message type.
Scenario #1: Send port, no ordered delivery
When a series of messages are published to the receiving web service and routed to the subscriber without ordered delivery, the result may look like this:
As you can see, even on my single development machine, the order gets mixed up. This is due to batch processing and the multi-threaded nature of BizTalk message distribution.
Scenario #2: Send port, ordered delivery turned on
To turn ordered delivery on, all we need to do is check a box on the send port (this assumes that your inbound transport can deliver messages in order. Examples include MSMQ, SOAP and HTTP).
The result in the database looks like this:
So you can see that all the messages are in sequential order.
Scenario #3: One send port no ordered delivery, one send port WITH ordered delivery
I wanted to prove that ordered delivery only impacted the corresponding port. When sending the message into BizTalk with two send ports subscribing, the data looked like this:
“Subscriber #1” kept everything in order, but “Subscriber #2” delivered messages a bit more haphazardly.
Scenario #4: Non-ordered delivery send port, no retries, and error in service
So what if the service raises an error? When there are no retries, and the send port doesn’t have an ordered delivery requirement, the result looks like this:
The bad message (#12) is simply skipped (because it is now suspended).
Scenario #5: Ordered delivery send port, retries enabled, and error in service
How about for an ordered delivery send port? What if there’s an error in the service call, and retries are turned on for the port? Also, the “cancel if error” is turned off. Check this out:
The messages queue up waiting for the first one to retry, and hopefully succeed. We’ll see more of this in a moment.
Scenario #6: Ordered delivery send port, NO retries, and error in service
This time, there’s an error in the service, and no retries. Also, the “cancel if error” flag is still turned off. The result is:
So the messages still get delivered AFTER the bad message has been encountered. That’s expected, since we told BizTalk to keep going, even for the ordered delivery port.
Scenario #7: Ordered delivery send port, NO retries, “cancel if error” turned on, and error in service
What if we reproduce the previous scenario, BUT, turn “cancel if error” on? This flag can be set on the send port here:
What you’re telling BizTalk is that if any message fails for this ordered delivery port, stop processing until this error can be corrected. This is useful if you’re concerned that an “insert new contact” message failed, but you expect a “modify existing customer” to be following. Clearly the “modify” message will fail unless the “insert” gets figured out. The result of this?
Processing stopped after message #11 because message #12 failed. The send port is still “started”, but you’ll find a suspended message. If you open it up, you’ll see that all following messages are queued up until the offender gets resolved.
Scenario #8: Send port stopped, and restarted for a non-ordered delivery send port
What if we KNOW that a downstream system is unavailable (e.g. maintenance) and want to prevent the inevitable failure of delivery? Maybe we want to shut off the send port, queue up the messages, and once the unavailable system is back online, open the distribution pipeline again. For a non-ordered delivery port, after restarting a stopped send port, the output looked like this:
The messages are in a crazy order, as you can see.
Scenario #9: Send port stopped, and restarted for an ordered delivery send port
Final scenario. What if we stop an ordered delivery port, and then start it back up later? Do we retain the correct ordering? When I sent five messages in, I got a suspended instance with five messages:
If I send in three MORE messages, they actually get added to this suspended instance:
Once I finally turn the subscriber service back on, all messages were sent in the same order received.
So there you go. Ordered delivery is a fairly powerful concept, and not particularly hard to do with BizTalk. It greatly impacts performance because the send port won’t send subsequent messages until a delivery confirmation is received, but, in many cases that performance impact is outweighed by business requirements.
Technorati Tags: BizTalk