Using ASP.NET SignalR to Publish Incremental Responses from Scatter-Gather BizTalk Workflow

While in Europe last week presenting at the Integration Days event, I showed off some demonstration of cool new technologies working with existing integration tools. One of those demos combined SignalR and BizTalk Server in a novel way.

One of the use cases for an integration bus like BizTalk Server is to aggregate data from multiple back end systems and return a composite message (also known as a Scatter-Gather pattern). In some cases, it may make sense to do this as part of a synchronous endpoint where a web service caller makes a request, and BizTalk returns an aggregated response. However, we all know that BizTalk Server’s durable messaging architecture introduces latency into the communication flow, and trying to do this sort of operation may not scale well when the number of callers goes way up. So how can we deliver a high-performing, scalable solution that will accommodate today’s highly interactive web applications? In this solution that I build, I used ASP.NET and SignalR to send incremental messages from BizTalk back to the calling web application.

2013.02.01signalr01

The end user wants to search for product inventory that may be recorded in multiple systems. We don’t want our web application to have to query these systems individually, and would rather put an aggregator in the middle. Instead of exposing the scatter-gather BizTalk orchestration in a request-response fashion, I’ve chosen to expose an asynchronous inbound channel, and will then send messages back to the ASP.NET web application as soon as each inventory system respond.

First off, I have a BizTalk orchestration. It takes in the inventory lookup request and makes a parallel query to three different inventory systems. In this demonstration, I don’t actually query back-end systems, but simulate the activity by introducing a delay into each parallel branch.

2013.02.01signalr02

As each branch concludes, I send the response immediately to a one-way send port. This is in contrast to the “standard” scatter-gather pattern where we’d wait for all parallel branches to complete and then aggregate all the responses into a single message. This way, we are providing incremental feedback, a more responsive application, and protection against a poor-performing inventory system.

2013.02.01signalr03

After building and deploying this solution, I walked through the WCF Service Publishing Wizard in order to create the web service on-ramp into the BizTalk orchestration.

2013.02.01signalr04

I couldn’t yet create the BizTalk send port as I didn’t have an endpoint to send the inventory responses to. Next up, I built the ASP.NET web application that also had a WCF service for accepting the inventory messages. First, in a new ASP.NET project in Visual Studio, I added a service reference to my BizTalk-generated service. I then added the NuGet package for SignalR, and a new class to act as my SignalR “hub.” The Hub represents the code that the client browser will invoke on the server. In this case, the client code needs to invoke a “lookup inventory” action which will forwards a request to BizTalk Server. It’s important to notice that I’m acquiring and transmitting the unique connection ID associated with the particular browser client.

public class NotifyHub : Hub
    {
        /// <summary>
        /// Operation called by client code to lookup inventory for a given item #
        /// </summary>
        /// <param name="itemId"></param>
        public void LookupInventory(string itemId)
        {
            //get this caller's unique browser connection ID
            string clientId = Context.ConnectionId;

            LookupService.IntegrationDays_SignalRDemo_BT_ProcessInventoryRequest_ReceiveInventoryRequestPortClient c =
                new LookupService.IntegrationDays_SignalRDemo_BT_ProcessInventoryRequest_ReceiveInventoryRequestPortClient();

            LookupService.InventoryLookupRequest req = new LookupService.InventoryLookupRequest();
            req.ClientId = clientId;
            req.ItemId = itemId;

            //invoke async service
            c.LookupInventory(req);
        }
    }

Next, I added a single Web Form to this ASP.NET project. There’s nothing in the code-behind file as we’re dealing entirely with jQuery and client-side fun. The HTML markup of the page is pretty simple and contains a single textbox that accepts a inventory part number, and a button that triggers a lookup. You’ll also notice a DIV with an ID of “responselist” which will hold all the responses sent back from BizTalk Server.

2013.02.01signalr07

The real magic of the page (and SignalR) happens in the head of the HTML page. Here I referenced all the necessary JavaScript libraries for SignalR and jQuery. Then I established a reference to the server-side SignalR Hub. Then you’ll notice that I create a function that the *server* can call when it has data for me. So the *server* will call the “addLookupResponse” operation on my page. Awesome. Finally, I start up the connection and define the click function that the button on the page triggers.

<head runat="server">
    <title>Inventory Lookup</title>
    <!--Script references. -->
    <!--Reference the jQuery library. -->
    <script src="Scripts/jquery-1.6.4.min.js" ></script>
    <!--Reference the SignalR library. -->
    <script src="Scripts/jquery.signalR-1.0.0-rc1.js"></script>
    <!--Reference the autogenerated SignalR hub script. -->
    <script src="<%= ResolveUrl("~/signalr/hubs") %>"></script>
    <!--Add script to update the page--> 
    <script type="text/javascript">
        $(function () {
            // Declare a proxy to reference the hub. 
            var notify = $.connection.notifyHub;

            // Create a function that the hub can call to broadcast messages.
            notify.client.addLookupResponse = function (providerId, stockAmount) {
                $('#responselist').append('<div>Provider <b>' + providerId + '</b> has <b>' + stockAmount + '</b> units in stock.</div>');
            };

            // Start the connection.
            $.connection.hub.start().done(function () {
                $('#dolookup').click(function () {
                    notify.server.lookupInventory($('#itemid').val());
                    $('#responselist').append('<div>Checking global inventory ...</div>');
                });
            });
        });
    </script>
</head>

Nearly done! All that’s left is to open up a channel for BizTalk to send messages to the target browser connection. I added a WCF service to this existing ASP.NET project. The WCF contract has a single operation for BizTalk to call.

[ServiceContract]
    public interface IInventoryResponseService
    {
        [OperationContract]
        void PublishResults(string clientId, string providerId, string itemId, int stockAmount);
    }

Notice that BizTalk is sending back the client (connection) ID corresponding to whoever made this inventory request. SignalR makes it possible to send messages to ALL connected clients, a group of clients, or even individual clients. In this case, I only want to transmit a message to the browser client that made this specific request.

public class InventoryResponseService : IInventoryResponseService
    {
        /// <summary>
        /// Send message to single connected client
        /// </summary>
        /// <param name="clientId"></param>
        /// <param name="providerId"></param>
        /// <param name="itemId"></param>
        /// <param name="stockAmount"></param>
        public void PublishResults(string clientId, string providerId, string itemId, int stockAmount)
        {
            var context = GlobalHost.ConnectionManager.GetHubContext<NotifyHub>();

			 //send the inventory stock amount to an individual client
            context.Clients.Client(clientId).addLookupResponse(providerId, stockAmount);
        }
    }

After adding the rest of the necessary WCF service details to the web.config file of the project, I added a new BizTalk send port targeting the service. Once the entire BizTalk project was started up (receive location for the on-ramp WCF service, orchestration that calls inventory systems, send port that sends responses to the web application), I browsed to my ASP.NET site.

2013.02.01signalr05

For this demonstration, I opened a couple browser instances to prove that each one was getting unique results based on whatever inventory part was queried. Sure enough, a few seconds after entering in a random part identifier, data starting trickling back. On each browser client, results were returned in a staggered fashion as each back-end system returned data.

2013.02.01signalr06

I’m biased of course, but I think that this is a pretty cool query pattern. You can have the best of BizTalk (e.g. visually modeled workflow for scatter-gather, broad application adapter choice) while not sacrificing interactivity and performance.

In the spirit of sharing, I’ve made the source code available on GitHub. Feel free to browse it, pull it, and try this on your own. Let me know what you think!

About these ads


Categories: BizTalk, General Architecture, SOA, WCF/WF

17 replies

  1. Can you please advise what version of BizTalk was used in this sample, i have BizTalk Server 2010 installed and could not open the BizTalk Project.

  2. I have re-created all the artifacts as per the design and everything works perfectly fine except the send port not calling the web application’s WCF service. Can you please walk me through the process of calling the publishresponse Operation from BizTalk Send Port.

    • Send port uses WCF-BasicHttp adapter. Address is http://localhost/IntegrationDays.SignalRDemo/InventoryResponseService.svc. The SOAP action is:

      [BtsActionMapping xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”]
      [Operation Name=”PublishResults” Action=”http://tempuri.org/IInventoryResponseService/PublishResults” /]
      [/BtsActionMapping]

      • Thanks for the Quick Response, I have done this but the still the orchestration is in the Dehydrated status. To Debug i have provided the Backup Transport to FILE and the XML files are placed in the folder after 3 retries without any problem with actual data from Orchestration Transformed message. Any Idea why the orchestration is going to the Dehydrated status?

      • What errors are in the Event Log? If the orchestration is dehydrating then it’s likely because the send port is failing one of its retries …

      • Man you are fast responding – This is what the event log states. is the OperationContract arguments have to match the Schema on the BizTalk Orchestration send?

        The adapter failed to transmit message going to send port “SendInventoryResponse” with URL “http://localhost/IntegrationDays.SignalRDemo/InventoryResponseService.svc.”. It will be retransmitted after the retry interval specified for this Send Port. Details:”System.ServiceModel.FaultException: a:ActionNotSupportedThe message with Action ‘<BtsActionMapping xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”>
        <Operation Name=”PublishQuotes” Action=”http://tempuri.org/IInventoryResponseService/PublishResults” />
        </BtsActionMapping>’ cannot be processed at the receiver, due to a ContractFilter mismatch at the EndpointDispatcher. This may be because of either a contract mismatch (mismatched Actions between sender and receiver) or a binding/security mismatch between the sender and the receiver. Check that sender and receiver have the same contract and the same binding (including security requirements, e.g. Message, Transport, None).
        at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.RequestCallback(IAsyncResult result)”.

        • So one thing I didn’t point out in the blog post is that I actually created the SignalR piece FIRST, and then did an “add generated items” to BizTalk in order to get the right WCF schemas and binding file. Consider doing that to get past any mismatch problem. The SignalR piece can’t be done until BizTalk development is done because the ASP.NET app needs the BizTalk service endpoint, but I ended up building enough of each to connect them!

      • Richard, The “add generated items” tip worked great and i was doing one more mistake by not exactly specifying the Send Port Operation name from “Operation_1″ to “PublishResults”. These changes made the solution work perfectly fine. Thanks for all your support, this SignalR is very revolutionary and have solved the long time Web application Problem.

  3. Works like a charm. Thanks. In one of my previous contract we exposed BizTalk orchestrations as a WCF service and it was consumed by a front end java web app. The BizTalk orchestration calls a variety of back end services and we implemented a call back functionality to update the status to the web application in an asynchronous fashion. Had we known this 2 years before we coiuld have implemented this. How long SignalR has been in the technology space? any idea?

  4. Hi Richard,

    Great article. It describes exactly what I need.

    I try to run your demo solution, but I don’t understand this part “the WCF Service Publishing Wizard in order to create the web service on-ramp into the BizTalk orchestration.”. The input and output body message are defined “use=literal”.

    Does it mean that we don’t use the predefined schema here?
    And where can I define “use=literal” in the wizard? Because I need to select an input and an output schema.

    Thanks

  5. Richard,
    Never mind. I have found it: create a new method with only the request does the trick. It works perfectly. Thanks for sharing.

Trackbacks

  1. Scott Banwart's Blog › Distributed Weekly 193
  2. Favorite Books and Blog Posts of 2013 | Richard Seroter's Architecture Musings

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 264 other followers

%d bloggers like this: