System Integration with Nintex Workflow for SharePoint 2007 (Part II)

[Series Overview: Part I /  Part II / Part III]

In my previous post I briefly described what the workflow tool from Nintex looks like and how to use it within the SharePoint environment.  Now, let’s see how to actually perform our first system integration scenario through the use of web services.

One big caveat before I start: Nintex currently only has support for consuming ASP.NET Web Services, not Windows Communication Foundation.  I consider this a pretty big gap, but for now, let’s work with what’s available.

In this demonstrated scenario, I have a SharePoint 2007 list which holds all the vendors a company uses to provide IT services.  Because core vendor data is stored in a different CRM system, we want to store a minimum of information in the SharePoint list and retrieve critical data from the CRM system via a web service.  So, once I add a record to the SharePoint list, I want to look up the details for that vendor in the CRM system and put the vendor “contact name” into the SharePoint list.

I start out with a simple ASP.NET service which fronts the CRM system.  Because I’m writing a blog post and not a real system, I simply hard coded a response object.

 

In my SharePoint list, I added a new Nintex workflow and dragged a “Call web service” and “Set field value” activities to the flow.  On this first pass, I’m just going to dump the entire web service XML response value into a SharePoint list field.

Note that the web service response has to go somewhere, so I first set up a workflow variable (VendorDetails) to hold the XML data.  I’ll use the other variable later to hold a specific value from the XML response.

The “Call web service” activity has a pretty comprehensive set of configuration options.   First you specify the endpoint URL.  The available service operations are then auto-populated in a list.  For dictating the service payload, you have two choices.  First, for services with simple type parameters, you can enter in each value from either list values or workflow variables using the SOAP Builder.

 

The other option, which is handy, is the SOAP Editor where you can shape the SOAP content yourself.  Note that you can still insert values from either lists or workflow variables into this interface.

As for the service response, you can choose to apply an XSLT transform and then select a workflow variable to stash the value.   You also get the option to catch exceptions and store those messages.

After using the “Set field value” workflow activity to take the service result and copy it into the SharePoint list field, we’re ready to publish the workflow.  I added a new row to the list and kicked off the workflow.

As we’d hope for, the full XML payload from the service is thrown into the SharePoint list field.

Note that XML payloads are wrapped in an “<xml>” node by the workflow activity.  It’s great that we can call a service, but just dumping the XML result is not particularly friendly to someone who is viewing this information.  What we want to do is parse the XML response and yank out JUST the “VendorContact” node.  So, I went back to our existing workflow and added a new “Query XML” activity to the process.  This activity lets me parse XML content stored either in a URL, workflow variable or list field.

You can process the XML via either XSLT or XPath.  In my case, I want to do an XPath query to find the node I’m looking for.  I then took the result of that query and stored it in a workflow variable. 

Finally, I have a “Set field value” activity which takes this new workflow variable and copies its data to the SharePoint list.  After once again publishing the workflow and kicking it off, we can see that not only do we have the XML blob, but now we have another field that just stores the name of the vendor contact person.

Summary

The ability to punch out to external systems is a valuable aspect of a full featured business process.  The Nintex workflow product does a fine job making service invocation a fairly straightforward task.  Now, the lack of WCF integration is a concern, but hopefully one being actively addressed.  However, because the “Query XML” activity can accept a URL, it seems possible that I could mash up RESTful services via this toolset.  I’ll have to try that.

The final post in this series will cover the native BizTalk Server integration in the product.  Stay tuned.

Technorati Tags: SharePoint

System Integration with Nintex Workflow for SharePoint 2007 (Part I)

[Series Overview: Part I /  Part II / Part III]

If your organization uses MOSS 2007, hopefully you’ve taken a look at what the folks at Nintex have to offer.  My company recently deployed their workflow solution, and I thought I’d take a look at how to execute system integration scenarios as part of a Nintex Workflow.

In this first post, I’ll take a short look at the general product toolset.  The second post will show off web services integration, and the final post will highlight their native BizTalk Server integration.

First off, what is Nintex Workflow?  It’s a solution hosted within the SharePoint environment that allows you to graphically construct robust workflow solutions that play off of SharePoint lists and libraries.  While you can build workflows in SharePoint using either WF or SharePoint Designer, the former is purely a developer task and the latter really exposes a linear, wizard driven design model.  Where Nintex fits in is right in the middle: you get the business-friendly user experience alongside a rich set of workflow activities (including any custom ones you build in WF).

Design Experience

Once the Nintex module is installed and enabled in your SharePoint farm, then any list or library has a set of new options in the “Settings” menu.

If I choose to create a new workflow, then I am given the option to select a pre-defined template which has a default flow laid out for me.

Once a template or blank workflow is chosen, I have a plethora of “Workflow Actions” available to sketch out my process.  For example, the Integration category has options such as “Call web service”, “Execute SQL”, “Query LDAP”, “Send/Receive BizTalk” and “Call Workflow.”

There are nine categories of workflow activities in all, including:

• Integration
• Libraries and lists (e.g. “Check out item”, “Create list”, “Set field value”)
• Logic and flow (e.g. “For each”, “Run parallel action”, “State machine”)
• Operations (e.g. “Build dynamic string”, “Wait for an item to update”)
•  Provisioning (e.g. “Add User to AD Group”, “Provision user on Exchange”)
• Publishing (e.g. “Copy to SharePoint”)
• SharePoint Profiles (e.g. “Query user profile”)
• Sites and Workspaces (e.g. “Create a site”)
• User Interactions (e.g. “Request approval”, “Send a notification”, “Task reminder”)

Note that you can turn off any individual action you wish if the capability exposed is too risky for your particular organization.

Using these activity shapes, I can draw out a simple process made up of decisions, notifications and approvals.

Each activity can be configured (and re-labeled for later readability) per its function.  When an activity requires data to act upon (as most do), those values can often either be retrieved from (a) a hard coded value, (b) a workflow-specific variable, or (c) content from any list on the site.  For the “Request approval” activity, I have all sorts of options for choosing where to get the approver list from, which means of approval to require (all, single, first, vote) and where to store the assigned tasks.  What’s also cool is the “Lazy Approval” setting in Nintex which allows you to respond to a notification email with a single word or phrase to indicate your response.  In the image below, notice that I used a value from the list (in red text) as part of my task name.

The configuration experience is pretty similar for each activity.  For the most part, I’ve found it to be fairly intuitive although I’ll admit to actually having to open the Help file a few times.

Runtime Experience

You can choose what the start up option should be for the workflow.

Then, from the “Actions” menu, you can publish the workflow and make it available for use.  Pretty darn easy.

Then, when you add/change data in the list or library, the workflow is either automatically or manually triggered.  Just like with any SharePoint workflow, you have a column added to the list which keeps you up to date on the status of the workflow (which you can drill into).  Whatever task list is associated with the workflow is also populated with tasks assigned to individual users or groups.

 

Users can also add the “My Workflow Tasks” web part to a page which will show only the tasks for the active user.

Users can also browse into the running workflow and graphically see what’s been completed so far, how long each step took, and what comes next.

Analysis Experience

Just like the previous image, we can drill into a completed workflow and analyze how it ran and the duration of a given step.

As for reporting, the product comes with plenty of canned reports on a per-site or all-site basis that address topics such as: Approver Performance Statistics, Workflows in Progress, Workflow Performance and much more.

You can display these reports either graphically or tabularly as a web part.  For the graphical reports, you can choose line, bar or pie chart.  The charts actually rely on Microsoft Silverlight (for the 2-D representation) and are pretty snazzy and configurable.

Summary

This was a very simple, but hopefully adequate, walkthrough to show you around the software.  This technology has lots to offer and integrates nicely with the Microsoft stack of products (including Live Communication Server).   Note that nothing I did here required a lick of programming or even a particularly technology-centric background.  And because the design surface is hosted within the SharePoint environment itself, you get a very rapid, accessible means for building and deploying functional workflows.

If you have a SharePoint sandbox, consider downloading the free trial and playing around.

In the next post, I’ll show you how to do some simple system integration via web service calls from a workflow.

Technorati Tags: SharePoint, BizTalk

My Book "SOA Patterns with BizTalk Server 2009" is Up for Pre-order

Well look at that.  I just happen to check my publisher’s website today and noticed that they put my book description online and made it available for pre-order.

The book, entitled SOA Patterns with BizTalk Server 2009, is due out by May 2009 (hopefully sooner).  I’m still in the process of editing the book and responding to technical review comments, but we’re nearing completion of that phase.  I have three fantastic technical reviewers (Charles Young, Zach Bonham, Ewan Fairweather) who have added a ton of great comments that I’m almost done wading through and addressing.

This publisher (Packt Publishing) specializes in short, targeted technical books that address specific topics.   I tried to be “short” but still went over my suggested page count by nearly double.  Oh well, hopefully that’s better for the readers. 

The high level chapter summaries are …

• Chapter 1 – Building BizTalk Server 2009 Solutions – A brief walkthrough of creating a new BizTalk project from scratch
• Chapter 2 – Windows Communication Foundation Primer – An overview of WCF with examples of developing, hosting, and consuming WCF services
• Chapter 3 – Using WCF Services in BizTalk Server 2009 – An explanation of the marriage between BizTalk and WCF and how to both expose and consume WCF services within BizTalk Server
• Chapter 4 – Planning Service Oriented BizTalk Solutions – A discussion of the core aspects of SOA, the types of services one can construct, available message exchange patterns, and how service-oriented principles apply to BizTalk solutions
• Chapter 5 – Schema and Endpoint Patterns – How to build effective schemas and endpoints depending on the type of service constructed
• Chapter 6 – Asynchronous Communication Patterns – A look at options for asynchronous communication and how to exploit this pattern in BizTalk solutions
• Chapter 7 – Orchestration Patterns – We analyze the role of orchestration in service solutions and see how to build loosely coupled workflow processes
• Chapter 8 – Versioning Patterns – We see here how to effectively version our SOA solutions and cleanly introduce updates to our BizTalk components
• Chapter 9 – New SOA Capabilities in BizTalk Server 2009 – A first look at the latest additions to the BizTalk product. This includes the new UDDI v3 registry, WCF SQL Server Adapter, and ESB Guidance 2.0

So there you go.  Once the book actually hits the shelves, I’ll make another mention of it on the blog.

Technorati Tags: WCF, BizTalk, SOA

Presentations from 2009 Microsoft SOA/BPM Conference Available Online

Hadn’t noticed this before, but found the complete collection of videos and presentations from this year’s SOA & BPM Conference.  I didn’t make it up to Redmond for this, so it’ll be nice to peruse the content which covers topics such as:

• Customer case studies
• Designing services for “Dublin”
• Using BAM for Service and SLA monitoring
• Supporting WS*, REST and POX simultaneously with WCF
• SOA patterns from the field
• Lap around “Oslo”

Check it out.

Technorati Tags: WCF, BizTalk, SOA

New "WCF Adapter FAQ" Whitepaper From Microsoft

The folks over at the BizTalk Adapter Development blog pointed out their new paper which answers a number of questions about the WCF adapters for BizTalk Server 2006 R2.

This collection of FAQs address topics such as:

• Deciding which WCF adapter to use
• When to use the two custom adapters
• How do messages flow in from WCF adapters and flow back out
• How to preserve the entire inbound message arriving via a WCF adapter
• Accessing WCF message properties within a BizTalk solution
• Decision criteria when choosing the WCF LOB Adapter SDK vs. the old BizTalk Adapter Framework
• How BizTalk uses the WCF adapters
• The difference between WCF behaviors and BizTalk pipelines

There are plenty more besides these.  If you are doing WCF development alongside your BizTalk solution and need to integrate the two, you really should check this out.  The document is well written and nicely explains the concepts.

Technorati Tags: WCF, BizTalk

What I Learned at the 2009 Microsoft MVP Summit

Just returned from the Summit, and had a really great time meeting colleagues and seeing interesting content.  Without breaking any NDAs, I thought I’d share some thoughts on what I saw this past week.

• The CSD/BizTalk MVP group remains a smart bunch of guys with a diverse set of backgrounds.  I shared a hotel room with Yossi Dahan, and was thrilled to spend time with the likes of Mick Badran, Michael Stephenson, Saravana Kumar, Stephen Thomas, Leonid Ganeline, Ben Cline, Jon Fancey, Kevin Smith, Alan Smith, Mikael Håkansson, Jon Flanders, Matt Meleski, Tim Rayburn, Kent Weare, Bill Chestnut and many others.  The practical experiences these guys have with customers should be milked dry by the product teams.
• I finally spent some time on Azure (thanks to Steve Marx’s session) from a development platform perspective (i.e. not .NET Services) and I am quite impressed by the developer experience.  This seems to be something Microsoft will really bang on when trying to differentiate its offering from those of others in the “cloud computing” space.
• The Summit was also the first time I’ve REALLY looked at “M” since last year’s Summit, and I think it has potential to be useful.  I made this observation in a previous blog post, and plan to actively mess around with “M” for both data solutions and to emit actionable content.  The interesting thing is, the lengthiest work doesn’t seem to be writing the textual DSL as much as constructing the parser to do something with the output.  That is, in the examples referenced in the link above, “M” models are used to create orchestrations and BizTalk build processes, respectively.  Once the “M” part is done, both Yossi and Dana had to write the parser which took the MGraph and cycled through the objects to generate the final output (i.e., ODX file and executable deployment script).  Not a bad thing, just something that’s seemingly glossed over.
• We had a “BizTalk Next” brainstorming session, and I was happy to see some big ticket items on the minds of the BizTalk architecture team.  After sitting on the same runtime engine for 5 years, and mostly surface-level changes during the past 2 releases, it seems like “put up or shut up” time for these guys.  And they seem to be working hard to be in the former category.
• Ewan Fairweather led a strong session on BizTalk performance with a special focus on the impact of virtualizing BizTalk solutions in Hyper-V.  Ewan authored the BizTalk Server R2 Hyper-V Guide and BizTalk Server Performance Guide.  The results of Ewan’s recent exploits with his giant server farm will be making their way into public documentation shortly.  A few takeaways (which I was assured aren’t NDA!): (a) performance of “inline sends” vs. “logical ports” within orchestrations is nearly 4x faster with regards to throughput, (b) the performance impact of virtualization on BizTalk solutions is fairly negligible and (c) making a few well-known platform tweaks can nearly double performance.  Look for more approachable guidance in the near future (vs. massive, detailed whitepapers) as the team realizes that with such a developer-centric product as BizTalk, you need to remember that you aren’t always talking to your standard system administrators.

As for non-content related observations …

• Tim Rayburn can sing a mean Black Hole Sun, as experienced during one of the social parties of the week.
• Those two white guys who sang California Love?  Not so much.
• Mick Badran is a wild man, and a helluva great guy.  However, I worry that any “guys night out” with him always ends with something like “… and that’s how we ended up in a Chinese prison with two 85 year old women and $450 in nickels.”
• People need to stop with the corny gifts to Ballmer and the general Q&A shenanigans.  You’re embarrassing yourself folks.
• Conference attendees will take pictures of ANYTHING.  Seriously, random speakers, crowds, buses (inside and out), signs, buildings, hotel lobbies, plant life.  And what’s with the incessant picture taking of NDA slides?  SLIDES, people.  Write down the content if it’s that freakin’ important.  I’m convinced that these people are either (a) IBM spies or (b) crafting the most sleep-inducing scrapbook since my 3rd grade trip to the Liberty Bell.

Great time overall, looking forward to doing it again.

Technorati Tags: BizTalk, MVP

Interview Series: Four Questions With … Jesus Rodriguez

I took a hiatus last month with the interview, but we’re back now.  We are continuing my series of interviews with CSD thought leaders and this month we are having a little chat with Jesus Rodriguez.  Jesus is a Microsoft MVP, blogger, Oracle ACE, chief architect at Tellago, and a prolific speaker.  If you follow Jesus’ blog, then you know that he always seems to be ahead of the curve with technology and can be counted on for thoughtful insight. 

Let’s see how he handles the wackiness of Seroter’s Four Questions.

Q: You recently published a remarkably extensive paper on BAM.  Did you learn anything new during the creation of this paper, and what do you think about the future of BAM from Microsoft?

A:  Writing an extensive paper is always a different experience. I am sure you are familiar with that feeling given that these days you are really busy authoring a book. A particular characteristic of our BAM whitepaper is the diversity of the target audience. For instance, while there are sections that are targeting the typical BizTalk audience, others are more intended to a developer that is really deep with WCF-WF and yet others sections are completely centered on Business Intelligence topics. I think I learned a lot in terms of how to structure content that targets a largely diverse audience without confusing everybody. I am not sure we accomplish that goal but we certainly tried

I think BAM is one of the most appealing technologies of the BizTalk Server family. In my opinion, in the next releases, we should expect BAM to evolve beyond being a BizTalk-centric technology to become a mainstream infrastructure for tracking and representing near real time business information. Certainly the WCF-WF BAM interceptors in BizTalk R2 were a step on that direction but there are a lot of other things that need to be done. Specifically, BAM should gravitate towards a more integrated model with the different Microsoft’s Business Intelligence technologies such as the upcoming Gemini. Also, having interoperable and consistent APIs is a key requirement to extend the use of BAM to non Microsoft technologies. That’s why the last chapter of our paper proposes a BAM RESTful API that I believe could be one of the channels for enhancing the interoperability of BAM solutions.

Q: You spoke at SOA World late last year and talked about WS* and REST in the enterprise.    What sorts of enterprise applications/scenarios are strong candidates for REST services as opposed to WS*/SOAP services and why?

A: Theoretically, everything that can be modeled as a resource-oriented operation is a great candidate for a RESTful model. In that category we can include scenarios like exposing data from databases or line of business systems. Now, practically speaking, I would use a RESTful model over a SOAP/WS-* alternative for almost every SOA scenario in particular those that require high levels of scalability, performance and interoperability. WS-* still has a strong play for implementing capabilities such as security, specifically for trust and federation scenarios, but even there I think we are going to see RESTful alternatives that leverages standards like OpenID, OAuth and SAML in the upcoming months. Other WS-* protocols such as WS-Discovery are still very relevant for smart device interfaces.

In the upcoming years, we should expect to see a stronger adoption of REST especially after the release the JSR 311 (http://jcp.org/en/jsr/detail?id=311 ) which is going to be fully embraced by some of the top J2EE vendors such as Sun, IBM and Oracle.

Q: What is an example of a “connected system” technology (e.g. BizTalk/WCF/WF) where a provided GUI or configuration abstraction shields developers from learning a technology concept that might actually prove beneficial?

A:  There are good examples of configuration abstractions in these three technologies (BizTalk, WCF and WF). Given the diversity of its feature set, WCF hides a lot of things behind its configuration that could be very useful on some situations. For instance, each time we configure a specific binding on a service endpoint we are indicating the WCF runtime the configuration of ten or twelve components such as encoders, filters, formatters or inspectors that are required in order to process a message. Knowing those components and how to customize them allows developers to optimize the behavior of the WCF runtime to specific scenarios.

Q [stupid question]: Many of us have just traveled to Seattle for the Microsoft MVP conference.  This year they highly encouraged us to grab a roommate instead of residing in separate rooms.  I’ve been told that one way to avoid conference roommates is to announce during registration some undesirable characteristic that makes you an lousy roommate choice.  For instance, I could say that I have a split personality and that my alter ego is a nocturnal, sexually-confused 15th century sea pirate with a shocking disregard for the personal space of others.  Bam, single room.  Give us a (hopefully fictitious) characteristic that could guarantee you a room all to yourself.

A:  My imaginary friend is a great Opera singer We normally practice signing duets after midnight and sometimes we spend all night rehearsing one or two songs. We are really looking have our MVP roommate as our audience and, who knows, maybe we can even try a three-voice song.

Seriously now, given work reasons I had to cancel my attendance to the MVP summit but I am sure you guys (BizTalk MVP gang) had a great time and drove your respective roommates crazy

As always, I had fun with this one.  Hopefully Jesus can say the same.

Technorati Tags: BizTalk, WCF, REST

Good Example of Oslo "M" Modeling for BizTalk-related Use Cases

While at the Microsoft MVP Conference this week, I’ve added “modeling in ‘M’” to my “to do” list.  While I’ve kept only a peripheral view on “M” to date, the sessions here on “M” modeling have really gotten my mind working.  There seem to be plenty of opportunities to build both practical data sets and actionable content through textual DSLs. 

One great example of applying “M” to existing products/technologies is this great post by Dana Kaufman entitled A BizTalk DSL using “Oslo” which shows how one could write a simple textual language that gets converted to an ODX file.  It’ll be fun watching folks figure out cool ways to take existing data and tasks and make them easier to understand by abstracting them into a easily maintained textual representation.

Update: Yossi Dahan has also posted details about his current side project of creating an “M” representation of a BizTalk deployment process.  He’s done a good job on this, and may have come up with a very clean way of packaging up BizTalk solutions. Keep it coming.

Technorati Tags: Oslo, BizTalk