Congrats to the Connected Systems Division for getting BizTalk Server 2006 R2 out the door. Now that we’re done with that, here’s my humble “wish list” for BizTalk Server vNext. I realize that development is well under way, but, hopefully some of these requests can make it in.
Design Tools
- High level modeling tool. Nothing the team doesn’t know already, but I want a tool/view that let’s me architect the BizTalk solution at a broader level. Much like the BizTalk Server 2006 Administration Console introduced “application management”, I want a similar metaphor for “application architecture.”
- Modeling tools that support industry standards. The BizTalk team has done a great job in embracing industry standards for developed artifacts (e.g. XSD, XSLT, SOAP, WSDL, XML), and I’d love to see a similar embrace of the design environment. Specifically, UML and/or BPMN support in the above-mentioned higher level modeling toolset.
BizTalk Administration
- Option for subscriber throttling. I need to be able to pick any orchestration or any send port and tell the engine not to instantiate more than X number of them at one time. Many smart folks have come up with various solutions (e.g. singletons, ordered delivery, etc), but I can’t see why it’s too technically challenging to force the XLANG engine (or EPM) to verify running instances vs. throttle count prior to instantiating a new instance.
- Stronger dependency visibility. I’d like to be able to open the BizTalk Administration Console, view a host, and see every artifact that uses it. Likewise, I’d like to be able to view a schema and see each map that references it. I need more ways to find out which artifacts have dependencies on others so that I can better plan application upgrades or retirements.
- “Application” level permission controls. Right now, when our team adds someone to the “Operators” group, they have free reign over any application deployed in the environment. That makes me a tad nervous. Too easy to accidentally terminate someone else’s suspended messages, or see message content that they shouldn’t. I’d like the option to allow department-level administrators to own, manage and troubleshoot specific applications in the BizTalk environment.
- Web-based Administration Console. While it’s fairly simple to do a “Admin only” install of BizTalk on a desktop machine, I’d appreciate a web-based management console that let’s me perform a subset of standard tasks. Easier to provide access to multiple administrators (only if the isolated ownership point above is enacted), and if you wanted to get fancy, you’d AJAX the UI and provide near-real-time updates of running and suspended instances without a manual refresh.
- Better subscription analysis. It’s great that the Subscription Viewer is now part of the Admin Console, but I need more criteria to search for. For instance, I’d like to be able to search for any subscription built upon a particular message type. If I need to change a schema namespace, which subscriptions will it impact? Same with searching for subscriptions by port names, etc. Again it comes back to impact analysis of changes.
- Additional subscription operators. Right now, I can’t create a subscription based on a field NOT existing in the schema. I can only do “exists”. I’d also like subscriptions based on “contains” where I could route messages (without orchestrations) where a “customer ID” contains a particular substring.
- More health metrics in the Admin Console. Specifically, I would find it useful if there was a portion of the Administration Console where I would be notified if host throttling thresholds were approaching, if a particular application was backlogged, etc. I know that I can find out this information using performance counters, or MOM, but I’d like to have the Admin Console be more of a “one stop shop.”
Adapters
- Updated core adapters. It’d be great to refresh some of the core adapters with new capabilities. I’d like to see the FILE adapter support XPath-based file name tokens. If I want the output file name to contain a field from the message, it’d be much easier to manage this at the adapter level rather than introducing orchestrations or custom pipelines. For the SMTP adapter, it should be much easier to do dynamic addressing. To dynamically choose the “To:” address, I have to do an orchestration with a dynamic port. And instead of just setting the “To:” address, I also have to use the BRE or custom component to grab the SMTP Host, Subject, etc. Often, the only “dynamic” piece of the email is the address. Seems like lots of improvements are possible for the SQL adapter. I’d like an “after poll” process option (like the Oracle adapter), and support for querying tables/views instead of requiring a stored procedure (or updategram). Seems like the Oracle adapter has more features than the SQL Server one.
- More browsing, less typing. One of the top 5 improvements in BizTalk 2006 was the addition of the “browse” button in the FILE ports. Why am I still typing URLs in the SOAP/HTTP ports, or typing settings for the SharePoint adapter? Why can’t we browse more settings instead of relying on me to inevitably type the values incorrectly?
Development
- Refresh auto-generated schemas. I love that I can update a “web reference” in Visual Studio.NET with no problem, but I absolutely dread changes to auto-generated BizTalk schemas (SQL stored procedure, Siebel business object, etc) since I have to walk through the Generate Schemas wizard again even for a simple update to the data source. I’d love to right click on the Oracle database view XSD schema, and choose “refresh schema from source” and have the update automatically taken care of.
- Option to automatically GAC referenced assembly. I know that I could add post-build steps on my .NET component libraries which would GAC the component for me. But, how great would it be if the BizTalk project properties page had a choice to “GAC all referenced assemblies”?
- Orchestration unit test. I don’t know how you’d implement this, but even a simple test of an orchestration process involves a full deploy, build ports, etc. Sometimes I would like a quick process logic test without going through the whole deployment production.
- “Construct Blank Message” in orchestration. Seems that I often come across folks who use XmlDocument variables or maps to simply create a new, empty BizTalk orchestration message. For instance, I may want an empty message that I pass to the BRE, which in turn fills in all the fields I want. Or, I create an empty Oracle query schema, and use a distinguished field to actually set my query filter. I’d like a “construct blank message” which instantiates a message WITHOUT using a transform or “message1 = message2” assignment.
That’s all I’ve got for now. Thoughts? Any of those requests seem outlandish?
Technorati Tags: BizTalk
Richard Seroter's Architecture Musings 
How about better namespace support in Orchestrations (ala Using). I hate seeing 5 levels of dotteed notation 🙂
Good call. I’d also like to see namespace aliasing for the mapper so that “inline C#” can call external assemblies.
Here are some of the features I would like to see in vNext version:
1. Build-in scheduler to trigger any service/process (listeners, orchestrations). Ability to call any listener iteratively or based on timer including HTTP/S to make a call to a web-service in addition to triggering orchestrations/business process flows directly based on timer.
2. Checks for duplicates based on promoted properties in the document. A Configuration utility to say if certain combination of fields is not unique (with respect to messages already received), then system should flag the message as duplicate message.
3. Automated deployment tool to push code to multiple machines or move from staging to production. It should check the target system configurations, compares versions, automatically prompts for machine specific details like host name, paths etc.
4. Certificates and other shared objects loaded from a SAN disk than local disk in a multi server cluster deployment. The same applies to all run-time config files or binaries. (i.e, run time artifacts and config files will be on SAN disk and all machines read from this one copy)
5. Very detailed logs to trace every step in the process not just error logs.
6. Strong user management features with granular permissions like role based access etc.
7. Resubmit/Reprocess options. Ability to do “playback” as a new transaction altogether, and resume a failed process from the point of failure after the issue is resolved.
8. Explicitly specify a schema on a listener or ability to promote properties so the listener can bind a document instance to a partner based on data inside certain promoted fields. For example if I get XML documents from multiple partners with the same RootNode, I can group the partners into say “Customers” and then say check for SenderID/Qualifier, ReceiverID/Qualifier and any other logic and let the system figure out partner/schema resolution.
9. Support for FTP over SSL, FTP over SSH, AS1 and AS3.
10. Ability to set inactivity parameters in File, FTP, HTTP and other listeners and trigger an alert. For example if a File Adapter is polling a location every 15 min and there is no document available to pickup in the last 30min (or whatever internal user specified) raise an alert so we can pro-actively check upstream systems or call trading partners why they didn’t send documents.
11. Unified web-portal framework to view B2B transactions and Business processes (public and private) and drill down on status and key document data fields that are promoted.
12. Ability to monitor system metrics like Memory, CPU usage etc and also correlate them with message flows. This is useful for operations to check systems health and monitor resource utilization.
13. Trading Partner Management capabilities with concepts of Sender/Receiver and TPA.
14. Ability to proactively specify what changes to existing runtime artifacts/components will impact what existing configurations/applications/projects that are deployed in a particular server/server cluster.
15. Support for “Store and forward” pub/sub message model. Subscribers should be allowed to poll messages from the store than the current push only model.
16. Ability to turn any BizTalk artifact like schema, map etc as a web service.
17. Native support for JMS.
Some more features for vNext:
1. Local settings for Time Zones when viewing at transactions and process instances in the web-portal.
2. File and FTP adapters should support file masks with optional File Archive option to disk for easy recovery. 6. Certificates and other shared objects loaded from a SAN disk than local disk in a multi server cluster deployment. The same applies to all run-time config files or binaries. (i.e, run time artifacts and config files will be on SAN disk and all machines read from this one copy)
3. Unified web-portal framework to view B2B transactions and Business processes (public and private) and drill down on status and key document data fields that are promoted. (Look at IBM WebSphere Partner Gateway or webMethods 7.1 Trading Networks Portal…..BizTalk is not even close to match their feature set in terms of B2B portal functionality)
4. TPA specific exception management capabilities. Trading Partner Groups (such as Suppliers, 3PL Providers, WHS Applications etc) with group level configurations for Business Processes, Messaging Configurations etc.
5. Powerful event triggers (like trigger a service based on a File create event with a matching file name if required etc without picking or processing that file) in addition to scheduled triggers. Check out TIBCO for reference.
6. Tool in BizTalk Admin to deploy BizTalk assemblies, certificates and components across multiple machines in a BizTalk group. For example if we have 4 nodes in the group and we set one node as the master where we push assemblies from SourceSafe and update Biztalk configurations in that node, is there a way to copy and deploy them from this node to others so they are in Sync automatically through some preset configuration? webMethods, TIBCO and other vendors had build-in tools where this process can be automated.
7. Provide a MMC plugin to BizTalk administrator to show the components deployed on each Node in the group (with assembly name, version, location, assembly type , date of deployment etc) and also provide a feature to deploy, add, delete objects directly and also provide options to compare versions between machines.
I want the mapper that Eddie Churchill demonstrated as the ‘New Sexy Mapper’ to be implemented……..
See http://channel9.msdn.com/showpost.aspx?postid=126990
DMR,
Great additions. I particularly like the ideas of central assembly management, better scheduling abilities, and playback.
Here’s a few pain points for me I’d like fixed:
1. Mapper – highlighted links when selecting a funtoid. This was there in 2002 – why was it removed?! Can save so much time and avoids introducing bugs. So simple yet so beneficial.
2. Mapper – ability to prompt for input file when testing the map – current 2004/2006 is a single file which is painful to change when testing a range of files – too many mouse clicks. What I would like is a way to specify whether you want to be prompted (as you were in 2002) or whether you would like to use the file you have specified in the map’s properties.
3. Automated builds – why aren’t more people complaining about how hard this is with BizTalk??! In my last job, we spent about a quarter of a project purely trying to get BizTalk projects building as part of the nightly builds. We finally got it (it was a hackfest), but it is ridiculously hard due to:
a) BizTalk projects not being supported by MSBuild (they have their own non standard structure)
b) Embedded assembly versions scattered throughout so many artefacts
The BizTalk project file should comply with the MSBuild requiements, and either these embedded versions should be stored elesewhere, or, if that is too hard/inflexible, there should atleast be build tasks provided by MS which perform the updates to maps/binding files/project files etc. etc. for you
4. Better ways to manage how specific partners interact for a particular transaction type. We wrote a system which performs a lot of logic which I think BizTalk should be able to do out of the box. E.g. when partnerA sends to partnerB for transaction type of Orders, the message should be routed via pipelineXYZ, then to orchestrationABC, then through mapAAA etc. I can’t see why the trading partner system can’t be greatly enhanced to support this type of architecture. Then things become more of a configuration change. If you have a large number of partners, creating individual ports for each etc. becomes an admin chore.
All great sugestions and i have three more simple ones.
For starters, the BizTalk Administration tool keeps asking me questions such as: Are you shure, You are about to. Why not add a checkbox “Ask this again?” in the very same dialog. For development that would be great.
Second of is when deploying for development purposes the reference dependencies should not be so hard. Especially if one is redeploying (build and deploy) the referenced projects in the same batch (solution). It is W E R Y anoying that one has to manually remove a lot of artifacts until you can deploy your actual change and test it.
Third of is Web Service and WCF messaging in conjunction to envelopes in BTS. The envelope handling in BTS requires a xsd:any element for the actual message and on the other hand Web Services and WCF requires a “specific message”. This is a P A I N. The XMLReceive pipeline does the work it should, but the XMLTransmit pipeline freaks out if you do not have an xsd:any element instead of a “specific message” within your envelope. So the lay down is: Use Web Services and WCF and not envelopes O R use envelopes and obsolete transports like file transport and FTP. Please Please Please do something to the XMLTransmit pipeline.
In the best of worlds one could have overridden the standard pipeline/functoid/adapter but, NO. Everyting is “sealed” and/or “friend”. Why is that?
Enough of critisism. BTS is great and can be even better.
One last comment so we will actually have four suggestions…
The development environment, Visual Studio X, always seems to have a dilemma regarding what’s changed or not. Does the BTS bits get read from source files, what’s open in an editor or what’s compiled latest? I can’t figure it out. So whenever a change does’nt produce the desired result: I begin with closing all windows, try. Build, try. Restart Visual Studio, try. Yeha, now it worked. This is very anoying especially for beginners who are feeling totaly lost. Let the beginers in so the product can spread.
When writing I came up with one more improvement (so five now).
That is in the administration and deployment area. It is just great with “Target environments” but why can’t i define and choose them when I’m developing. When I sit at my office (developing) i know the addresses to the test systems, when i do so, i also know the addresses to the customers test and production systems. I just love to enter them so that it is done without the hassle of export binding and “Import Resource”. That way the solution is 99.9% done when some one is deploying at the customers sites, the 0.01% is for my missprint of an ip address or a folder name, easily corrected on site.
If possible a comment shape that i need badly…
3 years ago and still nothing from Richard’s list in two versions of BizTalk. It is going to be annoing to send these feedbacks to MSFT without any response 😦
Sorry, Richard, for my sentiments.
Hi Richard,
I don’t want to use blog comments as a marketing tool. But I thought it will be relevant to mention about BizTalk 360 here. Since it addresses some of the problems you highlighted here. Examples
1. Web based administration tool. We have gone one level above and it’s a web based RIA application
2. Fine grained authorization, you can provide permission at application level using BizTalk 360, you can even control things like providing only read-only access.
3. Complete auditing/governance. Everything the user does on the environment is audited. Examples start/stop host instances, BizTalk artifacts like send ports/receive locations/orchestrations, terminating/resuming suspended instances.
There are more features than what’s listed here.
Regards,
Saravana