Q: Do you architect BizTalk solutions differently when you have a beefy, multi-server BizTalk environment vs. an undersized, resource-limited setup?
A: In a word, no. I’m a big believer in KISS (Keep It Simple Stupid) when architecting solutions and try to leverage as much of the in-built scaling capabilities as I can – even with a single server, you can separate a lot of the processing through dedicated Hosts if you build the solution properly (simple techniques such as queues and direct binding are easy to implement). If you’re developing that solution for a multi-server production set-up, then great, nothing more to do, just leverage the scale-out/scale-up capabilities. If you’re running on a 64-bit platform, even more bang for your buck.
I do however think that BizTalk is sometimes used in the wrong scenarios, such as large-volume ETL-style tasks (possibly because clients invest heavily in BizTalk and want to use it as extensively as possible) and we should be competent enough as BizTalk consultants/architects/developers to design solutions using the right tool for the job, even when the ‘right’ tool isn’t our favorite Microsoft integration platform….
I also think that architects need to keep an eye on the development side of things – I’ve lost count of the number of times I’ve been asked by a client to see why their BizTalk solution is running slowly, only to discover that the code was developed and QA’d against a data-set containing a couple of records and not production volume data. We really need to keep an eye of what our end goal is and QA with realistic data – I learnt the hard-way back in 2006 when I had to re-develop an orchestration-based scatter-gather pattern overnight because my code wasn’t up-to scratch when we put it into production!
Q: Where do you prefer to stick lookup/reference data for BizTalk solutions? Configuration files? SSO? Database? Somewhere else?
A: Over the last several years I think I’ve put config data everywhere – in the btsntsvc.exe.config file (a pain for making changes following go-live), SSO (after reading one of your blog posts in fact; it’s a neat solution, but should config data really go there?), in various SQL Server tables (again a pain because you need to write interfaces and they tend to be specific to that piece of config).
However about a year ago I discovered NoSQL and more recently RavenDb (www.ravendb.net) which I think has got amazing potential to provide a repository for lookup/reference data. With zero overhead in terms of table maintenance coupled with LINQ capabilities, its make a formidable offering in the config repo area, not just for BizTalk, but for any app requiring this functionality. I think that anyone wanting to introduce a config repository for their solution should take a look at NoSQL and RavenDb (although there are many other alternatives, I just like the ease of use and config of Raven).
Q: What are you working on besides BizTalk Server, and what sorts of problems are you solving?
A: Good question! I tend to have so many ideas for personal projects bouncing around my head at any one time that I struggle to stay focused long enough to deliver something (which is why I need one of these on my desk – http://read.bi/zUQYMO. I am however working on a couple of ideas:
The first one is an internet proxy device based around the PlugComputer (see http://www.plugcomputer.org/) – which is a great little ARM based device that runs various flavors of Linux – to help parents ‘manage’ their children’s internet use, the idea being that you plug this thing into your broadband router and all machines within your home network use it as the proxy, rather than installing yet more software on your PC/laptop. I’ve almost produced a Minimum Viable Product and I’ll be asking local parents to start to beta test it for me in the next week or so. Amazingly, I’m starting to see my regular websites come back much quicker than usual, partly because it is running the caching proxy Squid. This little project has re-introduced me to socket programming (something I haven’t done since my C days at University) and Linux (I used to be a Linux SysAdmin before I moved into BizTalk).
My second project is really getting up to speed on Azure which I think is an absolutely amazing solution, even better than Amazon’s offerings (dare I say that?), simply because you don’t have to worry about the infrastructure – develop and deploy the thing and it just works. So I can learn Azure properly, I’m writing a RosettaNet handler (similar to the BizTalk RosettaNet Adapter), however I hope that some of this stuff will come out of the great work being done by the Windows Azure Service Bus EAI & EDI Labs Team in a similar vein to the EDI functionality being delivered on top of Azure.
I also continue to maintain the BizTalk Message Archiving Pipeline Component (shameless plug: download a free trial at www.atomic-scope.com/download-trial/), supporting existing customers and delivering great functionality to small and large customers worldwide.
Q [stupid question]: I saw that an interesting new BizTalk blog was launched and its core focus is BizTalk Administration. While that’s a relatively broad topic, it still limits the number of areas you can cover. What are some hyper-specific blog themes that would really restrict your writing options? I’d suggest BizTalkConcatenateFunctoidTips.com, or CSharpWhileLoopTrivia.com. What about you?
A: I actually investigated BizTalkHotfixes.com a while back as a website dedicated to, well, BizTalk Hotfixes. At the time I was really struggling to find all of the BizTalk Hotfixes relevant to a particularly obscure customer problem and couldn’t find an authoritative list of hotfixes. This issue has gone away to a certain extent now that we have CU’s for the product, but I think the idea still has legs, especially around some of the more obscure adapters (see http://www.sharepointhotfixes.com/ for example) and it might be something to resurrect in the future if I ever get the time!
As for BizTalk Administration, it sounds like a narrow topic, but I think it’s just as important as the Dev side, especially when you think that the health of the underlying platform can make or break a solution. I also think admin specific content is also beneficial to the large number of SysAdmins who inherit a BizTalk platform once a solution goes live, simply because they are the ‘infrastructure guys’ without any formal or informal BizTalk training. I do quite a few health checks for clients where the underlying infrastructure hasn’t been maintained, causing major problems with backups, ESSO, clustering, massive data growth etc. The work produced by the BizTalk360 chaps is really helping in this area.