A first look at Windows Azure Service Bus EAI and EDI Labs

End of last year, Microsoft released a CTP of its Windows Azure Service Bus EAI and EDI solution. Cloud is hot these days and we see more and more cloud based integration solutions appear (SnapLogic, IBM Cast Iron, Dell Boomi,...) or traditional EAI and EDI vendors that release a cloud offering (Tibco Silver, Software AG Cloud Ready,...). Although Microsoft is also a traditional EAI and EDI vendor with its well-known and widespread BizTalk product, the Windows Azure EAI and EDI solution doesn't seem to have anything in common with the existing BizTalk code base. On the contrary, over time Windows Azure Service Bus EAI and EDI will be the future of BizTalk according to this talk. BizTalk will be rearchitected on top of Windows Azure Service Bus EAI and EDI and other Azure components, so how far are we from this new situation?

It's clear that the Windows Azure Service Bus EAI and EDI CTP is functionaly not as complete as BizTalk yet, but I expect some of the missing pieces to be added or completed in the final release. One feature I'm worried about though is logging, tracking and monitoring. This is probably one of the best features in BizTalk, existing out of multiple SQL Server databases (Health and Activity tracking, MessageBox, BAM) that are supplied out-of-the-box. This currently seems to be missing completely in the design of the Windows Azure Service Bus EAI and EDI Labs. The Windows Azure CTP Management Portal portal doesn't provide any monitoring GUI and the SDK doesn't provide an API to log messages for tracking/resubmission or to log errors to allow monitoring. According to this thread on the forum, Windows Azure Service Bus EAI solutions are not yet intended for asynchronous scenarios as supported by BizTalk, but only for synchronous scenarios in which the client is repsonsible to capture and handle an exception in Azure. Nevertheless, even in a synchronous scenario monitoring and tracking are vital, for example when using the XML one-way bridge in the Azure Service Bus EAI and EDI CTP, where no repsonse is sent back to the client which means there is no way to determine if a message was successfully routed to the targetted backend system. The same remark applies for the receive endpoints of EDI agreements. These are implemented as black-boxes and can be used as input for a Service Bus bridge to translate from EDI to XML, but when something goes wrong with the EDI - or XML message in the chain of process steps, there is no way to determine the cause of the problem.

I tried to work around the missing logging, tracking and monitoring functionality by using Azure Service Bus Queues for debugging purposes, but I encountered the same monitoring issue here, even though queues are already available for quite some time. There is no graphical interface available in the Azure portal to have a peek in the Azure Service Bus queues and there don't seem to exist any tools yet that cover this functionality.

This lack of logging, tracking and monitoring functionality is not only painful during development, I also wonder how you will be able to troubleshoot issues in a production environment. It seems that all software vendors forget about the "design for operations" aspect and add this in later releases, so I'm looking forward on how Microsoft will tackle this in the future.

Author: Kristof Lievens