BizTalk Integration Summit in Seattle

Hi,
 
I had the luck this year to attend the BizTalk Integration Summit in Seattle, Washington.
This blog post is about how this went for me and what I will remember from it, based on the notes I took.

First of all we were about 300 people attending. We had Partners, Customers and off-course Microsofties. Somebody told me that we were 1/3 partners, 1/3 customers and 1/3 Microsoft.

From Belgium we only had a CodIt delegation and myself representing i8c. Our Dutch colleagues had more people present and I had the pleasure to get to know better the Motion10 people.

Also all the Microsoft Integration MVP’s were present, so I had the chance to meet some in person.

These are my notes from day 1:

BizTalk Integration Summit 2013 Agenda
Time	Session	Speaker
Thursday, November 21 (Day 1)
8:50am - 9:00am	Welcome to BizTalk Summit 2013	Mark Mortimore, Microsoft
9:00am - 10:30am	Delivering Modern Enterprise Applications	Scott Guthrie, Microsoft Vivek Dalvi, Microsoft
11:00am - 12:30pm	Using Windows Azure BizTalk Services to Accelerate Application Integrations	Karthik Bharathy  & Sameer Chabungbam, Microsoft
1:30pm - 2:30pm	Business Process Management	Alok Jain, Microsoft
2:30pm - 4:00pm	What's New in BizTalk Server 2013?	Jorge Gomez Basanta, Paul Larsen, & Mark Mortimore, Microsoft
4:30pm - 5:15pm	Best Practices for Enterprise Clinical Integration	Alan Scott, HCA

Delivering Modern Enterprise Applications

The trend today is: Mobility, Apps, Big data and cloud.
The challenge is how to connect and collaborate.

The Microsoft Could OS is the common platform to get this to work. It enables the creation of modern applications.
 
API Mgmt is new to the Cloud OS so we definitively have to look into that.
Mimeo.com (are live with the Windows Azure BizTalk Services preview)

We have Cloud EAI: in order to bridge the on-premise LOB’s and to integrate with SaaS solutions.
We also have Cloud EDI to extend the EAI with B2B functionality.

   I think Cloud EAI will always be used in conjunction with Cloud EDI.

At the summit they announced that Windows Azure BizTalk Services is now general available.
Customers should now no longer hesitate to start using it.

The road ahead will focus on on-premise (BizTalk Server) updates and cloud integration with quarterly updates. Microsoft stays committed to BizTalk Server.

The Key trends that were mentioned are:

• Experience is important since integration = complex. At Microsoft they have the experience J
• SaaS is getting more and more important for Cloud integration
• Hybrid applications are a reality.

Microsoft aims at making everything available out-of-the-box, so that we only need to concentrate on configuring the integration solution. We will have to wait and see this happen.
 
Still, everything will be extensible. 1st party experience for 3rd party ==> They expect a marketplace to be put in place, which will be a great opportunity for partners.

They announced that BPM will become part of WABS next to EAI en B2B.
A common management tooling infrastructure will be put in place.

Release:

• BizTalk server every 2 years
• R2 minor release (platform, key features) every 2 years

Meaning one year a major and the next year a minor release.

• CU every quarter
• WABS refresh every 3 months (at least)

Microsoft is really committed to let BizTalk and WABS evolve and finally merge into 1 product.

BizTalk Server 2013 R2 will be released in the first half of 2014. It will allow the product to align itself with:

• VS 2013
• SQL 2014
• WS 2012 R2

It will also contain Healthcare improvements.
 
One thing I learned from this summit is that Healthcare is a big thing for BizTalk (certainly in the US).

The WABS - GA also contains isolation and predictable performance. We now can have an isolated environment per subscription.
EDI is now available is all WABS editions!

WABS what is next:

• AAD
• Scheduled backup and restore capability
• EAI
• Adapter extensibility (this is the main focus)
• Light weight BizTalk adapter service
• Pull from queue and topics
• B2B
• EDIFACT
• Custom code in EDI bridge
• BPM
• Workflow on Azure
• Rules engine designer
• Process designer 

WABS Futures:

• BAM
• Partner directory
• Self service partner on-boarding
• Managed file transfer

 It seems that the future looks very bright for WABS.


Using Windows Azure BizTalk Services to Accelerate Application Integrations

GA: It also includes some extra performance counters
TPM migration tool to migrate BizTalk Server parties.
EDI in all editions, no longer limited to the Premium version!!

They also made it possible to have more than one user deploying WABS solutions per subscription. It is even so that per e-mail one now can manage several subscriptions.

Backup/restore tool.

Next:

• Edifact
• Pull service bus
• Edi bridge agreement association
• Scheduled backup/restore
• Workflow

They showed us an EDIFACT demo including batching. Now we only need to be allowed to participate in the TAP program to get hold of the bits and start doing WABS EDI projects in Belgium.
 

Business Process Management

This is new in the WABS. They only started developing it, so it will take another year before we see anything of this.

Model & Execute & Monitor business processes.
Those can be document, people or system (integration) centric.

Model:

• Process modeller
• Standard notation. They are thinking of using BPMN notation. I say they should use the BPMN notation!
• Automated business processes
• Define check-points and kpi's
• Repeatable and flexible

Key capabilities:

• BPMN notation
• Unified tooling
• Process, Rules, BAM
• EAI, B2B, BP
• Rich set of out-of-the-box activities
• Extensible, third party extensions
• .net workflow
• Support for custom code

Rule engine:

• Declarative
• Logic in Rules, process in workflow
• Key capabilities
• BizTalk BRE goes to the cloud
• Vocabulary and versioning included

What is missing: SCORECARDS!!! I think they should take this opportunity to implement this too.

Monitor:

• BAM
• KPI's and alerts
• Track messages and status
• Identify bottleneck in Real-time
• Message tracking for troubleshooting
• BAM Portal

Deploy to an artefact store.
A marketplace for the activities will become available. Third party activities will also be possible.
Management portal for deploy/config/start/stop of the business processes.
A BAM Portal for tracking business processes.

A TAP program will become available next year around the time of the summit I presume.

Well, BPM is coming to WABS, so we will no longer be restricted to WF or SharePoint for workflow solutions. WABS will include BPM built on top of WF.

What's New in BizTalk Server 2013?

 

Coming:

• SFTP: proxy support, SSO
• REST: JSON, Empty message support exposing REST
• Service Bus: more authentication options
• ESB toolkit: BRI revolver, SSO configuration
• Healthcare: MLLP 64 bit, dynamic adapter, better troubleshooting experience, HL7 2.6, freetext data type

Those will be included in the BizTalk Server 2013 R2 version.

Best Practices for Enterprise Clinical Integration

HCA presented their impressive BizTalk implementation.

As with any BizTalk installation they also learned from their mistakes J

I found the first day the most interesting part of the summit. A lot of new stuff announced and we know where they are heading with BizTalk and WABS.

For those of us investing in BizTalk and WABS the future looks bright.
 

 

These are my notes from day 2:

BizTalk Integration Summit 2013 Agenda
Time	Session	Speaker
Friday, November 22 (Day 2)
9:00am - 10:00am	One Microsoft Integration – Focus on Solutions	Mark Mortimore & Hector Rodriguez, Microsoft Eric Stott, Stott Creations
10:30am - 12:30pm	Deep dive on EAI	Sameer Chabungbam, Microsoft
10:30am - 11:00am	Applying Cloud Integration Patterns in the real world	Sam Vanhoutte, Impack Alliance
11:00am - 11:30am	Going live on Windows Azure BizTalk Services Preview - Tales from the bleeding edge	Kent Brown, Nimbo Dwight Goins, Nimbo
11:30am - 12:00pm	Management and Monitoring of your BizTalk Environments using BizTalk360	Saravana Kumar, BizTalk 360
12:00pm - 12:30pm	Running at Microsoft scale on Biztalk technologies	Eric Watson, Microsoft
1:30pm - 2:30pm	Learn about Service Bus	Ziv Rafalovich, Microsoft
1:30pm - 2:00pm	Exposing Enterprise Data Using APIs, BizTalk, and Windows Azure Service Bus to the Mobile World	Simon Barere, SOA Software
2:00pm - 2:30pm	BizTalk Server 2013 Performance Optimization and Lessons from the field	Ashish Bhambhani, Sandeep Kesiraju, & Todd Rivers, Microsoft
2:30pm - 3:00pm	SOA Governance for BizTalk Server  - Concepts and Practice	Andrew Slivker, Nevatech
2:30pm - 3:30pm	Deep dive on EDI / B2B	Karthik Bharathy, Microsoft
3:00pm - 3:30pm	Healthcare Integration with BizTalk Server and Corolar from Dapasoft	Ramraj Rajkumar, Dapasoft

One Microsoft Integration – Focus on Solutions

On-premise revenue counts as azure revenue. Interesting fact.

We need to connect to a business unit manager (list will become available), in order for us and them to do more business J
One integration stack: mobile, BizTalk, API, services bus, workflow.
Again focus on Healthcare industry.

"why" integration reduces friction

"opportunity" reduce waste, less defects, new customer, ..
"cloud" ROI 1.7x compared to on-perm, 40% less consulting J

Move to hybrid cloud: trend for 2015
One consistent hybrid platform: private, cloud
Experience = app + devices + data + security.
ESB helps Healthcare solutions to consolidate workflow and make them more generic.

Deep dive on EAI

Overview of the WABS EAI capabilities

• Validation
• Enrichment
• Transform: xslt based but in the future it will also support JSON and .NET data

.NET object model for transforms is not available for development (maybe in the future)

• Routing is the first match gets the message: that is why we have a sequence number in the routing list
• For PUB/SUB routing we need to use service bus topics
• BizTalk Service Explorer VS plugin

http://visualstudiogallery.msdn.microsoft.com/1f75a6a6-a54e-44eb-8b11-1b5ea8928754?SRC=VSIDE
It allows to debug a bridge via the plug-in. Wauw J

• Extension is possible via message inspectors, scripting functoids and custom wcf bindings

IMessageInspector interface, PipelineProperty attribute to include custom design properties
Custom WCF Binding

• In order to deploy custom assembly we need to set copy local to true!
• They are working on a framework to provide our own source adapter framework

Focus on the system NOT on the framework
 
PowerShell: Management samples available.
  

Learn about Service Bus

Loosely coupled apps

Message oriented middleware
Hybrid solutions

Push/event model: OnMessageOptions (cliënt.OnMessage) OnSession is coming

               Replaces the fact that we need to loop to receive messages.
Shared Access Signature authentication (SAS)
QueueClient Peek allows us to peek for messages.
Retry on transient errors MessagingFactory RetryPolicy
TopicDescription, QueueDescription EnablePartitioning

Service Bus 1.1 (on-premise) compatible with SDK 2.1

AMQP 1.0

Paolo's samples on hybrid solutions

Next:

• Reliability: SLA, dedicated resources
• Lazy writing (push to DB later) results in better throughput
• Device to cloud, the next big thing for Service Bus
• Smart mobility: 2018 30% smart cars
• Smart grid (power)
• Smart home
• Smart building
• Smart factory
• Smart logistics
• Intelligence Delivery via backing services

Q2 2014 Customer adoption program for Device to cloud

Deep dive on EDI / B2B

Dev max 10 agreements

Basic max 25 agreements
Standard max 250 agreements
Premium max 1000 agreements

TpmContext to create agreements @ runtime, all about using WCF Data Services

Windows Azure BizTalk Services Explained book coming…

Including the Xbox one supply chain solution @ Microsoft:

http://msdn.microsoft.com/en-us/library/dn521038(v=bts.10).aspx

 

I found the second day less interesting, still it made it even clearer that WABS is ready for the integration world and that we also should take the device to the cloud thing, seriously.

Finally, the sessions have been recorded and will become available as will be the presented slide decks, so stay tuned. 

Koen Van Oost

Business Integration Architect  @ i8c
 

WS-ReliableMessaging (WS-RM) between IBM and SAP back-end

More than 10 years ago, SAP started service enabling its applications. SAP implements a large number of web services as documented on SAP Enterprise Services Workplace.

SAP distinguishes 3 types of services:

• B2B: for exchanging data with business partners
• A2A: one-way, reliable services for application-to-application
• A2X: two-way request/respone for interaction with front-end application and use in composite applications

The A2X and A2A services are directly supported by SAP back-end systems. Their implementation is done in SAP's proprietary programming language ABAP, leveraging the SAP Proxy Framework.

The A2X request/response web services are standard SOAP web services. The asynchronous services communicate typically via SAP's integration solution, Process Integration or SAP PI. This reliable communication between the A2A web services and SAP PI is based on a SAP specific protocol (XMB, based on SOAP with Attachments and a number of custom SAP header fields).

But more interesting is the support of SAP's A2A web services for WS-ReliableMessaging. WS-RM is one of those many WS-* standards. WS-RM is a protocol for reliably delivering messages over SOAP. SAP supports versions 1.0 and 1.1 of WS-RM.

We recently connected WebSphere ESB 7.5 with SAP using WS-RM. The communication from SAP over WS-RM to the IBM ESB was rather trivial. WS-RM was well supported. One attention point was the configuration of a WS-Addressing action for each operation.

Going van WebSphere ESB to SAP was a bit trickier. Messages from WESB over WS-RM 1.1 to SAP always failed with the error message "Only the anonymous value for wsrm:AcksTo is supported". The WS-RM 1.1 uses the URI http://docs.oasis-open.org/ws-rx/wsrm/200608/anonymous.

When switching back from WS-RM 1.1 to 1.0, we finally got the communication from WESB to SAP working. The anonymous URI is http://w3.org/2005/08/addressing/anonymous is then used.

It seems that SAP has an error (bug) in its implementation of WS-RM 1.1, only accepting the URI http://w3.org/2005/08/addressing/anonymous to specify "anonymous".

SAP is also coming up with an alternative for WS-RM, called "SAP-RM" or "plain SOAP". It simply boils down to a HTTP POST with filtering of duplicate messages. This mechanism has been available in SAP's integration solution PI but is now also being supported in modern SAP back-end applications. We only tested this with soapUI, not yet from IBM WESB.

Author: Guy

Configuring Software AG webMethods WmMonitor archiving for Oracle RDBMS

Configuring archiving/deletion for SAG webMethods WmMonitor can be a bit confusing although it is documented in the webMethods Monitor User’s Guide that can be found on http://documentation.softwareag.com. In this article I try to clarify a number of aspects which may not always be clear from the documentation.

This article is based on a demo setup with an Oracle RDBMS, in which I created 2 oracle users:
- one for the operational tables which I called "WM9" (for simplicity, I created "All" tables in this schema using the SAG webMethods Database Component Configurator)
- one for the archive tables which I called "WM9ARCH" (all "Archive" tables have been created in this schema using the SAG webMethods Database Component Configurator)
Because this is a demo setup, I granted both users the DBA role in Oracle, but this is of course not a good practice and even acceptable in other environment.

The webMethods version I used is 9.0.1.  On my Integration Server (IS) I created a JDBC pool for the user WM9, which I linked to the functional aliases: DocumentHistory, ISCoreAudit, ISInternal, ProcessAudit and ProcessEngine. Also for CentralUsers I used the same user WM9, but the SAG installer creates a separate JDBC pool for this functional alias. Another JDBC pool was created for the user WM9ARCH that is linked to the Archiving functional alias.

On the home page of the WmMonitor package, I configured the archiving to use the Stored Procedures. In my experience this is much more stable and provides a better performance than the JDBC archiving that runs on the IS itself.

Next you need to run the service pub.monitor.archive:setOperationParameters in the package WmMonitor and specify the PROCESS_SCHEMA and ISCORE_SCHEMA input parameters. The value of these parameters should correspond with the name of the Oracle user whose schema contains the operational tables, in this case WM9. This service will insert the input parameters in the table OPERATION_PARAMETER in the schema of the user linked to the Archiving functional alias of the IS, in this case WM9ARCH. So make sure the Archiving functional alias of the IS is set and that the Archiving tables have been installed in the schema of the user used for the Archiving functional alias.

Another important configuration step is to grant the Oracle user used for the archiving schema, in this case WM9ARCH, CRUD permissions to the operational tables, in this case in the schema of user WM9. This can be done by executing the following Oracle SQL statement as user WM9:
"GRANT SELECT, INSERT, UPDATE, DELETE ON <table_name> TO WM9ARCH;"
Note that WM9ARCH is the name of the Oracle user used for the archiving schema in this particular case. You need to execute this statement for each table mentioned in appendix A of the webMethods Monitor User’s Guide. This is required, even if both Oracle users have the DBA role, because the stored procedures that perform the archiving use the EXECUTE IMMEDIATE construct to execute the SQL DML statements and the DBA role is not sufficient to grant all necessary permissions. Note that the stored procedures are executed using the Archiving JDBC pool of the IS, which pull the data from the operational Oracle schema to the archiving schema.

Once these configuration steps have been completed, you should be able to successfully archive or delete data from the webMethods Monitor tables.

Author: Kristof Lievens