Web Service incompatibilities

SOAP Web Services have lost quite some of their popularity: too complex, incompatibilities etc.  My answer is always that 1) SOAP just adds a very simple envelope around the request and response messages and 2) SOAP does work fine when you stick to the rules (a copy of a slide I use in my training classes):

Just recently I had encountered 2 nice examples of SOAP incompatibilities.
 

Cookies and SOAP

While investigating the web services API of a cloud SAAS application, encountered another example how things should not be done.  First of all it was not "stateless" but required the use of a login and logout operation. With security not based on standard HTTP basic authentication or WS-Security, but a proprietary scheme:

  <urn:credential>

    <urn:companyId>company-id</urn:companyId>

    <urn:username>user-name</urn:username>

    <urn:password>password</urn:password>

  </urn:credential>

But then came the surprise: the login operation returns a session handle which is actually a cookie! The cookie is to be passed as an HTTP header in each subsequent web service.  Had seen many ways to make web service implementations incompatible, but this is one for the top 5!  Obviously most web service clients require some hack to pass this cookie along the SOAP request.
 

Doc/literal with 2 parts

A more subtle challenge came recently by at a customer: the IBM DataPower ESB refused to import the WSDL file an Oracle product.  The web service used the document/literal style and one of the operations had a request message consisting of 2 parts.  So who was wrong and who was right: IBM or Oracle?

SOAP went through some growing pains in the beginning. The initial idea was an RPC mechanism whereby an operation could have multiple parameters. These parameters are passed as multiple parts in a request and response message. But with a better understanding of XML and XML schema's, the world move to a model whereby XML documents were passed. Microsoft introduced the document/literal wrapped style whereby the root contains the name of the operation.

<soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope">

  <soap:Body>

    <OperationName>

      actual XML document...

    </operationName>

  <soap:Body>

</soap:Envelope

</soap:Envelope>

So my initial response was, document/literal web services should only have one part and Oracle is wrong. But a colleague pointed to the fact that Oracle would not implement web services that violate the standards. And indeed, the IBM article clearly explains that a document/literal web service can have multiple parts in a message.

The WS-I Basic Profile was an initiative to sharpen the rules and states: "R2201 A document-literal binding in a DESCRIPTION MUST, in each of its soapbind:body element(s), have at most one part listed in the parts attribute, if the parts attribute is specified.". So the Oracle web service is not WS-I basic compliant but does not violate the SOAP/WSDL specifications.

Again a situation where one has to go for workaround, this time in the DataPower ESB. Had IBM implemented the specs correctly and/or Oracle stuck to the widely accepted ways-of-working and the WS-I Basic profile, everything would have worked smoothly.

 

Author: Guy

Live from Software AG Processworld 2012: Day 2

On day 2 of ProcessWorld I attended some sessions that were focusing on the 9.0 release. The first one was about a general introduction to webMethods 9.0. It started with a short overview of the new products: webMethods cloudStreams, webMethods Pulse, Command Central, ActiveTransfer, etc. After that they dived into the new functions of the existing components like IntegrationServer, Broker, Optimize, etc. The most important ones are:

For integration server:

• Support for webservice reliable messaging, SFTP and JSON. 
• Worry-free updates with maintenance mode support. You can put the integration server in maintenance mode to perform updates. 
• Local development. 
• BigXML and service caching thanks to the implementation of TerraCotta. 
• WebMethods Command Central Integration. 

For Trading Networks: 

• Community management & partner on-boarding 
• Integration of the new webMethods ActiveTransfer product. 

For BPM: 

• Improved architecture by OSGI enablement of the my webMethods Server. 
• Event-enabled BPM processes can feed webMethods Pulse, Business Events & Mashzone. 
• BPMN 2.0 support for sub-processes & debugging 

Optimize:

• Improved usability & management of KPI data. 
• Enhanced dashboards with MashZone. 
• Improved reliability. 
• WebMethods Command Central Integration 

CentraSite: 

• New business UI for occasional & non-technical users 
• Easy keyword search 
• Event-based automatic promotion across life-cycle states 
• Simplified infrastructure 
• WebMethods Command Central Integration 

Nirvana messaging: 
Something I forgot to mention in my previous blog post is that Nirvana also supports priority messaging and message throttling for bandwidth limitations.
 

Important to mention are the following system requirements for the whole webMethods 9.0 suite.

• There will be ipV6 support 
• Only 64-bit operating systems will be supported 
• The standard JDK is Java 7 
• The ability to define checkpoints & the rollback of deployments  

Another new webMethods product that I didn’t mention yesterday is webMethods ActiveTransfer for Managed File Transfer. There are two major components: ActiveTransfer server & ActiveTransfer proxy. The ActiveTransfer server needs to be installed on an IntegrationServer in the trusted zone of your IT infrastructure. The server part allows you to:

• Configure and manage all server instances centrally. It allows you to define IP banning rules and/or filename or folder restrictions. 
• Manage users and their access privileges by prioritizing file transfers, restrict file transfer actions and inheritance of user settings from templates. 
• Perform post-processing and alerting by the definition of file transfer criteria based triggers. Based on these triggers IntegrationServer services can be invoked or native scripts can be called. 
• Perform scheduled and event-driven transfers 
• Accelerate file transfers in high latency environments by definition of multiple active transfer tunnels. This allows file transfers speed up to 20 times faster. 
• Suspend and resume file transfers by the definition of checkpoints 
• Provide centralized monitoring to keep track of all transfers.

 

The ActiveTransfer proxy server part runs on a reverse gateway IntegrationServer in the DMZ and communicates with third parties.

Author: Dimitri Van Kerckhoven

Live from Software AG Processworld 2012: Day 1

The day started with general keynotes from the CEO and CTO of Software AG. During these keynotes the so called “4 forces, shaping the future of IT” of Processworld 2012 were introduced:

• Big Data    
• Social/Collaboration 
• Cloud 
• Mobile

All of these 4 forces are integrated in the new ARIS 9.0 and webMethods 9.0 releases by a number of new products. The one that were addressed today are the following:

• ARIS connect: a complete new product focusing on the social and collaboration part. It is an addition to the existing ARIS product line. During the session a demo has been given showing how multiple persons can collaborate in an ARIS process. A person can invite someone else to collaborate in an ARIS process, meaning the person can put comments, can view the part he or she is responsible for and so on.
  At this moment there are different ARIS products, each of them having their own version. With ARIS 9.0 there will be only one ARIS client with version number 9.0 and extension packs in case specific features are wanted. 
• webMethods pulse: is focusing on the social and mobile part. This event-driven solution is based on the webMethods ESB and allows us to subscribe to business events and to get warned in real time on our mobile phone or tablet. 
• Cloud streams: is responsible for cloud integration. While the webMethods SalesForce adapter is only focusing on integration with SalesForce, cloud streams comes out of the box with a lot more SAAS connectors (salesforce, google, …). It has also advanced security, traffic analysis and SAAS governance on board.
  I discussed the comparison of the salesforce adapter with the cloudstream solution with a software AG expert at the cloud booth. The main advantage of cloudstreams is that it delivers a shorter time to market compared to the adapter. With cloudstreams they provide a framework that can easily adopt a new version of the API that is availaible in the cloud (cfr the updates of the salesforce partner WSDL).
  Cloudstreams can be installed as an integration server package. 
• Nirvana messaging: A java based messaging system for webMethods with the same functions as the broker + additional API's for javascript, iOS, android, ... Tests have shown that the nirvana messaging solution is up to 10 times faster than the broker (including guaranteed delivery). So there is a chance that it will replace the broker in a far future.
• Command Central allows us to administer almost all webMethods components via one central GUI. It allows administrators to: 
• Perform landscape management by stopping or starting, monitoring, perform license management, basic alerting of webmethods components. 
• Manage fixes by comparing fixes across servers and install fixes on the servers. 
• Perform central configuration across products, compare the configuration and provide access to the legacy UI. 
At this moment they are still working at the product, since the central configuration is still a work in progress. Although the GUI is one way to access command central, it also can be accessed via command line, service API and even a mobile API. For the command line you can give a command to perform a specific operation or XML scripts can be used to apply changes using the command line.

Command central provides a standard API, independent of the underlying runtime. This means you can configure for instance a JNDI connection in CC in one way, whether it needs to be applied on integration server of MWS.
In the future it also will be possible to install products using CC.
On top is this all you can define templates, manage substitution variables and apply these templates to target servers.  

Author: Dimitri Van Kerckhoven

Crossvista: ALM and concurrent development for webmethods (part 2)

Components

CrossVista as a whole contains the following parts:

• CVCM package: the core TEAM server product 
• CVCM agents: packages that need to be installed on every integration server TEAM needs to connect to. Will be done automatically in the background. 
• TeamVCS package: package for concurrent development, to be installed on every development integration server 
• Team API: API to access the team services 
• CvTunneler.jar: tunneling for webMethods reverse proxy 
• TeamVCS plugin for eclipse (webMethods Designer)

 

System Requirements

CrossVista can be run on the latest versions of the webMethods platform: 6.5, 7.1, 8.0 and 8.2. Many version control systems are supported out of the box: CVS, SubVersion, VSS, ClearCase, TFS, but you need to mention the version! Only a limited number of versions are supported and not following these requirements will make CrossVista possibly not work. Since the CVCM and TeamVCS packages contain java server pages, the WmTomcat package needs to be enabled on every integration server containing one of these packages.

Installation

Installation of CrossVista is very straightforward since the core products are nothing more than just packages that need to be deployed on an integration server:

• Install the CVCM package on one and just only integration server. This integration server will become the TEAM server. There are no recommendations whether this can be installed on an existing integration server, hosting other packages or whether a separate integration server needs to be taken for a good operation. 
• Install the TeamVCS package on every integration server that can be categorized as development machine. 
• Install the TeamVCS plugin for webMethods Designer in case you want to develop BPM or CAF projects. 
• Assign users to the TeamAdministrators and TeamUsers groups and ACLs in integration server. These groups and ACLs will be created when you install the CVCM package. 
• Give non administrators that need to access Team server or TeamVCS access to WmTomcat. By default only Administrators can access the WmTomcat package functionality. You can add them in the web.xml file under web/WEB-INF of the WmTomcat package.

 

CrossVista Terminology and workflow

Two import names need to be kept in mind before you start working with CrossVista:

• A TEAM project: a definition of all the constituents 
• A release: an instance of a project

As you can see you can make the link with webMethods deployer naming. You define a deployer project, where each project can have multiple builds.

In order to work correctly with CrossVista, the following workflow needs to be followed:

• Define a new TEAM project. In this project you define the constituents that form the project (packages, configuration files, sql files, …) 
• Upload the project in the version control system. Here a new release will be created and the release will be stored in the development repo 
• Promote the project. Promotion means moving the release from one repo to another using promotion rules. In these promotion rules you change the configuration and target servers to be conform to the new environment. Promotion does not mean you deploy the release! 
• Deploy the project. Here you select the release from the repo that corresponds to the physical environment on which the release will be deployed. 
• Analyze the project

 

Features overview and their strengths and weaknesses

• Automated release cleanup. This sounds promising, but the feature only allows cleanup after x days. This means that when you don’t make any new release during x days, all your releases will be deleted 
• One repo per environment. A good feature to keep separate releases per environment. You always need to be aware in which you are working. This feature is prone to mistakes. 
• Almost everything can be part of a release: 
• IS services 
•  Extended settings 
•  Adapter connections 
•  Schedulers 
•  Users, Groups, ACLs 
•  Broker clients 
•  TN, MWS, BPM resources 

• Dependency definition between projects. Very useful feature. You can define the type of dependency: latest release, a specific release or a range of releases. Team server detects the missing release and will automatically add the missing release. 
• Visual diff between releases. Probably the best feature in Team server. It allows you to compare two different releases visually for flow services, document types, java services and flat files. It also allows you to make a live compare between a release and live integration servers.  
• Create a new release based on an existing one. You can make a new release, which is a subset of an existing release or you can merge different releases. This seems to be a dangerous feature, because you need to be aware very well of what you are doing. A mistake can be made quickly. 
• Site definition and check whether sites are in sync. You can group multiple integration servers in a site and check whether this site is in sync with another one. By default integration server also provides this feature when they are running in a cluster. 
• Built in scheduled tasks. This feature doesn’t seem to be very useful. You can create scheduled tasks for automated upload, deploy, promotion, etc. The question is why would you do that? The TEAM server solution seems too complex to me to automate such things. 
• Easy to keep track of changes since release notes can be added 
• Out of the box support for many version control systems, but you need to be aware of the version you are working with. 
• Automated creation of revisions, triggered by the lock/unlock operation in developer. Doesn’t seem to be very useful since a good practice is to make a full release when you are finished. Can act as a backup mechanism. 
• Concurrent development. Nice feature in case you need it, but I haven’t found an environment so far where this feature has been missing. 
• Support for both shared development (one integration server) and distributed development. The TeamVCS feature seems to be very useful in case of distributed development, but not necessary when development is being done on one machine. In that case you can start making releases directly from within team server when development has been finished.

Author: Dimitri