i8c - Business Integration Architects

Troubleshooting a HTTPS TLSv1 handshake between Microsoft software and Webmethods 8

2012-02-27T14:27:00.000+01:00

The problem

Microsoft software (biztalk, wfetch, IE,…) all have a problem when performing a HTTPS TLSv1 handshake to Webmethods 8.0.1 running on a JVM 1.6 server. The problem manifests itself as an error on the client side (biztalk):
“The underlying connection was closed: An unexpected error occurred on a send.”
Note: IE will actually catch this error and retry using SSLv3 so to the end-user it seems that IE works but it actually exhibits the same behavior.
Server specifications:

Webmethods: windows 2003, JSE 1.664 bit, v8.0.1
Biztalk: windows 2008

Recognizing the error

The first step is recognizing the error on our system so we can identify it when trying to reproduce it.

Webmethods

First step is setting all the logging to “TRACE” for thewebmethodsserver logger. The server log/error log does not show any indication that the failed connection took place.

JVM

The JVM offers a few system attributes to enable security debugging:

java.security.debug
javax.net.debug

Setting “java.security.debug=all” in the server.bat (and running from CLI) produces a stupendous amount of logging at startup which made it ratherunusable (after 10minutes it wasn’t even at the first startup log you expect in server.log), especially because I couldn’t redirect the output of the CLI to a file (for unknown reasons)
Setting “javax.net.debug=all”(or ssl)in the server.bat actually results in no additional output whatsoever.This may point to a custom webmethods implementation for HTTPS.

Wireshark

We installed wireshark on the server to capture the actual traffic going back and forth.We needed some additional configuration to actually make heads and tails from the encrypted traffic (source:http://support.citrix.com/article/CTX116557)

Go to edit > preferences > protocols > SSL
Add <ip-of-webmethods-server>,<https-port-of-webmethods-server>,TCP,c:/path/to/private-key.pem
- E.g. 10.0.0.1,443,TCP,c:\my-private-key.pem
- Use “;” to separate multiple entires and make sure there are no spaces after each comma
- Or in newer versions of wireshark:

Fill in an SSL debug file (e.g.c:\ssl-debug.log)
Restart wireshark and the debug file you configured should show something like:
- ssl_init found host entry<configured-entry>
- ssl_init addr‘<ip>' port '<port>' filename '<name>' password(only for p12 file) '(null)'
- Private key imported: KeyID<id>..
- ssl_init private key file<path>successfully loaded
If the key could not be imported (wrong format or the likes) you will see an error in the log file

In our case the private key was in DER format so we converted it to PEM using openssl:
$ openssl rsa -in private-key.der -out private-key.pem -outform PEM–inperform DER
Now that we can capture the HTTPS traffic,we add a filter to only see the traffic from the client:
ip.src == <client-ip> || ip.dst == <client-ip>
You can also add a port filter if necessary:
tcp.port == <local-port>
You can see the followingtrafficin wiresharkwhen the client attempts to connect:

Reproducing the error

Now that we can recognize the error on our system, we need to reproduce the error so we can narrow down the problem.

Firefox

Firefox 9.0.1 will perform the entire conversation in SSLv3 which presents no problem to the server:

Internet Explorer

Internet explorer would seem to work from the client perspective as the HTTPS connection issuccessful;however wireshark tells us a slightly different story:

As you can see, it actually tries a TLSv1 connection which fails in the same way as the BizTalk one did, but then simply retries using SSLv3 which is successful.

WFetch

Microsoft provides a tool to test HTTPS connections: WFetch. If we tell it to connect to the server using TLSv1 (labeled TLS 3.1 because SSL 3.1 == TLS 1.0)we get an error as you can see:

Wireshark shows exactly the same pattern as the BizTalk failed connection.
If you toggle the connect option to “https” it will actually work on windows XP because the initial handshake uses SSLv2:

If you toggle it to “https” on windows 2008(and presumably vista/7)it will try the initial handshake with TLSv1 and fail(note that SSLv2 is disabledin windows 2008 due to security concerns).

OpenSSL

With OpenSSL you can also emulate a client (using the s_client option) to test a server:
$openssl s_client -verify 6 -state -msg -showcerts -connect 10.32.54.5:4483 > ssl-output.txt
The output in “ssl-output.txt” shows us that this command uses SSL 2.0 for the initial handshakeand thenproceeds in TLSv1:

However you can force openssl to use tls1:
$openssl s_client -verify 6 -state -msg -showcerts-tls1-connect 10.32.54.5:4483 > ssl-output.txt
Surprisingly, the TLSv1 handshake works for OpenSSL:

The server shows us this:

Java Client

The following code will allow you tomake a rather low level SSL socket connection in java where you can choose the supported cypher suites and protocols:
publicstaticvoidmain(String...args)throwsUnknownHostException, IOException {
SSLSocketFactory factory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket socket = (SSLSocket) factory.createSocket("10.32.54.5", 4483);
socket.setEnabledProtocols(newString [] {"TLSv1"});
socket.setEnabledCipherSuites(newString [] {"TLS_RSA_WITH_AES_128_CBC_SHA"});
socket.startHandshake();

Writer writer =newOutputStreamWriter(socket.getOutputStream());
PrintWriter printer =newPrintWriter(writer);
printer.println("GET /invoke/wm.server:ping HTTP/1.1");
printer.println();
printer.flush();

System.out.println(StreamUtils.toString(socket.getInputStream()));

writer.close();
}
There does not seem to be a way to enable or set specific TLS extensionsfor the requestthough.

Analysis of the error

We surmise from the above that:

It is not just a bug in TLSv1 handshakes as OpenSSL works
All Microsoft based software seems to be impacted suggesting they use a central configuration which leads to this error

Analysing the “unexpected error” in wireshark shows us this:

TheRFC (http://www.ietf.org/rfc/rfc2246.txt)does not really shed any more light on the “unexpected message” error.

Comparing the request to the successful OpenSSL TLSv1 handshake, we notice that OpenSSL sends along 27 suggested cipher suites (a selection of them in the screenshot):

As opposed to theMicrosoftclientswhichsend a much shorter list:

None of these suits ismentioned in the JSE6 documentationhttp://docs.oracle.com/javase/6/docs/technotes/guides/security/SunProviders.html#SupportedCipherSuitesbut it may be that there are functional aliases that should work.

Fixing the error

Enable Renegotiation

More about the renegotiation bug can be found in the sources at the end of this document, but hereis a very short summary:

A renegotiation security error was discovered recently (http://www.oracle.com/technetwork/java/javase/documentation/tlsreadme2-176330.html) which triggered an update to both the JVM and to Webmethods to disable renegotiation by default, you can reenable them using the following system properties:
- JVM:sun.security.ssl.allowUnsafeRenegotiation=true
- Webmethods:
  - watt.ssl.iaik.clientAllowUnboundRenegotiate=true
  - watt.ssl.iaik.serverAllowUnboundRenegotiate=true
Renegotiation has to be enabled on both client and server to work

We enabled all renegotiation parameters on the server to no avail, the BizTalk client still failed.While we weren’t able to check if the client had renegotiation enabled, their TLS extensions would seem to indicate that they do:

However, even if enabling renegotiation was the fix, itdoes not negate the inherent security risk in doing so in a production environment.

Upgrade the JVM

Java 7 was released a little while ago and it contains a number of improvements for TLS communication:http://docs.oracle.com/javase/7/docs/technotes/guides/security/enhancements7.html
However Webmethods does not support it so we can’t run it on a production system.Additionally it seems that Webmethods bypasses the default implementation anyway so it may not impact the error.

Add support for the windows ciphers to the existing JVM

Just in case the lack of a common cipher is indeed the problem, we tried adding another security provider: bouncy castle. We addeditto the security file (jvm\w64_160\jre\lib\security\java.security):
#
# List of providers and their preference orders (see above):
#
security.provider.1=sun.security.provider.Sun
security.provider.2=sun.security.rsa.SunRsaSign
security.provider.3=com.sun.net.ssl.internal.ssl.Provider
security.provider.4=com.sun.crypto.provider.SunJCE
security.provider.5=sun.security.jgss.SunProvider
security.provider.6=com.sun.security.sasl.Provider
security.provider.7=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.8=sun.security.smartcardio.SunPCSC
security.provider.9=org.bouncycastle.jce.provider.BouncyCastleProvider
And added it to the classpath in the server.bat filein the hopes that it would contain a working cipher. No luck there.

Add support for the JVM ciphers to windows

In the same vain, we looked into adding ciphers to the windows machine (as a client-side fix).There seems to be no easy way to add a cipher to windows though the process in C is explained here:http://msdn.microsoft.com/en-us/library/windows/desktop/bb870930%28v=vs.85%29.aspx

Install latest core patch in Webmethods

Although it would seem like the most logical step, installing a core patch for webmethods is also the most invasive solution which is why it is not at the top of our list.
Empower contains (at least) one reference to a handshake failure:https://empower.softwareag.com/sl24sec/SecuredServices/KCFullTextASP/viewing/view.asp?KEY=081375-10412598&DSN=PIVOTAL&DST=TCD&HL=1&QUERY=tls|1.0|handshake&SessionID=608289137
It actually points to a problemfor integration server 7.1.3 instead of 8.0.1, but it advises the installation of the latest core fix to solve the problem.
Installation of core fix 20 fixes the problem though we do still do not know exactly which part of the client handshake triggered the error.

Sources

An incomplete list of sources:

Author: Alexander Verbruggen

Invoking stored procedure from DataPower ESB

2012-02-15T12:41:00.000+01:00

On a recent project, a few stored procedures had to be exposed as web services. Actually, the stored procedures already took an XML message as input and returned an XML message as output.

Initial plan was to write some .Net components to expose the stored procedures in the SQL Server 2008 database as web services. The web services themselves would then be exposed via a DataPower ESB.

But with the DataPower ESB already in place and being capable of directly invoking the stored procedure (see also), the .Net components and underlying Windows servers could be avoided.

A WSDL was created based on the XML input and output messages. That WSDL was the starting point to create a Web Service Proxy (WSP) on DataPower. Within a rule of the WSP policy, a Transform step (XSLT) is executed that will invoke the stored procedure using a DataPower specific extension function.

A standard Web Service Proxy will invoke a back-end web service. The rule of the Web Service Proxy immediately returns - wihout invoking any back-end web service - by setting the variable service/mpgw/skip-backside to 1.

Below the XSLT used to invoke the stored procedure using the dp:sql-execute function. We obtain the XML part from the soap:Body and pass that on to the stored procedure. The respone XML is wrapped in a SOAP Envelope.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:dp="http://www.datapower.com/extensions"
xmlns:xxx="http://xxx.com/"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
extension-element-prefixes="dp" >
<xsl:template match="/">
    <xsl:variable name="statement">exec stored_proc ?</xsl:variable>
    <xsl:variable name="result">
      <dp:sql-execute source="'DS_DB'" statement="$statement" >
        <arguments>
          <argument name="request_id" type="SQL_XML" mode="INPUT">
            <xsl:copy-of
             select="./soap:Envelope/soap:Body/xxx:Operation" />
          </argument>
        </arguments>
      </dp:sql-execute>
    </xsl:variable>
    <soap:Envelope>
      <soap:Body>
        <xsl:copy-of select="dp:parse($result)" />
      </soap:Body>
    </soap:Envelope>
</xsl:template>
</xsl:stylesheet>
Note on the last few lines: the XML response from the stored procedure was contained in a CDATA segment. With the utility function dp:parse() the XML content within the CDATA could be obtained.

Conclusion: one of the great features of Integration tools (ESB, EAI) is their built-in database connectivity. The database adapter of an ESB allows querying, updating and invoking of stored procedures. Database adapters typically also support polling (staging) tables for new or modified records.

Author: Guy

Running Oracle Weblogic Server on Windows: a shutdown class

2012-01-26T18:41:00.000+01:00

Overview

Running Oracle Weblogic server on Windows can be done in two ways:

You can start Oracle Weblogic Server from the command line. This is good for test purposes but not very likely to be used in a production environment.
You can install Oracle Weblogic Server to run as a windows service using the java wrapper beasvc.exe (or beasvcX64.exe on windows 64-bit)

The problem with the last option is that, if you stop the service, the Java Virtual Machine (and thus the Oracle Weblogic Server) simply gets killed. This can result in corrupt files, database connections that aren’t closed or even transactions that are started but won’t be stopped. A possible workaround is to stop each Weblogic server in the domain from the admin console. This not only time consuming but also not the preferred way of working.

In order to provide a proper shutdown, the beasvc.exe has an option called “stopclass”. It specifies a java class with a stop method that will be invoked at the moment the windows service gets stopped. Although you can write any kind of java code in the shutdown class, we will focus on writing code for a decent shutdown for both admin servers and managed servers.

The shutdown class works as follows:

It sends a request over HTTP to the Admin server
The Admin server sends the shutdown request to the corresponding server.

This means that the admin server sends a request to itself in case of a shutdown. The shutdown request for a managed server passes first via the admin server back to the managed server.

Configuration

First of all the beasvc.exe service wrapper needs to be aware of the shutdown class. This can be done by adding the -stopclass option (for example: -stopclass: be.i8c.systemtools.weblogic.ServerStopper)

Secondly, two options need to be set on each server in the domain:

You have to enable tunneling. You do this by ticking “Enable Tunneling” under Domain > Environment > Servers > “Server” > Protocols > General

You have to ignore http sessions during shutdown, otherwise the server keeps on waiting for ever until no more http sessions are active. Enable the option “Ignore Sessions During Shutdown” under Domain > Environment > Servers > “Server” > Control > Start/Stop

Code

package be.i8c.systemtools.weblogic;

import java.io.File;

import java.net.URL;

import java.net.URLClassLoader;

import java.util.Date;

import java.util.Iterator;

import java.util.Properties;

import java.util.Set;

import javax.naming.Context;

import weblogic.jndi.Environment;

import weblogic.management.MBeanHome;

import weblogic.management.runtime.ServerRuntimeMBean;

import weblogic.management.runtime.ServerStates;

public class ServerStopper

{

public static void stop()

{

Properties prop = System.getProperties();

try

{

MBeanHome home = null;

//url of the Admin server

String url = "";

if (prop.getProperty("weblogic.management.server") != null)

{

// this property will be set automatically on a managed server to indicate the location of the admin server

url = prop.getProperty("weblogic.management.server");

}

else

{

// the admin.url is a property to indicate the location of the admin server on the admin server. It refers to itself, but is necessary to perform a proper shutdown. You can add it manually as a –D property when you install the server as a windows service. The name of the property can be any random name.

url = prop.getProperty("admin.url");

}

// username and password to perform the shutdown operation

username = "username";

password = "password";

ServerRuntimeMBean serverRuntime = null;

Set mbeanSet = null;

Iterator mbeanIterator = null;

// Set ContextClassloader to prevent assertions

URL[] urls = { new File("/").toURL() };

Thread.currentThread().setContextClassLoader(new URLClassLoader(urls));

Environment env = new Environment();

env.setProviderUrl(url);

env.setSecurityPrincipal(username);

env.setSecurityCredentials(password);

Context ctx = env.getInitialContext();

home = (MBeanHome)ctx.lookup("weblogic.management.adminhome");

mbeanSet = home.getMBeansByType("ServerRuntime");

mbeanIterator = mbeanSet.iterator();

while(mbeanIterator.hasNext())

{

serverRuntime = (ServerRuntimeMBean)mbeanIterator.next();

try

{

// we have to add this check, since the iterator will list all the servers, both admin and managed. If we don’t add this check a shutdown of all the servers will be performed in case we only stop one server

if (serverRuntime.getName().equalsIgnoreCase(prop.getProperty("weblogic.Name")))

{

if(serverRuntime.getState().equals(ServerStates.RUNNING))

{

// we perform a normal shut down in case the server is in the RUNNING state

serverRuntime.shutdown();

}

else

{

//This is the same as a forced shutdown, but not the same as killing the JVM. We don’t do a normal shutdown in case the server is not in the RUNNING state.

System.exit(10);

}

catch (Exception e)

{

// You can do nothing over here. In case the shutdown call will fail, windows will wait for the service to end, but that won’t happen. It will kill the service anyway after a while.

}

catch (Exception e)

{

// We get here in case the admin server couldn’t be reached

System.exit(10);

}

Author: Dimitri Van Kerckhoven

How to: automatically remove LOCKFILE on server start (WINDOWS)

2012-01-20T10:01:00.000+01:00

Have you ever had that annoying issue when your server started up and the LOCKFILE still existed in the webMethods folder?

I’ve created a small script for windows which tackles this issue.

set jobname=softwareAGwebMethodsIntegrationServer_8.0

set location=C:\SoftwareAG\IntegrationServer\LOCKFILE

for /F "tokens=3 delims=: " %%H in ('sc query "%jobname%" ^| findstr " STATE"') do (

if /I "%%H" NEQ "RUNNING" (

IF EXIST %location% GOTO REMOVEFILEANDSTART

)

:REMOVEFILEANDSTART

del %location%

net start "%jobname%"

Variable jobname is your servicename which you set to start automatic

To find this name open Services and double click the job name

The name entered behind Service name is the name you need to enter for this variable.

Variable “location” is the location of the LOCKFILE, enter the full path.

Explanation of the code :

set jobname=softwareAGwebMethodsIntegrationServer_8.0

set location=C:\SoftwareAG\IntegrationServer\LOCKFILE

Declaration of the variables.

for /F "tokens=3 delims=: " %%H in ('sc query "%jobname%" ^| findstr " STATE"') do (

if /I "%%H" NEQ "RUNNING" (

IF EXIST %location% GOTO REMOVEFILEANDSTART

)

Loop over the result of sc query "%jobname%" and search for the word STATE

If the result of STATE is different from RUNNING then execute function REMOVEFILEANDSTART

:REMOVEFILEANDSTART

del %location%

net start "%jobname%"

Delete the LOCKFILE and start the service.

To use this script, create a scheduled task which is triggered on server startup.

Author: Jeroen W.

Writing a Mule JMX Agent

2012-01-15T19:04:00.000+01:00

A client of ours uses Mule ESB mainly as a mediator component that will throttle, translate and call external parties. Some of these parties are however not so reliable when it comes to response times. There were several utilities in use to check what the response time was of a specific third party. All of these mechanisms could only provide a rough estimate however. Since Mule was the system executing the call to the third party, we found that these kind of statistics should come out of Mule instead. Next question was: how to maintain these and get them out of Mule? To expose statistics, JMX seems a logical choice.

Mule ESB comes with a number of JMX Agents: http://www.mulesoft.org/documentation/display/MULE3USER/JMX+Management. These give some great insight in the system. At our client this info is already being used to display statistics in Zabbix, not only from Mule but also from HornetQ JMS. Zabbix is used to monitor the Average service execution times, JMS Queue depth, memory usage, CPU usage and so on.
These agents however do not expose the information that we want so we we need to write our own.

But first we need to collect the statistics For this we created a statistics object that will keep track of the number of calls, minimum, maximum and average duration and also the average of the last 500 calls. This last metric allows us to have an average which does show peaks over time. To capture the call and add the call info to our statistics object we used AOP:

@Around("execThirdPartyCall")
public Object execute(ProceedingJoinPoint pjp) throws Throwable{
   long timeBefore = System.currentTimeMillis();
   Object result = pjp.proceed();
   try{
      getStatistics(pjp).addMuleCallInfo(
         System.currentTimeMillis() - timeBefore);
   }catch(Exception e){
      LOGGER.error("Error adding MuleCallInfo to Statistics Object.",e);
   }
   return result;
}

This statistics object than keeps all this info but does not calculate all the metrics: min, max, avg, avg500. Because adding the call info is on the call stack, we want to keep these methods as short as possible. The actual calculation is done when the info is requested via an MBean.

So now we come to the main part of this post: how to create an MBean and register it to the Mule MBeanServer. I've worked with Spring and MBeans before and I do like the annotation driven mechanisms that Spring offers for auto registrating your MBeans.
Mule however does not use annotations but a method that is more reflection based. The main class in Mule that provides this logic is org.mule.module.management.agent.ClassloaderSwitchingMBeanWrapper. This class needs an interface for introspection and a concrete class in order to create an MBean instance.

The first thing you do is create an interface that will define the MBean attributes and methods. In the example there are 4 read-only attributes. The JMX_PREFIX will be used later on to define the JMX objectName:

public interface StatisticsMBean {
   String DEFAULT_JMX_PREFIX = "type=Thirdparty.Statistics,name=";
   long getHttpCallLast500Average();
   long getHttpCallLast500Minimum();
   long getHttpCallLast500Maximum();
   int getTotalNumberOfCalls();
}

Second, you need to define an implementation of this interface:

public final class StatisticsService implements StatisticsMBean {
    private Statistics Statistics;

    public StatisticsService(Statistics statistics) {
        this.statistics = statistics;
    }
    @Override
    public long getHttpCallLast500Average() {
        return statistics.getAvgLast500HttpCall();
    }

    @Override
    public long getHttpCallLast500Minimum() {
        return statistics.getMinLast500HttpCall();
    }

    @Override
    public long getHttpCallLast500Maximum() {
        return statistics.getMaxLast500HttpCall();
    }

    @Override
    public int getTotalNumberOfCalls() {
        return statistics.getTotalCallsMuleCall();
    }
}

Third you need to create the actual agent. A few things to note about the code below:

the fields are missing and also some methods that are left empty anyway
only the registerMbean part is in here, if you have some fancy hot deploy setup, you'll need to provide an unregisterMBean part
a MuleContextListener is used to make sure that Spring has finished initializing before doing any work
the Statistics object below is an Enum, because we keep statistics for multiple third parties

public final class StatisticsAgent extends AbstractAgent {

   @Override
   public void initialise() throws InitialisationException {
      if (initialized.get()) {
         return;
      }
      //get mbeanserver
      if (mBeanServer == null) {
         mBeanServer = ManagementFactory.getPlatformMBeanServer();
      }
      if (mBeanServer == null) {
         throw new InitialisationException(
             ManagementMessages.cannotLocateOrCreateServer(), this);
      }
      try {
         // We need to register all the services once the server has initialised
         muleContext.registerListener(new MuleContextStartedListener());
      } catch (NotificationException e) {
         throw new InitialisationException(e, this);
      }
      initialized.compareAndSet(false, true);
   }

   protected class MuleContextStartedListener implements
       MuleContextNotificationListener {
      public void onNotification(MuleContextNotification notification) {
         if (notification.getAction() == MuleContextNotification.CONTEXT_STARTED) {
            try {
               registerMBeans();
            } catch (Exception e) {
               throw new MuleRuntimeException(
                   CoreMessages.objectFailedToInitialise("MBeans"), e);
            }
         }
      }
   }

   private void registerMBeans() throws MalformedObjectNameException, 
      NotCompliantMBeanException, InstanceAlreadyExistsException, MBeanRegistrationException {
      Statistics[] statisticsAr = Statistics.values();
      for (Statistics statistics : statisticsAr) {
         ObjectName on = jmxSupport.getObjectName(
                           String.format("%s:%s",
                           jmxSupport.getDomainName(muleContext, false),
                           StatisticsMBean.DEFAULT_JMX_PREFIX + statistics.name())
                          );
         StatisticsService statisticsService = new StatisticsService(statistics);
         ClassloaderSwitchingMBeanWrapper mBean = new ClassloaderSwitchingMBeanWrapper(
                           statisticsService,
                           StatisticsMBean.class, 
                           muleContext.getExecutionClassLoader()
                           );
         logger.debug("Registering StatisticsAgent with name: " + on);
         mBeanServer.registerMBean(mBean, on);
      }
   }
}

As a last step you need to add your agent to the Mule configuration as follows:

<mule>
   <custom-agent class="management.StatisticsAgent" name="statistics-agent"/>
</mule>

And that's it, once you start up Mule, you will see your custom agent appearing and your MBeans are available via JMX.

That is how our operations guys can now provide a nice screen with the call durations over time:

Author: Jeroen Verellen

XSLT transformations and smooks driven CSV parsing in JBoss ESB

2012-01-10T18:54:00.000+01:00

In this last entry that completes the video series of our JBoss ESB introduction, we explore XSLT transformations and basic smooks parsing.
First up is the XSLT transformation:

Link: http://www.youtube.com/watch?v=vkd12IHJMGY

Parsing flat file into usable XML formats is a rather common requirement for integration platforms. Out of the box JBoss ESB comes with smooks.
The following video shows you how to parse a CSV file using smooks.

Link: http://www.youtube.com/watch?v=Rme4Fv-eqUo

Author: Alexander

webMethods Process performance analyzing

2012-01-03T18:14:00.000+01:00

We all get in touch with different processes on daily basis. Most of the times there are tools available to monitor if a process ran successfully. But in many cases the performance of these processes aren’t measured before they are deployed to the production environment.

We noticed the same problem with some of our customers. There were processes that should deal with a large amount of traffic. So it was critical that the performance was as good as possible.

How did we get started?

We recognized different steps in the process

Validation
Xslt transformations
Database transactions
Queuing
…

We created SQL scripts to measure the time these process steps ran

Based on custom audit data from the customer
Based on the webMethods audit tables (service execution)

We analyzed the results with our custom excel template

Performance Analyze:

We wanted to see if there were bottlenecks in our process and if the load had any effect on the process.

We noticed that all the steps ran a bit longer on a heavy loaded server than on a dedicated one. This seemed to be normal. Abnormal behavior would be when a server started to freeze and the more instances ran, the longer it would take. The curve you can see below is a normal curve. The average runtime doesn’t start to be longer and longer over time.

We also noticed that there wasn’t a normal curve in the queuing step. After some debugging we noticed that though the process was async.

The webMethods triggers were defined as serial. The processing curve improved after changing these triggers.

Author: Pieter Van de Broeck

Dealing with huge BizTalk DTA Databases

2011-12-27T10:29:00.000+01:00

Unexpectedly, it can sometimes occur that you notice that your BizTalk DTA database has grown from a small tracking database to a huge database taking up all disk space on your server. I had the same issue at a client, and have noticed that this can have quite some consequences: BizTalk server stopped processing messages, no backups could be taken anymore from the databases, etc. Not really something you want to happen to a critical environment.

Therefore following little guideline that you can use as a precaution, as well as a fix while dealing with a huge DTA database.

1. Make sure you have enough disk space

The DTA database can grow quite large from one moment to another, so its best to take into account a quite large disk where the database will be stored. A DTA database stays, according to Microsoft guidelines, healthy in size until 15GB. Everything above 15GB is considered problematic and needs to be dealt with.

When adding the numbers in terms of size of the DTA db together with the other BizTalk databases, make sure you have around 30GB of disk space allocated for the databases itself. When storing the backups of the databases also on the same disk, take at least 40GB, but take into account that this is the absolute minimum!

2. Enable the DTA Purge and Archive job

The DTA Purge and Archive job in SQL Server will clear completed and failed messages after a certain given time. We make a separation here between 2 situations: a situation where there is no problem yet with the database and one where we’re struggling with a huge DTA database and disk space issues:

No db problem, just precautionary

By default, the DTA Purge and Archive job will call the BackupAndPurge stored procedure. This call to the stored procedure will take some parameters:

exec dtasp_BackupAndPurgeTrackingDatabase
1, --@nLiveHours tinyint, --Any completed instance older than the live hours +live days
0, --@nLiveDays tinyint = 0, --will be deleted along with all associated data
1, --@nHardDeleteDays tinyint = 0, --all data older than this will be deleted.
'[path to the backuplocation]', --@nvcFolder nvarchar(1024) = null, --folder for backup files
null, --@nvcValidatingServer sysname = null,
0 --@fForceBackup int = 0 –

The first 3 parameters indicate the following: Amount of hours that a completed instance will be kept in the database; Amount of days that a completed instance will be kept in the database; Amount of days that a failed instance will be kept in the database.

The first 2 (both for completed instances) will be added up, so you can configure to keep completed instances for example for 2 days and 3 hours. Everything after this time will be removed.

DB problem, disk space issues

When struggling with a huge DTA db and disk space issues, the above stored procedure which is called might give problems, because the backup of the db can’t be made anymore since there is no more space left on the disk to put the backup on.

When doing this, just to be better safe than sorry, stop all BizTalk services while this script is running and enable them again after step 3.

In this case, we need to change the DTA Purge and Archive job. Instead of calling the PurgeAndArchive procedure, we will call the Purge procedure, without having to deal with a backup of the database.

Replace the original procedure call in the step of the DTA Purge and Archive job by this:

declare @dtLastBackup datetime
set @dtLastBackup = GetUTCDate()
exec dtasp_PurgeTrackingDatabase 1, 0, 1, @dtLastBackup

And after having done this, run the job. BEWARE! This can take quite some time to finish if this script hasn’t run for quite some time. Give it some time to finish (in my case, it took about 3min to finish and it wasn’t that long since that script had run).

3. Shrink the database

This is a step that is often forgotten, but after doing the Purge of the database, we’re not finished yet. The purge will clear the rows that were filled in the DTA db, but will not remove the empty rows itself. Therefore, you might see a little change in size, but not the big change that we’re looking for.

What really matters here is shrinking the DTA database. This will remove all empty rows and elements in the database which are not in use anymore. This will free up a significant amount of space and shrink the database significantly.

Just right click on the BizTalkDTAdb and choose Tasks – Shrink – Database as pictured below.

You will be presented with a window where you can see how much space exactly will be cleared. In my case here, only 158MB will be cleared since I’ve already optimized my DB.

Just click OK and let it do its work. BEWARE again: also this can take a lot of time. In my optimization scenario, I had 2,5GB that would be cleaned and took me up to 10 minutes to execute, so be patient, it will be worth the wait.

After this is done, the window will just disappear.

In case you’ve disabled your BizTalk services/Host Instances, you can restart them here again.

4. Finish it up

Just do a quick check on the size of your database. Normally, you should see a huge difference in db size, depending how much affected your database was of course.

When having any problems or questions, shoot in the comments.

Author: Andrew De Bruyne

Introduction to Android App development (Devoxx 2011)

2011-12-10T10:00:00.000+01:00

This session started with an explanation of what android is and how it was built.

Android is a platform developed by open handset alliance.

The platform is built in different layers.

Lowest layer based on linux used for hardware
Top of linux libraries and android runtime
Application framework

As you all know Android development is done in JAVA, but underneath this layer lies a code which is called “dex”.
When you want to run your app on an android phone, the class files will be converted into a dex file so it can be used on any android phone (depending on the apps requirements).

For each app we can reuse existing functionality and therefor a subset of java se is available and an android specific api.

Taking this into account we can do amazing things, but remember it’s just a phone.
So we cannot develop apps which need a massive amount of memory or space as this is limited on a phone. We also need to think about garbage collection.

How to start developing?

There are plugins for eclipse and emulators for testing. (http://www.android.com/ for more information on how to install the plugin)

How does an android project look like?

Some important files

Mail.xml : Describes content what to see

R.java : is the class file which is automatically generated when compiling your project. This file zwill be converted into dex code so your app can be used on an android device.

Android manifest file : this file cannot be better described as the property of the app. This file is read on installation, can prompt for access to allow connections to the internet, current location ,…
This file is also used on the android market to filter apps.

strings.xml : when displaying static fields, you create a variable here and refer to this var that will be used to display. (key-value pair)

When creating a new project, you have to create an Activity.java file. This is the root file which is called when the app is opened.

Layouts: you can create specific layouts for portrait or landscape mode.

Lifecycle of apps

Android has a set of rules to kill apps to free-up memory. So when developing apps we need to take in mind so “save” the state of our device before it will be flipped. When flipping your device, android will trigger the lifecycle so your state will be killed and if you haven’t saved the state it cannot be restored.

SQLite

This is the databases used in android. Each app will have its own database.

Hope this helps you on your way to develop your first android app.

Author : Jeroen W.

SPDY protocol

2011-12-05T17:06:00.000+01:00

The new Google SPDY protocol is another attempt to make the web more efficient and reliable. The SPDY protocol introduces an extra layer between HTTP and TCP/IP (actually SSL/TLS) that primarily allows for multiplexing and parallelizing multiple HTTP requests over a single SSL connection.

The SPDY protocol is not some lab exercise but used in production! The Google Chrome browser uses the SPDY protocol (or should we say extension?) to communicate with most Google's applications. SPDY remains mostly a Google thing, with no adoption by other big names (except for Amazon EC2 it seems).

With SPDY, the Chromium browser needs to establish fewer SSL connections. But more importantly, the Chrome browser can launch many HTTP requests in parallel, no longer restricted by a maximum number of TCP/IP connections.

But this made me think: could this have a positive impact on how service consumers are implemented? Similarly to a browser parallizing the retrieval of web content, (web) service consumers should also try to parallize as much as possible.

The HTTP request/response model that underlies web services should not lock us into a synchronous RPC paradigm wherey a requestor blocks waiting for a response. To fully leverage this potential of parallellism, we must move to non-blocking, AJAX like model programming model.

While reading about SPDY, I encountered 2 links worth looking into:
- Recent blog entry by F5 that is a critical review of the SPDY protocol.
- Article that starts with SPDY but goes much further into wild (?) and interesting ideas to re-engineer the workings of the Internet in a more dramatic.

Author: Guy

How to use drag-and-drop for creating a reusable processing rule in IBM Datapower

2011-11-29T17:05:00.000+01:00

The basic method of creating a reusable processing rule is by dragging the cursor over a part of the existing processing rule. For a very small group of actions this is fine. If you want to make the rule a bit more advanced you probably want to create the processing rule from scratch. This can be achieved by creating a ‘Call processing rule’ action and create the reusable processing rule starting from there.

The problem with this approach is that you cannot use the Datapower drag-and-drop interface you use for all other processing rules. The GUI presents you the same view you get for all processing rules in the ‘Objects’ menu. It lists the different actions, but that’s it. No overview of the used contexts on mouse-over, or other details of the actions besides their name.

A simple solution to overcome this problem is to create the reusable rule in a Multi-Procotol Gateway Policy.
Go to the page ‘Multi-Protocol Gateway Policy’ by following the link in the menu on the left (Services>Multi-Protocol Gateway>Multi-Protocol Gateway Policy).
Create a new policy that will contain all future reusable rules. In my example I called it ‘ReusableRules’.

You can open this policy and add as much rules to it as you like. For the match action you can pick any existing match (f.e. all (url=*)). This match is only used in this policy. When calling a processing rule from within another rule the match action is not used.

To call this processing rule from within the ‘Call processing rule’ action you just have to select your newly created rule in the dropdown-box as you can see in the screenshot below.

Author: Tim

Technical: Using REST web services with 2-legged OAuth on datapower.

2011-11-24T07:48:00.000+01:00

2-legged OAuth consists of:
•   The client signs up to the server. The server & client has a shared secret.
•   The client uses these key to gain access to resources on the server

What is REST ?

GET method
The client(secret=password) sends the following request to datapower.
Request: GET http://testname:1010/testname?name=KIM
Header: Authorization: OAuth    oauth_nonce="12345abcde",
                                                  oauth_consumer_key="Kim",
                                                  oauth_signature_method="HMAC-SHA1",
                                                  oauth_timestamp="1319032126",
                                                  oauth_version="1.0",
                                                  oauth_signature="m2A6bZejY7smlH6OcWwaKLo7X4o="

oauth_nonce = A randomly-generated string that is unique to each API call. This, along with the timestamp, is used to prevent replay attacks.
oauth_consumer_key = the client’s key.
oauth_signature_method = the cryptographic method used to sign.
oauth_timestamp = the number of seconds that have elapsed since midnight, January 1, 1970, also known as UNIX time.
oauth_signature = The authorization signature.

On datapower a Multi-protocol gateway needs to be configured allowing requests with an empty message body.

The secret needs to be shared between client and datapower. To make everything easy I stored the shared secret in an xml file in the “local:” file system of datapower.

The next step is verifying the signature. The best way to verify the signature is recreate the signature with the shared secret and then compare the two signatures. Of course to do this, some xslt skills are recommended.
This is the signature base-string that will be signed with the shared secret.

GET&http%3A%2F%2Ftestname%3A1010%2Ftestname&name%3DKIM%26oauth_consumer_key%3DKim

%26oauth_nonce%3D12345abcde%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1319032126%26oauth_version%3D1.0

How do we get to this?
If you look closely you see that there are 3 distinguished parts.

GET&
http%3A%2F%2Ftestname%3A1010%2Ftestname&
name%3DKIM%26oauth_consumer_key%3DKim%26oauth_nonce%3D12345abcde%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1319032126%26oauth_version%3D1.0

The first part is fairly straightforward.
   GET = the http method used.

The second part is the URL which is url-encoded.
We’ll need to percent encode all of the parameters when creating our signature. These "reserved" characters must be converted into their hexadecimal values and preceded by a % character. The rest of the characters are "unreserved" and must not be encoded.
The reserved characters are:
!   *   '   (   )   ;   :   @   &   =   +   $   ,   /   ?   %   #   [   ] white-space
These are the percent-encoded values of the above characters:
%21 %2A %27 %28 %29 %3B %3A %40 %26 %3D %2B %24 %2C %2F %3F %25 %23 %5B %5D %20
Be careful with URL encoding functions built into languages, they may not encode all of the reserved characters, or may encode unreserved characters.
   <xsl:variable
   name="BasicUrl"
   select="dp:encode(string(
   dp:variable('http://testname:1010/testname’)
       ),'url')
   "/>

The third part gets a bit more complicated to do with xslt, because the number of parameters and OAuth header fields can change.
We need to extract the Oauth header from the request.
     <xsl:variable
      name="OAuthParams"
      select="dp:request-header('Authorization')"
     />
The URL parameters are also necessary. The parameters in this example are “name=KIM”. Put them all together and alphabetically organize them. You will get a string like this:
name=KIM&oauth_consumer_key=Kim&oauth_nonce=12345abcde&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1319032126&oauth_version=1.0
And again url-encode this string using the datapower function(dp:encode).If you now concatenate the 3 parts with a “&” you will get the signature base string.

The next thing we need to calculate the signature is the shared secret. OAuth standard uses a concatenation of the shared secret & token(<secret>&<token>). Since we are working with the 2-legged OAuth we don’t need the token and we can leave it empty. So in the example we used the shared secret “password”. Important is that you add the “&” behind the secret: ”password&”. Now we have the necessary information to calculate the signature.
   <xsl:variable name="Calculate_Signature"
    select="dp:hmac('http://www.w3.org/2000/09/xmldsig#hmac-sha1',
    ‘password&’,
   <SignatureBaseString>)"/>

Finally we can compare the signature of the OAuth header with our own calculated signature. Don’t forget to validate the timestamp against a predefined limit e.g. 5 minutes. If all checks out, the client is authorized to use the web service.

link to check OAuth signature

POST method is coming soon…

Author: Kim

Exposing a webservice in JBoss ESB

2011-11-15T18:46:00.000+01:00

Webservice-based communication is of course at the core of SOA and as such the support should be easy and robust.
As the video demonstrates it is very easy to expose a service as a webservice and optionally demand validation.
Additionally it shows you how to enable WS-Security.

There are other ways to work with webservices:
- you can expose native jboss webservices
- create proxy webservices
- poll remote webservices

Link: http://www.youtube.com/watch?v=xflkWQZZsHE

Author: Alexander Verbruggen

OAuth

2011-11-14T22:45:00.000+01:00

With the upcoming Devoxx conference, I did some reading last weekend. With Fri Nov 11 as a national holiday in Belgium - because of the end of World War I - I had some extra time. Looked a bit into most recent development around HTML 5 and Android development.

Quickly I ended up diving deeper into REST. Must confess that I was very WS-* minded and was not really impressed by REST initially. But with the incompleteness of WS-* and the success of REST, I'm changing my mind.

So I ended up browsing through the book "Restful Java with JAX-RS". This REST stuff triggered me into looking into different REST API's, including the one from Dropbox. And Dropbox security is based on OAuth, which triggered me to dive (back) into OAuth.

Looked for an OAuth book on Safari and Amazon, but none (yet?) avaialble. So I ended up re-reading chapter 9 of the the book "REST in practice". By the way, very good book, I like it. Some great links while looking around:
- The introduction on OAuth
- Good OAuth introduction by Yahoo
- Google Oauth Playground, so see OAuth live in action

While looking into OAuth, I started making the comparison with WS-Security and SAML in particular. With OAuth, no XML signing nor XML canonicalization, the option to use HMAC instead of keypairs and certificates. So simpler, but not simple!

Note: one of my I8C colleagues (Kim) just finished project on DataPower appliance to implement OAuth support

Author: Guy

jBPM Orchestration in JBoss ESB

2011-11-08T18:39:00.000+01:00

The third installment in the JBoss ESB video series features the jBPM module which introduces service orchestration and human task flows.
The first video shows you how to set up a simple jBPM process:

Link: http://www.youtube.com/watch?v=jFBEqhAuOCw

The second one covers a few additional topics like variables passing back and forth between jBPM and the ESB services as well as user-driven branching:

Link: http://www.youtube.com/watch?v=9Mhd7VO0-oA

Author: Alexander Verbruggen

Custom java service in JBoss ESB

2011-11-02T17:35:00.000+01:00

This second video in the JBoss ESB series demonstrates how to create a simple custom java service that prints out the message content it receives.

Link: http://www.youtube.com/watch?v=TfgUW0D-aAo

Author: Alexander Verbruggen

Oracle Openworld 2011 through he eyes of an I8C employee

2011-10-27T09:04:00.000+02:00

In the beginning of October I was one of the 45,000 attendees from 117 countries.

If someone would ask me next year what was on the 2011 Open World I will definitely remember these key words: CLOUD and EXA.

Exalytics and Supercluster were anounced during several keynotes. These new children of the EXA family will allow Oracle even better to power private clouds and their own public cloud. These machines are impressive and they are definitive an asset to your corporation ... if you can pay for them :-)
In any case the specs for Exalogic, Exadata and Supercluster are pretty amazing. Who would have thought a few years back that you could have a machine that can load a few TB of data in memory?

As mentioned before Larry Ellison also announced the Oracle Public Cloud. Their cloud offering is mixed in that way that it offers a SAAS model for CRM and HCM fusion applications and Oracle Database and a PAAS offering for Custom Java applications. Maybe the riot with Salesforce was only to get extra attention for this anouncement ;-)

While the cloud wars continued I was trying to get a roadmap for fusion middleware for the next years. I however could not seem to find it, which was a pity.

A keynote that did provide a roadmap was at JavaOne. For java 8 the most interesting items are for me: addition of lambda expressions, Java modularity and completion of the HotSpot/JRockit JVM convergence project. Also nice to hear was that Twitter will join the OpenJDK!

At openworld some of the most interesting sessions I followed were:

A hands on workshop for Oracle CEP: this is a really interesting tool. Even though you can freely download and play with it, it is nice to just walk in a room to play with the tool and get all the information that you want, straight from the PTS guys.
A more in depth talk on Oracle Coherence gave me some more insight in this product. A distributed cache is something that is used in many applications nowadays, mainly to boost performance to new limits. In middleware and more specifically in ESB solutions this technology is interesting to cache configuration data, offload frequently used data from the database, cache service results and sometimes it is even used for messaging.
Continuous integration for SOA and BPM projects: this was a very interesting session on how to do more Test Driven Development (TDD) in an Oracle SOA environment. They also showed how to use popular tools like ant and Jenkins.
There were also some panel sessions during the conference. The most interesting I found the one on OSB and how to tune this ESB for peak performance. By the questions from the audience you can really see that this ESB is widely used in massive deployments.

For me personally the only downside of Open World are the 25 hours I need to spend in the airplane :-)

Author: Jeroen V.

Random errors when using MQSeries adapter in Clustered Environment

2011-10-24T20:18:00.000+02:00

In the following article, we will be describing an error that we've encountered in a clustered MSCS Cluster with MQSeries installed, using Microsoft BizTalk 2006 R2.

The Problem

Well, it's not exactly one error we've encountered. The problem actually exists of multiple random warnings that seem to be popping up in the event viewer when using the MQSeries adapter (using the MQSAgent2 COM+ component) whilst sending messages towards this adapter. Strangely enough, we also noticed these warnings popping up when there is absolutely no traffic going on on the server.

Some samples of errors:

The adapter "MQSeries" raised an error message. Details "The remote procedure call failed and did not execute. (Exception from HRESULT: 0x800706BF)"

--

The adapter "MQSeries" raised an error message. Details "Unable to cast object of type 'System.__ComObject' to type 'Microsoft.BizTalk.Adapter.MQS.Agent.MQSProxy'."

--

The adapter failed to transmit message going to send port "prtSendSecLendMsgStatusSLT_MQSeries" with URL "MQS://BDAMCAPP100/MQPRD205/FIAS.QL.SLT_BT_IN.0001". It will be retransmitted after the retry interval specified for this Send Port. Details:"The remote procedure call failed and did not execute. (Exception from HRESULT: 0x800706BF)".

The Impact

It might seem that these warnings are "just" warnings, but whenever one of these warnings is encountered, it causes the sending of messages towards the MQSeries adapter to fail. This will trigger the BizTalk retry mechanism which works with a certain amount of retries. These retries can be scheduled by minutes, where the lowest value of this configuration is 1.

This means that, when an problem occurs and the message needs to be resent, there will be a minimum delay of 1 minute on every message. In an environment where large amounts of messages are being processed, this can be a huge painpoint.

The Solution

After investigation, we seem to have found that the source of this issue can be found on network level of the servers. Idle TCP sessions on the network of the client are closed within 1 hour. This seems to be a common default setting on a lot of environments, and is certainly not a bad thing.

The problem is that the default TCP Keep Alive interval in Windows seems to be 2 hours. So when a Keep Alive package is sent towards the cluster through an idle connection with more than 1 hour idle time, it will return an error since the TCP session on the network is already closed.

Obviously, the most logical thing to do to avoid these errors is changing the Keep Alive timeout time on the Windows (BizTalk) servers. Tis can be done by changing a registry value on your BizTalk servers, followed by a reboot.

The following link describes which value to set:
http://technet.microsoft.com/en-us/library/cc782936(WS.10).aspx

Good luck!
Andrew De Bruyne

Installing an old E1 adapters in recent versions of webMethods

2011-10-16T20:12:00.000+02:00

Let’s say you have a current webMethods version 6.5 and you’re upgrading to version 8, but all subsystems are not upgrading.

The webMethods Enterprise one adapter (8.96 one-off Adapter for XPI 8.94) which came with the installation of webMethods 6.5 will be replaced when you install a new Enterprise one adapter in version 8.

It can be that your JDE environment is old and that the adapter of version 6.5 is the latest supported version. How can it be used now on webMethods version 8.
First step: Export the current Enterprise one adapter from version 6.5 (Developer -> export) and save it on your machine.

Now copy this export in the replicate -> inbound folder of your new version 8 installation.
Before you can install this package, you need to install the PSFT_PackageManagement package first.
Once that package is installed, open it’s pub page.

Click install inbound releases and select your exported zip file.

Select that exported zip to be installed and click save changes

The Enterprise one adapter from version 6.5 is now installed and your new webMethods version 8 installation can work with old JDE systems.

Author : JeroenW

TUCON2011

2011-10-11T22:06:00.000+02:00

Last week TUCON2011 took place in Las Vegas, USA. As we are the only TIBCO partner, with its HQ in Belgium, Integr8 Consulting could not be absent this year! So together with my colleague, we went out to the far-west of the USA.

This year TUCON started with the keynote speakers and the BIG idea track. The focus of these BIG idea tracks is to hear from TIBCO customers and TIBCO visionaries, how they are using TIBCO technologies and how it provides them with the 2-second advantage. For those out there not knowing what the 2-second advantage is, here is a link to a video from Vivek (TIBCO CEO) explaining the 2-second advantage.

Putting the right information on the right time in the right context

These are my 4 keywords I remember from my TUCON visit:

Tibbr
Context
Silver
Mobility

If it’s one thing that is clear, TIBCO is investing a lot in its social computing tool, tibbr. Tibbr is a tool; build specific for the work-space, which will allow you to follow subjects, applications and event streams. Using this approach it will be possible to put the information you receive, directly in the right context when and where you want it.

This brings me to the second keyword, context. One of the keynote speakers stated ‘what if you have a million of data events, but you can’t place it’. And that’s what it’s all about. If you can place the data at the right time in the right context, it will provide you with a 2-second advantage.

The third keyword is Silver. Silver is TIBCO’s brand name for its cloud services like you have ActiveMatrix for the SOA/BPM platform. Next to Tibbr, Silver will become a focus point for the future. You will get Silver Mobile, Silver Fabric, Silver Spotfire and many more. All providing you with Cloud services to start with Tibco technology in just a matter of seconds (at least in theory).

Now the fourth option will look a bit strange but this might become the biggest shifting of enterprise communication since the rise of the email (maybe a bit exaggerated) . With the upcoming boost of the smartphones, ipads, smart devices (like smart grid readers, which read your electricity usages, and send it to your electricity provider), new technologies are needed that will make it possible to use your mobile device as…. well as a workstation. If you’re interested on how Silver Mobile will work:

Silver Mobile will provide you with a platform that runs on your Android, iPhone or BlackBerry. Using the platform, you will receive a common API that you can use in your mobile framework (jQuery Mobile, etc…) when building your own app
Using Silver Fabric you can push, from the cloud, your apps to the Silver Mobile platform on your company’s iphones, blackberries, or androids.
The example shown on TUCON was showing the status of your BW applications on your iphone. In case something went down, you got a notification using the native notification bus from your mobile device.

In the sessions of Orange, we saw a M2M (Machine-2-Machine) example on how Orange is using mobile communication technology. Now imagine about the possibilities when you think again about ‘context’. You can analyze the date from a mobile device and correlate this context with build in embedded devices like GPS, etc… to provide you with … the context of the data.
To conclude, enterprises will have to adapt their architectures to provide more and more a context-aware mobility delivery architecture in order to ‘please’ there customers.

Announcements

Next to the BIG idea tracks, day 2 and 3 are technology tracks that provided us with some insights of the current developments done by TIBCO. Now I will start this chapter, like TIBCO started each session. Every information provided is purely informational and does not legally bound us / TIBCO to any delivery

I tried to follow a diverse schema trying to know as much as I can and these are the things that are still in my mind:

The ActiveMatrix platform will be extended with a rule engine, TIBCO ActiveMatrix Decisions. This product is build based on BusinessEvents and exposes it rules as services which can be used for example in a BPM process of ActiveMatrix BPM
TIBCO ActiveSpaces 2.0 data grid. Woow, was I overwhelmed with this technology. To be honest, I didn’t really know this technology, but from what I’ve seen on TUCON, I immediately want to start with it!
BusinessWorks plugins. Instead of adapter based technology, TIBCO is coming more and more with external plugins that can enhance BusinessWorks. Examples are tibbr, SalesForce, ActiveSpaces and an Aspect plugin that you could use for AOP programming in BW!
Hawk! What Hawk?? Yes indeed, Hawk is coming with a nice web interface that will provide you with a better overview of your TIBCO Administrative domains. And as most products will do, Hawk will also integrate with Tibbr and maybe Spotfire in the future.
Nimbus. This newly acquired technology provides you with a tool that can be used for documenting your business processes in a way your business can understand them. Don’t see it as an automation engine but rather as a tool for documenting (discovering) your business process. If TIBCO will connect AMX BPM and Nimbus, information is exchanged between them, it might become a strong product bundle.
TIBCO EMS / FTL. During the engineering roundtable, it came to a discussion on how the future will look like for messaging. In the end there is no real answer. EMS is still the messaging solution if you want a guaranteed reliable messaging solution. FTL is the future for TIBCO if it concerns really fast messaging.

If you visited TUCON this year, maybe you joined some other sessions and got other ideas then myself. Please share your experiences. By sharing information and putting it in the right context, we can get that 2-second advantage!

Author: Günther

JBoss ESB - A simple file poller

2011-09-20T20:39:00.000+02:00

A while ago I took JBoss ESB for a spin to see how it compares to other tools we frequently use (both in terms of features and usability). I created a few how-to videos which can be used as a reference to implement some common usecases. Though they assume a working JBoss ESB server and the appropriate eclipse tooling, they are generally pretty straight forward.

This first video demonstrates how to set up a (very) simple file poller. Subsequent tutorials will highlight more advanced features like XSLT transformations, jBPM and some smooks-based csv parsing. Please note that it is best viewed at 1080p or original resolution.

Link: http://www.youtube.com/watch?v=GOKFl79W1To

Author: Alexander Verbruggen

Migrating Trading Networks data between v6.5 and v8 on different servers

2011-09-10T19:00:00.000+02:00

The client I’m currently working for requires a new webMethods 8 environment, but this has to be a copy of an existing 6.5 environment, which should stay. They created new servers so I could install all required components on this new machine for a version 8 with Trading Networks.

They required a clean install, but needed to have the code and TN data from the version 6.5 on the new version 8 installation. The installation itself was easy, just as the setup of the environments (e.g. ports, security, LDAP, …)

When I came to the point where I wanted to migrate the Trading Networks data, I came up with some issues. The biggest problem was that we have a current table wm65tn which holds all the data and we had to create a new one wmTN for the version 8, as they still want to use the version 6.5.

The first thing I tried was to deploy the Trading networks settings. This was already an issue as the old environment was behind a firewall and this needed to be opened for traffic towards the new version 8 environment. Once the firewall was opened I could finally deploy all TN settings, but this wouldn’t work.

WebMethods Deployer won’t deploy TN data between different versions (6.5 and 8).

Next I tried to Export and Import all data into the new version. Again the same issue appeared, Not allowed to copy data between 6.5 and 8.

My next option was to install a second database version 6.5, copy all the data there and then run a migration for that table.

So with this last step in mind I started. First bumble on the road, the disk where all tables was stored didn’t have enough free space. After creating the required space I could create the database. (wmTN) After the database was created. I disconnected the old version 6.5 table and copied the MDF and LDF file (SQL server).

Now we need to disconnect the wmTN table and rename the log files to the required log file names which are linked with this table.

Now the new database has the same data as the old one. So we can connect the tables again to the SQL server.

Next step is to upgrade the database according to the webmethods documentation.

Unfortunatly there was a step which mentioned “if errors occured, contact Software AG Customer Care” and yes lucky me, I can contact software AG.

For getting this error :

[wm-cjdbc40-0034][SQLServer JDBC Driver][SQLServer]Foreign key 'fk_PtnrUser_PtnrID' references invalid table 'Partner'.

MIGRATE TRADINGNETWORKS [31] FAILED

After some mails and telephone calls with webMethods support the issue is still not resolved.

They managed to solve the foreign key issue by just removing this, but when that issue is solved we got this issue.

*** Data Migration from TN 6.5 to TN 7.1 started...

* Creating Partner-User mappings...

java.sql.SQLException: [wm-cjdbc36-0007][SQLServer JDBC Driver][SQLServer]Invali d object name 'PartnerUser'.

at com.wm.dd.jdbc.base.BaseExceptions.createException(Unknown Source)

at com.wm.dd.jdbc.base.BaseExceptions.getException(Unknown Source)

at com.wm.dd.jdbc.sqlserver.tds.TDSRequest.processErrorToken(Unknown Sou

rce)

at com.wm.dd.jdbc.sqlserver.tds.TDSRequest.processReplyToken(Unknown Sou

rce)

at com.wm.dd.jdbc.sqlserver.tds.TDSRPCRequest.processReplyToken(Unknown

Source)

at com.wm.dd.jdbc.sqlserver.tds.TDSRequest.processReply(Unknown Source)

at com.wm.dd.jdbc.sqlserver.SQLServerImplStatement.getNextResultType(Unk

nown Source)

at com.wm.dd.jdbc.base.BaseStatement.commonTransitionToState(Unknown Sou

rce)

at com.wm.dd.jdbc.base.BaseStatement.postImplExecute(Unknown Source)

at com.wm.dd.jdbc.base.BasePreparedStatement.postImplExecute(Unknown Sou

rce)

at com.wm.dd.jdbc.base.BaseStatement.commonExecute(Unknown Source)

at com.wm.dd.jdbc.base.BaseStatement.executeUpdateInternal(Unknown Sourc

at com.wm.dd.jdbc.base.BasePreparedStatement.executeUpdate(Unknown Sourc

at com.wm.app.tn.util.migrate.migratedata_to_tn_71.createPartnerUsers(mi

gratedata_to_tn_71.java:137)

at com.wm.app.tn.util.migrate.migratedata_to_tn_71.main(migratedata_to_t

n_71.java:47)

This issue is related to the partner table (which holds all the partner profiles). The first thought was that it was related to the unknown profile because it has an id of 00000000000000. But the contact person of SAG confirmed that this wasn’t the issue as this is the default entry that comes with every TN database.

At this moment the ticket is already open for more than one week and still no solution found.

Today I received another call from SAG and they said to have found the issue. I recovered the original 6.5 TN database. So I could follow the correct guidelines from SAG without touching anything to the table’s structure (removing foreign keys).

The problem was that the original 6.5 database was created with a user specific for this table and the table had a schema defined.

Solution:

I had to create a user which had a default schema equal to wm65tn (which was the old value) and then run the upgrade script via the command line.

dbConfigurator.bat -a migrate -d sqlserver -c TradingNetworks -v latest -l <db_server_URL> -u <existing_db_user> -p <password> -fv 10

Now the table was upgraded to version 8 and all new tables have the correct schema (wm65tn).

Now run this command migratedata_from_tn_7-1 6.5 from the correct folder as this copies the data from the old table to the correct new table. (all partner profiles, agreements, …)

E.g : Partner to PartnerUser

Now a restart of the environment is needed, unless your environment was already down, and the data is available in TN version 8.

Author : Jeroen W.

BizTalk is dead, long live BizTalk

2011-09-06T05:56:00.000+02:00

On the WPC (World Partner Conference) 2011, Microsoft revealed it’s future plans for their integration stack. It seems that BizTalk as product name will disappear in the nearby future…

A new integration platform (stack) is being built in the cloud, offering us most of the BizTalk functionality. The same stack will also be made available on-premise, thus replacing BizTalk as we know it today.

It is up to us now, to learn utilizing the new integration stack in the cloud as on-premise. This will keep us busy for the next 5 to 10 years, I presume.

Does this mean that BizTalk will no longer be supported? No! It just means that the integration stack as we know it today will remain fairly unchanged (they keep it aligned with their Windows, SQL and .NET stack) until the new product becomes available on-premise. How long will this take? I assume we will see the new on-premise product within 2 to 3 years. The name of this new product is not yet determined, so maybe “BizTalk” as name may continue to live J

This link gives a good overview of what we may expect. And the WPC presentation video.

Author: Koen