Saturday, June 4, 2011

How to do NTLMv2 authentication in TIBCO BusinessWorks

As a proof of concept I had to test if TIBCO could perform authentication from its BusinessWorks suite to a Microsoft Dynamics CRM web service using ‘Integrated Windows Authentication’.
TIBCO BusinessWorks has all the necessary tools for connectivity, transformation and orchestration of processes but unfortunately it has no support for Integrated Windows Authentication. But I don’t consider it as a flaw of TIBCO BusinessWorks. Integrated Windows Authentication is specific to Microsoft products and the protocol that is currently in scope for the POC, NTLM, is a proprietary protocol.

What is the goal of the POC?

Authenticate TIBCO when calling the Microsoft Dynamics CRM web service. The authentication needs to be done using the NTLMv2 protocol. The account I use is a designated system account for TIBCO, which has received the correct access.



How did I start?

A lot of developers think: ‘what I do, I do better’. Well, I am more in favor of ‘use instead of build’. So first I started to find solutions on the internet that might do the trick for us. Since that didn’t work out well, I started to use some libraries that implement NTLM and to see if it works with TIBCO BusinessWorks.

I also wanted to find a solution as fast as possible. So instead of trying to investigate further on why something doesn’t work by the book, I just tried a different library/application.


So here is a summary of things I’ve tried:



Proxy solutions:

NTLMAPS: This is a tool that was used at a client side but stopped working for them after they switched to a new Active Directory domain. For my POC, and using the latest NTLMAPS version, I constantly received a 401 error back. So I had to quickly give up on this.

CNTLM: A rewrite of NTLMAPS and I managed to get authenticated when I was trying it from a Non-MS browser like chrome or Firefox. However the tool was prompting me for credentials for user authentication, which were then used for NTLM authentication. I quickly tried to configure my SOAP Request-Reply activity using HTTP Authentication and a correctly set Identity but unfortunately it didn't work. I didn’t investigate further on this.


Since the proxy solutions did not work out well, I tried to use a Java Code activity and tried to use libraries implementing the NTLM protocol.



Client solutions:


According to the documentation on Apache it should support NTLMv2 but I didn’t manage to get it to work. Although following the guidelines, authentication was always failing with a 401 error. Maybe I was doing something wrong but since TIBCO BusinessWorks is also using (an older) HTTPClient in its third party library repository, I decided not to investigate further on this.  Just to be sure that an upgrade would cause a nasty side effect.



On http://devsac.blogspot.com/2010/10/supoprt-for-ntlmv2-with-apache.html I found an interesting article about configuring the HTTPClient 3.x of Apache with the JCIFS library to get NTLM support.

I didn’t try this one because on the site of JCIFS, they themselves recommend to use the Jespa library if you’re looking for full NTLM support.


Unfortunately the Jespa library is not open-source and has some limitations when you integrate directly with Active Directory. However in my situation I only needed a small portion of this library. I needed to establish a connection and needed a provider that will authenticate against the NTLMv2 protocol. So for my POC there is no impact.



Proxy setup

I’ve made a small TIBCO BW project, which can act as a proxy, between TIBCO BusinessWorks and MS Dynamics CRM web services. This service is working identically as the NTLMAPS application. It will sit as a proxy between the Soap Request-Reply activities and the endpoint.
 
How does the Forward request activity look like?


1)     First I defined some input parameters so I could dynamically configure my process:

 
2)     Configuring the Java Code: Updating the import statements
import java.util.*;
import java.io.*;
import java.net.URL;
import java.security.PrivilegedExceptionAction;
import jespa.http.HttpURLConnection;
import jespa.security.PasswordCredential;
import jespa.security.RunAs;
3)     Configuring the Java Code: Add an inner class

This class will perform the POST action and return the soap reply.
public class HttpPost implements PrivilegedExceptionAction
{

private URL url = null;
private HttpURLConnection conn = null;
private OutputStreamWriter wout = null;
private BufferedReader rd  = null;
private StringBuilder sb = null;
private String line = null;
private String responseMessage = null;
private int responseCode = 0;
private String responseBody = null;
private String endpoint;

public HttpPost(String endpoint){
       this.endpoint = endpoint;
}

public Object run() throws Exception
{

       url = new URL(endpoint);
       conn = new HttpURLConnection(url);
       try {
       conn.setDoOutput(true);
       conn.setDoInput(true);
       conn.setRequestMethod("POST");
       conn.addRequestProperty("SOAPAction", soapAction);
       conn.addRequestProperty("Content-Type", contentType);
       conn.setReadTimeout(timeout);

       // Set the input
       wout = new OutputStreamWriter( conn.getOutputStream() );
       wout.write(soapRequest);
       wout.flush(); // this triggers the POST
       wout.close();

       // Get the response
       rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
       sb = new StringBuilder();
       while ((line = rd.readLine()) != null) {
             sb.append(line + "\n");
       }
       rd.close();
      
        } catch (IOException ioe) {
             System.err.println(ioe.getMessage()); // such as '404 Not Found'
             rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
             sb = new StringBuilder();
             while ((line = rd.readLine()) != null) {
                    sb.append(line + "\n");
             }
             rd.close();
        } finally {
             responseCode = conn.getResponseCode();
             responseMessage = conn.getResponseMessage();
             responseBody = sb.toString();
             conn.disconnect();
             wout=null;
             rd = null;
             sb = null;
             conn = null;
        }
        return null;
}

public int getResponseCode()
{
       return this.responseCode;
}

public String getResponseMessage()
{
       return this.responseMessage;
}

public String getResponseBody()
{
       return this.responseBody;
}

}

4)     Implement the invoke function
org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger("bw.logger");
HttpPost t = new HttpPost(endpoint);
RunAs.runAs(t, new PasswordCredential(domain + "\\" + userName, password.toCharArray()));
logger.info("Server replied with HTTP status code: " + t.getResponseCode() + " " + t.getResponseMessage());
soapReply = t.getResponseBody();

Using the proxy class

When configuring my Soap Request-Reply message, I only need to configure a Proxy Configuration which points to my HTTP Receiver. My HTTP Receiver will forward the request and returns back the correct response.



Update: As some readers have commented, there seems to be a bug inside the above code. 

The updated project can be downloaded here. This project has updated java code that improves the handling of the soap request/response. You'll have to change the global variables so the authentication group is updated with your login credentials.
Also note that since BusinessWorks version 5.10, Tibco has added NTLM authentication support. See the release notes at https://docs.tibco.com/

Author: Günther

Tuesday, May 31, 2011

SoftwareAG ProcessWorld 2011 Day1

I see conferences like ProcessWorld as a class-room training. You can get a lot of (high level) info from the whole software stack in just 2 days. Unfortunatly, like training, there are less interresting parts that you have to sit through. But you as a blog reader, can dive right into the good stuff:

- Process roundtrip
I saw some process roundtripping, but I still have to make it to the boot of Process Transformation to see if it goes round all the way. The integration with ARIS seems nice, when you change the model in Designer, you can notify the analyst to verify the changes.

-Mobile Technolgy
I had a interesting discussion with someone from Matismo, the company bought by SAG to complement their stack with mobile-apps. Apparantly they can generate not only mobile apps for the major platforms (Android, IPhone iOS, Blackberry), they can also generate Flash and HTML5. We won't see see anything from it in the near near future, simply because the aquisition is that recent.
For me personaly I don't see the purpose for the app, I would choose to create a mobile webApp. Most of the gui's from SAG are web-based, so that would be more consistent. However the guys from Apple didn't like Flash, so their you might have a problem (BPM, Mashup)... maybe will have to wait until HTML5 is the defacto standard.

-TerraCotta/Cloud
Terracotta will leverage the abilty to scale easily. This will be the backbone of their cloud offerings, however I don't have any specifics on that (yet). When talking whith the guy at the Terracotta boot, he was all praise for the technology. So I asked him why we didn't see the technology everywhere. Apparently it is used in a lot of code and it is even used on IS's. So if anyone has experience with it, I'm interested to hear about it.

-Deployment 8.2
They are working hard on changing the deployment cycle. With version 8.2 you now have the option of deploying from builds. Builds can be created from scripts (ant), which gets the assets from a VCS, file-system or other location. The assets itself are defined through ADL's (asset discription language). This way they can keep dependencies between different assets: ex. a process uses several services.
So you have a setup where the developer checks in its code, and the script runs nightly/weekly/... builds of the latest version and puts them into a repository, most of the cases a filesystem (not yet in centrasite as this will be for a future release). That repo than acts as a source for deployer.

-Complex Event Processing
This looks very interresting, they have the engine that runs in the same osgi container as the IS. They are not the same process, you could theoretically run the engine seperatly, but most of the time you would need the IS for connection to other systems. The engine itself is JMS in/ JMS out, it works with broker documents as events. These are processed by rules which are defined in SQL. You can then monitor events through Aris Mashup (which looks like a nice and powerfull tool - unfortunally on flash).

Hopefully the next day will be as interresting as the first.
Auf wiedersehen.

Author: Stefan De Wandeleir

Monday, May 23, 2011

Azure Service Bus: next step with enhanced Messaging

Microsoft has the vision to make a complete Service Bus available in the Azure cloud.  One basic and essential component of an integration solution is Messaging.  At TechEd 2011, a session was dedicated to this Messaging part.

A screenshot from the presentation.   Also interesting to see the other pieces of the service bus that are on their way: Service Management and Integration.







The queuing solution has support for
and all the other goodies when using a Messagnig solution: reliable communication, asynchronous, variable timing, throttling and more.

In this pre-release (Community Technology Preview), the max. message size is 256 kB and queue size of max. 1 GB. This is too small for quite some B2B transactions.  From that we derive that the focus of the Messaging and thus Service Bus is (currently) more focused on use by Azure applications and not for B2B communication between partners (yet).

Author: Guy

Wednesday, May 18, 2011

JVM performance tuning - part 1

Many integration solution are based on the Java Virtual Machine.  Having a good understanding of the underlying machinery is essential for a well functioning integration platform.  Tuning of the Java Virtual Machine is an essntial part of that.  In a series of blog posting, we are going to focus on the tuning of the JMV.  Let's get started with part 1.

The importance of tuning the Java Virtual Machine (JVM) is based on a few characteristics of the virtal machine:
  • The virual machine layer makes abstraction of the underlying operation system and thus increases the overhead
  • This overhead can cause Java applications to run slower than equivalent programs written in a low level language
  • Java's advantages like memory management, exception handling, multithreading, ... add costs
In order to understand the ways in which the JVM can be tuned, several aspects needs to be cleared  out first.

Java process memory
The JVM runs as a process called java on an operating system. It runs as a single process, meaning it doesn't share memory with other processes. The size of this process is determined by 2 major blocks:
  • The java heap space (often the largest part)
  • Other memory
    • The permanent generation: contains metadata: data about classes and methods, statics, ... If too many classes are loaded a “java.lang.OutOfMemoryError: Permgen space” error may occur. The size of the permgen can be configured using the -XX:PermSize=value (initial value) and -XX:MaxPermSize=value (max value)
    • Code generation: the translation of byte code into native cod
    • Socket buffers: for TCP connections. Each connection has a send and a receive buffer
    • Thread stacks: every thread has a stack. The size of the stack can be configured using the -Xss option. The default size of a thread stack is determined by the combination JVM/OS (320k for java 1.6 on windows 32-bit and 1M for java 1.6 on windows 64-bit). The larger the number of threads, the larger the total thread stack size will be. Two possible memory errors associated with the thread stack size are “java.lang.OutOfMemoryError: unable to create new native thread”. This means the JVM wants to associate a stack with a new thread, but the java process size on operating system level can't be expanded. A “java.lang.StackoverflowError” occurs when the thread stack size is too small or when recursive code is called too often.
    • Direct memory space: the JVM allows to address memory directly outside the java heap.
    • JNI code and JNI allocated memory: for JNI programs
    • Garbage collection: introduces additional overhead
The java heap space is divided in two generations. Each generation contains objects of a different age. The java heap space generations are:
  • The young generation: new objects are created in the young generation. The object clean upof the young generation is called a minor garbage collection (minor GC). The younggeneration has two spaces:
    • Eden space: new object are created in this space. New objects that are too large to fit in here are created in the tenured space of the old generation.
    • Survivor space: divided in a “to” and a “from” survivor space. Contains objects that survived at least one minor collection. It gives objects an additional chance to “die”before being moved to the tenured space. One survivor space contains such objects,while the other is empty.
  • Old generation: has one space: the tenured space. Contains objects that survived severalminor collections. Object clean up of the old generation is called a full or major garbage collection (full GC).
The permanent generation resides outside the java heap space and contains metadata, statics and
objects responsible for a good working of the jvm garbage collector.

Author: Dimitri

Friday, May 6, 2011

Why Software AG My webMethods Server doesn’t show Microsoft Active Directory group members from trusted domains

Today we ran into the following problem at a customer of ours: a couple of security groups had been setup in Microsoft Active Directory (AD), containing users from a trusted domain. Unfortunately, these users were not visible for our My webMethods Server (MWS) that is linked to AD using an LDAP Directory Service. When we checked the Group Members for the group in the MWS User Management Groups portlet, the group members were not visible.

When looking a bit deeper into this problem, we discovered a number of possible causes for this problem.


First of all we discovered that the members of these groups, being users of the trusted domain, were located in a branch of the directory that was not located under the base DN that we configured for the corresponding Directory Service in MWS. AD places the LDAP entries for these users in a top level container called ‘ForeignSecurityPrincipals’. The AD documentation on MSDN teaches us that in AD DS, each domain naming context (NC) contains a well-known Foreign Security Principals container. This container holds objects of class foreignSecurityPrincipal. These objects represent security principals from trusted domains external to the forest, and allow foreign security principals to become members of groups within the domain. This might already explain why MWS doesn’t show the group members, but there is more.

Even after correcting the base DN for the MWS Directory Service, the group members are still not visible in MWS. This is because the LDAP objects for these users in AD are based on different object classes than the standard user objects, more precisely the class 'foreignSecurityPrincipal'. We configured our MWS to identify LDAP users based on the 'person' object class, which contains attributes such as sAMAcountName (User ID), sn (Last Name), givenName (First Name), ... that are not present on the foreignSecurityPrincipal class by the way. Therefore of course, MWS will not recognize these objects as users which explains why we can’t see these group members in MWS. If you would like to troubleshoot the values of an LDAP object’s class or other attributes, a free LDAP browser client such as JXplorer can be of great help by the way.

So the conclusion is that MWS doesn’t support AD users from trusted domains through the LDAP Directory Service. 



Author: Kristof Lievens