Wednesday, May 2, 2012

Test Driven Development on Oracle Service Bus

As a Java developer I have learned to appreciate Test Driven Development (TDD). It helps you to deliver quality software solutions and it gives you confidence when you make changes later on. Your test code will help you to make sure that no regression arises when changing your production code.
An intro on TDD: http://www.agiledata.org/essays/tdd.html

When working with ESB products you work on a different abstraction layer. You are more focused on configuring a message flow, detailing how a specific type of message needs to be routed, transformed, filtered and delivered. You are not as close to the code as a java developer hence it is more difficult to test the complete flow. In addition to that, ESB products are very focused on connectivity to for example an Oracle Database or a SAP backend system. This makes it even more difficult to test because it requires these external systems to be available during testing and also to produce predictable results.

Oracle Service Bus (OSB) is such an ESB. I will not go into depth about the product here, but important to know is that it is very much oriented towards Web Services and XML. These standards will help us to test our integration logic.

In the below picture you can see the setup for our tests. The following steps happen during a test run:

A test request is read from a file and sent using a HTTP client to OSB
OSB runs the integration logic as it would for any request
The business service it invokes is configured with a test endpoint. An embedded HTTP server is used for this purpose.
The test endpoint can assert the incoming request, wait for a configured period of time and return a configured response.
OSB will return a response to the test client as well, which can then in its turn assert the response.

Java libraries used in this setup: JUnit, XMLUnit, Hamcrest, Jetty HTTP client and server.

Some of the code for executing this test:

First question to expect is of course: why the custom code and not SOAP UI?
Indeed you can use SOAPUI as SOAP client and SOAP mock server. The reasons for not doing this is because you have a lot more control. When I start up a test I can change the behaviour of the mock server:

I can choose to let it return different responses
I can let it wait so that OSB will not have a response in time; allows me to test the exception handling and retry mechanisms.

I can also assert the request that comes from OSB to make sure it is as expected. In case I have to call multiple business services I can have one service return a sucessfull response and have another one time out.
All this more complex testing is a lot harder, if not impossible, to configure using SOAP UI.

Author: Jeroen V

Friday, April 27, 2012

Microsoft Installer Custom Actions User Impersonation

Problem

When creating an installer with custom actions you might run into some security issues when executing it on a Windows Vista/7/2008 or later OS.
This is because custom actions will be executed in the context of the user running the Windows Installer Service being the SYSTEM user.
This behavior is enforced from Windows Vista on and can give you authorization problems for certain tasks you want to perform in your custom actions.

Solution

To overrule this default behavior and run the custom actions as the impersonated user that is executing the MSI we will have to flip the msidbCustomActionTypeNoImpersonate bit that is on by default.
In a Visual Studio Setup project it is however not possible to set this flag through the Properties Window.
To solve this we must create a post-build script that will flip this bit:

// CustomAction_Impersonate.js <msi-file>

// Performs a post-build fixup of an msi to change all deferred custom actions to Impersonate

// Constant values from Windows Installer

var msiOpenDatabaseModeTransact = 1;

var msiViewModifyInsert = 1

var msiViewModifyUpdate = 2

var msiViewModifyAssign = 3

var msiViewModifyReplace = 4

var msiViewModifyDelete = 6

var msidbCustomActionTypeInScript = 0x00000400;

var msidbCustomActionTypeNoImpersonate = 0x00000800

if (WScript.Arguments.Length != 1)

{

WScript.StdErr.WriteLine(WScript.ScriptName + " file");

WScript.Quit(1);

}

var filespec = WScript.Arguments(0);

var installer = WScript.CreateObject("WindowsInstaller.Installer");

var database = installer.OpenDatabase(filespec, msiOpenDatabaseModeTransact);

var sql

var view

var record

try

{

sql = "SELECT `Action`, `Type`, `Source`, `Target` FROM `CustomAction`";

view = database.OpenView(sql);

view.Execute();

record = view.Fetch();

//Loop through all the Custom Actions

while (record)

{

if (record.IntegerData(2) & msidbCustomActionTypeInScript)

{

//We must flip the msidbCustomActionTypeNoImpersonate bit only for deferred custom actions

record.IntegerData(2) = record.IntegerData(2) & ~msidbCustomActionTypeNoImpersonate;

view.Modify(msiViewModifyReplace, record);

}

record = view.Fetch();

}

view.Close();

database.Commit();

}

catch(e)

{

WScript.StdErr.WriteLine(e);

WScript.Quit(1);

}

This script file (CustomAction_Impersonate.js) must be placed in the same folder as your setup project (Setup.vdproj) and add the following PostBuildEvent in your setup project:
cscript.exe "$(ProjectDir)CustomAction_Impersonate.js" "$(BuiltOuputPath)"

Build and run your setup project and you will notice that all the custom actions will now run as the impersonated user that executes the MSI.

Author: Christophe

Monday, April 23, 2012

General Access Denied error (0x80070005) in Hyper-V Manager when starting VM

I ran into the following error when I tried to start a VM in Hyper-V Manager recently:

To solve this issue I had to grant Full control to the folder containing all the VM's files (Virtual Hard Disks, config,...) using the following command:

icacls <VM directory> /grant "NT VIRTUAL MACHINE\0BA74464-10AF-4709-AAE9-4C1B196C08ED":(OI)(CI)F /t

After executing this command you can verify if it was successful by checking the Security properties of the directory where the VM files are located through Windows Explorer. The user that corresponds with the Virtual machine ID in the error message should have Full control of the folder where the VM files are located.

Author: Kristof Lievens

Tuesday, April 17, 2012

Graceful shutdown of a webMethods Integration Server through custom developed java program

Out of the box a webMethods Integration Server is shutdown via the Administrator screen.

Following code snippet with allow you to gracefully shutdown a webMethods Integration Server through the usage of a custom developed java program.

Java code snippet :

import java.io.*;

import com.wm.app.b2b.client.*;

import com.wm.util.*;

import com.wm.data.*;

public class IntegrationServerShutdown

{

public String hostName = null;

public String port = null;

public String userName = null;

public String password = null;

Context context = null;

public boolean connected = false;

public static void main(String[] args)

{ String input = null;

//Create an instance of a b2bServer

IntegrationServerShutdown b2bServer = new IntegrationServerShutdown();

b2bServer.hostName=args[0];

b2bServer.port=args[1];

b2bServer.userName=args[2];

b2bServer.password=args[3];

System.out.println("");

b2bServer.connect();

if(!b2bServer.connected)

System.exit(0);

b2bServer.shutdown();

}

public void connect()

{ context = new Context();

try

{ context.connect(hostName + ":" + port, userName, password);

System.out.println("Connected to " + hostName + ":" + port);

connected = true;

}

catch(ServiceException e)

{ System.out.println("Could not connect to " + hostName + ":" + port);

System.out.println(e.toString());

connected = false;

}

public void disconnect()

{ if(context != null)

context.disconnect();

}

public void shutdown()

{ try

{

System.out.println("Shutting down IS Server... ");

IData inputs = IDataFactory.create();

IDataCursor myCursor = inputs.getCursor();

myCursor.insertAfter("bounce", "no");

myCursor.insertAfter("timeout", "0");

myCursor.insertAfter("option", "force");

context.invoke("wm.server.admin","shutdown",inputs);

System.out.println("IS Server shutdown complete.");

System.exit(0);

}

catch(ServiceException e)

{ System.out.println("IS Server shutdown failed.");

System.out.println(e.toString());

System.exit(1);

}

Remarks :

When compiling and running the java program make sure to include files wm-isclient.jar and mail.jar in your class path. File wm-isclient.jar can found in the “common/lib” folder of an Integration Server whereas file mail.jar is localized in the “ext” sub folder of the “common/lib” folder.
Usage of the IntegrationServerShutdown program is IntegrationServerShutdown <IS hostname> <IS port> <IS account with administrative privileges> <IS account password>
At this moment the java program is working with at least webMethods Integration Server v7.1.2 and v8.2. Specifications contained herein are potentially subject to change so use them at your own risk.

Author: Johan De Wulf

Thursday, March 22, 2012

Daily or Weekly statistics on what has been processed in webMethods

This arctivle will describe how we can get some daily or weekly statistics from the database (MsSql or Oracle) of webMethods.

Let us start with the Dynamic SQL Adapters. These are needed to run a query which will return us the statistics of a specific period.

Input

Output

Query to enter in the Dynamic Adapter.

Oracle

SELECT a.servicename,

sum(decode(a.status,1,1,0)) as TotalStarted,

sum(decode(a.status,2,1,0)) as SuccessCount,

sum(decode(a.status,4,1,0)) as OutstandingFailures,

sum(decode(a.status,32768,1,0)) as Resubmitted,

round(avg(a.duration)/1000,3) as AvgRunTime,

max(a.duration)/1000 as MaxRunTime ,

min(a.duration)/1000 as MinRunTime ,

round(AVG(dbms_lob.getlength(a.pipeline)),2) as AvgPipeSize

FROM wmiscoreaudit.wmservice a

where a.audittimestamp > ?

and a.audittimestamp < ?

and a.status in (1, 2, 4, 32768)

group by a.servicename

SQL

SELECT a.servicename,

sum(cast(CASE WHEN a.status = 1 THEN 1 ELSE 0 END as bigint)) as TotalStarted,

sum(cast(CASE WHEN a.status = 2 THEN 1 ELSE 0 END as bigint)) as SuccessCount,

sum(cast(CASE WHEN a.status = 4 THEN 1 ELSE 0 END as bigint)) as OutstandingFailures,

sum(cast(CASE WHEN a.status = 32768 THEN 1 ELSE 0 END as bigint)) as Resubmitted,

round(avg(a.duration)/1000,3) as AvgRunTime,

max(a.duration)/1000 as MaxRunTime ,

min(a.duration)/1000 as MinRunTime ,

round(AVG(cast(DATALENGTH(a.pipeline)as bigint)),2) as AvgPipeSize

FROM wmiscoreaudit.dbo.wmservice a

where a.audittimestamp > ?

and a.audittimestamp < ?

and a.status in (1, 2, 4, 32768)

group by a.servicename

Output can be formatted as HTML with following XSLT service.

<?xml version="1.0"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="html" encoding="UTF-8"/>

<xsl:template match="/">

<xsl:param name="reportheader"/>

<html>

<head>

<!—some CSS styles à

</STYLE>

<title>Webmethods System Status Report</title>

</head>

<body>

(This Email is formatted to be viewed in a Html enabled Email Client)

Webmethods Service Runtime Stats

Note: The day/week of the data in this report is in GMT timezone

Started - Total number of times the Service started. Partial data from current day is not included.

Completed - Total number of times the Service completed successfully. Partial data from current day is not included.

Failed - Total number of times the Service failed due to an error. Partial data from current day is not included.

Resubmitted - Total number of times the Service was resubmitted. This will subsequently result in addition to above counts. Partial data from current day is not included.

<table>

<TBODY>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">ServiceName</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Started</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Completed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Failed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Resubmitted</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgRuntime</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgPipelineSize</TH>

</TR>

<xsl:for-each select="Statistics/wmServiceStats">

<xsl:choose>

<xsl:when test="position() mod 25 = 0">

<TH ALIGN="LEFT" NOWRAP="NOWRAP">ServiceName</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Started</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Completed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Failed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Resubmitted</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgRuntime</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgPipelineSize</TH>

</TR>

</xsl:when>

</xsl:choose>

<xsl:choose>

<xsl:when test="position() mod 2 = 1">

</TR>

</xsl:when>

<xsl:otherwise>

</TR>

</xsl:otherwise>

</xsl:choose>

</xsl:for-each>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Grand Total</TH>

</TR>

</TBODY>

</table>

Daily/Weekly System Runtime Reports will automatically be generated from WebMethods Environment.

Please contact the webMethods Administrator for any questions or concerns about the report.

</body>

</html>

</xsl:template>

</xsl:stylesheet>

Now once this is done a service can be created to enter a number of days (e.g.: 1 for daily reports, 7 for a week). You can take the current day and set is as the toTime input parameter of the query and for the fromTime you can subtract the number of days from today’s date.
Format of those parameters should be “yyyy-MM-dd 00:00:00”.

E.g. for daily statistics: now = 2012-03-08 13:48:35

toTime will be 2012-03-08 00:00:00

fromTime will be 2012-03-07 00:00:00

Which will generate the statistics results for everything processed yesterday

Once this is done and you can execute the adapters we created above and with that result you can generate the html(after conversion to XML)and send it out via mail if you want.

This has been tested on webMethods v8.

Authoer : Jeroen W.

i8c - Business Integration Architects

home

Wednesday, May 2, 2012

Test Driven Development on Oracle Service Bus

Friday, April 27, 2012

Microsoft Installer Custom Actions User Impersonation

Problem

Solution

Monday, April 23, 2012

General Access Denied error (0x80070005) in Hyper-V Manager when starting VM

Tuesday, April 17, 2012

Graceful shutdown of a webMethods Integration Server through custom developed java program

Thursday, March 22, 2012

Daily or Weekly statistics on what has been processed in webMethods