i8c - Business Integration Architects: webMethods

Showing posts with label webMethods. Show all posts

Monday, February 18, 2013

Looking up Tasks faster and more efficiently in webMethods

In case you want to locate tasks (that use standard business data) on the Task Engine to which the webMethods Integration Server is connected to, an invoke to built-in service "pub.task.taskclient:searchTasks" is used. On the other hand, when working with indexed business data fields service "pub.task.taskclient:searchTasksIndexed" is called.

From a past experience an invoke of service "pub.task.taskclient:searchTasks" resulted in a long running sql query on the related Database source. This eventually led to a "java.net.SocketTimeoutException: Read timed out" scenario between Integration Server and Task Engine. A result set of 140 active tasks took more than 90 seconds to get returned causing the Integration Server per default to disconnect the ongoing Task Engine query after 60 seconds.

One option is to put on the Integration Server extended setting "WSClientConfig.SocketTimeout" in place and increase the timeout value (in milliseconds).
A better, more elegant and formal solution is to replace invoke "pub.task.taskclient:searchTasks" by "pub.task.taskclient:searchTasksIndexed" and put boolean businessData to false. The replacement service will take advantage of the built-in index capabilities whereas "pub.task.taskclient:searchTasks" doesn't.

When using service "pub.task.taskclient:searchTasksIndexed" instead of "pub.task.taskclient:searchTasks" a similar result set was returned in a couple of seconds instead of causing an operational problem for the integration.

Author: Johan

Tuesday, April 17, 2012

Graceful shutdown of a webMethods Integration Server through custom developed java program

Out of the box a webMethods Integration Server is shutdown via the Administrator screen.

Following code snippet with allow you to gracefully shutdown a webMethods Integration Server through the usage of a custom developed java program.

Java code snippet :

import java.io.*;

import com.wm.app.b2b.client.*;

import com.wm.util.*;

import com.wm.data.*;

public class IntegrationServerShutdown

{

public String hostName = null;

public String port = null;

public String userName = null;

public String password = null;

Context context = null;

public boolean connected = false;

public static void main(String[] args)

{ String input = null;

//Create an instance of a b2bServer

IntegrationServerShutdown b2bServer = new IntegrationServerShutdown();

b2bServer.hostName=args[0];

b2bServer.port=args[1];

b2bServer.userName=args[2];

b2bServer.password=args[3];

System.out.println("");

b2bServer.connect();

if(!b2bServer.connected)

System.exit(0);

b2bServer.shutdown();

}

public void connect()

{ context = new Context();

try

{ context.connect(hostName + ":" + port, userName, password);

System.out.println("Connected to " + hostName + ":" + port);

connected = true;

}

catch(ServiceException e)

{ System.out.println("Could not connect to " + hostName + ":" + port);

System.out.println(e.toString());

connected = false;

}

public void disconnect()

{ if(context != null)

context.disconnect();

}

public void shutdown()

{ try

{

System.out.println("Shutting down IS Server... ");

IData inputs = IDataFactory.create();

IDataCursor myCursor = inputs.getCursor();

myCursor.insertAfter("bounce", "no");

myCursor.insertAfter("timeout", "0");

myCursor.insertAfter("option", "force");

context.invoke("wm.server.admin","shutdown",inputs);

System.out.println("IS Server shutdown complete.");

System.exit(0);

}

catch(ServiceException e)

{ System.out.println("IS Server shutdown failed.");

System.out.println(e.toString());

System.exit(1);

}

Remarks :

When compiling and running the java program make sure to include files wm-isclient.jar and mail.jar in your class path. File wm-isclient.jar can found in the “common/lib” folder of an Integration Server whereas file mail.jar is localized in the “ext” sub folder of the “common/lib” folder.
Usage of the IntegrationServerShutdown program is IntegrationServerShutdown <IS hostname> <IS port> <IS account with administrative privileges> <IS account password>
At this moment the java program is working with at least webMethods Integration Server v7.1.2 and v8.2. Specifications contained herein are potentially subject to change so use them at your own risk.

Author: Johan De Wulf

Thursday, March 22, 2012

Daily or Weekly statistics on what has been processed in webMethods

This arctivle will describe how we can get some daily or weekly statistics from the database (MsSql or Oracle) of webMethods.

Let us start with the Dynamic SQL Adapters. These are needed to run a query which will return us the statistics of a specific period.

Input

Output

Query to enter in the Dynamic Adapter.

Oracle

SELECT a.servicename,

sum(decode(a.status,1,1,0)) as TotalStarted,

sum(decode(a.status,2,1,0)) as SuccessCount,

sum(decode(a.status,4,1,0)) as OutstandingFailures,

sum(decode(a.status,32768,1,0)) as Resubmitted,

round(avg(a.duration)/1000,3) as AvgRunTime,

max(a.duration)/1000 as MaxRunTime ,

min(a.duration)/1000 as MinRunTime ,

round(AVG(dbms_lob.getlength(a.pipeline)),2) as AvgPipeSize

FROM wmiscoreaudit.wmservice a

where a.audittimestamp > ?

and a.audittimestamp < ?

and a.status in (1, 2, 4, 32768)

group by a.servicename

SQL

SELECT a.servicename,

sum(cast(CASE WHEN a.status = 1 THEN 1 ELSE 0 END as bigint)) as TotalStarted,

sum(cast(CASE WHEN a.status = 2 THEN 1 ELSE 0 END as bigint)) as SuccessCount,

sum(cast(CASE WHEN a.status = 4 THEN 1 ELSE 0 END as bigint)) as OutstandingFailures,

sum(cast(CASE WHEN a.status = 32768 THEN 1 ELSE 0 END as bigint)) as Resubmitted,

round(avg(a.duration)/1000,3) as AvgRunTime,

max(a.duration)/1000 as MaxRunTime ,

min(a.duration)/1000 as MinRunTime ,

round(AVG(cast(DATALENGTH(a.pipeline)as bigint)),2) as AvgPipeSize

FROM wmiscoreaudit.dbo.wmservice a

where a.audittimestamp > ?

and a.audittimestamp < ?

and a.status in (1, 2, 4, 32768)

group by a.servicename

Output can be formatted as HTML with following XSLT service.

<?xml version="1.0"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="html" encoding="UTF-8"/>

<xsl:template match="/">

<xsl:param name="reportheader"/>

<html>

<head>

<!—some CSS styles à

</STYLE>

<title>Webmethods System Status Report</title>

</head>

<body>

(This Email is formatted to be viewed in a Html enabled Email Client)

Webmethods Service Runtime Stats

Note: The day/week of the data in this report is in GMT timezone

Started - Total number of times the Service started. Partial data from current day is not included.

Completed - Total number of times the Service completed successfully. Partial data from current day is not included.

Failed - Total number of times the Service failed due to an error. Partial data from current day is not included.

Resubmitted - Total number of times the Service was resubmitted. This will subsequently result in addition to above counts. Partial data from current day is not included.

<table>

<TBODY>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">ServiceName</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Started</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Completed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Failed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Resubmitted</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgRuntime</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgPipelineSize</TH>

</TR>

<xsl:for-each select="Statistics/wmServiceStats">

<xsl:choose>

<xsl:when test="position() mod 25 = 0">

<TH ALIGN="LEFT" NOWRAP="NOWRAP">ServiceName</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Started</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Completed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Failed</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Resubmitted</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgRuntime</TH>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">AvgPipelineSize</TH>

</TR>

</xsl:when>

</xsl:choose>

<xsl:choose>

<xsl:when test="position() mod 2 = 1">

</TR>

</xsl:when>

<xsl:otherwise>

</TR>

</xsl:otherwise>

</xsl:choose>

</xsl:for-each>

<TH ALIGN="LEFT" NOWRAP="NOWRAP">Grand Total</TH>

</TR>

</TBODY>

</table>

Daily/Weekly System Runtime Reports will automatically be generated from WebMethods Environment.

Please contact the webMethods Administrator for any questions or concerns about the report.

</body>

</html>

</xsl:template>

</xsl:stylesheet>

Now once this is done a service can be created to enter a number of days (e.g.: 1 for daily reports, 7 for a week). You can take the current day and set is as the toTime input parameter of the query and for the fromTime you can subtract the number of days from today’s date.
Format of those parameters should be “yyyy-MM-dd 00:00:00”.

E.g. for daily statistics: now = 2012-03-08 13:48:35

toTime will be 2012-03-08 00:00:00

fromTime will be 2012-03-07 00:00:00

Which will generate the statistics results for everything processed yesterday

Once this is done and you can execute the adapters we created above and with that result you can generate the html(after conversion to XML)and send it out via mail if you want.

This has been tested on webMethods v8.

Authoer : Jeroen W.

Monday, March 12, 2012

Concept : suspending triggers automatically when target system is down

With this question we came up with a concept to automatically detect if there is one of the target systems down and then suspend the triggers used for this connection.

Once the system detects that this target system is back online, we will enable the triggers again and the normal process can continue.

Why did we start this concept?

We’ve seen that we cannot guarantee the 24/7 uptime of the target systems and we wanted to have some dynamic tool which will take care of these downtimes (scheduled or not) so we shouldn’t have to worry about any failed transactions and resubmit them. This in some cases can be a hard task if there are thousand failed messages.

Concept requirements:

1. Database

We need to identify the relationship between the trigger and the connections. Why? One connection can be used in different packages and also one package can have multiple triggers pointing to different target systems.

We will store this data in a database which can be easily implemented and cached as well.

Info required for the database
Full path name of the trigger
Package name of the trigger
Type of connection (e.g. : JDBC, SAP,…)
Full path name of the connection
identifier to make it unique if we have multiple target systems for 1 trigger. So we can identify all triggers to one target system.

2. Monitoring Job

We also need to have a job running every x time to check if all connections are still active and responding. We will create a dummy query per type of connection so we can see if the target system is still responding, if not there is an issue and we should not send any data towards that IZ.

Of course we also need to build in some security checks like, what if there was a network blips why the dummy query didn’t ran successfully. So we will have to execute this a 2^nd time to make sure that there is definitely an issue, but we have to delay this request with some time (e.g. one minute) to know there are no one time issues.

Let’s say we’ve detected a target system which is down. We now have to start another thread which will suspend the triggers and connection for that target system. Why disabling the connection? It’s not required, but easier to monitor if a connection has been disabled. As in a good environment we won’t have any disabled connections.

3. Recycle Job

This is the job which will check if the target system is back online and responding. In this we can try to enable the connection again. If this doesn’t work that means the target system is still not responsive. If the connection get’s enabled again we can enable the triggers again and resume normal processing. If not, re-execute this job again. The purpose of this job is to check every X time if the connection is back online. Take in mind we will not execute this job every seconds which will be an overkill of the machines. But we can let it wait one minute and then try again.

4. Queue Monitor

If we suspend processing of documents they will remain in the broker, so we will have to monitor the broker so it will not grow to large and even in the worst case the broker can crash.

We can use Optimize for that of develop custom code to check the total size of the broker and alert if it reaches a predefined limit. We could even make it “smart” and release the trigger that is causing the broker to pill up and once the broker is back to a “safe” size, we can see if the target is back online or not, if not we can suspend this trigger again.

Now with this information in mind we can develop some concept. Will keep you posted on the progress and post some code samples as well.

Author : Jeroen W.

Friday, January 20, 2012

How to: automatically remove LOCKFILE on server start (WINDOWS)

Have you ever had that annoying issue when your server started up and the LOCKFILE still existed in the webMethods folder?

I’ve created a small script for windows which tackles this issue.

set jobname=softwareAGwebMethodsIntegrationServer_8.0

set location=C:\SoftwareAG\IntegrationServer\LOCKFILE

for /F "tokens=3 delims=: " %%H in ('sc query "%jobname%" ^| findstr " STATE"') do (

if /I "%%H" NEQ "RUNNING" (

IF EXIST %location% GOTO REMOVEFILEANDSTART

)

:REMOVEFILEANDSTART

del %location%

net start "%jobname%"

Variable jobname is your servicename which you set to start automatic

To find this name open Services and double click the job name

The name entered behind Service name is the name you need to enter for this variable.

Variable “location” is the location of the LOCKFILE, enter the full path.

Explanation of the code :

set jobname=softwareAGwebMethodsIntegrationServer_8.0

set location=C:\SoftwareAG\IntegrationServer\LOCKFILE

Declaration of the variables.

for /F "tokens=3 delims=: " %%H in ('sc query "%jobname%" ^| findstr " STATE"') do (

if /I "%%H" NEQ "RUNNING" (

IF EXIST %location% GOTO REMOVEFILEANDSTART

)

Loop over the result of sc query "%jobname%" and search for the word STATE

If the result of STATE is different from RUNNING then execute function REMOVEFILEANDSTART

:REMOVEFILEANDSTART

del %location%

net start "%jobname%"

Delete the LOCKFILE and start the service.

To use this script, create a scheduled task which is triggered on server startup.

Author: Jeroen W.

home

Monday, February 18, 2013

Looking up Tasks faster and more efficiently in webMethods

Tuesday, April 17, 2012

Graceful shutdown of a webMethods Integration Server through custom developed java program

Thursday, March 22, 2012

Daily or Weekly statistics on what has been processed in webMethods

Monday, March 12, 2012

Concept : suspending triggers automatically when target system is down

Friday, January 20, 2012

How to: automatically remove LOCKFILE on server start (WINDOWS)