Tuesday, May 13, 2014

BizTalk Health Statistics - Scripts

Sometimes you want to know how many messages have passed through BizTalk. Often this question pops up from a business perspective where they want to know how the use of the environment is evolving, in either positive or negative directions. It’s also useful to set out guidelines on how much data your environment can handle and to create upgrade plans when emergency levels are being reached.

There are four kinds of statistics that we always try to gather. These are: 
  • - Messages passing through per day (In & Out)
  • - Instances (orchestrations & pipelines) executed per day
  • - Usage of host instances
  • - Largest message sizes

The statistics are gathered by executing SQL scripts on the Tracking Database.

Messages per day

SELECT datepart(dd, [dtInsertionTimeStamp]) as dateMsg, datepart(MM, [dtInsertionTimeStamp]) as monthMsg, count(Convert(char(10), [dtInsertionTimeStamp], 108)) as ant
FROM [BizTalkDTADb].[dbo].[dta_MessageInOutEvents] WITH (NOLOCK)
WHERE [dtInsertionTimeStamp]
BETWEEN convert(datetime, '2014-02-10 00:00:00', 120) AND convert(datetime, '2014-02-16 23:59:59',  120)
GROUP BY datepart(dd, [dtInsertionTimeStamp]),datepart(MM, [dtInsertionTimeStamp]) ORDER BY [dateMsg]ASC, [monthMsg]

The “BETWEEN” part can be filled in with a timeframe of several days (in case your tracking database keeps all data on these days. A result as the following will be received:

dateMsg
monthMsg
ant
10
2
277262
11
2
381427
12
2
427297
13
2
291388
14
2
285235
15
2
3290
16
2
672

Instances per day

SELECT  datepart(dd, [dtInsertionTimeStamp]) as dateMsg, datepart(MM, [dtInsertionTimeStamp]) as monthMsg, count(Convert(char(10), [dtInsertionTimeStamp], 108)) as ant
FROM [BizTalkDTADb].[dbo].[dta_ServiceInstances] WITH (NOLOCK)
WHERE [dtInsertionTimeStamp]
BETWEEN convert(datetime ,'2014-02-10 00:00:00', 120) AND convert(datetime, '2014-02-16 23:59:59', 120)
GROUP BY datepart(dd, [dtInsertionTimeStamp]), datepart(MM, [dtInsertionTimeStamp] )ORDER BY [dateMsg] ASC, [monthMsg]

Also in this script, the “BETWEEN” part can be filled in with a timeframe. Following result is received:

dateMsg
monthMsg
ant
10
2
127369
11
2
183848
12
2
201597
13
2
138431
14
2
132828
15
2
1536
16
2
314

Usage of host instances

SELECT count(Convert(char(10), [dta_ServiceInstances].[dtInsertionTimeStamp], 108)) as ant, [dta_Host].[strHostName]
FROM [BizTalkDTADb].[dbo].[dta_ServiceInstances] WITH (NOLOCK)
INNER JOIN [BizTalkDTADb].[dbo].[dta_Host] WITH (NOLOCK) ON
[dta_ServiceInstances].[nHostId]=[dta_Host].[nHostId]
WHERE [dtInsertionTimeStamp]
BETWEEN convert(datetime ,'2014-02-10 00:00:00', 120) AND convert(datetime, '2014-02-10 23:59:59', 120)
GROUP BY [dta_Host].[strHostName] ORDER BY [dta_Host].[strHostName] ASC

In this script, a choice can be made in terms of grouping. When you want to have just the numbers of a timeframe (for example: between feb 01 and feb 15, there were 23,765 calls of MQHostReceive), just add the timeframe in the “BETWEEN” field. If you want the data grouped by day, the script needs to be executed once for every day you want the statistics. Set the “BETWEEN” to 2014-02-12 00:00:00 and 2014-02-12 23:59:59 for example, and you will get the following results:

ant
strHostName
303
NWND_BAM
301
NWND_BizTalkServerAdministation
23217
NWND_BizTalkServerApplication
15
NWND_ClusteredHost_RCV
18
NWND_ClusteredHost_SND
48481
NWND_MQConnectionHost_RCV
18127
NWND_MQConnectionHost_SND
10836
NWND_SAPProcessingHost
164
NWND_SAPLendingHost
2653
NWND_SIEBELProcessingHost
16746
NWND_TECHNICOMProcessingHost
6508
NWND_WCFHost

Largest message(s)

SELECT TOP 5 [dta_MessageInOutEvents].[nMessageSize], [dta_SchemaName].[strSchemaName]
FROM [BizTalkDTADb].[dbo].[dta_MessageInOutEvents] WITH (NOLOCK)
INNER JOIN [BizTalkDTADb].[dbo].[dta_SchemaName] WITH (NOLOCK) ON
[dta_MessageInOutEvents].[nSchemaId]=[dta_SchemaName].[nSchemaId]
WHERE [dtInsertionTimeStamp]
BETWEEN convert(datetime ,'2014-01-01 00:00:00', 120) AND convert(datetime, '2014-01-01 23:59:59', 120)
ORDER BY [nMessageSize] DESC

This script will get a top 5 of all the largest messages that passed through BizTalk in the given timeframe. A result like following will pop up:

nMessageSize
strSchemaName
28272537
http://NWD.DataSchema.Issuer.Schemas#Issuers
28272537
28272537
28255369
28255369

If you got any remarks, please leave them in the comment section!

Andrew De Bruyne (@draitnn)

Friday, May 2, 2014

BizTalk MQSeries Adapter COM+ Error 80004005

When connecting BizTalk to Websphere MQ, you can choose between two options:

-       MQSC adapter (Client based)
-       MQSeries (Server based)

My current client chose to use the MQSeries adapter to connect to Websphere MQ. The core of this adapter is a COM+ component running on the BizTalk server which will handle all processing.

Issue

Normally, all processing runs smoothly. But sometimes, when executing a new installation of an application on BizTalk, we started receiving the following error-message:

“Retrieving the COM class factory for remote component with CLSID {} from machine xx failed due to the following error: 80004005”

Also, we noticed that all messages that were sent to the MQSeries adapter, were staying in status Started with the same error. All processing was gone and nothing was going to or coming from MQ.




When first checking the running component, we didn’t see anything weird. Everything was running fine. We looked into the issue for quite some hours but didn’t find anything that was causing the problems.

Solution

The problem itself lays in the fact that the COM+ component can recycle itself but that the recycle doesn’t work properly. So we don’t see any “extra” failing instances of the component but things change when we boot up the process explorer:




We see two instances of the component! We can tell this for sure since the MQSAgent2 is the only thing running under this specific account (check the properties of a process – right click – properties). So there is an instance running correctly, but an old one, which isn’t visible in the running, processes on the Component Services, but still present in the windows processes. This faulty component is still taking in all the incoming requests.

Only thing to figure out now is the process id of the component that is currently running. This process id is visible in the components mmc.


It’s now just a matter of killing the other component to resolve the issue.

Extra

The other measure we took to avoid extra downtime of the environment was raising the pool size of the component. This means that when a component starts to recycle another component is started which will automatically take in all the new requests while the other is still recycling. This can be set in the properties of the component.

Any questions can be put in the comments, or you can contact me through twitter directly.

Andrew De Bruyne (Twitter: @draitnn)

Friday, April 25, 2014

Bug in WebMethods OneData MDM 9.5 standard configuration

When you install WebMethods OneData MDM 9.5 using the Software AG installer, 2 database connections will be created by default as shown in the image below:
1) STG: connection to the Work Area
2) PRD: connection to the Release Area



The parameters for both connections are by default configured using JNDI. The corresponding full config details can be found in the Tomcat config file <install dir>\profiles\ODE\configuration\tomcat\conf\context.xml. These contain JDBC URL, username, password,... Although the username is specified in the JNDI context in the context.xml file, it is also by default specified in the connection details in OneData as shown in the image below.



The value of the User-ID parameter seems be generated automatically and composed out of the connection prefix you define during the installation in SAG Installer, concatenated with the suffix "_wa" for the work area connection and "_ra" for the release area connection.

Although it seems that this configuration is working fine, this is actually not 100% the case. It turns out that when the value in the User-ID parameter doesn't correspond with the actual Oracle username (which is very likely), some functionality like the "Schema Update" to create new Objects from existing tables in the Release Area will not work (existing tables are not shown in OneData UI). So in order to avoid potential unexpected behavior in OneData you should update the User-ID parameter after installation to the actual Oracle username (so identical to the username in context.xml).

Another solution is to not use the JNDI config and copy all the configuration details from the context.xml file to the corresponding fields in the Connection Parameters form in OneData.

This problem also seems to occur in OneData 9.0. I haven't been able yet to verify it in version 9.6. but I couldn't find any record of this error on Software AG Empower, so the bug probably still exists in version 9.6 also.

Author: Kristof Lievens

Monday, April 7, 2014

Fixing HTTP redirect for ARIS 9.5 Design Server after an IP address change

I recently moved a Windows Server 2012 virtual machine running ARIS 9.5 Design Server, from my laptop running VMWare Workstation to a VMWare ESX 5.5 host. My Windows Server 2012 VM has one single network adapter and changing the VM host implied that the IP address of the VM had to be changed because the VM hosts were running in different netwerk segements (my Windows Server 2012 network connection had DHCP enabled, so the IP address changed automatically actually).

After moving my VM, I was able to start ARIS Design Server without a problem, but to my surprise, my browser was redirected to my old IP address every time I tried to connect to the ARIS Design server at default port 5480, even when I used the new IP address of my server or "localhost" as hostname. Turns out that the old IP address was still used by the load balancer component in ARIS Design Server. To fix this problem, you have to update the "httpd.conf" configuration file in the folder <ARIS install root>\server\bin\work\work_loadbalancer_m\httpd\conf. Look for the "ServerName",  "RewriteCond" and "RewriteRule" parameters and update them with the new IP address. After restarting the ARIS Design Server, the redirect in my browser worked correctly.

Author: Kristof Lievens