JCaps 5.1.2 JMS IQ Manager database files

A blog entry on the JMS IQ manager that comes with (Sun) JCAPS, triggered by a problem that we recently encountered with it.


First a small explanation of how the JMS IQ Manager database works
The JMS IQ Manager uses the JMS IQ Manager database to store persistent messages. The database is also used to store messages that are larger than can be kept in the JMS IQ Manager memory. By default, JMS clients are configured for persistent messaging; therefore, in a default configuration, the database is used to store these messages. The messages are stored until they are consumed or until the duration set for the maximum time to live for a live message expires, which is 30 days by default.

The database resides in the message server folder on the Logical Host. The database consists of a number of database files called segments. A segment is a diskspace store that is memory-mapped on the server. The segments act together to form the equivalent of a sequential database. By default, these files are named stcms*.dbs.

The JMS IQ Manager creates four segments in the database when it starts up initially. The default size of a segment is 8 MB on Windows and 16 MB on UNIX. The JMS IQ Manager creates as many segments as necessary. Before running your Java CAPS Project, it is important to set the segment size to a larger value than the largest transaction the JMS IQ Manager may need to process. The JMS IQ Manager cannot accommodate a transaction that is larger than the segment size.

Configurations such as the database filenames, segments size, the maximum and minimum number of segments created can all be specified.

The JMS IQ Manager database resides in the directory specified by the JMS IQ Manager Data directory property. The standard location for the database files is as follows: ..\logicalhost\is\domains\domain_name\stcms\instance1\stcms*.dbs

Figure 1: shows the default Sun SeeBeyond JMS IQ Manager settings (This means that when the JMS IQ Manager starts up, the database consists of four segments; the total size of the database is 32 MB on Windows):

Figure 2: shows the JMS IQ Manager Database Structure:


On startup, the JMS IQ Manager performs the following operations:

1. It allocates sufficient disk space to hold the minimum number of segments. Figure 2 shows a JMS IQ Manager allocation of four segments, numbered 7ef325 through 7ef328.
2. As messages arrive, they are appended to the first segment until the segment is full. When a segment is full, the JMS IQ Manager stores subsequent messages in the first free segment. Figure 2 shows that the third segment, 7ef327 in file stcms7ef327.dbs, is almost full.
3. If there is no free segment, the JMS IQ Manager allocates a new segment if possible.
4. Memory is freed up as soon as the consumer has acknowledged the message or committed the transaction.
5. When all messages in a segment have expired or been removed from their queues, the JMS IQ Manager cleans up the segment, freeing it for re-use. In Figure 2, the first segment (7ef325) has several segments that are white, indicating the slot is marked eligible. The segment is therefore almost ready for cleanup.

Problem we noticed in JCaps 5.1.2:
In JCaps 5.1.2 we noticed that sometimes transactions from weeks/months ago were processed again without any particular reason. After some research we discovered that the transactions were not resend by any source system, so the problem was located somewhere in the JCaps system.

After a lot of research we noticed that the problem only occurred when the number of dbs files was increasing and not cleaned up. We checked all queues and they were all empty.

So the JMS IQ Manager database files were not cleaned properly. With high throughput there is an overlap of the commit actions which may cause the disk cache to “not” being flushed. There are little intervals between an entering message being stored inside a segment file and the subsequent commit, and the interval between a message being retrieved and the subsequent commit. Dealing with a large number of publisher or just a high level of input, the overlap of commit actions will prevent the IQM from flushing its disk cache and sending an acknowledgement to the subscribers. Try to split your publishers/subscriber over multiple message servers.

Due to the fact that JMS IQ Manager database files are “not” being flushed, it can happen that messages are reprocessed!

You can clean the JMS IQ Manager database files:

1. Stop all ‘applications’ which can post new messages on the queues
2. Check if there are no more message in the queues using f.e. the Enterprise Manager
3. Stop the JCaps domain
4. Delete all JMS IQ Manager database file (*.dbs) in directory <JCAPS Installation Dir>\logicalhost\is\domains\<Domain>\stcms\instance1
5. Start the JCaps domain

Author: Vaast