Command Line JMX + SSL

The tool I use for command line JMX and which I think really works well is http://wiki.cyclopsgroup.org/jmxterm

You can download and start the single jar as shown below. In this example I'm connecting to a HornetQ instance.

[jeroen@homesrv~]# java -jar jmxterm-1.0-alpha-4-uber.jar -l service:jmx:rmi:///jndi/rmi://192.168.1.3:10995/jmxrmi
Welcome to JMX terminal. Type "help" for available commands.
$>

Next you can fetch a list of all the available beans (not showing all the beans):

$>beans #domain = JMImplementation: JMImplementation:type=MBeanServerDelegate #domain = com.sun.management: com.sun.management:type=HotSpotDiagnostic #domain = java.lang: java.lang:name=Code Cache,type=MemoryPool ... #domain = java.util.logging: java.util.logging:type=Logging #domain = org.hornetq: org.hornetq:address="hornetq.notifications",module=Core,name="notif.fb6d27d5-02ac-11e0-ae9b-1cc1de6fb76e",type=Queue ... org.hornetq:module=JMS,name="TEST.IN",type=Queue org.hornetq:module=JMS,type=Server

To see the available attributes and operations on a specific MBean you can use the info command:

$>info -b org.hornetq:module=JMS,name="TEST.IN",type=Queue #mbean = org.hornetq:module=JMS,name="TEST.IN",type=Queue #class name = org.hornetq.jms.management.impl.JMSQueueControlImpl # attributes %0 - Address (java.lang.String, r) %1 - ConsumerCount (int, r) %2 - DeadLetterAddress (java.lang.String, rw) %3 - DeliveringCount (int, r) %4 - ExpiryAddress (java.lang.String, rw) %5 - JNDIBindings ([Ljava.lang.String;, r) %6 - MessageCount (int, r) ... %12 - Temporary (boolean, r) # operations %0 - void addJNDI(java.lang.String jndiBinding) %1 - boolean changeMessagePriority(java.lang.String messageID,int newPriority) %2 - int changeMessagesPriority(java.lang.String filter,int newPriority) %3 - int countMessages(java.lang.String filter) ... %20 - int sendMessagesToDeadLetterAddress(java.lang.String filter) #there's no notifications

To get the actual value of an MBean attribute, use the get command

$>get -b org.hornetq:module=JMS,name="TEST.IN",type=Queue MessageCount
#mbean = org.hornetq:module=JMS,name="TEST.IN",type=Queue:
MessageCount = 0;

To invoke an operation use the run command:

$>run -b org.hornetq:module=JMS,name="TEST.IN",type=Queue resetMessageCounter
#calling operation resetMessageCounter of mbean org.hornetq:module=JMS,name="TEST.IN",type=Queue
#operation returns:
null

$>get -b org.hornetq:module=JMS,name="TEST.IN",type=Queue MessageCount
#mbean = org.hornetq:module=JMS,name="TEST.IN",type=Queue:
MessageCount = 0;

As you can see the tool is very easy to use. BUT, since we are talking about Java Management Extensions which let you control, in this case, a HornetQ server, I think some security is in order.
The JMX agent on the JVM is by default started with TLS/SSL and username + password authentication. More information on how to set this up is on: http://download.oracle.com/javase/1.5.0/docs/guide/management/agent.html#auth. In the example above, security was switched off, which is easy to do by setting the following properties:

com.sun.management.jmxremote.authenticate=false
com.sun.management.jmxremote.ssl=false

Now to enable TLS/SSL and username + password on the JMX agent set the following properties, first 2 were set before as well:

com.sun.management.jmxremote com.sun.management.jmxremote.port=10999 com.sun.management.jmxremote.password.file=$HOME/jmxremote.password javax.net.ssl.keyStore=hornetq-srv1-keystore.jks javax.net.ssl.keyStorePassword=changeit

The jmxremote.password is following the template under java.home/jre/lib/management/jmxremote.password.template

On the client side, in this case jmxterm, you need to set the following properties:

javax.net.ssl.trustStore=jmxclient-truststore.jks
javax.net.ssl.trustStorePassword=changeit

This is all following the java specs and should work, but when we start with the following command

[jeroen@homesrv~]# java -Djavax.net.ssl.trustStore=jmxclient-truststore.jks -Djavax.net.ssl.trustStorePassword=changeit -jar jmxterm-1.0-alpha-4-uber.jar -l service:jmx:rmi:///jndi/rmi://192.168.1.3:10999/jmxrmi

The following error pops up:

constituent[0]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/jmxterm.jar
constituent[1]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/commons-beanutils.jar
constituent[2]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/commons-cli.jar
constituent[3]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/commons-collections.jar
constituent[4]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/commons-lang.jar
constituent[5]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/commons-logging.jar
constituent[6]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/commons-io.jar
constituent[7]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/jcli.jar
constituent[8]: jar:file:/root/jmxterm-1.0-alpha-4-uber.jar!/WORLDS-INF/lib/jline.jar
---------------------------------------------------
java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLKeyException: RSA premaster secret error
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2327)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:279)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
at org.cyclopsgroup.jmxterm.cc.SessionImpl.doConnect(SessionImpl.java:85)
at org.cyclopsgroup.jmxterm.cc.SessionImpl.connect(SessionImpl.java:49)
at org.cyclopsgroup.jmxterm.cc.CommandCenter.connect(CommandCenter.java:110)
at org.cyclopsgroup.jmxterm.boot.CliMain.execute(CliMain.java:139)
at org.cyclopsgroup.jmxterm.boot.CliMain.main(CliMain.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchStandard(Launcher.java:353)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:264)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.uberjar.boot.Bootstrapper.bootstrap(Bootstrapper.java:209)
at org.codehaus.classworlds.uberjar.boot.Bootstrapper.main(Bootstrapper.java:116)
Caused by: javax.net.ssl.SSLKeyException: RSA premaster secret error
at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.(RSAClientKeyExchange.java:97)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:673)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:230)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:529)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:465)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:884)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1120)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:623)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
... 25 more
Caused by: java.security.NoSuchAlgorithmException: SunTlsRsaPremasterSecret KeyGenerator not available
at javax.crypto.KeyGenerator.(DashoA13*..)
at javax.crypto.KeyGenerator.getInstance(DashoA13*..)
at com.sun.net.ssl.internal.ssl.JsseJce.getKeyGenerator(JsseJce.java:223)
at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.(RSAClientKeyExchange.java:89)
... 37 more

It looks like the uberjar is doing something fishy with the classloaders which make that the standard sun security provider is no longer available. Bummer ...

This is the only explanation I can think of. Because when connecting with jconsole, like below, I only need to enter the username/pwd. No other issues.

[jeroen@homesrv~]# jconsole -J-Djavax.net.ssl.trustStore=jmxclient-truststore.jks -J-Djavax.net.ssl.trustStorePassword=changeit service:jmx:rmi:///jndi/rmi://192.168.1.3:10995/jmxrmi

I guess for command line jmx with SSL I'll have to look elsewhere ...
If someone has ideas please post :-)


Authored by: Jeroen