Monitoring and Management using JMX

Apache Karaf provides a complete JMX layer.

You can remotely connect to a running Apache Karaf instance using any JMX client (like jconsole).

The Apache Karaf features provide a set of MBeans, dedicating for the monitoring and management.

Connecting

Apache Karaf exposes a complete MBean server that you can use remotely.

The default port number is 1099.

The JMX URL to use by default is:

service:jmx:rmi:///jndi/rmi://localhost:1099/karaf-root

You have to provide an username and password to access to the JMX layer. The JMX layer user the security framework, and so, by default, it uses the users defined in etc/users.properties.

You can change the port numbers of the JMX layer in the etc/org.apache.karaf.management.cfg configuration file.

Configuration

The Apache Karaf JMX management layer is configured in the etc/org.apache.karaf.management.cfg configuration file:

################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# The properties in this file define the configuration of Apache Karaf's JMX Management
#

#
# Port number for RMI registry connection
#
rmiRegistryPort = 1099

#
# Port number for RMI server connection
#
rmiServerPort = 44444

#
# Name of the JAAS realm used for authentication
#
jmxRealm = karaf

#
# The service URL for the JMXConnectorServer
#
serviceUrl = service:jmx:rmi://0.0.0.0:${rmiServerPort}/jndi/rmi://0.0.0.0:${rmiRegistryPort}/karaf-${karaf.name}

#
# Whether any threads started for the JMXConnectorServer should be started as daemon threads
#
daemon = true

#
# Whether the JMXConnectorServer should be started in a separate thread
#
threaded = true

#
# The ObjectName used to register the JMXConnectorServer
#
objectName = connector:name=rmi

  • rmiRegistryPort property contains the port number of the JMX RMI registry. Default is 1099.

  • rmiServerPort property contains the port number of the JMX RMI server. Default is 44444.

  • jmxRealm is the security realm to use as authentication backend. By default it uses the karaf realm.

MBeans

Apache Karaf provides a bunch of MBeans.

The MBeans object names have the same format:

org.apache.karaf:type=[feature],name=[instance]

Installing additional Apache Karaf features and external applications can provide new MBeans.

The following MBeans list is non exhaustive:

  • org.apache.karaf:type=bundle,name=*: management of the OSGi bundles.

  • org.apache.karaf:type=config,name=*: management of the configurations.

  • org.apache.karaf:type=diagnostic,name=*: creation of dumps containing the current Apache Karaf activity (used for diagnostic).

  • org.apache.karaf:type=feature,name=*: management of the Apache Karaf features.

  • org.apache.karaf:type=http,name=*: management of the HTTP service (provided by the http feature).

  • org.apache.karaf:type=instance,name=*: management of the instances.

  • org.apache.karaf:type=jdbc,name=*: management of the JDBC service (provided by the jdbc feature).

  • org.apache.karaf:type=jms,name=*: management of the JMS service (provided by the jms feature).

  • org.apache.karaf:type=jndi,name=*: management of the JNDI service (provided by the jndi feature).

  • org.apache.karaf:type=kar,name=*: management of the KAR file.

  • org.apache.karaf:type=log,name=*: management of the log service.

  • org.apache.karaf:type=obr,name=*: management of the OBR service (provided by the obr feature).

  • org.apache.karaf:type=package,name=*: details about packages exported/imported.

  • org.apache.karaf:type=service,name=*: management of the OSGi services.

  • org.apache.karaf:type=system,name=*: management of the Apache Karaf container itself (halt, restart, etc).

  • org.apache.karaf:type=web,name=*: management of WebApplications (provided by the war feature).

  • org.apache.karaf:type=wrapper,name=*: management of the service wrapper (provided by the wrapper feature).

RBAC

Apache Karaf provides a complete Role-Based Access Control to the JMX MBeans and operations.

Whenever a JMX operation is invoked, the roles of the user are checked against the required roles for this operation.

The access lists are defined in configuration file in the etc folder.

The relevant configuration is prefixed with jmx.acl and based on the JMX ObjectName that it applies to.

For instance, specific configuration for a MBean with the object name foo.bar:type=Test can be placed in the etc/jmx.acl.foo.bar.Test.cfg configuration file.

More generic configurations can be placed in the domain (e.g. jmx.acl.foo.bar.cfg) or at the top level (jmx.acl.cfg).

A simple configuration file looks like:

    # operation = role
    test = admin
    getVal = manager, viewer

In this configuration, Apache Karaf looks for a:

  1. Specific match for the invocation, e.g. test(int)["17"] = role1

  2. Regex match for the invocation, e.g. test(int)[/[0-9]/] = role2 In both cases, the passed argument is converted to a String for the comparison. If any of the above match, the search stops and the associated roles are used.

  3. Signature match for the invocation, e.g. test(int) = role3 If matched, the search stops and the associated roles are used.

  4. Method name match for the invocation, e.g. test = role4 If matched, the search stops and the associated roles are used.

  5. A method name wildcard match, e.g. te* = role5 For all the wildcard matches found in the current configuration file, the roles associated with the longest match are used. So if you have te* and * and the method invoked is 'test', then the roles defined with te* are used, not the ones defined with *.

If no matching definition is found, the most specific definition always takes the precedence.

You can find some configuration examples:

  • Only a manager can call GC on the Memory MBean:

# etc/jmx.acl.java.lang.Memory.cfg
    gc = manager

  • Bundles with ID between 0 and 49 can be stopped only by an admin, other bundles can be stopped by a manager:

# etc/jmx.acl.org.apache.karaf.bundles.cfg
    stop(java.lang.String)[/([1-4])?([0-9]/] = admin
    stop = manager

The etc/jmx.acl.cfg configuration file contains the global configuration for the invocation on any MBean that doesn’t have a specific configuration:

# etc/jmx.acl.cfg
    list* = viewer
    get* = viewer
    is* = viewer
    set* = admin
    * = admin

By default, all "read-only" operations (list*, get*, is*) can be performed by a viewer, whereas the "read-write" operations can be performed only by an admin.

The org.apache.karaf:type=security,area=jmx MBean can be used to check whether the current user can access a certain MBean or invoke a specific operation on it. This MBean can be used by management clients (monitoring tools, etc) to decide whether to show certain MBeans or operations to the end user.

JMX-HTTP bridge with Jolokia

It’s not always easy to use a JMX client with the RMI protocol.

Some monitoring tools (Nagios, Zabbix, …​) are not native JMX clients.

But most of them can use HTTP.

More over, you may want to write your own application/web application. In that case, HTTP and JSON can be very interesting and easy to remotely manage Apache Karaf.

Jolokia can be installed in Apache Karaf as a remote JMX-HTTP bridge.

First, you have to install the http feature:

karaf@root()> feature:install http

Now, you can install directly the Jolokia bundle:

karaf@root()> bundle:install -s mvn:org.jolokia/jolokia-osgi/1.1.5

By default, Jolokia is listening on the port 8181 (see the WebContainer (JSP/Servlet) page for details about the HTTP configuration).

If you point a browser on http://localhost:8181/jolokia you will see a JSON output like:

{"timestamp":1387552680,"status":200,"request":{"type":"version"},"value":{"protocol":"7.0","agent":"1.1.5","info":{"product":"felix","vendor":"Apache","version":"4.2.1"`}

You can manipulate the Apache Karaf JMX layer via HTTP and JSON, via system tools (like curl, jmx4perl, monitoring tools (supporting HTTP/JSON), or web applications.

For instance, you can send a JSON request to get details about the current Apache Karaf heap memory usage.

The format of the request is:

{
    "type":"read",
    "mbean":"java.lang:type=Memory",
    "attribute":"HeapMemoryUsage",
    "path":"used"
}

We can send this JSON request using curl and get the result:

curl -d "{\"type\":\"read\",\"mbean\":\"java.lang:type=Memory\",\"attribute\":\"HeapMemoryUsage\",\"path\":\"used\"}" http://localhost:8080/jolokia/ && echo ""

You can find details on the Jolokia website and in the documentation.

Apache Karaf Decanter

Apache Karaf Decanter provides a complete monitoring solution including data history, turnkey dashboards, SLA and alerting support.