Configuration Support for GlassFish V3

Kedar Mhaswade, Lloyd Chambers, Jerome Dochez, Kohsuke Kawaguchi

Introduction

This document focuses on ideas about integrating configuration support for GlassFish V3. Some of the ideas will be implemented into HK2 codebase and others will be part of the V3 Config support. Few goals that we tried to satisfy are:

Extensibility
Support for JMX/AMX
Dynamic reconfiguration
Batching configuration changes and allowing for role based administration
Provision of Validation

Reader is assumed to be familiar with basic principles of V3, Java 5 and JMX.

Overall Scheme

At some level, a V3 distribution is a set of HK-2 modules. This document discusses the need of configuring these modules or just the objects that create these modules. The configuration of these objects can optionally be stored in domain.xml. When a server runtime starts up, the modules come into being (according to a set of contracts) and a habitat of configuration objects comes into existence. The information from the domain.xml is injected into the configuration objects according to certain annotations and/or interfaces available in modules. The configuration objects themselves may well be the MBeans that are exposed for management. The exact distinction is not yet finalized.

Let's take a snap-shot of such a runtime. Typically, this is how it looks like:

You have a few modules started (those that satisfy the @ModuleStartup contract). Lets say there are two modules: Grizzly and WebContainer. These modules are configured by certain configuration objects, which are in turn specified by certain Elements and Attributes in domain.xml. Needless to say, each configuration object is an instance of a Java Class (or implementation of a Java Interface).

A set of configuration objects and corresponding MBeans identified by domain.xml elements like: http-listener and virtual-server are read from domain.xml. Let's work with following snippet placed at a convenient location (yet to be determined) in domain.xml:

 
		<http-listener id="ls1" port="8080" address="0.0.0.0" secure="false" vs="vs1"/>
		    <property name="GrizzlyBufferSizeInBytes" value="8192">
		<http-listener id="ls2" port="8181" address="0.0.0.0" secure="true" vs="vs1">
		    <ssl alias="verisign-1"/>
		<http-listener>
		<virtual-server id="vs1" hosts="www.abc.com">

Thus, at this point in time, there are five MBeans in the runtime. They have defined interfaces and ObjectNames. It is important to note that the MBeans might have a number of management operations not evident from the domain.xml. These include operations like creation and deletion of a child element. Another subtle point is that these MBeans are arranged in a flat ObjectName patterns that are predefined with certain conventions. This is required to look them up according to a hierarchy. This is achieved by introducing certain property names in the property list of ObjectName. For example:
- type specifies the class name of the configuration object, like org.glassfish.web.HttpListener
- parent specifies the type of the logical parent of a given configuration object.

With this snap-shot, this section outlines how administrative tools (via configuration infrastructure) carry out basic administrative operations. A basic administrative operation is any of:

Modification of an existing element in terms of attribute values.
Addition or Deletion of an existing element by name.
Listing elements of the same class (e.g. all http-listener elements).
Accessing an element attribute, a set of element attributes, an element or a set of elements through a (convenient) dotted name specification.
Invoking a long-running operation that accesses (or mutates) a set of configuration objects that may or may not be known a priori. (The thought process for this is yet to be complete).

Here is how this is proposed to be done:

Administrative interfaces find out about the MBean of interest (somehow). For example, http-listener, ls1, in above snippet. Consider that its ObjectName is on1. Interfaces also establish an MBeanServerConnection (mbsc) to the MBeanServer of the admin server.
The meta-data of this MBean is made available through mbsc.getMBeanInfo(on1).
Five operations are of essence:
1. mbsc.setAttribute(on1, attribute1), where attribute1 is: "port" -> 9080 (instead of 8080)
2. mbsc.setAttributes(on1, list1), where list1 is: "port"->9080, "address"->"mymachine"
3. mbsc.invoke(on1, "createchild" ...)
4. mbsc.getAttribute(on1, "port")
5. mbsc.getAttributes(on1, list1)
At the MBeanServer level, when it knows which MBean(s) are getting modified or accessed, the MBean(s) is placed in a unique Transaction. When an MBean is placed in a Transaction this way, it is immediately cloned and all the modifications made to the MBean are made on the clone. Thus, during the Transaction, a copy of MBean corresponding to ls1 above is modified and/or accessed. While a particular MBean is in a Transaction and the operation might mutate the MBean state, all the other requests for Transaction that may or may not involve that MBean fail (See the unresolved question #8 to decide whether we can do better).
During an in-flight Transaction, an MBean or a set of MBeans is modified. The system knows both the old and modified state of the MBeans along with the state of rest of the configuration. The Validation phase then commences under these circumstances, in the same Transaction. Till now, only the copies of MBeans are modified. With every modification of the MBean, a JMX AttributeChangeNotification is emitted to which a Transaction-specific JMX NotificationListener listens. Thus, this listener is responsible for maintaining the list of changes (Delta) during the Transaction.
When the Validation fails, the MBeans/configuration objects in the Transaction are invalidated and error is reported.
When the Validation succeeds, it means few things (and this is something where lot of discussion should happen):
1. The domain.xml is kosher and can be flushed.
2. Co-located View: When the (copy of) configuration objects are shared between config infrastructure and affected module (say Grizzly here), a successful change in state implies that the runtime actually responded to the change and adapted to it. Thus, it is not only valid, but also currently applied. This is a significant deviation from how things happen in GlassFish V2. Call this item as the Co-located View.
3. Distributed View: When the (copy of) configuration objects are not shared between config infrastructure and affected module (this is the classic enterprise case, where admin server sees/makes the changes in modules running in a cluster), a change in state implies that only validation is successful. Thus, in this view, since domain.xml is flushed, the runtime system is not modified (has not adapted to the changes). Call this item as the Distributed View.
  It is important that we understand and agree upon the differences in Co-located and Distributed Views. And if we don't agree, we need to modify the design.

Unresolved Questions

Should MBeanServer be injected into the configuration objects? Most likely, MBeanServer needs to be singleton because that is what is associated with remote JMX Connectors. This is better, for it gives us a control over what we need to with MBeanServers and MBeanServer Interceptors.
Do we use VetoablePropertyChangeEvent or JMX AttributeChangeNotification or both?
Where should the configuration of an arbitrary class (a module that is new) be specified in domain.xml? In other words, should we retain the referential structure of the domain.xml?
Since there is no schema, what should be done with default values of attributes? How can admin tools get their view?
How to do the long-running operations?
Should the Validation be delegated to the module classes?
Should the read-only operations be done outside a Transaction?
Should the entire set of MBeans be locked while a Transaction is in flight or we can do something better? In other words, locking semantics have to be carefully done.