Writing a post-delegation classloader

22

Sometimes in Java you need to load a set of classes that are isolated from the classpath. For instance, you might need to support the capability to plugin a library to your main application and you want to be able to isolate the libraries used by the plugins from your main application.

In this situation, you need to instantiate each plugin in a classloader that subverts the normal classloader delegation hierarchy. Java classloading usually works by first checking the parent for a loaded class, then checking the current classloader if the class is not found. This is the recommended and expected behavior for a classloader.

However, in the plugin scenario, you want just the opposite behavior. You want to check your own classloader first so that the plugin’s libraries are used before any used by the main application. You then must delegate to the parent classloader after checking your own classloader so that any shared interfaces or libraries and all the system classloaders are still checked.

A key design point is that the plugin should integrate with the main application entirely through interfaces or classes that are defined in the parent classloader. Most commonly, the main application defines interfaces and the plugin returns implementations of those interface. Usually the plugin is loaded in the main application via reflection to force the use of the plugin classloader.

It is very important that the plugin not include these shared interfaces in its classloader as the post-delegation classloader will load those instead of the ones in the main application. Classes (even from the same identical jar) loaded in different classloaders are seen as different classes by the Java VM. This will typically cause a Java VM to freak out and throw a LinkageError. So, don’t do that.

In the rest of this post, I’ll show how to implement the classloader and call the classloader, and then put it together in an example that demonstrates two versions of a class being used in the same application.

First, the classloader. We will extend URLClassLoader as it provides a ton of useful capability for loading classes from a classpath defined as a set of URLs. Normally when you create a classloader, you are expected to override the findClass method. This is the method that actually takes a class name and returns a Class. However, we’re fine with the findClass implementation, we really want to manipulate the delegation order and that is defined in the loadClass method.

The implementation will look something like this:

    public Class<?> loadClass(String name) throws ClassNotFoundException {
        // First check whether it's already been loaded, if so use it
        Class loadedClass = findLoadedClass(name);
        
        // Not loaded, try to load it 
        if (loadedClass == null) {
            try {
                // Ignore parent delegation and just try to load locally
                loadedClass = findClass(name);
            } catch (ClassNotFoundException e) {
                // Swallow exception - does not exist locally
            }
            
            // If not found locally, use normal parent delegation in URLClassloader
            if (loadedClass == null) {
                // throws ClassNotFoundException if not found in delegation hierarchy at all
                loadedClass = super.loadClass(name);
            }
        } 
        // will never return null (ClassNotFoundException will be thrown)
        return loadedClass;
    }

Other than implementing the constructors, that’s it!

Let’s look at a sample application. First, let’s define a trivial application with plugins.

// Simple "common" class that will have multiple versions
public class CommonClass {
    public String toString() {
        return "Common class version 1";
    }
}

// Simple plugin interface
public interface Plugin {
    public String test();
}

// The application
public class Application {
    public static void main(String arg[]) throws Exception {
        // Use the initial version of CommonClass in the main application
        System.out.println("Main says: " + new CommonClass().toString());

        // Instantiate the PostDelegationClassLoader with a classpath pointing to a plugin
        URL[] classpath = new URL[] { new URL("file:../plugin/") };
        PostDelegationClassLoader classLoader = new PostDelegationClassLoader(classpath);

        // Use the classloader to instantiate the plugin in the classloader.  Since this is done 
        // by reflection in a classloader, there is no need for the main application to have 
        // the PluginImpl class at compile time, although it must have the Plugin interface.
        Plugin plugin = (Plugin) classLoader.loadClass("PluginImpl").newInstance();

        // Use the plugin, which will use a different version of CommonClass
        System.out.println("Plugin says: " + plugin.test());
    }
}

Then, we need to actually define the plugin and second version of CommonClass:

// Second version of common class used by the plugin
public class CommonClass {
    public String toString() {
        return "Common class version 2";
    }
}

// Plugin implementation which uses CommonClass
public class PluginImpl implements Plugin {
    public String test() {
        return new CommonClass().toString();
    }
}

We can then compile these two sets of code and run the application. Note that at runtime, the main application will not include the PluginImpl and CommonClass (version 2) code in the system classpath. Instead, they will be included only in the post-delegation classloader.

At runtime you will see:

Main says: Common class version 1
Plugin says: Common class version 2

This shows that two versions of the same class are being used in the same application without conflict through classloader isolation. This is a trivial example but this technique is often needed to use different versions of the same library in different parts of an application. This technique is used pervasively in servlet containers and application servers.

Attached here are two Eclipse projects encompassing the main and plugin parts of this example. You should be able to unzip these projects in an Eclipse workspace and be ready to run. All code is included.

Main (5 KB) Plugin (2 KB)

Comments

22 Responses to “Writing a post-delegation classloader”
  1. Bill Mooney says:

    For examples of how a plugin framework is used, you may want to checkout James or JMeter for examples of how this is done. I know these projects were enlightening to me.

  2. Old Curmudgeon says:

    Why not use something like OSGi and leave the messy details of ClassLoader implementation to someone else? Unless of course you like faffing about with things instead of getting some real work done! ;-)

  3. Alex says:

    Well, sometimes using OSGi (or similar) is just overkill. I recently had a situation where I needed to isolate one particular library in a pluggable part of my application. Implementing the whole app with OSGi (after the fact) wouldn’t make sense. But slapping in a post-delegation classloader worked great. At a future time when the particular plugin goes away, so does the classloader.

    Plus, I do enjoy faffing about with such things so I didn’t mind too much…. :)

  4. Liam says:

    You could try classwords, as well.

  5. prashant says:

    Hi,
    Good post.Got some knowledge reading your post.

    Thanks
    Prashant

  6. Sid says:

    Thanks for this post. I had pretty much the same problem and your solution work for me. I’m using your class loader code as well. Actually, this works beautifully for what I need.

  7. Torsten says:

    How do use newInstance, when the cosntructor of your class needs arguments?

  8. Torsten says:

    One more question. I try your solution with a plugin too. Before I see an output over java option -verbose:class for the loaded classes, after implementing your solution, I cannot see this anymore. Does this mean the classes are not loaded? Or do we have to reimplement this in the overwritten loadClass function?

  9. aha says:

    To understand what you mean i have to read basics of class loading first.

  10. Alex says:

    @Torsten: Regarding newInstance – you can only use this to instantiate an object with a default constructor. If you have arguments to pass, you will need to use reflection to look up the constructor and instantiate the object using the Constructor instead. An important note is that constructors cannot be defined in an interface, so there is no way to enforce the constraint that classes passed as plugins will have a particular interface.

    @Torsten: Regarding -verbose:class, I believe this setting will only list classes loaded in the system, ext, and boot classloaders (not positive on boot). To get that behavior, you would need to implement it yourself. Actually, in the version of this classloader that I have used myself, I added a boolean debug flag that was passed on the constructor and a way to enable debug mode. I use the debug flag to determine whether to print when a class is requested and whether it was loaded by this classloader or a parent. Often, this can help determine in a problem scenario where classes are being loaded in an unexpected classloader.

    @aha: Great article! Thanks for the link.

  11. Torsten says:

    Have tried your solution, but it does not solve my problem.
    I try to build an interface for an opensource framework in for an ide. The opensource framework consists of tree jar Files, from which two are other opensource framework.
    I can instanciate all of my interface calls with my post-delegation classloader. But the calls between the jar’s itself use the parent classloader. Therfore I get problems, because the ide brings along an older version of one of the jar’s in the parent classloader, so I get errors at runtime. Do you have a solution for this?

  12. Alex says:

    @Torsten: If you are constructing the classes from your own version of the jar in a post-delegation classloader, those should effectively shadow the older version in the IDE framework itself. Where you could get in trouble is if your framework is returning instances of the newer library that may meet up with instances loaded from the older version. What is the actual problem you’re seeing?

    And what lib is it? Some libs that let you plug things into them have some more esoteric problems. log4j is the classic example of this. If the IDE framework happens to be Eclipse, you might also want to look into buddy classloaders and osgi classloading which provide some more options. Important to note is that OSGi does not use parent delegation but relies on a sibling-dependency resolution scheme (as most module-based systems do).

    Another option might be to use jarjar to repackage your version of the lib in your own packages to avoid the conflict. Generally, I would prefer to avoid that as it makes build, deployment etc more complicated but it certainly works (lots of people do this).

  13. Robert Ross says:

    I’m having a specific problem with exactly this issue:

    “A key design point is that the plugin should integrate with the main application entirely through interfaces or classes that are defined in the parent classloader. Most commonly, the main application defines interfaces and the plugin returns implementations of those interface. Usually the plugin is loaded in the main application via reflection to force the use of the plugin classloader. ”

    However, I have been unable to resolve the problem just from reading this text. Basically, I’m using an Abstract base class rather than an Interface but I can’t seem to get the plugin to instantiate properly while using the abstract class defined in a different classloader. Is this situation only resolvable by using an Interface or can this be implemented somehow with an abstract base class as well?

  14. Alex says:

    The key issue I was trying to get at was that if you load the same class from two different plugins with post-delegation classloaders and they’re used together in the parent domain, if those classes ever meet, you’ll get a LinkageError of some kind.

    If you can describe your classloader structure and where the classes are getting loaded from, I might be able to pinpoint the error. Hard to tell just from your description.

  15. Robert Ross says:

    Understood. I have a hard time explaining the fulling breadth of the architecture in this little tiny text box. :-)

    Perhaps I could email you something?

  16. Robert Ross says:

    Understood. I have a hard time explaining the full problem in this little tiny text box. :-)

    Perhaps I could email you something?

  17. Robert Ross says:

    Sorry for double-post. Had a network hiccup.

  18. Bao Le Duc says:

    I dealed with the same problem. It works for me. Another interesting article in this subject in onjava.com

    Many thanks.

  19. Miles Parker says:

    Hi Alex,

    I came across the same issue quite a while back and got to the same solution, so it must be right. :) I’d just written it up and came across your post.

    http://milesparker.blogspot.com/2009/03/dynamically-loading-classes-using.html

Trackbacks

Check out what others are saying about this post...
  1. […] One place where class names show up that I think is unavaoidable (and worth the consequences) is the case of delayed loading. Sometimes, you want to use a plugin-like system and delay loading of classes, either because you want to dynamically choose dependencies or because you want to hide the loading of dependencies inside a custom classloader. In either case, you can’t refer directly to the class as a Class as that would cause it to be loaded at the point of reference. […]

  2. […] I’ve written before about using classloaders to isolate portions of your code. Lately, I’ve been in the thick of fighting some classloader issues so I thought I would add a few additional thoughts. […]