Open source frustration

6

Today was a day of frustration, primarily at the hands of two open source projects: Ant and CruiseControl. I’ve been attempting to set up two things – a sync from Perforce in an Ant build and a CruiseControl continuous integration server.

The Perforce task seems extremely simple in the Ant manual. Should look something like this:

	<property name="p4.port" value="host:6999"/>
	<property name="p4.user" value="user"/>
	<property name="p4.client" value="client"/>

	<target name="sync">
		<p4sync view="//depot/path" />
	</target>

So I tried it out:

sync:
   [p4sync] java.lang.NullPointerException

BUILD FAILED
C:\perforce\dev\alsync\build\build.xml:34: Problem exec'ing P4 command: java.io.
IOException: CreateProcess: p4 -phost:6999 -uuser -cclient -s sync //depot/path error=2

Not good. Got an NPE somewhere. I did get an error message with a command line though, which is a great place to start. So, I made sure that the command worked by itself, which it did. I tried a few other permutations and continued to get the NPE no matter what I tried. I tried using -debug on the Ant run and managed to get a stack trace at least:

        at org.apache.tools.ant.taskdefs.optional.perforce.P4Base.execP4Command(P4Base.java:297)
        at org.apache.tools.ant.taskdefs.optional.perforce.P4Sync.execute(P4Sync.java:105)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:105)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:357)
        at org.apache.tools.ant.Target.performTasks(Target.java:385)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1329)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1298)
        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1181)
        at org.apache.tools.ant.Main.runBuild(Main.java:698)
        at org.apache.tools.ant.Main.startAnt(Main.java:199)
        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)

So, I headed to the source. The top line is in P4Base, which is basically the base class for all the Perforce tasks, all of which basically just execute a command line to the p4 shell based on your args. The source file shows that the error is coming from the catch block at the very end of the class. If you’re too lazy to look at the code, the pertinent bits looks like this:

        try {

            // a gazillion lines of code  

        } catch (Exception e) {
            String failMsg = "Problem exec'ing P4 command: " + e.getMessage();
            if (failOnError) {
                throw new BuildException(failMsg);
            } else {
                log(failMsg, Project.MSG_ERR);
            }
        }

So, we know that given the error message, we’re seeing some kind of IOException. I’d sure kill to see e.printStackTrace() here though to know which of the gazillion lines of code is actually throwing the exception. I can deduce a few things from looking at the code and surely something has gone awry in creating or running the Process but who knows what or where.

My next thought was to reproduce in Eclipse since I could then debug into the code. Unfortunately, it actually worked in Eclipse. Arrghgh.

Examining the source history for P4Base, it said an NPE had been fixed in the recently release Ant 1.7, so I upgraded from Ant 1.6.5 to 1.7, which I’d probably want to do eventually anyways. No dice, same error.

I tried the Google of course, to no avail. At this point, I’m contemplating either patching the code or attaching a debugger. Not sure which is easier yet but I might be too lazy to do either. In any case, I think this is a good lesson in exception handling and logging and how a little more thought could have saved me at least a few hours. In this case, chaining the original exception into the BuildException would have gotten me the whole story, which presumably would help track down the NPE. Of course, I’m sure the code pre-dates exception chaining and just hasn’t been updated but the lesson serves.

CruiseControl was more of just an installation / documentation mess. And I’m sure that all of the good folks working on CC have been trying really hard, but I think the story on how you get the build and web server started up (even in light of the docs targeted at exactly that) is still too confusing. I managed to get the CC instance itself going pretty quickly, but I can’t figure out exactly which env vars should be set, working directories, etc.

On my very first attempt to start CruiseControl, it blew chunks because there was a space in my JAVA_HOME env variable. Never mind that c:\program files is the default install location for the JDK on Windows and many people have it installed there, making this a common use case. Even once I got it going, the built-in Jetty console dumps this scary error message every few seconds saying that the accept timed out. This makes you think you’ve done something horribly wrong, but is actually apparently the normal operation of Jetty based on a bit of searching. Seems like a bad choice for a default message strategy.

Anyhow, I’ll dive back into the fray tomorrow.

UPDATE: For those interested, Stefan Bodewig (an Ant committer) saw my post and had fixed the chained exception by the time I woke up this morning. He was also dead-on in his suggestion that I was missing p4 in my path. I had outsmarted myself in my .cmd file and it was a quick fix to get it running. So, Ant+Perforce is happy. Thank you Stefan!

Comments

6 Responses to “Open source frustration”
  1. The most likely case for the Perforce task failure is that p4 is not on the PATH when Ant is running. This would cause some subclass of IOException to be thrown.

    I’ve just modified the task to not swallow the exception here. Public ranting works almost as well as opening a bugzilla issue ;-) If you can build Ant from source, give it the current trunk of SVN a try.

  2. ######Perforce########

    Make sure your P4 installation is correct:
    1. Perforce needs to be installed on the your build machine. Check your PATH
    2. Run p4 client and make sure your clientspec has the right depot locations on the remote as well as your local PC
    3. Edit the temp file that comes after you run p4 client. It should look like this:

    # A Perforce Client Specification.
    #
    # Client: The client name.
    # Update: The date this specification was last modified.
    # Access: The date this client was last used in any way.
    # Owner: The user who created this client.
    # Host: If set, restricts access to the named host.
    # Description: A short description of the client (optional).
    # Root: The base directory of the client workspace.
    # AltRoots: Up to two alternate client workspace roots.
    # Options: Client options:
    # [no]allwrite [no]clobber [no]compress
    # [un]locked [no]modtime [no]rmdir
    # LineEnd: Text file line endings on client: local/unix/mac/win/share.
    # View: Lines to map depot files into the client workspace.
    #
    # Use ‘p4 help client’ to see more about client views and options.

    Client: yourmachinename
    Owner: youruser
    Host: yourmachinename
    Description:
    Created by youruser.
    Root: Define_Your_Root_Folder_Here
    Options: noallwrite noclobber nocompress unlocked nomodtime normdir
    LineEnd: local
    View:
    //depot/path… //yourmachinename/…

    4. Make the necessary modifications, save and close. P4 will update its clientspec.
    5. Verify P4 works OK. Run p4 –c CLIENTSPEC_NAME –u user -P passwrod sync -f

    #######CruiseControl########
    1. Download and install CruiseControl from http://cruisecontrol.sourceforge.net/. Extract the zip to the your local CC directory

    2. Configuration:
    Config.xml is part of the standard CruiseControl distribution. The config.xml file is in the root directory of CruiseControl. You will need to modify config.xml if you need to add a new CruiseControl project.

    3. Create a New CC project – Simply add a new project XML tag in the config.xml file. Use one of the existing projects as a baseline.

    4. Call a delegating build file – I recommend Maven2 over everything else out there:

    Give it a shot. I feel your frustration, but once you get over the installation quirks the system actually works like a steam engine [read:reliable].

  3. Oops, forgot to escape the XML. Here are the two examples for config.xml:

    3. Create a New CC project – Simply add a new project XML tag in the config.xml file. Use one of the existing projects as a baseline.

    <project>
    <modificationset quietperiod="${interval}">
    <p4 correctForServerTime="false" port="${p4_port}" client="${p4_client}" user="${p4_user}" view="${p4_view}"/>
    </modificationset>
    </project>

    4. Call a delegating build file – I recommend Maven2 over everything else out there:
    <schedule interval="${interval}">
    <maven2 mvnscript="${maven2.exe}" pomfile="${your_pom.file}"
    goal="clean package site … other goals" />
    </schedule>

  4. Alex says:

    Hi Stefan, thanks so much for your (amazingly) prompt attention! I’d say public ranting works way better than filing a bugzilla issue so far: I posted, went to sleep, and it was fixed when I woke up. :) I was planning on filing a bug (and providing a patch) when I got a little farther.

    Turns out the problem was indeed a p4 PATH issue. I was fooled as p4 is on my path and the command in question worked from the same command line I was running the build from. But, I have a build.cmd that wraps the call to Ant and cleans out the environment and that cmd file was resetting the PATH to a minimal state (wiping out p4 on the path).

    Also, I was able to try your fix, rebuild Ant, and see the full exception, so that will come in handy for someone else.

    This is definitely the up-side to open source! Bugs fixed while you sleep…

    Thanks!!!
    Alex Miller

  5. Alex says:

    Hi Levent,

    Thanks for the info. I actually got CruiseControl working with Perforce on the first shot yesterday. No problems there at all.

    Alex Miller

  6. Take a look at our Parabuild at http://www.viewtier.com/products/parabuild.htm

    Setting up Perforce builds with Parabuild takes two minutes.

Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!