Publish globally, trust locally
At Terracotta we use several dependency management systems. Our main source tree depends on Ivy and our Forge is heavily reliant on Maven.
Right now we’re starting the process towards our next release and got to the point of cutting the release branch. This of course, required our main source build system (using Ivy in this case) to download the Internet from ibiblio and beyond. For some reason, a handful of dependencies are now missing from ibiblio. I’m sure we’ll track it down and get things running in short order, but it’s a few more lost hours and a slight hold on development.
Prior to Terracotta, I worked on and built ground-up several build systems (mostly Ant-based) but had never used a dependency management system before. Since I’ve been here, I’ve done a whole buncha Maven and use our own internal build system on a daily basis. [Urge to rant about Maven fading...fading...rising...fading... :)] I’ve come to believe there’s actually a lot of value in fighting through and understanding your dependencies on a deep basis, actually. When you can just download anything from the net, you don’t bother trying to figure out why the heck you depend spongebob.util-1.9.jar anyways.
I personally have been bitten by the equivalent of the missing or broken Internet-based dependency several times, something I never experienced on my own build systems. Of course, I spent a lot of tortuous hours figuring out how to manage dependencies in my own build system and maintaining them over time, including problems with naming, versioning, integration into IDEs, etc. Clearly everyone has a dependency management problem and you choose whether to loosen your grip and go global or maintain a white-knuckled death grip but take control of your own destiny.
This led me to the thought that there is much good to come from publishing globally. The rapid rise and prominence of public module repositories with a well-defined scheme for jar naming, search, and indexing based on the Maven coordinate model is a freaking god-send. For my money, this is the hands-down best thing to come out of Maven, Ivy, etc. By all means, I want everyone out there to publish modules using these quasi-standards into public repositories and publish their Maven coordinates.
But from a use point of view, I can’t have a build system that depends on the Internet. You have to have a backstop in your organization that can take care of those dependencies in case of net drift or to shadow the net with a trusted local source. So, you really shouldn’t be using Maven without an enterprise or group level repository as well. There are also a number of open source and commercial repository systems that can address security and other issues as well. I haven’t used any of those but I gotta believe they will continue to rise in importance.
My personal belief is that we are currently in a transitional period. I think we’re in a post-Ant world. Don’t get me wrong, I love Ant and I’ve spent a lot of quality time with it. It’s not perfect but it’s a very capable build tool. I think Maven is pointing us in a new direction with two key differences – standard repositories and standard build flows.
I don’t think Maven actually owns the next world. There have been many fine and detailed posts in the last few months about Maven issues and even some admirable people like Don Brown working avidly to fix it. I don’t think I have anything unique to add to those posts – they cover the space pretty well as I’ve seen it. I’m still puzzling on a model that can make our lives better as developers. Any thoughts?
Hi! My name is Alex Miller and I live in St. Louis. I write code for a living and currently work for 
“white-knuckled death grip” would be a great name for a band.
Agreed. Ant does not scale and we have had problems with Maven as well. Groovy’s dynamic nature coupled with Ant and Ivy have yielded a very powerful, flexible and easy to use build system. Also, all dependencies reside in our own repository.
Michael, has anyone written anything on how to build a “build system” like the one you are talking about – with Ant, Ivy and Groovy?
hi Alex,
how about ANT + IVY (with a local repository) ?
What does Maven REALLY give that the above combination cannot ?
Thank you,
BR,
~A
Maven’s central repository is a convenience for users of Maven. No sane enterprise user relies on Maven central: they use a repository manager, mirror central, or selectively copy what they need. Would you put the sources for your projects anywhere but on your own infrastructure? Probably not. Maven doesn’t rely on the internet: it relies on Maven repositories and you should definitely setup your own for production environments. Having your own set or repositories is nothing new for Maven users.
Maven’s central repository is also not Ibiblio. Contegix hosts Maven’s central repository as Ibiblio was originally kind enough to donate the space, but couldn’t provide the required QoS. The policy on Maven central is no deletes: if you screw up a release then you do it again. Things should not disappear, and if they do this is generally irresponsible groups that we have given synchronization access that abuse the access and wreak havoc on users. Some choice moves have been re-deploying artifacts of a particular version which are not binary compatible, redeploying POMs which change dependency information, and doing releases where POMs have ephemeral repositories listed i.e. these brain dead folks wipe out the repository they just did a release with. So being kind has sort of back fired on us as only under extreme circumstances do we consciously allow an artifact removal. Say for a severe security concern. So if something “disappears” we would sure like to know about it because that synchronization partner will be getting JIRA access instead of rsync access.
Sorry, I didn’t mean to post that anonymously. At any rate I was also going to add that publish-globally and trust locally is how Maven was meant to be used from day one, and is in fact, how every corporate client I have uses Maven. Maven’s central repository should be used for the initial population of your internal repositories, then all the traffic from your developers to Maven repositories should be internal. The central repository is not the magical forest of artifacts. You should use your own.
One that belongs to the ‘next world’ is in my opinion Equinox P2 directed towards provisioning (Eclipse) OSGi bundles.
–
Tuomas
We’ve used Artifactory (http://www.jfrog.org/sites/artifactory/latest/) to overcome “download the Internet” problem, it does the job pretty well.
@Scott – agreed.
@Rob – I don’t know for sure, but Michael may be referring to Gant, which is Groovy+Ant. This would be near the top of my list to try if I were going to create a build system from scratch today. Rake is maybe a similar idea but using Ruby.
@anjan – one thing it gives you is a “standard” build workflow and set of commands for initiating that workflow. I think bringing regularity to builds across projects and companies is great. I’m a fan from a concept point of view. From an execution point of view, Maven is a mixed bag.
@Jason – thanks for the info. I’m not sure these best practices are as wide-spread as they should be based on what I’ve heard. :)
@Tuomas – I’d never heard of P2 before but it seems a little more focused on taking installable things and installing them across a bunch of machines. I’d consider the build process to be more how those installable things are created in the first place. But maybe I misjudged P2 based on a quick reading.
@Yardena – Artifactory does the job indeed
About Maven in general, it’s like democracy: “It’s not perfect but that the best we got!”