Language criticism

6

I read @codahale’s “leaked” assessment of Yammer moving some code from Scala back to Java with great interest today. I post about it somewhat hesitantly because it is a private thing that became public but I have lots of thoughts and I need to dump them somewhere. I am going to write this quickly and it will likely be twice as long as it should be. Sorry.

First, I think there is a great temptation to point at this particular note and use it as evidence that “Scala is bad” or whatever. While I know and respect Stephen Colebourne a tremendous amount, I have not particularly enjoyed the tone of his recent Scala criticism (which is not to say that I don’t think he makes valid points in some cases). I winced to see him waving the flag in this way.

I would say instead that this is an incredibly valuable evaluation of a young and promising language and I hope and believe that it will be taken that way by Odersky, etc. It is easy to find drive-by criticism of a language but much rarer to find someone that has lived in it for a long time and is able to deliver precise, detailed feedback of where the pain was greater than the value. I found Coda’s criticism to be exactly that and also written in a highly respectful tone.

I am unabashedly a fan of Clojure (if you follow this blog, my tweets, my conference talks, my nascent Clojure/West conference, this is no surprise). I believe some people might perceive Scala and Clojure as enemies and thus suspect that the Clojure community likes seeing Scala bashing.

Much the opposite. We all want the same thing – a way to write programs that allows us to be more productive, make fast flexible systems, and write cool stuff. I find much to respect and envy in Scala and its ecosystem. I see some of the smartest programmers I know loving Scala and finding it productive. I believe that Clojure and Scala (and all the other languages I’m not talking about) want a better way to do programming. In that, I see allies, not enemies.

Going back to Coda’s gist, he is honestly evaluating his choices and deciding where his team can do their best work *right now*. In the end, you’re balancing forces of time-to-market, performance, quality, future maintenance debt, on-ramping new people, etc. I don’t hear Coda saying “Scala is bad”; I hear him saying “Scala is not the best balance of our needs at this time”. Other people will make a different choice. They will make different choices at other times. That’s just being smart at running your business. Being able to coherently say “…because X, Y, Z” is very helpful so the language can evolve.

I saw some questions floating around about whether people using Clojure have hit this part of the hype cycle. Since we’ve been doing Clojure full-time at Revelytix for a couple years now and have hired people, done maintenance, etc, I think we’re as qualified as anyone to judge that. In general, I’d say that we certainly have encountered painful aspects of the language or the tools – when you are using tools in anger, you find things that suck. We have hit serious pain around AOT (ahead-of-time) compilation. We’ve seen some nasty classloader problems (linked to AOT). We’re having a more painful time than I expected moving from Clojure 1.2 to 1.3. Debugging sometimes sucks due to laziness, large stack traces, confusing error messages, etc. I yearn for refactoring tools as good as what I had in Eclipse with Java. Sometimes I want a static type. :)

On balance, we are not clamoring to change back to Java (or anything else) though. All of the painful things above are things we have worked through. By choosing a young language, you are implicitly accepting consequences like immature tooling or buggy libraries. I think we’ve tried to hit a good balance of powering through issues and giving back fixes or filing bugs where appropriate. Clojure is such a dynamic language that there is almost always a way to address a problem (even in the core libraries or the language) from your own code. Encapsulation (of data or code) is not held in particularly high regard in Clojure. The upside is that the engine is laid bare if you need to swap out a piston or something. In general, we have found our resulting code to be vastly preferable to what we could do in Java. Clojure gives you a set of composable tools that allow you to build systems to talk at the level of your problem and that is what I want in any language. For us, right now, Clojure is still a big win in that set of forces.

Comments

6 Responses to “Language criticism”
  1. Nirav Thaker says:

    I’m not easily sold on fallback and composability arguments (which are nice for most data driver apps btw) but I would be interested to learn how you tackle the overhead induced by JVM synthetics inflicted on these programming languages, if/any. For example, see how performance is impacted in non IO bound tasks even in plain Java: http://marc.info/?l=git&m=124111702609723&w=2, immutablity will only make it worse both in terms of GC and runtime (copying).

    (I feel comfortable asking this since I’ve read your experiences from the trenches. )

  2. Alex says:

    @Nirav: I’m not sure exactly what you mean by “JVM synthetics” but Clojure is (always) compiled to (usually) non-reflective JVM invokevirtual calls, just like Java. You can ask to be warned about reflective calls which you can fix by adding type hints to help the inferencer. Our experience has been that 90% of the time it does what you want and 90% of the other 10% doesn’t matter. The remaining 1% is easy to find (as it shows up in any hot spot profiler analysis or even just simple thread dumps) and fix.

    Making Clojure as fast (or faster) than Java code while still being dynamic and supporting abstractions is explicitly a priority of Clojure. I would not pretend that it succeeds in all cases but I think it succeeds more than you’d expect and is getting better all the time (in leveraging primitives for numerics for example).

    Clojure uses persistent data structures that leverage structural sharing to avoid copying on modification. These structures means modifications are not constant but log base 32 (n). For the vast majority of cases that is “good enough”. For those that aren’t, you can use transients to drop into a mutable mode, make bulk changes, then go back to persistent mode. I found dramatic benefits from using transients when serializing large amounts of data.

    Clojure creates more garbage than Java. GC’s a bitch. We haven’t done enough perf tuning yet that I can say much in detail. I could see it being a limiting factor in some apps. We’re not one of those right now. The issues mentioned in your link are real. Clojure doesn’t (can’t) “fix” the size of a Java object vs using raw memory.

  3. Nirav Thaker says:

    By JVM synthetics, I meant reflective inter-op overhead (proxies, generated field accessors etc.) when interacting with other java libraries – you already answered that, I wasn’t quite clear if clojure classes interact directly with java objects or there’s a layer. I’m resisting the urge to compare “Structural sharing” with java.lang.String and … Thanks for sharing your experiences.

  4. Miloskov says:

    Clojure(Lisp) is Art and beauty compared to the messy Scala and friends, We cant compare it. Scala got cleaver stuff but also to complex and verbose things.

    The balance is when you need static typing use Java and is like talking to the metal, Speed, Libs all around but when you need an abstraction with elegant Clojure, I love Lisp.

  5. Thanks for sharing your thoughts, does anyone have the email he shared? Is it out thr on web? I would be interested to read it.

  6. Alex Miller says:

    It’s linked on the page (the word “assessment”).