Clustered Scala
There has been a lot of discussion about the different concurrency models provided in languages like Java (shared-state) vs those provided in languages like Scala and Erlang (message-passing). The latter languages use an actor-based model that relies on setting up independent actors and (reliably) passing messages between them, rather than coordinating access to shared state. The actors run as lightweight thread-like entities and are expected to exist in large numbers (far more than the typical number of threads in a Java program).
The advantage of the message-passing model is that it is harder to encounter many of the common problems in shared-state models. Each piece of code is independent and deterministic which makes it easy to test. The pieces fit together in ways that are easy to model and reason about (well, easier than shared-state systems maybe).
Scala makes it easy to use actors and message-passing on top of the JVM. This gives us interesting new possibilities for using message-passing paradigms with the stability of the JVM. It may also let us do that a bit under the radar as most enterprise ops centers know how to deal with the JVM, when they might not be as comfortable running something like Erlang.
So, this lets us make interesting new systems based on message-passing, but how do we scale out those systems once they’re too big for a JVM instance? Today mad scientist Jonas Bonér announced support for clustering Scala actors using Terracotta.
Terracotta is typically used to cluster Java applications by sharing the heap and lock state across nodes. However, this is done using runtime bytecode manipulation and the most important bits are based on the JVM, not on Java. Since Scala is based on bytecode, it was possible to extend the Terracotta infrastructure to also handle Scala actors.
If you’re interested in Scala or Erlang actor-based concurrency, try it out with Terracotta to build a distributed version as well!
[ Note: I work at Terracotta, so I am entirely biased. But c'mon tell me this isn't cool! :) ]
Hi! My name is Alex Miller and I live in St. Louis. I write code for a living and currently work for 
Very cool. I was just searching the web for “clustered scala” the other day and didn’t turn up anything good.
Isn’t actor concurrency 2.0 and people move to 3.0 style concurrency, aka domain based concurrency aka fork/join, map/reduce?
Peace
-stephan
(with threads based security being 1.0)
s/concurrency/security/ :-)
s/security/concurrency/
Sorry to spam your comments, it’s saturday morning and I’m sitting in a cafe in Prenzlberg having breakfast. I’m not up to speed as it seams :-)
Interesting.
We are probably also going to try out Scala actors for db4o to run some of the internal processing in separate threads.
If this doesn’t give us the performance improvements we are hoping for, we may try to write our own message passing in Java. Ideally it should be possible to run an object in it’s own thread with just one line of code. All the boilerplate code for passing any Java object back and forth in a non-blocking and threadsafe way could be produced automatically by a bytecode generator.
I have dumped my thoughts to our forums here (requires login, I am afraid).
Terracotta is considering the possibility of supporting
the plugin officially but first wants to do a “market survey” and get a feel for if there’s enough demand.
I encourage you to blog the word and get people commenting for the good of hacker-kind ;-)
http://blog.terracottatech.com/2008/03/question_for_scala_users.html
Hello,
Happy new year to everyone !!!
On my side, if i had a wish in the tech area and after reading “Feeling Groovy” blog article here and this current one, i would make the wish for either Groovy or Scala to be well “supported” into Terracotta.
What are the news on them if there are some and what would you advice to follow among these two for Terracotta use ?
Alex
@Alex: At different times, people mostly outside Terracotta have explored clustering Groovy, JRuby, Scala, and Clojure all with partial success.
As an open-source company striving to pay it’s developers, we have to focus on big markets where the mass of developers are to get the biggest bang for the buck. For the moment, that means Java web apps built with Spring, Hibernate, Ehcache, Quartz, etc.
In the future (and maybe the not so far future), that may mean JRuby/Rails, Groovy/Grails, Scala/Lift, or whatever. So if you want those frameworks to be well supported, the best thing you could do is to implement it yourself. :) But the *second* best thing is to do what you can to ensure the popularity of those frameworks so that it makes business sense for us to spend developer time supporting them.
My hunch is that a year from now the Groovy/Grails story will be better than it is now.
All is well said :)
Yes, the best thing we could do is to implement it ourself.
It’s also one of the most pleasant way but not possible in the near future when we are completly new to the products and technology. I just switch to the java platform and start learning things like zk, java, jboss rules, terracotta, have to choose between groovy and scala and so on, that’s a lot and doing what misses in them is not realistic, so due to the deep learning curve and amount of work, when nobody brings the magic :) unhappily you simply have to drop and switch to something else than can be realistic :), so the more i search for solutions, the more i see that for me Terracotta will be for the next time, if there is a next time :)
Your “second thing” is a nice proposition, do you have a Terracotta contact who would take the hand of some of the technical project responsible ? I think here as example of ZK web framework, Jboss Rules, Scala, Groovy and so on… being a kind of evangelist is, for the moment, my only potential capacity in the near future :)
I see you might be confident into future solutions around JRuby/Rails or Groovy/Grails or Scala/Lift, but which is a solution for some people can a problem for some others as for example you may be interested into groovy or scala but not into grails or lift :), well that does not sound very confortable to take a decision, and if i have to make to choice regarding the language on the jvm, that sounds more or less like a bet on the future :)
@Alex: The best way to register your interest in a particular integration at Terracotta is to file a jira at http://jira.terracotta.org and encourage others to vote for it. That raises it’s visibility to the product mgmt team.
Currently, Groovy and Grails are the non-Java things we’re most likely to work on as there has been a noticeable trend of people moving from Spring/Hibernate/etc towards Groovy/Grails/GORM. There are existing issues for Grails (http://jira.terracotta.org/jira/browse/CDV-1023) and Groovy (http://jira.terracotta.org/jira/browse/CDV-1024) if you want to vote them up.