• Slides
• Video

And you can find info from Scott Bale’s talk on Fan here:

• Slides
• Video

If you’re interested in catching future video, you can catch the feed from blip.tv or we’re on iTunes.

First, I will be introducing you to a research language called Fortress aimed at people doing scientific computation on large (peta-size) systems. It is a language that explicitly tries to meet the challenge of running systems up to a million cores and is “infested” with parallelism. This is a language for tomorrow.

Next, Scott Bale will be talking about the Fan programming language. This language takes the best of languages like Java and C#, throws out the cruft, and bleeds in some modern features. Is it the next Java? Java++? Come find out…

Should be a great night with two languages making their debut at the Lounge.

Fortress is a language being developed as a research project at Sun. Most people that have heard of it seem to vaguely know it has something to do with Fortran. Turns out it really doesn’t have much of anything to do with Fortran other than a similar goal and intended user. Fortress is targeted at large-scale scientific computing (like peta-scale large).

One thing you’ll notice immediately about it is that it has a strong mathematical syntax. In many cases, things work differently than languages you may be more familiar with in favor of a declaration that is closer to normal mathematics. For example, instead of writing a * b to multiply you simply write a b. Putting two things next to each other is actually a special operator in Fortress called “juxtaposition” and can be overridden like many other operators. For strings, juxtaposition means concatenation. For a function, it means application.

In addition, many traditional mathematical notations have textual representations you can use in your program, including all of the Greek symbols (alpha, beta, etc), various brackets, summation, sub-scripts, names like b’, etc. The specification defines these textual representations and renderings, making full use of Unicode. A program called fortify is provided to convert a Fortress program into LaTeX.

I working on a Fortress presentation for the Lambda Lounge and lost a couple hours of my life the other night trying to do the conversion from LaTeX to PDF/image. On Mac there are several comprehensive packages for stuff like this. I tried to go with the smaller packages listed here instead of the huge 1.2 GB MacTex package but I ran into all kinds of problems with the missing St. Mary’s font of mathematical icons. I have since learned that those problems are resolved by using the full package which includes the font.

But I was also told of a handy online tool provided by the Fortress folks that will turn Fortress files into images and PDFs. That is saving me from installing the giant LaTeX package for the moment.

One activity we usually do is a performance tuning competition. Being all about clustering apps and making them scale, we think it’s important for everyone on the team to be a guru in performance tuning. We break everyone up into cross-functional teams and give them all the same app to work on. In this case, we were working on an app built by our field guys to simulate single-sign on that was using MySQL and Hibernate for the backing store of authenticated credentials and every team got a 6-node cluster to work on (we’ve got a big perf lab :).

For the next Terracotta release (3.1) we’re working on building the most kick-ass clustered Hibernate second-level cache in existence. It will be the Chuck Norris of Hibernate second-level caches. Anyhow, the test had two phases - a load phase (to fill the db and warm the cache) and a query phase. The baselines for phase 1 were somewhere around 45 TPS and no one actually could baseline phase 2 because the app would run out of memory and crash. Lots of room for improvement.

By the end of the competition, we had fixed some bugs in the app, made a bunch of configuration tuning changes for Hibernate and Terracotta, and even found a bunch of awesome improvements at various levels of Terracotta and the our cache implementation. Most teams were seeing 3,000-4,000 TPS in phase 1, were able to load the entire 3 million cache entries without memory problems and were seeing in the region of 60,000 TPS in phase 2. Pretty sweet. Some of the memory-related optimizations we found are completely generic and should allow us to be much more efficient in all Terracotta apps.

At the moment, we are in the second day of a product improvement competition. Generally we turn people loose and let them work on whatever they want with some focusing guidelines. Last time we did this we had people try some crazy stuff but we didn’t put a high premium on actually getting something fully productized because it was a shorter competition. This time, we put some more emphasis on making things that actually can go into the product right away and also a focus on usability and Hibernate users.

The competition’s not over yet but there is some awesome stuff in work. I suspect we’ll see even better Hibernate tooling come out of this (since several people are working on things for that) and several other usability and performance improvements. It’s definitely a good thing to have the performance tuning competition first so people get a taste of where some bottlenecks are and where the tooling wasn’t helpful enough.

All of this is scored by our judges (from other parts of the company) and we give out some prizes to those that have made the biggest impact. We’ll finish out tomorrow with a bunch of brainstorming and meetings. All in all, it’s a great team exercise and a lot of fun. It’s also completely exhausting. :)

The question is what does this program print when executed?

• a) 2 2
• b) 1 2
• c) 0 2
• d) something else</li>

It seems like the answer should be a) and you will actually see that in JDK 1.6+. However, in JDK 1.5 you’ll see b). Understanding why requires a bit of a jaunt through the CHM code, where dragons lurk. Also, it’s actually possible to see several possible answers (at least a and b) in JDK 1.5 depending on what keys and values you use.

map.entrySet()

This call on a CHM does the following:

All that does is create (and cache) an EntrySet instance, which is an inner (non-static) class specific to CHM. It has access to the real CHM state and implements the interface using that state.

new HashSet()

The HashSet is constructed with the EntrySet instance. A new HashSet with an initialization collection will do this:

So, the state from the source collection is added through addAll().

HashSet.addAll()

There is some extra stuff here to manage the modified flag needed for fail-fast iteration but basically, this code walks through the collection’s iterator calling next(), then add() on the set under construction. Here’s where things get interesting. We need to look at what the EntrySet uses for an iterator and what happens when you call next().

EntrySet.iterator()

Here we return yet another inner non-static class EntryIterator. EntryIterator is defined as follows:

This should give you some pause. We fully expect to see “extends HashIterator” - there are several key/value/entry iterator flavors buried in here and they share most of their implementation in HashIterator. We also fully expect to see “implements Iterator<Entry<K,V>>” since that’s what we’re doing. But seeing “implements Map.Entry<K,V>” should make you go huh? This iterator walks over Entry<K,V> instances in the CHM yet it itself is typed as an entry?

What does it do on Iterator.next()?

EntryIterator.next()

Here we see the iterator advance to the next entry (it walks through segments at the high level and entries at the low level) but in the end it returns this?

The key here is that the EntrySet’s iterator just stores the current key/value during iteration and returns itself as the entry object for every single actual entry. If you modify that entry, it writes through appropriately to the CHM. Presumably, this saves on the creation and destruction of lots of little Map.Entry instances that will be used just during iteration and typically never looked at again.

Therefore, when the EntrySet iterator is walked through in HashSet.addAll(), AND because of how EntryIterator.hashCode() is implemented:

Here the hashCode is created as the bitwise OR of the key hash code and the value hash code. So, regardless of what the key and value hashcodes are, a ^ b == b ^ a. In our case we used two strings “hello” and “world” swapped. Thus the second entry instance will be seen as an identical hash code (and instance) and will simply put over the first entry instance in the HashSet.

A subtly different bug will happen in the case where this property (of swapped key/value) doesn’t occur. In that case, both entries will end up in the HashSet (size == 2 as expected) but they are the same instance and were put into the set with different hash codes. It’s always a bad deal to put something in a HashSet or HashMap and then modify the object in ways that modify the hashCode(). This breaks assumptions in the HashMap class that can cause lost elements in the collection and other weird behavior.

As I mentioned, this same instance behavior is probably surprising to most people. It’s doc’ed here in Sun bug #6312706; as a general problem in several maps (CHM, IdentityHashMap, and EnumMap). That bug is marked as “fix understood” which is kind of surprising because this behavior is already changed and fixed in JDK 1.6. In 1.6, the EntryIterator is implemented as:

Here EntryIterator does not implement Map.Entry and next() returns a new instance of WriteThroughEntry, which causes the HashSet to be much happier on creation.

Interestingly, another smart colleague noted that the existing Terracotta ConcurrentHashMap instrumentation fixes the 1.5 implementation by creating a wrapper object in the iterator as well.