Java 7 Prototype - Surprises and Expectations

In my first week of testing the prototype, I found quite a lot of bugs, and some more subjective defects.

Neal Gafter seems amazingly fast at fixing bugs, so there's probably not much point reporting any bugs on this blog. I'll run through some surprises instead. If you want to see the bugs I found, take a look in the test/ directory in the prototype. Neal put some of my bug reports in as Clarkson2.java etc. It looks like he uses some automated testing on those, so if he breaks one he should know before I report it again!

If there's anything in this code that you don't understand, just ask. I mainly wrote it to test the prototype, so I wasn't overly concerned with writing good code. Most of these code samples are from bigger programs that I chopped down whenever I found a bug.

And now to the meat.

1. Definite assignment rules make it difficult to write recursive closures.

(1)
{int=>int} factorial={int x=>x<2 ? x : x*factorial.invoke(x-1)};

The above only works if factorial is a field, not a local variable, because factorial is not 'definitely assigned' on the right hand side of the =. I've been playing with Scala a bit recently (which has given me some good fuel for testing this prototype with), and it doesn't have this restriction. After a conversation with David MacIver, a Scala programmer rather than a dabbler like myself, we came to the conclusion that it's possible to cause a NullPointerException in similar code to the above in Scala.

Lo and behold, Neal replied with much the same argument - if the definite assignment rules were relaxed, the following code could fail, or even do undefined things:

(2) {int=>int} factorial=f({int x=>x<2 ? x : x*factorial.invoke(x-1)});

If f() invoked the closure, then factorial would be accessed before it had been assigned a value. In Scala that is guaranteed to give a NullPointerException, because all variables are assigned to null/0/false before being assigned their real value. In Java, the verifier would not load the code, because it would possibly use values that had been in memory before.

However, for the actual case above (not (2), (1)), it's easy to see that the closure is not invoked before factorial has been assigned a value. I think this idiom is useful enough to make a special case for, but then I'm not involved in writing either the code or the specification so I can't really estimate that task.

The obvious workaround would be:

{int=>int} factorial=null;
factorial={int x=>x<2 ? x : x*factorial.invoke(x-1)};

..except that local variable capture is not supported yet. The current workaround is to make factorial a field instead of a local variable.

If I ever get my head around combinators properly I think I'll have a better workaround.

2. Java 5's foreach statement cannot take a closure.

This code fails:

for (final String s2: {=>asList(args).iterator()});

This code works:

Iterable it={=>asList(args).iterator()};
for (final String s2: it);

The only difference is that I explicitly made closure conversion happen. Typecasting would not fix it (ClassCastException). This is apparently as specified, as a foreach is only specified to take an Iterable or an array. It's not a huge issue, and I can't think of any use cases, but it's still a surprise.

3. Autoboxing doesn't apply to function types.

{Integer=>Integer} f={int x=>x*2}; is a compile error.
{int=>int} g={Integer x=>x*2}; is a compile error. This doesn't look insane on its own, but because generic type parameters cannot be primitive types, it has repercussions. I can't write a generic map function and use it like this:

Iterable<Integer> mapped=map(asList(1,5,11,26),{int x=>x*x});

I have to use the more verbose and more indirect-looking {Integer x=>x*x}.

There's no way of writing a single generic method that converts a primitive function to a wrapper function, again because generic type parameters cannot be primitive types. I could write 8 individual methods, e.g.:

public static {Integer=>Integer} lift({int=>int} f)
{
 return {Integer i=>f.invoke(i)};
}

etc.

However, there's again no way of generalising those to make them work with {int,int=>int}, etc. There is no subtyping relationship between different function types, except the covariance and contravariance rules, quickly demonstrated here:

{Number=>Number} f={Object o=>5};

The above compiles because any Number input is obviously also an Object, and because the output, 5, is obviously a Number (autoboxing taken for granted).

The lack of autoboxing also means that sometimes you'll have to explicitly supply the generic types. Here's an example:

class Main
{
        public static void main(String[] args)
        {
                int i=id().invoke(5);
        }

        public static <T> {T=>T} id()
        {
                return {T t=>t};
        }
}

This code won't compile, because the compiler judges the <T> in the call to id() from main to be int, which cannot be a type parameter. Neal says this isn't a bug. This version compiles:

class Main
{
        public static void main(String[] args)
        {
                int i=Main.<Integer>id().invoke(5);
        }

        public static <T> {T=>T} id()
        {
                return {T t=>t};
        }
}

I think this is a bug; it seems that inference should pick Integer instead of int in the first case.

Another implication of the lack of autoboxing is that when the closure invocation syntax arrives, that Neal promised would let you reimplement the foreach loop yourself as a method, you won't be able to. I expect this will work, if you write your own foreach method:

foreach(Integer i: asList(3,4,5))
 System.out.println(i*i);

but this won't:

foreach(int i: asList(3,4,5))
 System.out.println(i*i);

4. Closures cannot implement generic methods.

This one is much easier to describe in code than words. Given:

interface Identity
{
 <T> T id(T t);
}

..it is impossible to implement this with a closure. I think Haskell allows something like this. Scala doesn't. A use for this would be in implementing some kind of Either type, where an Either<X,Y> may hold an X or a Y, and you don't want to actually have to test which it is, just call a method in Either, supplying a function to run if it holds an X, and a function to run if it holds a Y. The return type of that method should be generic.

Stay tuned for some examples that actually do work with the prototype, at least one of which is actually interesting!