Is Josh Bloch the biggest problem for closures?

CORRECTION: Neal has since informed me that he, since the presentation that this post references, has learned that there is no such JCP rule. I'll still keep this post, because it makes some useful points (and so do the comments).

Josh Bloch, with "crazy" Bob Lee and Doug Lea, proposed their Concise Instance Creation Expression solution for closures for Java, last year.

They didn't address returning from the enclosing scope from within the closure. Theirs is a trivial syntax change, probably fairly straightforward to implement but it wouldn't address simple cases such as finding the first line in a file that matches an expression. If you're already familiar with why CICE is insufficient, feel free to skip past the code here.

Here's the psuedocode:

forEachLine ln in "/etc/passwd" do
   if ln contains "root"
      return ln

Here's the usual Java 5:

BufferedReader reader=null;
try
{
    reader=...;
    String line;
    while ((line=reader.readLine())!=null)
    {
        if (line.matches("root"))
            return line;
    }
}
finally
{
    if (reader!=null)
        try
        {
            reader.close();
        }
        catch (IOException exception)
        {
            log(exception); //but don't rethrow it
        }
}

So, of course, as the good programmers we are, we write an abstraction for that:

forEachLine("/etc/passwd",new LineProcessor()
{
    public void processLine(String line)
    {
        if (line.matches("root"))
            return line; //won't compile
    }
});

There's a problem there, the 'return line;' is returning from the run method, not the enclosing method. Ok, we can solve that, let's make LineProcessor return a String, which is normally null unless we want to end the loop. Then forEachLine can also return a String, which will be the return value of the method.

return forEachLine("/etc/passwd",new LineProcessor()
{
    public String processLine(String line)
    {
        return line.matches("root") ? line : null;
    }
});

This works, albeit with some strange semantics, which could be made more obvious by adding a class to hold the result, but that's not important for this post.

So forEachLine now has to know that the LineProcessor it runs might want to return early. When callers of forEachLine don't want to return early, they still have to return null all the time. It's inconvenient; to not punish callers that don't want to return early, we now need two versions of forEachLine, one that supports early return, and one that doesn't. That's confusing. The bad smell wafts up, making you either overload, or use names like "forEachLineWithAPossibleEarlyReturn".

What does CICE do for this?

return forEachLine("/etc/passwd",LineProcessor()
{
    return line.matches("root") ? line : null;
});

That's right. It removes a bit of punctuation and the 'new' keyword. Without underestimating that contribution, it doesn't actually help to make the end result resemble the psuedocode. It would be hard to get back from this end result into the psuedocode, there's a loss in translation. It isn't expressive.

The BGGA proposal allows this:

forEachLine(Line line: "/etc/passwd")
    if (line.matches("root"))
        return line;

Now just to remind you of the psuedocode again:

forEachLine ln in "/etc/passwd" do
   if ln contains "root"
      return ln

It doesn't need a special version of forEachLine for each use case. It's generally better, more expressive, with one wart that I'll mention at the end.

So why does this matter? Neal Gafter, the proposal lead, used to work for Sun, he's well respected, so he can easily make this into a JSR. Or not.

Neal Gafter now works for Google. All the work he's put into BGGA has been on his own time, it's not allowed to be his "20% project" (Google employees spend 20% of their paid time on ideas of their own). He is not the JCP's contact for Google. Surprise, surprise, Josh Bloch is, and Josh has his own more limited, less expressive, proposal, as I've demonstrated just now.

The JCP has a rule that disallows individuals from being on the JCP if their employer is on it. I felt sorry for Neal at the end of his presentation on closures at Google. Here's a quote from the last few minutes (emphasis from original):

From the audience: "Is someone planning on opening up a JSR on this?"

Neal: "There's a long answer and, I'm afraid, a longer answer. I was planning on writing a JSR for this. Sun Microsystems actually asked me to write a JSR proposal for this. But I can't do that, because I am not Google's JCP representative. And I cannot do it as an individual contributor to the Java Community Process, because as a Google employee I cannot be an individual contributor to the Java Community Process.

Google's JCP representative is Joshua Bloch, and he has other ideas about what should be done in the Java language in this space. As far as I know he is not currently planning on submitting a JSR on this. My hope is that creating a prototype, which by the way, I'm doing on my own time, will be something that Sun can use as a justification for Sun creating a JSR to do this into the language, because I think that's the only way this will happen in Java at this point."

Here's a little of Josh Bloch on closures:

"I like closures. I think they work very well in languages that are designed around them, like Smalltalk and Scheme and so forth. I think closures in java are a good thing. We've basically had them since 1.1 in the form of anonymous class instances and those are a bit clunky so I definitely see a place for improving support for closures; on the other hand some of the proposals that I've seen floating around are overly complex; they involve massive changes to the type system, things like function types and I'm severely worried about pushing the complexity of the language beyond the point where Joe Java can't use the language anymore.

If we add more support for closures I think it has to be in the spirit of the current support, which means that closures should take the form of instances, of types that have a single abstract method, whether they are interface types such as Runnable , or class types such as TimerTask. I think it would be nice if a better syntax in the current anonymous class syntax were adopted and if requirement with regards to Final variables were somehow made more tangible, which doesn't mean necessarily relaxed; I think it's actually good that you can not close over non Final variables, but it's a pain to actually mark them final, so if they automatically marked themselves final which would be nice."

His points are all good, except that function types aren't really a big overhaul of the type system - they will be resolved into interfaces. They're no bigger a conceptual problem than array types are.

Also, I think that if I could write a forEachLine in the BGGA style, I'd be more than happy for novices to use it.

The wart I mentioned is that Tennents' Correspondence Principle is violated by the BGGA when a closure throws a checked exception, and the interface that the closure conversion converts it to doesn't include any checked exceptions.

Neal proposes an extension to the generic type system to allow writing interfaces whose single method throws 0, 1 or many checked exceptions, but for those cases where the interface is already fixed, such as SwingUtilities.invokeAndWait(Runnable), the exceptions cannot be passed back to the caller. The code fails to compile. This is an inconsistency with the rest of the proposal. You can return from within a Runnable closure, but you can't throw a checked exception from within it.

I think that it would be possible to achieve exception transparency without needing anything special on the interface, as checked exceptions are purely a compile-time concept.