Google without the Search - Google Reader

I use quite a few Google services, but I noticed for the first time that Google Reader has no search facility - I sometimes don't remember to bookmark something I've replied to, and then have to search for it again if I want to check for updates. It seems fairly odd for a search company to omit search. For a few hours on Google Reader yesterday, I noticed that in the List View the titles of the articles weren't shown unless I clicked on them. I just switched to the Expanded View for that period, so I'm glad I only had a couple of hundred posts to scroll through (and a mousewheel!). I also find Google Docs a bit weird, it's started telling me occasionally that another collaborator has made some edits, and the edits from the last few seconds will be removed. Not a huge problem, but I'm the only person with the rights to edit the document in question. Marking things as bold, or changing the fonts, etc., sometimes happens to other text besides the bit I've selected, too. Perhaps Google's code quality is suffering as the organisation grows, or perhaps there's too little (none?) testing before features are pushed out to beta projects. After all, Google is using beta users as testers, but we're also using them. The reason everyone was happy to use Gmail in beta was because it was so damn good. It's a shame that other Google services aren't treated as killer apps like Gmail was.

Sanitising some code

A simple but valuable refactor - converting an interface plus two similar implementations into a final class. This is taken from real code; I wrote this document while refactoring. Here I have an interface that represents an entry in an ARP table.

public interface ArpEntry extends Stringable
{
 MacAddress getMacAddress() throws CheckedIllegalStateException;
 void age();
 boolean dead();
}

Stringable has one method, asString() - I call that instead of toString(), so that I never get java.lang.SomeType@89345aef3 in logs, etc. getMacAddress() may fail, because there are two kinds of ARP entry - some have a MAC address, and some do not (in real implementations, the second kind have a zero MAC address). The ones that have no MAC address are entries to say that a request has been sent. That's why it throws an exception. age() will age the entry by a unit - approximately one second, but as we use this simulation for teaching purposes, we may allow slower students to slow this down. dead() tests to see whether the entry is dead (aged beyond a certain limit). Currently, I have two implementations, both anonymous classes:

public class ArpEntryUtility
{
 public static final int START_TTL=20;

 public static ArpEntry arpEntry(final MacAddress macAddress)
 {
  return new ArpEntry()
  {
   int timeToLive=START_TTL;

   public MacAddress getMacAddress()
   {
    return macAddress;
   }

   public void age()
   {
    timeToLive--;
   }

   public boolean dead()
   {
    return timeToLive<=0;
   }

   public String asString()
   {
    return macAddress.getRawValue()+"; expires in "+timeToLive+" seconds";
   }
  };
 }

 public static ArpEntry incompleteArpEntry()
 {
  return new ArpEntry()
  {
   private int timeToLive=START_TTL;

   public void age()
   {
    timeToLive--;
   }

   public boolean dead()
   {
    return timeToLive<=0;
   }

   public MacAddress getMacAddress() throws CheckedIllegalStateException
   {
    throw new CheckedIllegalStateException();
   }

   public String asString()
   {
    return "incomplete ARP entry - expires in "+timeToLive+" seconds";
   }
  };
 }
}

It's trivial to see duplication here. One approach would be to make the interface a superclass, or to add a superclass that implements the interface, as a base for common code. However, that isn't the only choice. Let's look at a more classic form of code reuse - calling a method. I'll add two methods, getTimeToLive, setTimeToLive, to the interface. These will do no validation, just pass things through. Don't panic, these methods won't live long.

public interface ArpEntry extends Stringable
{
 MacAddress getMacAddress() throws CheckedIllegalStateException;

 void age();
 boolean dead();

 int getTimeToLive();
 void setTimeToLive(int ttl);
}

Now we can implement age() and dead() as static methods in ArpEntryUtility, and call them from the two implementations. Of course, those method calls are still duplication - we can call the static methods directly and remove the methods from the interface. IDEA has a refactor for this whole paragraph - Make Static. It will sort out the callers for you too, changing entry.age() to ArpEntryUtility.age(entry). Now almost the only case-specific code is getMacAddress(). If I change getMacAddress() so that it returns Maybe<MacAddress>, then there's no need for the exception, and hence no need for getMacAddress() to be implemented differently between each implementation.

public interface ArpEntry extends Stringable
{
 Maybe<MacAddress> getMacAddress();
 int getTimeToLive();
 void setTimeToLive(int ttl);
}

Looking at all the use sites, I see that getMacAddress() is only used once, and in that case the exception is caught and converted to a Maybe anyway, so I've just made the use site simpler too, by chance. It looks almost like a struct now, the only case-specific code left is the asString() implementations. I can make that a static method that does different things based on the Maybe<MacAddress>. The two implementations now only differ in how they are constructed. One is passed a MacAddress, the other isn't. Easy to solve, pass a Maybe and now we only have one implementation. One interface, with one implementation. Needless indirection. Let's change the interface to be a final class, and the implementation to just be a constructor call. Finally, we can get rid of the getters and setter, making macAddress a public final field, and timeToLive a public field. That gets rid of some extra needless indirection.

public final class ArpEntry
{
 public final Maybe<MacAddress> macAddress;
 public int timeToLive=20;

 public ArpEntry(Maybe<MacAddress> macAddress)
 {
  this.macAddress=macAddress;
 }
}

public class ArpEntryUtility
{
 public static void age(final ArpEntry arpEntry)
 {
  arpEntry.timeToLive--;
 }

 public static String asString(ArpEntry arpEntry)
 {
  return arpEntry.macAddress.apply(new Lazy<String>()
  {
   public String invoke()
   {
    return "incomplete ARP entry - expires in "+arpEntry.timeToLive+" seconds";
   }
  },new Function<MacAddress,String>()
  {
   public String run(MacAddress macAddress)
   {
    return macAddress.getRawValue()+"; expires in "+arpEntry.timeToLive+" seconds";
   }
  });
 }

 public static boolean dead(ArpEntry entry)
 {
  return entry.timeToLive<=0;
 }
}

You might now decide to restrict the users of ArpEntry so that they have to access everything via the provided static methods. That's fairly simple, just merge the classes and make the fields private. However, I like to keep my 'bags of functions' separate from my instantiable classes. While I was refactoring, one of my automated tests started to fail. It actually took me about two hours to fix the problem. Inadvertently, I had made some of ArpEntry's client code more logical, the code that decides what to do with an outgoing ARP packet from a computer. However, some parent code to that, the code that decides whether to send an outgoing ARP packet) had a logic error, which I'd never noticed before. This refactoring wasn't as straightforward as it could have been, mainly because IDEA doesn't know much about the Maybe type. But overall, I'm pleased I spotted the logic error now, rather than when I'm closer to a release!

Ricky's Properties for Java

While I think there should be some language change to make properties really useful, it's worth looking at how close we can get to properties without changing the language, to work out what needs changing. Suppose you were to write a method called, say, setField, or setf for short, that takes a property and a value, and sets it.  This is fairly reasonable as something you might like to do with arbitrary properties, for, say, a GUI.

  <T> T setf(Property<T> property,T value)
  {
      property.value=value;
  }

This approach relies on having one object per property, so it's easy to see it as a memory leak.  It's not actually a memory leak, it's a memory overhead.  Suppose that you create 1000 objects, each of which take 100 bytes normally.  Now you change them to expose Property objects as public final fields, and each object now takes 1000 bytes.  It's only a (potential) leak if you need to create objects every time you use a Property.  What we do have now though, is n more objects, where n is the number of properties.  This isn't a hopeless situation; there is a possible VM-based solution, holding Property objects directly, i.e., without a pointer, as part of the object they're in.  This requires the size of a Property to be known by the verifier, so the actual Property implementation would need to be known at load time. While this might seem like a hopelessly early optimisation, it's worth thinking about now, because if properties do get implemented in the language, and they are completely flexible (so that you can replace a Property object at runtime), then we'll no longer be left with the possibility of this optimisation.  A halfway house would be to make it possible to prevent a Property object from being replaced, or for the VM to be smart enough to tell which Properties aren't going to be replaced.  Obtaining a Property object is tricky
If we want setf to work with properties defined by existing code, we should be able to recognise the getX/setX convention, and make those into Properties.  Let's look at how we can create a legacy Property using current (Java 5/6) code:

  Property<String> nameProperty=new LegacyProperty<String>(new GetterAndSetter<String>()
  {
    public String get()
    {
      return object.getName();
    }
  
    public void set(String s)
    {
      object.setName(s);
    }
  });

This gets a bit shorter with the BGGA closures syntax, or even method references, but it doesn't get any sweeter.  It's still pointless duplication. Another implementation would use reflection.  Property<String> nameProperty=new ReflectiveProperty<String>("name",object);  Obviously there are the usual problems with this, such as type safety not being guaranteed at compile time, performance, that it requires tool support if the programmer is to be certain that it is refactor-proof.  There is an extra problem caused by erasure of generic types; there's no way of knowing that nameProperty really is a Property<String>.  setName could take an instance of some 'Name' class.  This is not simply a case of choosing dynamic typing over static typing, because erasure doesn't give us a choice.  The reflective solution is not typesafe at all unless we either implement reification or give ReflectiveProperty a 'type token', in this case String.class.

  Property<String> nameProperty=new
  ReflectiveProperty<String>(object,String.class,"name");

It doesn't work with legacy bean-manipulating code Suppose I write a new class and don't write getters or setters, but instead expose my fields via Properties.

  class Person
  {
    public final Property<String> name=new DirectProperty<String>("unknown");
  }

Now any code that reflects on Person looking for getX/setX methods won't find any.  It's arguable that the code should use Introspector to introspect, rather than direct manipulation, and hence that I could provide a BeanInfo class for Person, but not all the code that manipulates beans uses Introspector. Erasure could make a List of Properties useless. Suppose you asked a bean for all its properties, either directly or via some introspector.  You'd get a List<Property<What?>>.  It cannot be Object.  It can be ?, though this would prevent set from being called.  It can be a raw type.  In any case, erasure will stop us from seeing the actual type of the property, unless we add a type token, as mentioned earlier. What needs to change? Now let's take the above and make it convenient to use by changing the language a little. 1.  All getX/setX pairs are properties.  This includes isX, read-only properties and write-only properties.  This allows new code to work with existing beans. 2.  All explicit property declarations generate getX/setX or isX/setX pairs at compile time.  This allows new beans to work with existing code. 3.  The generated code simply calls the Property's get/set methods.  There is no generated field in the declaring class, other than for the Property itself. 4.  A syntax is provided for getting at a named Property given the name of a bean.  This is statically checked for correctness. 5.  A syntax is provided for getting the value or setting the value of a property.  The '.' operator will suffice.  A field and a property cannot exist with the same name, which avoids compatibility issues with existing code. The easiest argument against this is also the easiest to refute, namely that it calls non-obvious code.  The same argument could be used to reject polymorphism.  Plus, there are already precedents in Java.  arrayElement[index]=value is an assignment that does more than it appears to - it checks bounds.  String concatenation calls .toString() on objects.  + promotes low primitive types to int.  These are all good things; there's nothing fundamentally wrong with calling non-obvious code, as long as it is possible to discover what code is actually called. 

The strongest argument in favour is readability - there should be no readability price for using properties.  Currently there is a price.

Closures without instances, and safer non-local returns

It's possible to implement synchronous closures without instances, and to make non-local returns in closures not subject to unchecked exceptions. This article explains why. I like using Google Docs as a word processor, but the last time I published to blogger automatically from it, I lost some formatting, and even a random backslash, so this time I'm not going to bother. The actual article is here. Comment on this blog.

Why Closures in Dolphin is a Good Idea

On Javalobby, Mikael Grev argues that, while he personally likes closures, and would use them, he would not want them to exist in Java.

He makes quite some deal out of keypresses - namely, the excessive amount of keypresses required for an anonymous class in Java. However, this misses the point somewhat - if something takes many keypresses to type, it will take many 'brain cycles' to parse. In fact, in even the simplest code, an anonymous class is so distracting from the intent of the code that it almost prohibits some excellent coding styles. That is demonstrated in this article.

Apologies for the code formatting - posting from Google Docs seems a bit dodgy. Here's the original. If you're not familiar with use cases for closures, I strongly suggest you take a look at Neal Gafter's blog , and the links he has on there, which, even if you disagree with the authors' points, the articles are at least very entertaining.

Closures Make Code Easier to Understand

Consider the following Haskell code:

map (+2) [1..10]

This is the usual Java code that's roughly equivalent:

int[] list=seq(1,10);
for (int a=0;a<list.length;a++)
    list[a]+=2;

Fairly readable, but it's not as expressive. It doesn't say 'add 2 to each element of a list from 1 to 10'. Instead, it says 'make a list. for each element of that list, add 2'.

That's two sentences, and the second has a sub-clause.

If I try to write the more expressive form (the Haskell version) in Java, I get something like:

result=map(new Function<Integer,Integer>()
{
    public Integer run(Integer input)
    {
        return input+2;
    }
},seq(1,10));

As you can see, the original less expressive form actually maps better onto Java than this better style. And not just in number of keypresses, but in readability. After all, I wrote it, you're just reading it, and you're probably cringing.

The (+2) syntax from Haskell is a way of specifying the + operator, but with one of its values pre-filled. There is a more verbose, probably more familiar, syntax, that resembles Java's impending closures more:

map (x -> x+2) [1..10]

This is an anonymous function that takes a value, x, and returns x+2. The word anonymous is important. The moment we 'simplify' things by giving names to such code snippets, e.g., a named class, or a field that holds the function, or even a local variable that holds the function, we're not actually simplifying, we're increasing the number of sentences needed to express ourselves.

Function<Integer,Integer> addTwo=new Function()
{
    public Integer run(Integer input)
    {
        return input+2;
    }
};

result=map(addTwo,seq(1,10));

Yes, this starts to look more attractive, but it's not really. It just means that we need to understand what addTwo is, as well as what map is and what seq is. We're adding to the number of things we either need to commit to the subconscious, or hold in primary mental space.

For this extremely trivial example, you might wonder what the big deal is. If this was all the benefit one could get from closures, I'd agree with Mikael. However, by making trivial code exceedingly trivial, you can make less trivial code trivial, and complex code, well, readable. Being able to understand more code at once means that you can spot mistakes in it better.

Closures help to keep your code DRY, and encourage excellence.

DRY is Don't Repeat Yourself. By making the above more expressive code also more attractive, you open yourself up to all sorts of optimisations (removal of repetition - I'm not talking about performance, though that does come into it somewhat).

Most operations on lists or Strings can be expressed in terms of mapping or folding (also called reducing). For example, joining a list of Strings to add colons in between is a fold:

result=fold(new String[]{"root","0","/bin/bash"},new Function<Pair<String,String>,String>()
{
    public String run(Pair<String,String> pair)
    {
        return pair.first()+":"+pair.second();
    }
});

Now, at first glance, that code is garbage. Let's add closures:

result=fold(new String[]{"root","0","/bin/bash"},{first,second => first+":"+second});

Now, if you understand that to 'fold' is to run a function on the first and second elements of a list, then run the function on the result of that and the third element, etc., then you'll probably quickly understand the code above - but the excess notation in the anonymous inner class version makes it harder to grasp. This makes using fold unattractive. fold and map are some of the best techniques available for working with lists of data. They are immensely flexible and scalable. Google's famous MapReduce algorithm is entirely based on them.

So, without closures, we are not likely to come up with algorithms like MapReduce - that is, we are actively discouraged from writing the best code. Of course, we are able to think outside of the programming language that we use, but it tends to be slightly harder to do. I doubt that many Java programmers think in terms of folds and then convert that into a suitable Java version. Instead, we think in terms of the Java version, and maybe realise later that it was another hand-coded fold implementation.

Further, by keeping code DRY, you keep maintenance costs low, e.g., if you have a bug in your withLock() implementation, your using() implementation, your withResource() implementation, etc., you can fix it in one place. If you didn't use those, but hand-coded (or IDE-generated) it every time, then you're fixing it in many many places.

I once looked through some of the JDK source, and found that most of the resource allocations don't follow the suggested best practice - the try..finally{try{close}catch{log}} idiom. I wager that this would not have been the case had closures existed from the start. Reusable solutions would have been more attractive - more convenient.

And Now to Refute Some Points in General

These are from Mikael:

"the benefits must be proven to be measurably greater than the costs". It's impossible to prove that, as the benefits and the costs both have humans as part of their variables.

"I would guess that the more advanced coders, the ones that is usually on the closure side, does this". In that statement, 'this' meant auto-generating code using an IDE for anonymous classes. That is probably true, but auto-generating code is a workaround for a missing language feature (not necessarily a feature that should be there though - it's only with this clause that I can make the generalisation). More advanced coders probably get a slight pang of 'this sucks' whenever they auto-generate an anonymous class, or getters and setters.

"That is unless you have to use one of the proposed syntaxes for handling exceptions thrown in the closure or have some funky return structure". Clearly, anonymous classes aren't going to be removed from the language, so if you find the syntax hard to understand, you can always revert to anonymous classes. I expect IDEs will provide automated routes to and from closures and anonymous classes.

"The solution to this aesthetics problem isn't closures though, it can be solved without adding complexity by just allowing a little syntactic sugar for the AICs." Even the syntactic sugar for AICs (anonymous [inner] classes) detracts from the expressiveness, and still discourages DRY and excellent code in the same way that AICs do now. Consider:

result=fold(new String[]{"root","0","/bin/bash"},new Function<Pair<String,String>,String>()
{
        return pair.first()+":"+pair.second();
});

It's not a lot better, it's still got a lot of verbosity that could be inferred (the type parameters to Function, the word Function itself). It's still distracting.

"Closures can do many things that AICs can't. Change the variables outside their scope for instance". Like with autoboxing, you could conceivably configure your IDE to prevent yourself from doing this. For most cases it won't matter. Neal Gafter explained the reasoning behind making code that's inside a closure behave the same as code that's outside it. It doesn't break the WYSIWYG nature of Java, because it's damn obvious that, say invokeAndWait{frame=new JFrame();} will assign to the nearest variable called frame.

"I still think that the AIC should only be working on a copy of the value." This could only promote out-of-sync bugs.

"The primary cost here is that Java developers need to learn new constructs. Constructs that are not very Java-ish and therefore will take some time to getting used to." That's not a cost, it's a benefit. Learning how to use generics benefits those who have. Generics didn't look very Java-ish, but they worked well. It actually helps programmers to learn new concepts.

"Remember that not all are as bright as you and you gain nothing from alienating the Java-Joes however good that feels for your ego." Actually, I do teach some new Java programmers, and I'd be much more comfortable introducing them to:

invokeLater{frame.setVisible(true);}

than:

invokeLater(new Runnable()
{
    public void run()
    {
        frame.setVisible(true); //and, er, you'll have to make frame final.
    }
});

Closures are simpler, for all levels of programmers.

"Take the much loved Collection framework. If it'd been closure-enabled from day one it would've been even better. Now you need to squeeze in closures". Or make sure that closures are implemented in such a way that they are useful with the framework. For example, we can implement a Comparator as a closure, and don't even have to say the word 'Comparator'. It's inferred. Type inference is good.

Collections.sort(list,(x,y => y.intValue()-x.intValue());

If the JDK had to include another version of sort that was closure compatible, which it doesn't, then I would agree with you.

"You could argue the same way [against] for anything that gives more power to the developer. #DEFINE is such a thing.". The use cases for #DEFINE in C and C++, such as inclusion of header files, portability (hoho), definitions of function-like macros, are largely eliminated by simpler features in Java. What features combined are simpler than closures, for all (or most) use cases that closures have?

"With closures you can code Java that doesn't look like Java and that isn't something I'd like for the Java community.". You could replace 'closures' with 'generics' in that statement, rewind a few years, rinse and repeat.

And just a humourous note: This is from Mikael's top ten tips on how to become a Rock Star Programmer: "Write smart cool compressed code constructs". He's joking, but surely that's, well, closures. From the same place: "Less code, in a smart way, means less to maintain.". Agreed. Smart doesn't mean 'hard to understand'.

"Frankly (sort of) defining new keywords on a developer level scares the bejesus out of me." I wonder whether there was a time that allowing developers to write their own functions (rather than having them hard-wired into the machine) was scary.

Shai Almog said:

"Generally I tend to be wary from features that are designed for "experts"". Closures in Java appear not to be designed for experts, but for programmers. It looks to me like every effort is being spent to make programming in Java better. The usage syntax is very compelling.

"VM changes are that much worse even worse than half baked implementations (e.g. generics)." I've found that generics were cooked for just long enough. They could use some extra features, some garnish, but they taste nice. The worst thing about them is that they're awkward to talk about in comments on other peoples' blogs, with the old < etc.

This one's hard to quote, but, Shai conjectured that a closure-accepting method would be hard to maintain, because average programmers wouldn't understand the method.

1. Don't let code into your codebase that is above ALL your staff.
2. IDEs could probably refactor it into an equivalent interface-accepting method anyway with no change to the use site.
3. It's already possible to write code that is above the level of other programmers, e.g., with generics, enums, finally (yes, there are programmers who don't get finally), etc.

Preventing NullPointerExceptions, Maybe

Null has always bothered me. I can write code without causing NullPointerExceptions, fairly easily, but without the techniques documented here, some still slip through. Of course, my automated tests are entirely comprehensive (joke), so there's no problem, right?

Wrong. Writing tests doesn't solve the problem that null exists in the first place. If we place a bollard in the middle of a street, and test all the cars to make sure that they can get around it without hitting the houses, that doesn't make the bollard acceptable.

One rule absolutely solves this. Assign a value to each field as soon as it's declared. A non-null value. To be picky, you'd have to also ban the new Object[x] form of array creation, and never give a local variable a null value. Let's not be picky.

The instinctive reaction to this is to say that you don't always have a value to put in the field, and therefore that null is the best value.

Partly true. However, null is not the best value. The likely first thought is the NullObject pattern. For example, if we have a java.sql.Connection field, we might set it up with java.lang.reflect.Proxy, so that we can call methods on the Connection, though they do nothing. This only hides the problem, in obscure runtime behaviour. Usually, we'd rather see clear runtime behaviour (a NullPointerException) than obscure runtime behaviour ("I thought I'd saved to the DB, but it was the NullConnection"). NullObject isn't going to help.

It's better to have a real distinction between a useful value and a useless value - one that forces you to 'check', or even checks for you. There are a couple of ways of doing this. The @NotNull and @Nullable annotations introduced by IntelliJ IDEA is one - though I haven't used those myself. Another way is possible, using only the Java language. Though it has to be said, the Java 7 language will make this more comfortable.

And Now To The Meat

The following concept was shamelessly stolen from Haskell.

Given a field that may have a Connection, or null, I'll change it to 'maybe a Connection', or Maybe<Connection>. There are two implementations of Maybe - one of them does have a Connection (well, T), and one of them has Nothing.

Then, rather than testing it to see whether it really has a Connection, I tell it what I want it to do if it has a Connection, and what I want it to do if it doesn't have a Connection. Oh, and for maximum flexibility, return me the result.

Let's go with a less flexible version for a moment, as an explanation.

interface Maybe<T>
{
    void apply(SideEffect<T> runThisIfTheresAnObject,Runnable runThisIfThereIsnt);
}

interface SideEffect<T>
{
    void run(T input);
}

So, when I call maybeConnection.apply(saveStuffToDB,initialiseConnectionAndSaveStuff), if maybeConnection is 'just' a Connection, it will call saveStuffToDB.run(connection), and if it is Nothing, it will call initialiseConnectionAndSaveStuff.run().

However, there are two problems with this approach. One is that I tend towards functional programming, and this stuff relies on side effects, so it irritates me. The other is that programming side effects with anonymous classes can really be a pain in Java, thanks to the 'final' requirement on enclosing local variables.

So what I really want to do is to change apply so that it returns something. I could make it return Object, but then I'm just reintroducing the old ClassCastException possibility. I could make Maybe take two type parameters, T and R, R being the return type of apply. However, that would mean that each Maybe would only be able to run 'functions' that return one type - impractical.

Generics allows you to declare type parameters on methods, not just whole classes/interfaces, so let's do that. If you don't like the look of this, skip to the bottom and eye up the alternative implementation (visitor).

interface Maybe<T>
{
    <R> R apply(Function<T,R> ifT,R ifNothing);
}

Let's just walk through that syntax. <R> just declares a type parameter. If you don't like that, simply ignore it. apply takes in a Function, which has one method, R run(T), and it takes an R. If there is a 'real' object, a T, the Function's run method will be invoked, and the R that it returns will be returned from apply. If there isn't a real object, then ifNothing is returned.

It's rather like encapsulating an if statement. By taking the responsibility for checking null away from the user of Maybe, we're taking the possible bug away too. Note that we're only taking it as far as Maybe - of course, if the two implementations of Maybe are broken, then the bug will be everywhere.

And now for example usage:

Maybe<Connection> maybeConnection=MaybeUtility.nothing();
... some code, might set maybeConnection to something else, might not.
String outputToUser=maybeConnection.apply(new Function<Connection,String>()
{
    public String run(Connection connection)
    {
        some code that uses a PreparedStatement etc. and returns a String.
    }
},"Er, some fool forgot to connect to the database.  Fire Fred");

What we're doing here is implementing dynamic dispatch. It's another way of implementing the visitor pattern. In fact, Maybe can be implemented easily via the standard idiom for the visitor pattern - the only reason I don't is that I like single-method interfaces. I find that they fit my thinking better. They also fit the closure proposal better, which is probably worth bearing in mind now.

Here's Maybe implemented with a more obvious visitor approach:

interface Maybe<T>
{
    <R> R accept(MaybeVisitor<T,R> visitor);
}

interface MaybeVisitor<T,R>
{
    R ifJust(T t); //in Haskell, the opposite of Nothing is Just, in terms of the Maybe type.
    R ifNothing();
}

Maybe and friends can all be found in Functional Peas, which is currently a placeholder for some useful bits and pieces of functional (or nearly-functional) code.

Yeah, but..

If you think that this is wasteful in terms of programmer time, I might agree with you - until we have good syntax for closures, using Maybe isn't syntactically that attractive. This can be dealt with to an extent - such as by reducing the need for null from the original code, or choosing the visitor approach. I will blog about techniques for doing that, probably under the heading 'Reducing Mutability'. Another way is to prefer function composition over always writing 'closures', which personally I'm not very good at that yet.

If you think this is useless, because people don't make mistakes if they test enough, I refer you back to the bollard analogy at the beginning.

If you think this is useless, because I am not on a large team, I'm young, I work in a University, and therefore don't know what I'm talking about, then please don't bother commenting, and have a nice life.

If you think that this is useful, but that your colleagues won't understand or agree, just discuss it with them. They might have a better idea.

Making equals(Object) type-safe

Update - Jean-Francoise Briere came up with a better solution - I've included it at the bottom. It's easy to end up comparing two obviously incomparable objects using the equals(Object) method. This is because equals is not generic. By incomparable, I mean that it is possible to tell from the source that they are incomparable, such as new StringBuffer().equals(""). The contract of equals does not permit throwing an exception when incomparable objects are compared, because comparing two incomparable objects is not a problem; comparing two incomparable references is. That is, it makes sense if you have an List<Object> and add an Integer to it, then look in it for a String, for the equals method to be called, so it should behave normally (of course, it's quite likely that the hashCode method will be called instead). So we can't really ask the objects themselves to help us out, unless we create lots of overloaded equals methods. E.g., an Integer would need equalTo(Integer), equalTo(Number) and equalTo(Object). Clearly a pain in the proverbial. We can instead ask the static type system to help us out.

First Bad Solution; Type-Specific Statics

Wrap equals calls up, say, in static methods.

public class Equaliser
{
   public static boolean equals(String first,String second)
   {
       return first.equals(second);
   }
}

There are two problems with this: 1. Repetition - you'd have to do this for every type. 2. It doesn't work if you put a supertype in, e.g., equals(Object,Object) would match any calls and you wouldn't notice.

Second Bad Solution - Naive Application of Generics

public static <T> boolean equals(T first,T second)
{
   return first.equals(second);
}

This will actually allow a comparison between an Integer and a String, because T is resolved to Object, unless you use the clunky qualifying syntax - ClassName.<Integer>equals(1,"blah"), which is so bad it's worth avoiding in most cases. Your favourite IDE will confirm that the unqualified version resolves T to Object when you hover over the call.

First Not-So-Bad Solution - Less Naive Application of Generics

interface Equalator<T>
{
    boolean isEqualTo(T other);
}

public static <T> Equalator<T> equalator(final T first)
{
   return new Equalator<T>()
   {
       public boolean isEqualTo(T second)
       {
           return first.equals(second);
       }
   };
}

This is called as: equalator(someReference).isEqualTo(someOtherReference), and will catch more bad comparisons. Of course, if your references (not your objects) are actually of type Object, then this won't be useful at all.

And Finally, The Same Thing But More Reusable

In my own code, I implement this as a (badly named) method, equalT:

    public static <T> Function<T,Boolean> equalT(final T first)
    {
        return new Function<T,Boolean>()
        {
            public Boolean run(final T second)
            {
                return first==second || first.equals(second);
            }
        };
    }

Now it's called as: equalT(one).run(two). I should probably get rid of the first==second part of that. Function is a type I defined in the publically-available functionalpeas package - it represents a function with one argument and one return value. Now I can pass equalT(someString) to a method that expects a Function<String,Boolean>, which can be handy, and explains the 'reusable' part of this section's subtitle. One really cool thing about it is that if you replace all your calls to equals AND all your code to == with this, including primitive==primitive, then you won't get any of those pesky problems caused by comparing references instead of objects. Have fun. Next time I'll look at eliminating NullPointerExceptions. No, really. Update Jean-Francoise Briere's solution is based on my 'naive implementation using generics', but is less naive:

public static <T,U extends T> boolean equalT(T t,U u)
{
    return t.equals(u);
}

This is better than my final solution, because it doesn't rely on creating a new object for each comparison, and there are less keypresses. Thanks, Jean-Francoise!

A fairer brevity comparison between Ruby and Java

As a Java programmer, I'm not completely convinced that brevity is always good. I know that I can write some pretty unreadable brief code. ~(~0>>>prefixLength) is a nice little example. It converts a prefix length, e.g., /24, in a network number, into a netmask, e.g., 255.255.255.0 (as an unsigned int). However, for this article, I'll put readability on the backburner, somewhat. When I stumbled across Sometimes Less is More by Peter Szinek, who appears to like being photographed with camels, I found that some of the Java code posted seemed to be written by, well, someone who didn't like Java. In this post I'll try to suggest better Java examples. I will omit imports and method declarations unless they are relevant. 1. Ruby:

10.times { print "ho" }
or
print "ho" * 10

Perl possibly has a more sane syntax, print "ho" x 10; - this way '*' doesn't mean both multiplication and repetition. I'm not too bothered either way on this. The article actually gave no Java equivalent, here's one: out.println(format("%1$s%1$s%1$s%1$s%1$s%1$s%1$s%1$s%1$s%1$s","ho")); out is System.out, and format is String.format, imported statically. Obviously if you make a method to do this, the calling code becomes very very short: out.println(repeat("ho",10)); repeat comes from cirrus.hibernate.helpers.StringHelper (found via Google Code Search). 2. Ruby: if 11.odd? print "Odd!" The article showed this as the Java equivalent:

if ( 1 % 2 == 1 ) System.err.println("Odd!");

However, this only works for positives and zero. I'd prefer:

if (1%2!=0) out.println("Odd!");

(-3%2==-1) Again, with prewritten methods, this can become clearer - there is a prewritten 'odd' method in the JDK 1.4 demos, should this prove hard to write yourself.

if (odd(11)) print("Odd!");  //let's assume print is a method that does System.out.println.

3. Ruby: 102.megabytes + 24.kbytes + 10.bytes Java: 102 * 1024 * 1024 + 24 * 1024 + 10 I'd prefer this a little: 102<<20 + 24<<10 + 10; or with a little predefinition: 102*MB+24*KB+10. I really wonder what the bytes method does in Ruby! 4. Ruby: print "Currently in the #{2.ordinalize} trimester" Java: System.err.println("Currently in the" + Util.ordinalize(2) + "trimester"); My suggested Java: out.printf("Currently in the %s trimester",ordinalize(2)); 5. Ruby: puts "Running time: #{1.hour + 15.minutes + 10.seconds} seconds" Java: System.out.println("Running time: " + (3600 + 15 * 60 + 10) + "seconds"); Suggested Java: out.printf("Running time: %d seconds", 1*HOURS+15*MINUTES+10); 6. Ruby: 20.minutes.ago Java: new Date(new Date().getTime() - 20 * 60 * 1000) Suggested Java: new DateTime().minusMinutes(20) - DateTime is from Joda Time. 7. Ruby: 20.minutes.until("2006-10-9 11:00:00".to_time) Java: Date d1 = new GregorianCalendar(2006,9,6,11,00).getTime();
Date d2 = new Date(d1.getTime() - (20 * 60 * 1000)); Suggested Java: new DateTime(2006,10,9,11,0,0).minusMinutes(20) 8. Ruby:

class Circle
  attr_accessor :center, :radius
end

Java: Too long, see the original article! Suggested Java:

class Circle
{
    public Coordinate center;
    public float radius;
}

"a simple class definition having 10 fields in Java will have 80+ lines of code compared to 1 lines of the same code in Ruby." For a lot of classes, many of those fields will be immutable anyway, or at least not exposed via getters/setters. In the case of an anonymous class, there are zero extra lines of code for them. 9. Ruby:

stuff = []
stuff << "Java", "Ruby", "Python" #add some elements

Suggested Java:

List<String> stuff=arrayList();
stuff.addAll(asList("Java","Ruby","Python"));

10. Ruby: stuff = [”Java”, “Ruby”, “Python”] Suggested Java: List<String> stuff=asList("Java","Ruby","Python"); 11. The author complains that you have to sort arrays using Arrays.sort(array) instead of array.sort() - however, this can become sort(array) via a static import. 12. I think the stuff about stacks and arrays would be solved by using java.util.Stack, which has pop/push/subList etc. I don't see a great need for the static implementation to appear as part of the object though. I prefer thin objects. 13. The author seems to think that adding 'nil' values to an array/list when you try to add to an index beyond the end of the array/list is a good thing. I rather like the little protection that Java gives in that if you want null values you have to add them yourself (except with the new String[10] syntax). 14. Ruby: File.read('test.txt').scan(/.*?\. /).each { |s| puts s if s =~ /Ruby/ } Suggested Java:

import static java.lang.System.out;
import java.io.*;

class Test {
        public static void main(String[] args) throws IOException
        {
                File file=new File("filename");
                byte[] bytes=new byte[(int)file.length()];
                DataInputStream input=new DataInputStream(new FileInputStream(file));
                input.readFully(bytes);
                String[] sentences=new String(bytes,"ASCII").split("\n");

                for (String sentence: sentences)
                        if (sentence.indexOf("Ruby")!=-1)
                                out.println(sentence);
                input.close();
        }
}

I expect that I could mimic the Ruby way, given time and inclination. The end result would be this, with supporting methods: File.read("test.txt").scan("\n").each(ifMatches("Ruby",print)); 15. Ruby:

          tree = a {
            b { d e }
            c { f g h }
          }

Suggested Java:

Tree tree=tree("a",
    tree("b",leaves("d","e")),
    tree("c",leaves("f","g","h"))
  );

And just as a general comment, I'd gravitate closer to Haskell, with its type inference and very powerful static type system, than towards Ruby, Python et al, because I like the freedom to be able to mess around with my code, knowing that there is an automatic eye-over-my-shoulder that's going to tell me when I do something stupid. This post is not to invalidate the original article; it's clear that the Ruby way of doing things is obviously much simpler in many cases. However, often the idiomatic Java way is not the best anyway. Personally I've been experimenting with functional programming from within Java (is this like asking your wife to dress up as someone else?), and I think closures could really help in any future 'brevity wars'.

Where does static really fit?

Graham Rocher suggests that closures cannot be added to Java because Collection cannot compatibly grow in number of methods (methods like forEach, any, all, etc.), and apparently adding static methods to the Collections class is not a solution in these halcyon days of dependency injection. He does acknowledge that static methods are sometimes useful, but so far he hasn't said where or why. He says that Math.abs is a poor example, because you never need an alternative implementation. However, StrictMath.abs is an alternative implementation (though not used often). Suppose that you wanted to be able to choose whether to use Math or StrictMath's abs implementation. You could make or use an interface that contains the 'abs' method, and make two implementations of it. I know that many blog readers find generics difficult to understand, and some even find anonymous classes tricky, so I'll be accommodating..

interface Abs
{
    double abs(double value);
}

final class LaxAbs implements Abs
{
    public double abs(double value)
    {
        return Math.abs(value);
    }
}

final class StrictAbs implements Abs
{
    public double abs(double value)
    {
        return StrictMath.abs(value);
    }
}

Graeme's argument is that, because substitutability is required, 'forEach' and friends should not be static, so that their implementation can be provided through dependency injection. It can be seen from the above example that, given static methods, it is possible to make versions that are DI-compatible, so the static implementation can then be ignored (as it is wrapped). Then, why can't the implementation of forEach be static? For the users who just want to use the implementation given in the SDK, Collections.forEach is fine. For those who want to be able to substitute it for others, wrapping it, or using a wrapped version provided by the SDK, is trivial. I'd suggest that you can safely implement any algorithm as static, and, if there is a need, you can provide a non-static way of accessing it.

Autoboxed closures?

In response to Graeme Rocher, whose blog doesn't seem to process comments particularly well.. Graeme said: "Java has been around for a decade now. It has jumped several iterations to Java 5 (with Java 6 coming shortly) and the APIs have progressed hugely. The implications of adding closures would be huge, you would have to go back and revisit ALL the Java APIs. I mean, if closures had been around since the beginning the collections API would be entirely different." I expect there could be some autoboxing to get around the API issue, e.g., a closure that takes no input and gives no return value could be automatically boxed to a Runnable. Ditto for the other one-method interfaces, possibly. It'd certainly be interesting if you could do that for your own interfaces too. insert usual moans about autoboxing Yes, autoboxing adds a bump to the learning curve, but if, when it is understood, it removes complexity when reading and writing code, then it is a good thing. I think the autoboxing feature added in Java 1.5 works well, if you understand its mechanisms, though obviously if generics were able to work with primitive types directly (or appear to), autoboxing would be a lot less useful.

Duck Typing in Java, and no reflection

interface CanQuack
{
 void quack();
}

interface CanWalk
{
 void walk();
}

<T extends CanQuack & CanWalk> void doDucklikeThings(T t)
{
 t.quack();
 t.walk();
}

You can pass anything to this method that implements CanWalk and CanQuack. Of course, this is optimally flexible when you have one method per interface, but that's not a problem. I can easily see the dependencies of doDucklikeThings, whereas if I had a larger Duck interface, e.g.:

interface Duck
{
 void quack();
 void walk();
 void eatAFish();
}

I wouldn't be able to tell by looking at the signature of a method that used Duck, whether it would call eatAFish. This works well for reducing coupling, and would probably be better than this Proxy-based attempt. There is no runtime cost, thanks to type erasure. It's rare that someone thanks Sun for type erasure, it seems, but I have got to grips with generics fairly well, thanks to Angelika Langer's excellent generics FAQ, and I find it very useful. I have applied this technique to a lot of the 20kloc program that I work on, IPSim (a network simulator), but I am still in this process. I hope you find this useful, or give me some damn good reasons why it isn't!

Eclipse 3.1 User Experience

I gave up using Eclipse in favour of vim and ant some time ago, mainly because I did some work from home, on a 233MHz machine (my faster personal machine was stolen last year), and Eclipse was too slow to be useful there. But now I came across a bug in a unit test, and I can't figure out how to write a unit test to find that bug, so I have to do some debugging. Not having Eclipse installed, the first thing I did was launch jdb and type help, which wasn't too 'help'ful. I googled for how to use jdb, and only found a site that seemed inaccurate or out of date. I had a play with ODB (Omniscient Debugger), but couldn't get it to instrument all my classes, just the particular one I was running. So, I'm back with Eclipse, a fresh download. My first thought is that it seems a bit easier to start up, I don't seem to have to tell it on the command line where to put its workspace, it has a nice dialog. I go to Project->New Project and it has an entry for creating the project from an ant build.xml. I'm staggered. So I go about it and it seems to understand the build.xml properly. Eclipse then decides that I've got 100 syntax errors, merely because I haven't told it that I'm using Java 1.5. Let me check that build.xml.. hmm, I didn't have a source="1.5", but the ONLY compiler available on my system is the 1.5 compiler, so Eclipse could easily have detected that and just moved on. The Quick Fix for the first error seems good though - change workspace compliance and JRE to 5.0, or change project compliance and JRE to 5.0. I select workspace. So Eclipse rebuilds and gives me the following warning: taskdef class edu.umd.cs.findbugs.anttask.FindBugsTask cannot be found Back to the build.xml.. etc. I can't see how Eclipse could fail to find that, seeing as ant can. But hey, maybe if I go to Quick Fix I can tell Eclipse where the findbugs task is. Hmm, Quick Fix is greyed out. Thankfully, that's just a warning, so I'll try to ignore this annoyance and continue with my work. All the other warnings seem to be pretty innocent, unused imports, etc. It does seem that Eclipse is hiding some warnings though, as the errors before and the warnings now list exactly 100. Maybe 100 is the limit. I hope it isn't hiding any important warnings. Eclipse seems to italicise my static import of netsim.java.lang.Assertion.assertTrue, which seems like a nice little feature. On further use of the debugger, I see that it is italicising all static methods, nice. Well, I found my bug, mainly that two parameters were the wrong way around, and I've exited Eclipse. I'll still be using vim and ant, but I will keep Eclipse around for its debugger. The build.xml parsing is very handy, but not perfect. I hope Eclipse can improve this. I'll still try other debuggers, but ODB left a sour taste in the mouth - poor documentation. I should have blogged that user experience. Maybe the next one.

A new design pattern (Farm) and a nifty use of annotations.

It struck me that a difficulty with using interfaces and factories in Java is that you need to know which factory goes with which interface. A naming convention can help here, but it might not be consistent. Instead, I suggest that you annotate your interface, to know about a default implementation. This immediately brings up "Interfaces shouldn't be bound to their implementations!!! This is unpure, evil code!!!". When you are navigating code, trying to find implementations for a particular interface can be annoying and time-consuming. I have set out here the most cohesive way I can think of for finding out which class to use as a default implementation when all you have is an interface. It doesn't 'bind' the implementation to the interface. It neither 'binds' nor mentions the implementation. The interface 'knows about' a factory. I experimented with non-annotation ways of doing this, such as static blocks in interfaces, xdoclet-based solutions (well, I thought about xdoclet), but I decided they were all ugly. I think what I have is the most elegant but pragmatic way. Time for some code. @DefaultFactory(BlobbyFactory.class) interface Blobby {         void doSomething(); } This says that BlobbyFactory is the default factory for the Blobby interface. So when you want a Blobby, you instantiate BlobbyFactory and call its newInstance method. As a convenience, I wrote a Farm class that you can use as follows: Blobby blobby=(Blobby)new Farm().newInstance(Blobby.class); The reason newInstance is not a static method is that you may wish to override the default factory, in some part of your application. So the unit testing part of your application might want a mock object instead of whatever BlobbyFactory gives you (actually a BlobbyImplementation in this case). Farm farm=new Farm(); farm.setFactory(Blobby.class,new MockBlobbyFactory()); then elsewhere: Blobby blobby=(Blobby)farm.newInstance(Blobby.class); In effect, a Farm is a Map with some nice methods for abstracting the rubbish away, and for using annotations when there is no map entry. The code is all available at http://lavender.cime.net/~ricky/farm.zip To build just what you need for using it, run 'ant farm.jar' and farm.jar will be produced. It's released under the BSD licence, because I want you to be able to use it. If there is some problem with my licence choice, let me know. To build an executable example jar, run 'ant farm-example.jar' and farm-example.jar' will be produced. The ant build is not very advanced, so you can work out how to do it using just javac if you don't want to use ant. javac com/rickyclarkson/farm/*.java com/rickyclarkson/farm/example/*.java java com.rickyclarkson.farm.example.Main should work, but I haven't tested that. (Use ant!) This isn't just a Java design pattern, I've also implemented it in C just for fun. Yes, I have a strange idea of fun.

Requirement Orientated Development

For a long time I have thought that most software solutions are too far removed from the problem they attempt to solve. The simple answer seems to be to identify the use cases and base the system around those. However, normal software development seems to move away from the use cases, so I have created a new development model, called Requirement Orientated Development, or ROD, which aims to make the use case part of the code. A system consists of a number of UseCases. Each UseCase has a number of Actions which can take inputs and give outputs, and can do some processing. A UserInterface (e.g., ServletUserInterface, SwingUserInterface, AtmUserInterface, UnitTestUserInterface) works out the possible Actions at any particular time and gives the user those to choose from. I have not yet dealt with use cases which can loop, and branching is done with duplication, so it is still quite naive, but I aim to make this progress. Am I reinventing old concepts, simplifying matters too much, or have I come across something worth following up? Let me know your thoughts. I'll post some code in another blog post sometime.

Lame Java Benchmarks - Freeing Memory

Someone said that Java doesn't free up memory to the OS during a program's execution, so this is a simple test of that assertion.

import java.util.ArrayList;
import java.util.List;

public final class Main
{
        public static void main(final String[] args) throws InterruptedException
        {
                allocate();
                System.out.println(Runtime.getRuntime().freeMemory());
                System.gc();
                System.out.println(Runtime.getRuntime().freeMemory());
                System.out.println("Here");

                Thread.sleep(60000);
        }

        private static void allocate() throws InterruptedException
        {
                List list=new ArrayList();

                for (int a=0;a<5000000;a++)
                        list.add(new Object());
        }
}           

The output of running ps aux repeatedly on Linux while this is running shows that the garbage collection never has an external effect. The output of the program is simply : 6207736 66317488 Here This demonstrates that the garbage collection is happening.