Stubrn, Maps and Groovy

I've added a new feature to Stubrn: Instead of implementing an anonymous class with methods or fields as shown here, you can now pass a Map<String,Object> as a holder to stubFor().

Stubrn - Some Ideas about Concise Stubs in Java

Motivation

There are a lot of cool Java Mock frameworks out there in the wild. I am a great fan of JMock myself, but nevertheless, they are Mock frameworks. As a matter of fact, sometimes i just want to use a Stub and not a Mock. That said, and not finding a proper solution to fit my needs, i started considering my demands. If you are as eager as me to see the results of this quest, feel free to read on!

Transparent Handling of Timestamp Differences with Hibernate

SQL does not define unified methods to calculate differences between timestamp values. Many database vendors provide somehow their own solution for this problem, but when it comes to generalizing the persistence layer with Hibernate, we are faced again with the missing transparent approach. This article shows a concise solution proven to work transparently on Oracle 10g and HSQL.

In our case, we had

1. A model class, let's call it Task. Task has two fields startTime & endTime, both of type java.sql.Timestamp. And several other fields of different primitive types.
2. Task was subject to a Grails application (Although the persistence layer was not GORM, but plain Hibernate with Spring due to several other reasons).
3. We had to display lists of tasks, paginated and orderable by either all fields or the duration, which is actually the difference between endTime and startTime.
4. In production mode the data is stored in an Oracle 10g Database
5. Unit & Integration Tests were done with an In Memory HSQL Database.

These requirements implied the following constraints:

• We have to avoid redundancy. Storing the duration also as a field of Task is not an option. If it could be calculated, then it has to be calculated.
• We want to handle pagination and ordering of the fields and duration similar. We could have introduced some dispatching logic, like if duration is not envolved we use query A otherwise we use query B. But that's not a proper solution and most likely not very well maintainable. We just need a single query which can cope with both cases. Ultimately this means, we have a property name and tell Hibernate to order the result according to this property.
• We need mechanism to cope with Oracle & HSQL.

The solution we found consists of several aspects. Let's go through them top down:

Adding a property formula to the Hibernate Mapping

Since we have to handle Hibernate properties and the calculated value in the same way, we need to add duration as an "artificial property" to the Hibernate Mapping of Task. Luckily Hibernate provides a mechanism for this use case: Properties could consist of a formula, which is basically the call to a valid HQL function. We simply need to define

<hibernate-mapping>
    <class name="foo.Task" table="TASKS">
        ...
        <property 
            name="duration" 
            formula="MY_DIFF_TIME_SECONDS(START_DATE, END_DATE)" 
        />
        ...
    </class>
</hibernate-mapping>

What do we do here? We assume there is a registered function called MY_DIFF_TIME_SECONDS(x,y), which calculates the difference between two timestamps in seconds. This method is called with Task's properties START_DATE and END_DATE.

Adding custom functions to the Hibernate Dialects

We have expected that MY_DIFF_TIME_SECONDS(x,y) exists. Since it is not given by an Intelligent Creator for free, we need to take care that such a function will be created. Hibernate itself does actually the same thing. There are different Hibernate Dialects for different database vendors. Those dialects register several types and function which have individual names in the database with a unified name for Hibernate. We just have to apply this to our function as well.

    public class MyOracle10gDialect extends Oracle10gDialect {

     public MyOracle10gDialect()
     {
      super();
      registerFunction(
          "MY_DIFF_TIME_SECONDS", 
       new SQLFunctionTemplate(
           Hibernate.INTEGER, 
           "MY_DIFF_TIME_SECONDS(?1,?2)"));
     }
    }

This is the example of our custom tailored Oracle Dialect. We extend Hibernate's Dialect, make sure that super() is called and just add our new function. We do the same with MyHsqlDialect by extending HSQLDialect but just don't show it here.

Implementing the functions in the Database

Up to now we've created the unified facade for the Java world using Hibernate, now we need to look at the other side: Calculating timestamp differences in both Databases. This is, as outlined above, the actually problem we are facing. It is possible to do that in Oracle and HSQL, but with two completely different approaches. But since we have unified everything from Hibernate's point of view, we simply have to take care that the databases get a function MY_DIFF_TIME_SECONDS(x,y) to calculate our timestamp differences.

Let's do it for Oracle first:

    CREATE OR REPLACE FUNCTION MY_DIFF_TIME_SECONDS(
        start_date IN DATE,end_date in DATE)
    return NUMBER
    IS
    BEGIN
      if start_date is null 
      then 
        return null;
      end if;
      if end_date is null 
      then 
        return null;
      end if;
      return (end_date - start_date)*24*60*60;
    END;
    /

This is PL/SQL and it does what we need. Besides some defensive null checks we simply utlize the Minus operater, which is also defined on Oracle's date types. Its result is a fraction of a day, hence we need to shift the result to become our difference in seconds.

HSQL does not know PL/SQL (lucky dude!). But it enables us to add arbitrary Java functions which could be called from SQL. Fine thing:

        
    import java.sql.Timestamp;
    import org.joda.time.Period;
    
    public class HsqlStoredFunctions {

     /**
      * Returns the difference between two timestamps in seconds.
      * 
      * @param start the early time
      * @param end the later time
      * @return if start and end != null then the difference is returned,
      *         otherwise null is returned
      * 
      */
     public static Integer diffTimeInSeconds(Timestamp start,Timestamp end){
      if(start == null){
       return null;
      }  
      if(end == null){
       return null;
      }

      Period period = new Period(start.getTime(),end.getTime());

      return period.toStandardSeconds().getSeconds();
     }
    }

This is the function. It has to be static, but that's actually the only requirement. We make it aware to HSQL by:

    public class HsqlDataSourceInitializer implements InitializingBean{

     private JdbcTemplate jdbcTemplate;
     public void setDataSource(DataSource dataSource){
      this.jdbcTemplate = new JdbcTemplate(dataSource);
     }

     @Override
     public void afterPropertiesSet() throws Exception {

      String aliasDiffTime = "CREATE ALIAS MY_DIFF_TIME_SECONDS FOR " +
        " \"mypackage.testing.HsqlStoredFunctions.diffTimeInSeconds\"";
      jdbcTemplate.execute(aliasDiffTime);
     }
    }

Here we see actually two quite interesting features. First the function is registered with the shown SQL statement. Secondly we can also see when we do this registration. Just reconsider: We use HSQL only in our testing scenario. In our particular case the whole persistence layer (and all other layers of the application as well) is nicely decoupled via Spring. Hence the difference between the production and the test environment is the Spring Bean configuration of the DataSource. But since the whole setup is now silently down by Spring we need a hook to register our function when the In Memory Database is created. Therefore HsqlDataSourceInitializer is a Spring Bean which implements InitializingBean. The method afterPropertiesSet() is called after all Spring Beans have been instantiated. Since the DataSource of HSQL Database is nothing but another Spring Bean, we can safely assume that it is up and running.

Setting the sort order on Hibernate Criteria Queries

Now we have a brief look on how we set the order of Hibernate Criteria Queries to show that we have our required unified solution. We've got this nice helper class:

 
    final class CriteriaUtil {

     private static final String ORDER_ASC = "asc";
     private static final String ORDER_DESC = "desc";

     public final Criteria withOrder(Criteria criteria,String sort,String order){

      if(criteria == null){
       throw new NullPointerException();
      }

      if(order != null){
       if(!(order.equals(ORDER_ASC) || order.equals(ORDER_DESC) )){
        throw new IllegalArgumentException("Unknown order direction: " + order);
       }
      }

      if(sort != null){
       if(order != null && order.equals(ORDER_DESC)){
        criteria.addOrder(Property.forName(sort).desc());
       }
       else {
        criteria.addOrder(Property.forName(sort).asc());
       }
      }
      return criteria;
     }
    }

And we can use the attributes as provided by Grails' build-in pagination mechanism to extend our Criteria Queries.

Finally

Although we've just shown a solution for Oracle and HSQL, this approach should be applicable to other Databases as well, as long as it is possible to register custom functions and to calculate the timestamp difference in terms of these functions.

The Different Faces of Polymorphism

Others might think polymorphism is the skill used by Odo from Deep Space Nine when he shifts his form. But we computer geeks now it better. Polymorphism is the one stop show that makes writing software so much fun. Or to put in another way, it provides us with the power of generalization, loosely coupling and ultimately maintainability. But polymorphism does not just come in the shape of the too well known interfaces with its precious virtual methods. Different paradigms provide us with different mechanisms to wield the Sword of the Many Forms. In this article we will give an introduction to several of these incarnations. Namely C++'s compile time polymorphism and that what they call Duck Typing in Dynamic Typed Languages.

Let's start with the basic question: What the hell is polymorphism? It is the ability to specify behavioral contracts on the one hand and to fulfill them on the other. You can say what you expect from someone by defining an contract. He in turn could realize your contract however he wants, as long as he sticks to your defined expectations. With this simple mechanism we basically created the founding of what makes software development possible in the large scope.

Let's start simple.

Runtime Polymorphism

Yeah, we are familiar with this one. It's our daily work. Let us just exercise through it. Here is our contract:

    public interface Phone {
        void call(String phoneNumber);
    }

We simply expect a Phone. If someone provides us something fulfilling the Phone's demands, we can make a call:

    public final class Client {
        void makePhoneCall(Phone phone){
            phone.call("555-6832");
        }
    }

Then there is a very simple Phone implementation to complete the example

    public final class ToyPhone implements Phone {
        public void call(String phoneNumber){
            System.out.println("Ring Ring");
        }
    }

And finally the full scenario breaks down to:

    Phone phone = new ToyPhone();
    Client client = new Client();
    client.makePhoneCall(phone);

All in all not sophisticated, but just all we need to explain the general idea. You see: With the interface we define our contract. Our Client relies on that we give him something like a Phone, then he will just use it. How does it work? This is due to the way Java implements public methods. Actually they are virtual methods, which means, somewhere under the hood there is a vtable, which maps a class' method to a memory address where the implementation of that method could be found. This is all static runtime information. From an execution flow's point of view calling a method means getting the right address from the vtable and then jump there and continue (if we leave out compiler optimization...). Static means, that the compiler already knows where to look into the vtable, but it will contain the proper value only at runtime, of course. This is no magic at all...

Compile Time Polymorphism

Ah ha. Now it is getting more exciting. Let's play the same game, just with C++ meta programming:

    template<typename PHONE> class Client {
    public: 
     void makePhoneCall(PHONE& phone){
      phone.call("555-3223");
     }
    };

    class MobilePhone {
    public: 
     void call(const std::string& phoneNumber);
    };

    void MobilePhone::call(const std::string& phoneNumber){
     std::cout << "Hello?" << std::endl;
    }

    int main(){
     Client<MobilePhone> client;
     MobilePhone phone;
     client.makePhoneCall(phone);
    }

Let's see what we've got here. We see the client, yes that's fine. Ok, this is a templatized class, maybe we would not have expected it, but we just accept it for now. What else? Yes, the implementation of our Phone. This is a C++ class. But, wait: call() is NOT virtual, what the heck... hm let's wait a further second and finish the example. Ok, main() will run our example. And it does what we expect: Create an instance of the client, create an instance of the phone and then let the client do his call. But something is still odd... Yes! Where is the so called contract? It's not there!? And this compiles? Obviously yes... Hm... Now we owe some explanations!

What we see here is called meta programming used by the modern C++ guys and has been first described by Andrei Alexandrescu in Modern C++ Design. He explains how to utilize C++ template compiler for writing programs rather than the actual C++ compiler. Of course this blog post cannot satisfy all which has to be said about meta programming, but just so much: In our particular case we just imply an interface. You and I know, there is nothing which defines our contract explicitly. Psst! But the template compiler does not know that. It just knows that it creates a template instance for the type MobilePhone, and that PHONE and therefore MobilePhone is called on the method call(). While compiling the instantiated code the C++ compiler in turn does its usual job. MobilePhone has such method, hence everything works out fine.

I've heard people argue "What's that nonsense! Templates are for generalizing on types! Like the STL does it!" But think about it: Yes templates are used for generalization. For sure. But in this case we don't generalize on types, we generalize on behaviour. And it turns out more than fine. Are there any benefits? Yes! We've already mentioned the most crucial one: There are no vcalls involved. No vcalls means, the C++ compiler could apply plenty of its optimization strategies, which results in aweful cool runtime behaviour. Eager to see more? Boost and CGAL are awesome examples of modern C++ design.

Actually the C++ guys find this concept of compile time polymorphism that attractive, that they try to incorporate it into a future C++ standard under the term Concepts. Although they've currently postponed it from C++0x.

Duck Typing

Finally we come to our Dynamic Typed Languages. Here they say, if it talks like a duck and walks like a duck then it is a duck. What does this mean? Have a look.

The important aspect here is, besides others, that an instance is not checked for its type during compile time as it is done with Java or C++ for example, but only on runtime, just at the moment a method is called on that instance. For me as a Java Enthusiast it's obvious to give an example in Groovy:

    class Client {
        def makePhoneCall(def phone){
            phone.call('555-3126')
        }
    }
    
    class PhoneBooth {
        def call(def phoneNumber){
            println('Beep...')
        }
    }
    
    static def main(){
        def client = new Client()
        def phone = new PhoneBooth()
        client.makePhoneCall(phone)
    }

It's nearly the same game as with C++ meta programming. We have no explicit definition of our contract. This is just stated implicitly. But since method dispatching is done just when we execute call() on phone, and it's instance PhoneBooth owns such a method, everything works out fine again. But why is it helpful? Because it yields efficient code. Of course, yes, there are a lot of reasons why it is helpful to have a proper static type system. I will not deny it. I also favor it of course. But there are a certain amount of valid use cases, which have other desires, like implementation efficiency. Consider a unit test for example. Maybe you have decoupled everything loosely enough. Your class under test just knows other interfaces. But you still have to create stubs or mocks for those. This is acceptable of course (at least better than to avoid it), but even then it is boiler plate. It does not support the readability of your test. On the other hand, you can easily craft a stub for your interface out of maps and closures. Very dense code, very readable.

Finally

What have we learned today? Polymorphism is cool. And besides this there are different paradigms having different styles of polymorphism. Some have explicitly defined contracts, others don't. But they all obey to the same principle. A contract defines expectations. Implementations fulfill this contract.

Playing around with Java Scenegraph and Swing

I've just invested *some* hours to play around with

• scenegraph
• macwidgets

The results are pretty amazing, in my honest opinion. Basically I've got a SGPanel with an underlying SGNode tree. Most of the nodes are FXComponents and therefore contain swing components again. I did a quick screencast to demo the implemented effects. Currently I can slide pages to any direction in an iphone like style and overlay other components, in this case a semi transparent dialog, which has some fancy fade out effects. All in all quite cool for good ol' swing.

Using ant to manage larger Java projects

In this post I'll give a brief introduction to my experience of managing larger java projects with ant by sticking to the DRY principle.

One of the first steps during setting up a java project is surely the carefully planning of the build system. A build system's purpose is doing the always repeating tasks that occur during a project's life circle. The obvious ones are creating temporary directories, calling the compiler, packaging everything to a jar. If the complexity of the projects grows, it is quite certain that the requirements to the build system grow as well. Surely we want to include unit testing, javadoc generation. And what about automatic coverage analysis? Or ordering a pizza? Yes. Surely it should be able to cope with these challenges (maybe not the pizza thing, but i would love that...). There are roughly 2 approaches to get a build system in java: Either rely on the functionality your favored IDE provides, or use one of widely accepted build systems like ant or maven. In my opinion there are a lot of reasons not to choose the first one. Most importantly a particular project should not depend on a certain IDE. Maybe there's a common agreement in your company to focus on IDE, but this must not be the common case. As for every other aspect of software development as well, you should try to avoid dependencies, or least the unnecessary ones. On the other hand it is good to avoid complexity as well. If your source code builds in a shell, then it will most likely build every else. Hence, let's use some established build system like ant or maven.

I am using ant for about eight years now, and I am quite happy with it. My experiences are even that good, that I avoided the master the fully learning curve towards maven up to now. But this post is not about ant and maven. Actually I want to talk about how to maintain more complex projects. Let's start small. In a common ant project you will most likely have a (very simplified) structure like

/root 
    /src 
    /build 
        /bin 
        /jars
    build.xml

Your source code lives in src, it gets compiled to build/bin and jared to build/jars. You would have some targets, let's say init, compile, jar, which create some dirs, compile the sources and, who would have guessed, pack them. Anyways. But reality tends to be more complex than mickey mouse examples. If your project is larger, it could be a good advice to break your code up into different libraries. And even more important, with clear policies how to put things into libraries. There are several aspects which would affect this decision. Since your architecture is most likely organized in different components, it is wise to reflect this structure in your source code too. You should already have some policies for dependencies of your components, like

• more specialized components have to depend on more generalized ones
• don't rely on transitive dependencies
• permit cyclic dependencies

These rules are also valid on the actual library level.

But how should we reflect this in the directory layout? I've made good experiences with the following approach:

/root
   /libs
     /commons
        /src
        /build
           ...
        build.xml
     /derived
        /src
        /build
        ...   
        build.xml

But now we are getting to the root of the matter: All build files will more or less do the same. If we just copy them we would surely violate the DRY principle, and that's something we should avoid. Let's define what we need: We want to omit to redundantly specify the build process. Hence we want to have something like a template, which we always use, but, and that's important, it should leave enough freedom to make local modifications to this template. Why? Let's assume we have such a template mechanism specified somewhere and we call libraries with it. This is fine, as long as the libraries always have the same requirements. But what if we have one particular lib which differs from these requirements? let's say it consists of generated code, therefore it would have to call a source code generator after init but before compile? we could put this special target into the template as well and somehow ignore it as long as there's nothing to generate. But in my opinion this violates OCP. It would be better to provide some "hooks" when calling the template to do the special things.

Luckily ant has some nice tool in its toolbox to overcome this problem: The import target. This will let us specify some super build file, import its target and *tata* override some targets (if we want to). Let's think about import as ant's way of polymorphism. The super class (build file!) defines some abstract behaviour, the base class individualizes it to its specific needs. Therefore we will define the overall and general build steps in our template file and put wisely chosen hooks into it, which could be overridden by the importing build file. A very simplified super build file might look like

    <project name="Complex Ant Demo Master Libs">

        <!-- general library related properties -->


        <property name="dir.src" location="${dir.library}/src"/>
        <property name="dir.build" location="${dir.library}/build"/>

        <property name="dir.bin" location="${dir.build}/bin"/>
        <property name="dir.jars" location="${dir.build}/jars"/>

        <property name="file.lib.jar" value="${dir.jars}/${name.library}.jar"/>


        <target name="hook.init.pre"/>
        <target name="hook.init.post"/>

        <target name="init" description="Initializes the Library">

            <antcall target="hook.init.pre" />

            <mkdir dir="${dir.build}"/>    
            <mkdir dir="${dir.bin}"/>
            <mkdir dir="${dir.jars}"/>

            <antcall target="hook.init.post" />

        </target>

        <target name="hook.compiler.pre"/>
        <target name="hook.compile.post"/>

        <target name="compile" description="Compiles Sources" depends="init">
            <antcall target="hook.compiler.pre"/>

            <javac srcdir="${dir.src}" destdir="${dir.bin}"/>

            <antcall target="hook.compile.post" />
        </target>

        <target name="hook.jar.pre"/>
        <target name="hook.jar.post"/>

        <target name="jar" description="Creates Jars" depends="compile">
            <antcall target="hook.jar.pre" />

            <jar basedir="${dir.build}" destfile="${file.lib.jar}"/>

            <antcall target="hook.jar.post" />
        </target>    

        <target name="hook.clean.pre"/>
        <target name="hook.clean.post"/>

        <target name="clean" description="Cleans Up">

            <antcall target="hook.clean.pre"/>

            <delete dir="${dir.build}" />

            <antcall target="hook.clean.post"/>
        </target>

    </project> 

That's a long piece of example. But let's go through focussing on the interesting steps. First of all, we define some variables, which we will use later. Please note, that we implicitly assume there are variables called ${dir.library} and ${name.library}. We will see later where they come from. It's getting interesting in line 15. There we define those empty targets hook.init.pre and hook.init.post. But there purpose becomes clear in the init target below: First we call the (possible empty) pre hook, then we do our init action, finally the post hook. This mechanism is used in each target. (to be honest: I've seen this idea in the netbeans build system first...)

Let's assume we have library called "common", without any special treatment. Its build file is very simple now:

    <project name="Common Lib" basedir=".">

        <property name="dir.library" location="${basedir}"/>
        <property name="name.library" value="common"/>
        <property name="dir.project-root" location="${basedir}/../.."/>

        <import file="${dir.project-root}/build-master-lib.xml"/>

    </project> 

We simply specify those two implicit variables, just define the location for the super build file and then import it. Now we can call all the imported targets on common's build file. For instance

ant jar

would result in the appropriate jar file.

Finally consider our described specialized case. Do you remember? We want to call the code generator before compile. That's how we do it:

    <project name="A Generated Lib" basedir=".">

        <property name="dir.library" location="${basedir}"/>
        <property name="name.library" value="generated"/>
        <property name="dir.project-root" location="${basedir}/../.."/>

        <import file="${dir.project-root}/build-master-lib.xml"/>

        <property name="dir.gen" location="${dir.src}"/>

        <!-- overrides target from build-master-lib.xml -->
        <target name="hook.post.init">
            <mkdir dir="${dir.gen}"/>
        </target>

        <target name="hook.compiler.pre">
            <echo>Calling the source code generator</echo>
        </target>

        <target name="hook.clean.post">
            <mkdir dir="${dir.gen}"/>
        </target>

    </project>

We simply redefined the hooks to do all the housekeeping we need.

Of course this is only the first step in setting up a full blown, generically managed ant project. But this should be enough for now.