Wednesday, January 30, 2008

Modularity: what is it, and why is it important?

Modularity is taken for granted as something which has value in software engineering, perhaps without always a well articulated understanding of what it is, and why it is valuable. I came to this realisation when I found myself having in some difficulty attempting to explain why the modularity that Impala brings can be beneficial to a large application. Instinctively, I felt like I had a good understanding, but it's useful to put this understanding into words.

What is modularity?

I googled remarkably few articles dedicated to the subject. The definition in the wikipedia entry was quite good, from which I quote the relevant sections:

"Programs that have many direct interrelationships between any two random parts of the program code are less modular (more tightly coupled) than programs where those relationships occur mainly at well-defined interfaces between modules.
Modules provide a separation between interface and implementation. A module interface expresses the elements that are provided and required by the module. The elements defined in the interface are visible to other modules. The implementation contains the working code that corresponds to the elements declared in the interface."

Modularity in Impala

This definition fits in very well with how modularity is implemented in Impala. Modules fit into a hierarchy. An Impala module only depends directly on its parent. This means that two modules at the same level have no direct dependencies on each other. Modules communicate with each other through well defined interfaces - specifically Java interfaces defined in a shared parent module.

Child modules dynamically contribute their implementation to a proxy bean defined in a higher level. I'm also planning contributions to a service registry, as is the case with OSGi. The proxies can themselves be dynamically registered, although static wirings of these are necessary to support the traditional named bean style dependency injection.

Why is Modularity Important?

Modularity is a key weapon in reducing with complexity in a large project. In the same way that blocks of functionality within a component should be partitioned into classes according to responsibilities, so should parts of an application be partitioned into modules. Like classes in an OO model, each module should have a well defined - albeit higher level - set of responsibilities. Ideally, these responsibilities should be closely related to each other.

The benefits of modularity, in my view, are as follows:
  • by definition, it reduces the unnecessary cross dependencies between code for different parts of a system. This makes code easier to maintain and extend. Impala enforces this kind of modularity because classes at the same level of the module hierarchy are not visible to each other.
  • it's easier to specify the constituents of a modular system at a higher level. An Impala application is just a collection of modules. In practical terms, it is much more convenient to specify an Impala application as a small number of modules than as a large number of Spring configuration files.
  • it's easier to read and understand a modular system. Each module has a clear and relatively narrowly defined set of functions, which can be more easily documented and explained than in the case of a system which is not modular.
  • a dynamic modular system such as Impala also has the advantage that parts of the system can be separately added, upgraded and removed. This is not possible in a system which is not modular.
Without modularity, a large system rapidly degenerates into one which is extremely hard to maintain. I would go so far as to argue that a large system which is not modular is doomed to excessive complexity, maintenance issues, and an unnecessarily high bug rate.

Wednesday, January 16, 2008

Spring Exchange 2008 in London

Today I attended the Spring exchange, held in London. Not only is it good to get away from the day job occasionally, but Spring is obviously the critical base technology for Impala, so an opportunity to get a quick, thorough and free update from the horse's mouth, as it were, is very welcome. Thanks to Spring Source and organisers Skills Matter.

Rod Johnson delivered the keynote, which as usual found another interesting angle to his perennial topic - the success of Spring. This time, the theme was the "The Changing of the Guard". It was all about the disruption that was taking place in enterprise development as a whole, with the leading technologies of yesteryear, J2EE and .NET, with their "One Size Fits All" approach, increasingly viewed as inadequate. A bleak future awaits J2EE, is his view. It's hard to disagree. Spring of course has pride of place in this future, evidenced for example by the continual upward rise of "Spring and Java" job adverts. As you would expect, he had some interesting ideas and perspectives. But I did feel like his talk went on a bit long ... I'm told it was 59 slides long!

The second talker was Sam Brannen, who went through the new features of Spring 2.5, fairly succinctly although without too much fanfare. Spring 2.5 now targets Java 1.6, fully supports JEE 5, and ships OSGi compliant bundles. However, the biggest areas of development in Spring 2.5 core are the use of annotations, in particular the myriad of ways they can be used to set up Spring bean definitions, effectively reduce the amount of configuration code that you need to write, but not without a few health warnings having thrown in.

Some of the annotation mechanisms include:
  • JSR 250 common annotations, such as @PostConstruct, @PreDestroy and @Resource
  • the Spring @Autowire annotation
  • @Component, which identifies a class as a Spring bean
plus a whole bunch of others. While I am certainly a fan of annotations for specialised uses, such as in web frameworks such as Strecks, there is certainly the danger that such a sudden and vast proliferation of annotations will be confusing to many users. Personally, for most bread and butter bean definitions I think I will carry on preferring to use the basic XML definitions, with namespace support where appropriate.

On the whole, the annotations and new namespace implementations are really syntactic sugar which don't really tackle a real problem which remains with Spring: the absence of modularity as a first class concept at the heart of the framework. SpringSource aims to address this using OSGi, and I'm told that Spring OSGi will ship 1.0 next week. Nevertheless, Spring OSGi will not quite get to the crux of the problem, because a solution is needed now, and OSGi still needs to go through plenty of growing pains before it reaches maturity in the enterprise space.

The third talk before lunch was Dave Syer, who described the improvements to Spring's web offering. This is one area where they really have made impressive strides, and there is plenty more to come. Admittedly, it is also an area where some real catch up was necessary.

The bit I liked best was Spring @MVC, the new Annotation-based controllers, which combined the component scanning, offer a much neater way to wire up web applications. Couldn't help thinking back to my work on Strecks, and the similarity in the approach! JSF support is much improved, but I'm not terribly bothered about this. There's more AJAX'y stuff, and interestingly, gracefully degrading versions of the JSF components.

Both the afternoon sessions involved the impressive Adrian Colyer and Rob Harrop double act. One of the nice things about going to these kinds of conferences is you discover some unexpected things: this time, how they were using Eclipse MyLyn to group together different files used for different parts of their demos. A useful pedagogical technique. I liked Adrian's presentation on the internals of the Spring runtime, with some well crafted diagrams, and Rob had some good perspectives on how to use Spring in the field.

Something which really struck a chord for me was his advice on the structure of projects for large, complex applications, stressing the importance of modularity, and suggesting some practices for selecting between different configurations for development, testing and production. Yes, I'm back to my favourite themes: modularity and configurability, and of course, Impala. Not only does Impala bring modularity to Spring applications, but has some a couple of neat ways of tweaking module configurations without requiring any nasty build hacks.

Unfortunately, I didn't get to the final round table Q&A session due to home commitments, but if I had, I would have asked about Spring 3.0. There was surprisingly little spoken about Spring 3.0, and the plans they seem to have are evolutionary, not dramatic, at least as far as the core project is concerned.

A good day out, and I didn't even miss my train coming home!

Saturday, January 12, 2008

Spring users: seven reasons why you'll like Impala

Here are seven reasons why you'll like Impala if you like already Spring:
  1. Spring encourages interface based programming. Impala takes this one step further: modules communicate with each other using interfaces, and interface implementations are contained within modules.
  2. Modules help you to organise your code, and eliminate unnecessary cross-code and cross-bean dependencies. Modules provide a much simpler mechanism for identifying high level relationships between parts of an application than is possible using raw application context definition files and relationships between individual beans.

  3. What's not to like? There's virtually nothing that you can do in Spring that you can't do in Impala in exactly the same way as you would do it in vanilla Spring. You can still use virtually any of the Spring APIs, any Spring feature, technique, configuration mechanism, etc., unmodified! There's no new API or language to learn. You don't need to throw away any best practices. If anything, Impala makes it easier to enforce best practices, because it strongly encourages modular applications with well defined interfaces (based on Java interfaces).
  4. A small learning curve. To start benefiting from Impala, all you need to do is organise you project according to a well defined set of conventions, and to add a bean definition into the relevant Spring context files. Otherwise, everything is the same.

    Impala is not asking you to embrace a new programming model, set of APIs or even language. Simply leverage your Java and Spring knowledge, but to greater effect.
  5. Spring is great for test driven development in that it helps you manage dependencies better. Impala takes this a step further. The interactive test runner makes writing integration tests - still a pain point with plain Spring development - really easy, hence encouraging test driven development at every level.
  6. Tasks such as reloading configurations, updating application logic, refreshing logging configurations are simple with Impala. Impala leverages Spring's first class JMX support to simplify these tasks.
  7. Impala defines a convention-based project structure which makes sense both in terms of productivity and enterprise Java development best practice.

Monday, January 7, 2008

Spring's Petclinic Sample, Impala style

I've spent a bit of time converting the Spring Petclinic sample so that it runs with Impala. Petclinic it is actually a showcase for the various Spring data access technologies. I've cut down the Impala Petclinic sample to work with following configuration:
  • data access using Hibernate
  • MySQL database
  • simple service and web tiers
To run the application, follow these steps:
  • svn co http://impala.googlecode.com/svn/trunk/petclinic petclinic
  • Open Eclipse, with the workspace set to the checkout directory (petclinic)
Now set up the database:
  • First, from petclinic/db, run createDB.txt (as root). This creates the petclinic user
    mysql -u root -p < createDB.txt
  • Then as the petclinic user, insert the tables
    mysql -u petclinic -ppetclinic petclinic < initDB.txt
  • To insert data, run
    mysql -u petclinic -ppetclinic petclinic < populateDB.txt
To run the tests from Eclipse, simply run the main class AllTests as a JUnit test, which is in the project petclinic-tests.

To run up the web application, simply run the main class StartServer, which is in the project petclinic-web. This time, run it as a (main) Java application.

To run up the interactive test runner, run HibernateClinicTest as a Java application. Type u for usage. You can run this class as a standard JUnit test - it's part of the AllTests suite.

Next on my list is to create a sample based on Spring's Petstore sample (originally Clinton Begin's JPetstore application, a sample with a rather long history). This is more interesting, as it is a larger application with a greater variety of components, so should be an interesting showcase for Impala.

Sunday, January 6, 2008

When will Impala be "go public"?

At the moment, there is still no public release of Impala. There have been no announcements on any popular web sites, to the Spring mailing lists or forums, or to any other public forum. This is deliberate. This is not because the necessary features are not present. Most of the functionality needed for a public release is present, and the code is usable and good quality. There is is enough functionality to provide real benefit to a project using Impala.

What still remains is to be absolutely sure that the Impala internal interfaces are correct. For the last few months (yes, literally!), I have been doing extensive refactoring, with the result that I am progressing towards this point. If users come up with a whole new set of requirements and ideas on how it can be used, there should be a clearly defined set of interfaces to build upon.

The first public release of Impala won't be just before going 1.0. However, it will only occur when I have reached the point that I am satisfied that the interfaces are as correct as I can make them, to support extensibility and maintainability moving forward. I think I am quite close. The interfaces are in pretty good working order, and the organisation of the code seems to make good sense.

I need to spend a bit more time working on the interactive test runner, and on deployment issues. Most of the work I've been doing with Impala has been within Eclipse itself. And of course, samples and documentation, although the first public release doesn't need to depend on these.