Eclipse Indigo: I am touched

Screenshot11

I've downloaded Eclipse 3.7 final, codename Indigo, Java Edition (not JEE).
As expected it has Maven integration by default, and that's good but
is not the real good news.

The good news is that it works great, more than I could even dream.
The integration exists and comes without any negative effect: you can
just import Maven project by default.

What is great is that when you import a project, Eclipse analyzes the
POM and propose you to install some plugins afrter it: so while importing my
projects Eclipse asked me to install WTP integration and another plugin
dedicated to OSGI, for example. And after the restart, all the existing projects
take advanteges of the new plugin and I had to do nothing. Going to Preferences | Maven | Discovery will show all the catalog.

It's not finished. I installed Spring Tools Suite from
http://www.springsource.org/node/3151 and not only installation was
smooth and working, but after the restart all the spring based
projects gained the Spring Nature again without doing nothing. The
project explorer also show a nice organized view of a project.

From Eclipse Marketplace I quickly added Subclipse, EGit and MoreUnit
and I'm ready to go.

First Eclipse release I'm really happy about since, I thing, 3.2. At
those time was the Eclipse features to be important: new refactoring,
new automation etc... Since a few years what I was missing more the
lack of integration with the project management (maven) and
application configuration (spring).

Now the gap has been filled. Good work.

Artivio - The Making of - Overview

Artivio is project I made that I will launch for private beta in the next month. It is mostly a CMS for artworks: pictures, metadata, search... nothing more than this for now, but I'm actually quite satisfied of the state of the project, form different reasons:

First, Artivio now does well what it does. It's still basic but already useful and it works fine.
Second, the "platform" that is emerged allows me to create a new similar site with reasonable low effort and with a well defined architecture.
Third, the status of the code is quite good including a good automated test base, allowing me to make modifications quickly.
Finally I've been able to successfully apply a lot of technologies and techniques that have proved to work.

The Making Of

Artivio is a made with many open source technologies, a few I master since some years like Wicket, other that I became expert during the last year: Restlet, OSGi, some JMS with ActiveMQ, MongoDB. Also, For some of these technology there is something I want to talk about and I'll promise here to write these articles:

  • JMS: event based in-app communication
  • Wicket: automatic form panels generation from XML
  • OSGi and Karaf: light war deployment
  • Restlet: light web frontend and REST API backend.
  • MongoDB: integration with Elasticsearch (so, Lucene)

I've already written the article about Wicket so I'll probably publish that first, and there is already a generic introduction to the Event Based Application (1)

Also, Artivio now is a lot more "cloudified": lives on a Amazon EC2 instance, stores pictures on S3, the database is hosted at MongoHQ and the User Administration is delegated to Pipelean Accounts (2) that offer a per user authentication, authorization and OAuth connection with third party services.

Code Quality

But today is Sunday and I'm spending some time making project cleanup (mainly removing unused classes) and analyzing the status of the code.

Just looking and using it, it seems good: for example, today I've been able to add the "Remove Picture" and the "Force Picture Upload during Bulk" features in less than one hour and it has been smooth: implementation involved four classes in nice direct flow: a new Link in the page, one line of code per class involved with the last one with 4 lines to update the database and eliminate two files from S3 storage.

What do automated analysis say? Sloccount says there are about 3100 lines of Java code in three projects (core, api, web), with 808 lines of XML for Spring IOC configuration, Maven POMs and the HTML templates also, I think. Without the test tree of the source, numbers are, for Java only

core: 236
api: 1026
web: 1328

That makes less then 2600. What does this mean? Not much, but it's good that the codebase does not explode.
PMD does not find any priority 1 or 2 problem that is not the Logger that's not static final (just final).
Findbugs also shows less then 8 issues per project, nothing major.
Copy Paste detector, from PMD Suite, also did not find nothing but a couple of real code duplication which I was already aware of.

That is all. Now I promise to publish one article per week from the list above. Stay tuned.


1. My Event Based Application article
2. Pipelean is my other main project, get in touch at http://pipelean.com or, well, just write me :)

Asynchronous non blocking apps

Since a year I'm developing my applications following the usual principle of "non duplication" but pushing it to an higher level then just code duplication.
I'm moving to eliminate the "runtime duplication", or maybe I something that can be called like that.
This lead me to decoupled modules that communicate through messages, of course. This is pretty normal in the world of software integrations, is a well known Integration Pattern. But I wanted to put this into a single, standard application, like a CMS that manage images. There are two way I can achieve this.

Scenario

Let's say that there is a message broker I can use, external to the application, that just works. The app receive a file uploaded by the user. It's an image I need to:

  1. handle the file upload to file system
  2. convert file into expected formats
  3. store file somewhere into the cloud (S3 or such)
  4. make the image available to the system  (associate with database)

Now point 1 is handled by the web framework and is synchronous (but some tricks on the user interface side that are outside the scope of my analysis now)
Simplest solution is to do 2,3 and 4 in a row: that stuck the gui of course. I should have a scheduled thread to search for new files and performed 2, 3, 4 so that the GUI becomes free as long as upload is complete, and the "backend" perform the rest of the processing: there's the scheduled thread to tune and is not that nice, is it?

What I'd rather do today? I'll send a message, I do delegate. This is nice.

At the end of point 1, the gui send a light message to the broker saying "there's a new file called ..." and it send this to the right recipient(s). The receiver performs the conversion and send a new message, this time to a new component that perform the upload to S3 and when it's finished send a last message that is received by the original application that performs the association between the stored blob and the database: now the image is available and we can eventually notify the user, maybe with some cool user interface async message.

Push further

Now if the broker is something like ActiveMQ, the components have some listener to different queues or topics and that's done. What is the next question?

Where does those components lives? In the same process of the application? Or maybe somewhere else?
It can be both. If they live with the application they can be some spring beans that register a listener on the right topic at startup. But this is a solution that does not satisfy me.
Cause there's duplication, and not enough decoupling.

The duplication is that each new app will have to have it's own instances (and configuration) of the same components. Also, if the app is hanged, all those components are and there is no reason for this to be.
Better make them leave somewhere else, and the message broker (and the right queue/topic) is the only link between who procuce a message (say: upload is done!) and the consumer (ok, I'll do perform the conversion!) and so on.


There's a better environment than java?

Now, this is something I've done many times in the last year and I'm trying to make it more "standard" for all my apps. What I'm wiondering is: there a more natural way to do this? I mean, something more deeply integrated with the language?

In java, there are a lot of good tools to do this but everything only exists in my mind and some docs. This kind of architecture really is not supported by the environment, and unit testing that is possible but hard to do an maintain.

I'm talking, maybe, about the actor model. Or something that is possible with node.js. Are those technologies really more suitable to create programs the way I just narrated? If so, what is the dark side of the moon?