The Lazy Dev

Yesterday a friend asked me a suggestion to quickly put online a form to gather email of people interested in a service he is promoting.
I told him to go with http://wufoo.com. Or Launchrock, but it's still in private beta. Anyway, my advice was to use something existent and not to reimplement it.

He was a little concerned: "but does the Wufoo logo will appear? Does not people think that if we have to used something like that we will not be able to build the service we are promising to?"

Is that a reasonable concern?
As an engineer and a guy that follows daily the evolution of the web, I think exactly the opposite. People that would reimplement a whole registration mechanism just to gather emails are not good candidate to build software that works and sticks.
Cause they make some basic and common mistakes:

1. reinvent the wheel
2. over-engineering

In using a Wufoo like solution, you prove to know your world, using the right tools for the right purpose and not just reimplement everything with your php/rails-mysql or java-oracle or whatever is your usual development stack over and over again.

Also, as a third bonus point, this is the very basic but indeed important example of what cloud computing means, and what "thinking cloud" means.

I am assembling the software developers team for a new, financed startup.

The final product will be a web service that will let user engage in videogame tournaments and matches and will focus the most famous games in the multiplayer scenario.

The team will build the new product from scratch. We are interested in experienced developers as well in younger ones. We are not considering applications from people with zero or almost zero experience.

The development will start in September 2011 in Milan. There's a very high probability that the team will be relocated in London after a few months of development or immediately after the public launch.

The development will require some months and the team will be dedicated and committed to the build and launch of the product. The company is interested in a team that will remain in the company after the project is launched, to continue working on improvements on the product.

There is not any strict technological requirement to apply for this position. My personal background is public but I will consider and choose technologies after the team is formed and not vice versa. Also, the team will be involved in the upfront analysis and extimation so to build the release schedule together.

For any information feel free to contact me personally.
Daniele Dellafiore
@ildella
ildella at gmail dot com

Starting in September, I'll work close with Telnext, which has been my client in the last year.

I've downloaded Eclipse 3.7 final, codename Indigo, Java Edition (not JEE).
As expected it has Maven integration by default, and that's good but
is not the real good news.

The good news is that it works great, more than I could even dream.
The integration exists and comes without any negative effect: you can
just import Maven project by default.

What is great is that when you import a project, Eclipse analyzes the
POM and propose you to install some plugins afrter it: so while importing my
projects Eclipse asked me to install WTP integration and another plugin
dedicated to OSGI, for example. And after the restart, all the existing projects
take advanteges of the new plugin and I had to do nothing. Going to Preferences | Maven | Discovery will show all the catalog.

It's not finished. I installed Spring Tools Suite from
http://www.springsource.org/node/3151 and not only installation was
smooth and working, but after the restart all the spring based
projects gained the Spring Nature again without doing nothing. The
project explorer also show a nice organized view of a project.

From Eclipse Marketplace I quickly added Subclipse, EGit and MoreUnit
and I'm ready to go.

First Eclipse release I'm really happy about since, I thing, 3.2. At
those time was the Eclipse features to be important: new refactoring,
new automation etc... Since a few years what I was missing more the
lack of integration with the project management (maven) and
application configuration (spring).

Now the gap has been filled. Good work.

Artivio is project I made that I will launch for private beta in the next month. It is mostly a CMS for artworks: pictures, metadata, search... nothing more than this for now, but I'm actually quite satisfied of the state of the project, form different reasons:

First, Artivio now does well what it does. It's still basic but already useful and it works fine.
Second, the "platform" that is emerged allows me to create a new similar site with reasonable low effort and with a well defined architecture.
Third, the status of the code is quite good including a good automated test base, allowing me to make modifications quickly.
Finally I've been able to successfully apply a lot of technologies and techniques that have proved to work.

The Making Of

Artivio is a made with many open source technologies, a few I master since some years like Wicket, other that I became expert during the last year: Restlet, OSGi, some JMS with ActiveMQ, MongoDB. Also, For some of these technology there is something I want to talk about and I'll promise here to write these articles:

• JMS: event based in-app communication
• Wicket: automatic form panels generation from XML
• OSGi and Karaf: light war deployment
• Restlet: light web frontend and REST API backend.
• MongoDB: integration with Elasticsearch (so, Lucene)

I've already written the article about Wicket so I'll probably publish that first, and there is already a generic introduction to the Event Based Application (1)

Also, Artivio now is a lot more "cloudified": lives on a Amazon EC2 instance, stores pictures on S3, the database is hosted at MongoHQ and the User Administration is delegated to Pipelean Accounts (2) that offer a per user authentication, authorization and OAuth connection with third party services.

Code Quality

But today is Sunday and I'm spending some time making project cleanup (mainly removing unused classes) and analyzing the status of the code.

Just looking and using it, it seems good: for example, today I've been able to add the "Remove Picture" and the "Force Picture Upload during Bulk" features in less than one hour and it has been smooth: implementation involved four classes in nice direct flow: a new Link in the page, one line of code per class involved with the last one with 4 lines to update the database and eliminate two files from S3 storage.

What do automated analysis say? Sloccount says there are about 3100 lines of Java code in three projects (core, api, web), with 808 lines of XML for Spring IOC configuration, Maven POMs and the HTML templates also, I think. Without the test tree of the source, numbers are, for Java only

core: 236
api: 1026
web: 1328

That makes less then 2600. What does this mean? Not much, but it's good that the codebase does not explode.
PMD does not find any priority 1 or 2 problem that is not the Logger that's not static final (just final).
Findbugs also shows less then 8 issues per project, nothing major.
Copy Paste detector, from PMD Suite, also did not find nothing but a couple of real code duplication which I was already aware of.

That is all. Now I promise to publish one article per week from the list above. Stay tuned.

1. My Event Based Application article
2. Pipelean is my other main project, get in touch at http://pipelean.com or, well, just write me :)

Since a year I'm developing my applications following the usual principle of "non duplication" but pushing it to an higher level then just code duplication.
I'm moving to eliminate the "runtime duplication", or maybe I something that can be called like that.
This lead me to decoupled modules that communicate through messages, of course. This is pretty normal in the world of software integrations, is a well known Integration Pattern. But I wanted to put this into a single, standard application, like a CMS that manage images. There are two way I can achieve this.

Scenario

Let's say that there is a message broker I can use, external to the application, that just works. The app receive a file uploaded by the user. It's an image I need to:

1. handle the file upload to file system
2. convert file into expected formats
3. store file somewhere into the cloud (S3 or such)
4. make the image available to the system  (associate with database)

Now point 1 is handled by the web framework and is synchronous (but some tricks on the user interface side that are outside the scope of my analysis now)
Simplest solution is to do 2,3 and 4 in a row: that stuck the gui of course. I should have a scheduled thread to search for new files and performed 2, 3, 4 so that the GUI becomes free as long as upload is complete, and the "backend" perform the rest of the processing: there's the scheduled thread to tune and is not that nice, is it?

What I'd rather do today? I'll send a message, I do delegate. This is nice.

At the end of point 1, the gui send a light message to the broker saying "there's a new file called ..." and it send this to the right recipient(s). The receiver performs the conversion and send a new message, this time to a new component that perform the upload to S3 and when it's finished send a last message that is received by the original application that performs the association between the stored blob and the database: now the image is available and we can eventually notify the user, maybe with some cool user interface async message.

Push further

Now if the broker is something like ActiveMQ, the components have some listener to different queues or topics and that's done. What is the next question?

Where does those components lives? In the same process of the application? Or maybe somewhere else?
It can be both. If they live with the application they can be some spring beans that register a listener on the right topic at startup. But this is a solution that does not satisfy me.
Cause there's duplication, and not enough decoupling.

The duplication is that each new app will have to have it's own instances (and configuration) of the same components. Also, if the app is hanged, all those components are and there is no reason for this to be.
Better make them leave somewhere else, and the message broker (and the right queue/topic) is the only link between who procuce a message (say: upload is done!) and the consumer (ok, I'll do perform the conversion!) and so on.

There's a better environment than java?

Now, this is something I've done many times in the last year and I'm trying to make it more "standard" for all my apps. What I'm wiondering is: there a more natural way to do this? I mean, something more deeply integrated with the language?

In java, there are a lot of good tools to do this but everything only exists in my mind and some docs. This kind of architecture really is not supported by the environment, and unit testing that is possible but hard to do an maintain.

I'm talking, maybe, about the actor model. Or something that is possible with node.js. Are those technologies really more suitable to create programs the way I just narrated? If so, what is the dark side of the moon?

What happened to me today.

The OSGi Manifest file generator tool suddenly started inserting a dot in the import package. A dot? A dot. Does not make sense, who tells him to do that?
This way the bundle won't start on the OSGi container. Why does this happen? All other 4 projects with the very exact same configuration work fine.

Two hours investigating the problem. In the end, I hacked the maven-bundle-plugin, discovered that is the underline tool, Bnd, that insert the dots, and hacked the maven-bundle-plugin to remove the dots.

In the meantime, I asked the question on twitter. http://twitter.com/#!/ildella/status/71189575145496576
@njbartlett answered me: "It can happen with classes produced by the Eclipse compiler that still have compile errors in them. "
http://twitter.com/#!/njbartlett/status/71192822333718528

Really? Really?
"It's because you drank a capuccino too late this morning".
"Hey, have you called your mom today? That's why it does not work"
"Look out, there's too much sun, so Bnd insert the dot".

These explanations to me make sense exacly like the one that has been provided by @njbartlett
The only difference is that he is right, that is exactly the problem that was affecting me.

But why? Why in the world this happen? And why specifically with Eclipse?
You can say it's easy, you just make the build compile. Yes, but it continue not to make sense.
And I still lost two precious hours dealing with some completely insane behavior.

That's bad.

1. how do I tell obr:deploy to ignore optional bundles?
2. how do I change the layout of "osgi:list" and "obr:list" to display symbolicName instead of the (useless) bundle name?
3. how do I instruct maven-bundle-plugin to create a war without the dependencies jars in the WEB-INF/lib folder?
4. how do I configure Export-Package to export ALL the packages in the bundle BUT the packages that the bundles already imports from some other bundle
5. why there are so few tutorial kind of documentation about OSGI?

Yesterday, February 28th 2011, Apache Karaf 2.2.0 OSGI container has
been released. It's a serious upgrade, the release notes say:

Some highlights include many new or updated commands, a LDAP JAAS module with login support, an updated feature service, a feature for Aries transaction, JPA & JNDI modules, expanded documentation, and new versions of most dependencies.

and here is a list of the most significative improvment I see:

• XML schema for features descriptor https://issues.apache.org/jira/browse/KARAF-53
• Pax Web 1.0 https://issues.apache.org/jira/browse/KARAF-338
• Allow the use of version ranges on dependant features https://issues.apache.org/jira/browse/KARAF-251
• encryption with Jasypt 1.7 https://issues.apache.org/jira/browse/KARAF-266
• Pax Logging 1.6 with SLF4J 1.6 support https://issues.apache.org/jira/browse/KARAF-396
• Command to manage users, password and roles https://issues.apache.org/jira/browse/KARAF-188
• 'extract' deployment option, with automatic detection of feature repositories https://issues.apache.org/jira/browse/KARAF-151

Also we have some minor upgrade to Aries, Spring, Jetty, Pax and so on
and something like 50 bug fixes, here's the complete release notes,
find out what's relevant for you.
http://karaf.apache.org/index/community/download/karaf-2.2.0-release.html