-k.

I live in the Dandenong Ranges, a beautiful part of Victoria with an abundance of nice twisty roads for a motorbike rider. Most days I commute to work on the bike. It cuts my travel time in half and there’s that undeniable aspect of fun. It was a late(ish) Thursday night, around 10:40 in the evening. I was riding back from dinner with a friend and closing in on home.

Living in the hills means there is always some wildlife around. I often see (and avoid) wallabies who like to bounce across the road at unpredictable moments. At the edge of my headlight beam possums and rabbits scurry for cover as my nice new Suzuki SV650s trundles down the road. As I rounded a corner I noticed a small wombat at the side of the road. Now let me tell you something about wombats, they look fat and slow, but they aren’t. Plus they’re as solid as a small bolder. As I approached he scampered out in front of me and into the path of my front wheel, flipping the bike and catapulting me over the handlebars. After a breif moment of fight I came to the ground landing hard on my shoulder.

I stood up and took stock of the damage to my bike and myself. My left shoulder was sore, figured it might be dislocated, but not badly. On initial inspection the bike looked reasonably unscathed. So I picked it up. Yup, I was pretty sure now that I’d dislocated my shoulder. The bike started straight up. Did some quick checks, bouncing the forces, making sure all the leavers still worked, and road the rest of the way home. A little grumpy at the fall, but feeling pretty lucky.

I parked the bike in the garage and took another look. The gear leaver was damaged and there were a few scrapes on the fairings. Nothing major and all relatively easy to fix up. I’d take a closer look in the morning.

I got inside and went to take my helmet off. My left arm really didn’t want to cooperate. My brother came up from the lounge and gave me a hand undoing the helmet straps and getting it off. I thought I’d better take a look at the shoulder. I tried to take my jacket off. A rush of pain. The world began to swim and I hit the deck. “Right, that’s it, I’m taking you to the hospital.” “She’ll be right. I think it’s just dislocated. I’ll go down and see the doctor in the morning. Let me take a look in the mirror.”

I pulled my shirt up to get a look at the shoulder, it looked ok. My collarbone on the other hand didn’t look so good. “Ok mate, you’re right, we better go down to the hospital.”

What happened to the wombat you ask? He ran away, seemingly unhurt.

Sensis holds a virtual monopoly (due to historical positioning) over the production of telephone directories in Australia. In “They’ve lost their Sensis” the Edward Mandla points out that the company has completely failed to innovate and build upon what has to be one of Australia’s most valuable data assets.

I wanted to ask the NSW Liberal Party a question and needed its phone number. I entered “NSW Liberal Party” into Sensis and was given a whole lot of links to articles that were not relevant. I entered the same information into Google and the first link was the NSW Liberal Party website.

Pay for position listings make the Australian Yellow Pages site almost unusable. Search for a G.P. in your area and you’ll be bombarded with breast implants in Sydney. Now that might be fine in some circumstances but I like my breasts the way they are and really I’m looking for someone to give me a flu shot.

Maybe the Australian government should look at splitting the company up. Allow multiple sales organisations to sell ads into a government operated book printing business. In the same way as any telco can provide my phone service, my long distance service, etc. Government should own the infrastructure, private business should be allowed to retail on top of it (don’t get me started on Telstra’s position by inheritance, or selling off power lines). Handing monopolistic government infrastructure over to private business isn’t good for consumers or competing businesses.

Agile Software Development, Principles, Patterns and Practices is a book that belongs on every software engineers bookshelf. It’s a book filled with the right balance of the pragmatic and theoretical.

Don’t let the title put you off. Though agile principles are dealt with there is a wealth of content concerned with the practical application of design patterns. Each design pattern is placed in an overall design context and the forces shaping it are clearly presented (unlike many of my own ideas). The pattern is then explained and it’s use demonstrated through code. The examples are numerous and, with one or two exceptions, well written. What I like most about the pattern coverage was the pragmatic context in which each pattern was placed. Unlike many other books that are presented in a “This is the way it’s done and I’m right ’cause I wrote this book and you’re an idiot if you don’t do it my way.” This book provides clear presentation of the forces that effect the pattern as well as the theory behind it and practical advice on it’s usage.

I can’t say I took a lot from the agile sections of the book, I’ve been using light weight processes for a long time now, and I’m fortunate (?) enough to be working with a guy who’s soapbox is TFD (I still don’t quiet grok it, though I love TDD). It did serve to re-enforce things I either already knew and had chosen to ignore or already knew and used.

If you don’t want to fork out the cash go find one of those bookstores with a cafe, grab a coffee (or your beverage of choice) and read the chapter on package design. You’ll be better for it and, like me, you might just buy the book.

Toward the end of last week I had three job offers on the table from three very good organisations. I’d quickly dismissed one of the organisations simply because it’s interview process had been very one way (they asked lots of questions, but were not as forth coming as I would have liked when I asked questions in return). The other two were much harder to separate. Both represented great opportunities. Both operated in a similar space consulting). Both had great people. Where they did differ greatly was size (the organisation I chose being significantly smaller) and approach (both to career development and client engagements). It was a though decision, but as my first week draws to an end I’m sure I made the right decision.

I’m now a thoughtworker. Something all thoughtworkers seem to be proud to stand up and say. Though the legendary “arrive to find your laptop on your desk” didn’t materialise for me (still waiting for Dell to deliver it) everything else went according to plan. People are genuinely interested in you, both personally and professionally. There is a passion for technology in everyone. There is a real sense of wanting to deliver true value to the client. There is a healthy level of skepticism when it comes to the “right” way to do things (in fact people seem to want to challenge many of the norms of consulting and software development). It’s exciting and so very different to the senseless politically driven organisation I was previously working for.

My son is almost 16 and nearing the end of his last “mandatory” year of school. He doesn’t want to go back next year. He can’t see the point of school. It’s a difficult thing to deal with. I’m passionate about him continuing on though the rest of high school and into university. I was talking with him last night and I explained it this way.

When you were five, if I’d just let you eat what you wanted then you would have lived on lollies and ice cream. Now you’re a little older you understand that you can’t live on lollies and ice cream. You know that although they taste good, you just need to have other things in your diet (like coke and meat pies ;)).

We went on to talk about things he might want to do. He doesn’t want a desk job, fair enough, but he has it in his mind that all jobs out of university are boring desk jobs. “I don’t want to sit in front of a computer all day.” He a gregarious kid, outgoing, happy and a bit of a charmer. I’m trying to show him that there are careers that will allow him to be involved with people, get outside, that going to university doesn’t mean doing a desk job.

It’s an ongoing process.

An updated version of Necessitas is now available. Necessitas is an Eclipse plug-in that adds a class-path container to manage a projects jars using the ivy dependency manager. You get the advantages of ivy without leaving eclipse.

Key features for this release:

• Start-up time greatly reduced (especially for poor bastards like me on a modem
• Update to Ivy 1.2a
• Takes advantage of the new “cache only” operations in 1.2a
• Will attempt to assign source achieves if they are available
• You can check for updated dependencies without downloading them
• Ivy messages are available while updating
• Lots of internal cleanup

It can be downloaded from the usual location (http://eclipse.oneill.id.au/updates/).

Spring MVC, Web Work, Cocoon, Tapestry et al provide us with great frameworks to build our web applications on, but are these frameworks the way forward or are they the last remnants of the old republic?

In a recent post to the Cocoon mailing list Stefano Mazzocchi (the father of Cocoon) asks if changes in client side technology are making these advanced web frameworks less relevant.

… and the more I learn how to driven the client, the less I feel the need for advanced server frameworks. Is it just me?

Stefano is right, things are changing, the web is becoming a true application platform (just take a look at how many patents google has taken out in this area over the last 12 months) and the current crop of frameworks are going to either change to meet our needs or fade into obscurity. It’s not just the frameworks but also the infrastructure on which they are built. As Greg Wilkins points out in his entry on continuations in Jetty 6

The advent of AJAX as a web application model is significantly changing the traffic profile seen on the server side. Because AJAX servers cannot deliver asynchronous events to the client, the AJAX client must poll for events on the server. To avoid a busy polling loop, AJAX servers will often hold onto a poll request until either there is an event or a timeout occurs.

The Servlet specification with its focus on simple request/response interactions will need to change to support asynchronous event based modes of interaction. The current crop of servlet containers use a thread per active connection which has worked in our simple request/response world may not be practical in an AJAX environment. NIO and it’s event based networking will be the key but the current servlet specification was written well before NIO and has no real support for event based processing of requests. HTTP itself isn’t all that AJAX friendly but with the web browser being the likely application client for the foreseeable future and the slow moving w3c we are going to have to live with it.

There is space to make a mark. To take a mental shift. To be apart of the evolution. It’s an exciting period to have some free time to investigate emerging technologies.

The project I’ve put my heart and soul into for the last two years has been cancelled.

I’d pretty much had enough of the project a few months back (see Time to move on?) but my manager, someone I respect, convinced me that things would change, well they did. The business is a fit of senseless stupidity and corporate political maneuvering killed the project. Not officially of course, that wont happen until July (next financial year)
. The official line is that it’s being “re-scoped”. All the contractors and many of the senior staff (one person in the entire team is remaining with the company) are being released to pursue other opportunities. I left on Friday.

The hardest thing about leaving the project is leaving the team. If you work in a corporate environment you often end up working with a large number of “also rans”. Our team wasn’t like that. It was filled with smart, energetic, committed people. We took everything that was thrown at us and turned it into a success (at least from a technical perspective). When problems arose we worked through to find the correct rather than the easiest solution. It paid off countless times and although discussions were at times heated everyone respected everyone else.

The final debriefs were hard. I sat in one meeting that involved the development manager from a downstream system. I have little time for fools and this team (with one or two exceptions) is full of them and the development manager is one of the nastiest pieces of work I’ve ever had the misfortune to work with, she a true obstructionist. Unfortunately she leads development for the public facing on-line sites so is able to use fear as a lever on the business. Anyway at one point our project manager was talking about the re-use of some of our work, and praised the development team for the quality of our output. She smirked, humphed and asked if there was any hardware she could steal from the project. When she was told no, she got up and left. I nearly lost it.

I’m proud of the work the team did, it’s a shame the system never made it to production.

• We built content storage system that was tested with 24.5 million pieces of content ranging from single lines of text to multi-megabyte image files
• We developed a method for streaming content to and from the repository that provided a predictable level of performance and memory usage
• We provided a sophisticated validation system that supported complex business rules with real-time (in editor) performance
• We created a event driven content distribution system
• We provided notification hooks for work-flow and other systems interested in repository changes and events

I’m now looking for my next challenge. I’ve spoken with old colleagues who have exciting things on the boil. The future is bright.

As a final ironic twist one of the external vendors (who was effectively part of the political power play that bought us down) has spoken to one or two people from the project about integrating our ideas into their product.

We recently switched our client database implementation from Mckoi to Derby due to some performance issues we’d run into. All was going well, performance was up significantly and everyone was happy, that was until Friday when one of the developers noted that performance had dropped a little then promptly went on holidays :).

It wasn’t until today that I got around to integrating the new database into the client application. Once I did I pulled some customer information from the server onto the client and waited, and waited, and waited, then the sinking feeling came. First I checked to see if I’d done something lame, like turned off the turbo switch (anyone else remember when PCs had turbo switches?), nope, everything pretty much checked out. Bum.

So out comes the trusty profiler, Yourkit, odd name, great profiler. Quick run through with yourkit collecting profiling information and it was apparent that we were compiling statements far more often than we should have been. The code seemed to be doing the correct thing. We use prepared statements almost everywhere, still the database log and yourkit where showing that each time the statement was used it was being recompiled.

Okay time to attach the Derby source and set a couple of breakpoints to get a better idea about what was going on. Nope the Derby source is compiled without debug symbols. That’s okay, it’s open source, I can recompile it myself. Now I’m only going to say one thing about the whole sordid process. Any ant build that has a property named sane that must be set to false to get the build to work has some very serious issues.

Now, finally, after the shock and horror of building Derby, I begin to trace execution. There are some prepared statements, there they are getting cached, all good. Let the application run and enter the area running slowly, what, the cache is empty, huh? I saw statements in there just a moment ago. Trace, click follow, ooohhh. It seems that one of the last little optimisations / sanity checks added by my holidaying colleague before departing, was to drop a constraint before running a series of operations. It was then added back (the database is accessed by a single user on a single thread) once everything was done. Unbeknown to him once you make a DDL manipulation call in a Derby transaction no statements will be cached and every call recalculates the plan. Eeeks, a minor modification to drop and restore the constraints in separate transactions did no end of good.

Derby’s documentation could have been a little clearer. It states that the following must be true for the prepared statement to be cached:

• The text must match exactly
• The current schema must match
• The Unicode flag that the statement was compiled under must match the current connection’s flag

Obviously we were violating the second point. It didn’t occur to me because the manipulation is done once at the start of the operation and the schema is the same for all calls after that. There is no mention of the fact that no statements will be cached until the transaction manipulating the schema commits. At least with the source available I was able to trace through and locate the cause.

I’m a big fan of Cedric Beust’s work but in “Testing private methods? You bet.” Cedric asserts that private methods should have tests because “if it can break, test it”. In my opinion this is just plain wrong and I believe that Cedrics own examples back me up.

Cedric provides us with a reasonably common scenario.

Imagine that you are writing a Swing table widget that lets you order your columns in several different ways. Your sorting algorithm is private and one day, you introduce a bug in it. If all you do is test the public methods, the widget will simply start showing wrong results and your tests won’t tell you anything more. For all you know, there might be a bug in your Swing code. If you unit test your (private) sorting algorithm, you will catch the error right away instead of wasting time “higher up the stack”.

This is variant on a theme that we’ve all come across. In the past I’ve seen people expose private methods as public just to test them and these methods then become a part of the public contract of the class. Funnier is the public method with a note in the javadoc that the method is really private and only exposed for testing purposes. Better yet is the use of reflection to call the a private method to test it. I could go on but I’m guessing you get my point.

My rule is simple. If it’s part of the public contract, test it, if it’s internal implementation, test is via the public contract.

So we now go back to Cedric’s problem. From the description doesn’t this sound like the perfect place to use a strategyy? Cedric’s need for a sorting algorithm sounds like an ideal place for a sorting strategy. You can then test your sorting algorithm simply and easily external to it’s usage in the widget. This breaks the design down neatly but still leaves us with a dangling pointer. The sorting strategy class is now a stand alone class that is not really part of the public contract of the application. We could make the class package protected and have the test in the same package but I like to have my tests in a separate package to ensure that I am just testing the public contract. So following the lead of the eclipse team, I put classes like this, ones that need to be public for implementation purposes but are not a part of the public contract in their own package, internal. Eclipse can even enforce that I don’t directly use classes found in any internal package.

In my opinion Cedric’s solution tests a bad design. No amount of testing will “fix” it. If you need to test a private method look at your design. If the method should be part of the public contract then make it so. If not, take another look at the design of your classes.