Saturday, June 28, 2008

Code Te Ching - Verse 46

When a tiger begins to run to parts unknown,
    she must first lean forward.
This is called instability.

When a tiger's paws land on the earth,
    they find their place of comfort.
This is called self-organizing.

When a tiger's eyes see, ears hear,
    the feet know to move.
This is called multi-learning.

When a tiger has her senses about her
    no trap can capture her feet.
This is called subtle control.

When a cub watches his mother run,
    he will mimic her movements.
This is called knowledge transfer.

Instability, self-organizing, multi-learning, subtle control, knowledge transfer.
This is the natural way of action.

Forced unnatural natural force? Madness!
Artificial process? Confusion!
The sage guides but does not manage.

Thursday, June 26, 2008

Cheap Tricks XV - Static Collections in Java

Being back in the Java game from Ruby, makes me miss a lot of the shortcuts. Despite the crushing pain I feel whenever I recognize a 5 line block of Java code that can be reproduced with one short Ruby line, being a professional Ruby developer was overall a great experience. It has made me so lazy, I have amassed quite the little toolkit to save precious lines. Here's another cheap trick around reducing the common need for a static block populated collections (NOTE: this isn't limited to Enums - if you are only dealing with Enums, just use EnumSet.of):
public class StateMachine

public static final Set<State> FINAL_STATES;
HashSet<State> fs = new HashSet<State>();
fs.add( COMPLETED );
fs.add( CANCELLED );
fs.add( ABORTED );
FINAL_STATES = Collections.unmodifiableSet( fs );
// ...
into a trim one-liner:
import static StaticUtils.sset;
import static StateMachine.State.*;

public class StateMachine

static final List<State> FINAL_STATES = sset( COMPLETED, CANCELLED, ABORTED );
// ...
Here's a code for a static list creator:
public final class StaticUtils
public static final <T> Set<T> sset( T...objects )
HashSet<T> al = new HashSet<T>( objects.length );
for ( T t : objects ) al.add( t );
return Collections.unmodifiableSet( al );
As one person pointed out, you can achieve the same effect for Lists via:
import static java.util.Arrays.asList;
import static StateMachine.State.*;

public class StateMachine

static final List<State> FINAL_STATES = asList( COMPLETED, CANCELLED, ABORTED );
// ...
Slight modifications can be made for handling maps:
public final class StaticUtils
// ...
public static final <K, V> Map<K, V> smap( Pair<K, V>...pairs )
HashMap<K, V> hm = new HashMap<K, V>();
for ( Pair<K, V> p : pairs ) hm.put( p.k, p.v );
return Collections.unmodifiableMap( hm );

public static class Pair<K, V>
private Pair(K k, V v) { this.k = k; this.v = v; }
private K k;
private V v;

public static final Pair<K, V> p( K k, V v )
return new Pair<K, V>(k, v);
Usage is similar, but bundles keys/values into a pair, the creation of which is shortcutted by the static "p" method.
import static StaticUtils.smap;
import static StaticUtils.p;
import static StateMachine.States.*;

public class StateMachine

static final Map<State, String> INCOMPLETE = smap( p(CANCELLED,"cancel"), p(ABORTED,"abort") );
// ...

Wednesday, June 18, 2008

The Flywheel and the Dead Sea

First thing's first. Some definitions:

The Flywheel Effect
Now picture a huge, heavy flywheel. It's a massive, metal disk mounted horizontally on an axle. It's about 100 feet in diameter, 10 feet thick, and it weighs about 25 tons. That flywheel is your company. Your job is to get that flywheel to move as fast as possible, because momentum -- mass times velocity -- is what will generate superior economic results over time.

Right now, the flywheel is at a standstill. To get it moving, you make a tremendous effort. You push with all of your might, and finally, you get the flywheel to inch forward. After two or three days of sustained effort, you get the flywheel to complete one entire turn. You keep pushing, and the flywheel begins to move a bit faster. It takes a lot of work, but at last the flywheel makes a second rotation. You keep pushing steadily. It makes three turns, four turns, five, six. With each turn, it moves faster, and then -- at some point, you can't say exactly when -- you break through. The momentum of the heavy wheel kicks in your favor. It spins faster and faster, with its own weight propelling it. You aren't pushing any harder, but the flywheel is accelerating, its momentum building, its speed increasing.
- Jim Collins in Good to Great

Now consider the above analogy concerning your software development team. Is it descriptive? Do they move via internal momentum, or need cattle-prodded to get work done? Moreover, consider this in terms of hiring software engineers. Good people want to work together - and they want to be valued. This is my positive analogy concerning the makeup of a development organization: if your group is filled with smart people it will attract other smart people, if they are empowered it will attract others looking to make a difference. Keep momentum high, and those incapable of keeping up will be flung off by the sheer force of the flywheel... what an attractive metaphor! Keep this in mind while reading the next one.

The Dead Sea Effect
The Dead Sea, of course, is a large body of water between Israel and Jordan, located well below sea level. The Jordan River empties into it; water leaves only by evaporation, which means that over the eons, the Dead Sea has become very salty (e.g., 8x saltier than the ocean). As such, it is generally unable to support life, except when spring floods temporarily lower the salinity.

Many large corporate/government IT shops — and not a few small ones — work like the Dead Sea...what happens is that the more talented and effective IT engineers are the ones most likely to leave — to evaporate, if you will. They are the ones least likely to put up with the frequent stupidities and workplace problems that plague large organizations; they are also the ones most likely to have other opportunities that they can readily move to.

What tends to remain behind is the ‘residue’ — the least talented and effective IT engineers.
- Bruce Webster

This is another common disease that plagues, not just IT shops, but companies that make their money on software. It tends to perpetuate when upper management has no real development experience, but I've seen it happen when those in charge are former developers themselves - the latter has always puzzled me, but considering middle-management's tendency to mask organizational problems that occur below them, I like to think that upper management is merely blissfully ignorant of the problem. but I digress.

Minding the Gap
What is at issue here is that, surprisingly, most development teams I encounter gravitate toward either end of the spectrum. The center of gravity is bimodal. Where one might conclude there is a bell curve associated with any particular company's talent pool, real productivity distribution is a little more lopsided. Certainly, overall talent may land somewhere around the middle, but a company entirely consisted of mid-range developers is still a dead sea. A pool of mediocre developers won't give you average results - you'll just get a decent maintenance staff with occasional updates - all the while lagging behind your more agile competitors. You must be a flywheel to move forward. It's like trying to break the Earth's gravitational field in a rocket... anything less than the threshold velocity and you'll tumble back to Earth. You must have a top developer to advance, and you must have a field of them to attract and retain others, and to keep each other motivated.

Google started out with top notch talent. The two founders were computer science PHDs, and naturally that helped bootstrap their organization. They were able to hire the top talent, because they recognized them. Only talent can recognize talent (Corollary 2 of the Dunning-Kruger Effect).

But this is not all you require. You need to have a reason for talent to want to work with you. Remember: at this point in time, skilled developers are at a premium. They are not desperate for your job. Anyone who is, you don't want. This has a couple corollaries:

You must pay competitive wages for today. This should go unsaid - but it can't be. Don't be cheap. Equity sharing is also good. Give your people a reason to care. Don't fall into the trap of believing because you hired your first crop of developers three years ago at 60k, that a 5% raise is sufficient to match today. Don't wait for your top developers to put in their notice to start offering the raises. You must pay competitive wages to keep talent.

Money is ok, but the most attractive thing you can offer great developers is the opportunity to work with other smart, motivated people. Along with that, from personal experience, the ability to work with new technologies is also a plus (though not always necessary or possible - if you can offer it, make this known).

Attracting good developers is only part of the issue - then you must retain them. Attraction without retention of talent creates a dead sea.

ATG started out great. Also founded by two PHDs they invented JHTML, later licensed by Sun and converted to JSP. But anymore - well - how many great developers have heard of ATG? This may sound like an unfair comparison, but I don't think so. They both started out attracting great talent, but not anymore. Having worked rather closely with ATG, as well as others who have worked in and with the company, there is nothing attractive about working there. Innovation is stifled, and that knowledge floats around the aether.

Retaining developers is easy - it's all about empowerment. Beyond that, assuming you are not attracting new developers by lying (about their situation, benefits, etc.), simply allowing talented people to be valued is often enough.

Retention of talent without motivation stops a flywheel.

Motivation is, of course, the force that causes the flywheel to turn. Keep the bar high, give talent ownership in the outcome. If you can attract top talent, and keep them motivated, retention handles itself. If you can keep talented people empowered, they will motivate themselves. And this is precisely the point. You don't need to do anything - hence the flywheel. Once it's up and running, it moves on it's own - the momentum makes it an unstoppable force.

Starting the Flywheel
Up to here, I've been pretty vague - purposefully brushing with large strokes. I want to drive home to idea that attraction, retention, and motivation are all required to create a self-sustaining flywheel of talent. So what if you're a company leaking talent, or are already a dead sea, what specifically can you do?

Step 1: Pinpoint internal talent. Ask around. Don't just ask the bosses, ask the developers' peers. All developers will know who the badasses of your organization are. Leveraging your developers allows them to have a voice in the business process - it's a great motivator, which is always great for retention. But what if you are new company, or don't have any top talent? This is more difficult - since as stated above, I believe only talent can spot talent. The next best thing to do is ask around, multiple sources, and rely on reputation - though this is the worst of the two options. It's like trying to find a good auto mechanic: you either be an expert yourself, or rely on reputation. Obviously the former is much better - and you'll probably go through several mechanics before you find a good one anyway. Reputations can be lies.

Step 2: Once you've found some top developers to meet with, leverage that talent to attract new people. Smart people tend to know other smart people. Despite the rumors, great developers are social - there are good ol' boys clubs everywhere. Chances are, your top developers are involved in other projects - either small side projects with others, or top open source projects. You never can tell. Find out. This is how you will attract the best.

Step 3: Generate mindshare. It's well known that Google Engineers are alloted 20% of their time to work on pet projects. That's where gmail, maps, movies, and other projects came from. Why let Google have all the fun? Sure, you may not be able to spare a whole day a week - but I don't care what you do, or how tight your deadlines are, every business can spare 4 hours a week for their developers to write open source tools, corporate blog, manage an external corporate bulletin board.

Make your place a fun and interesting place to work for developers - and great developers like to brag (who knows why... it must be written in our awesome DNA). Create an open source arm and release the code of internal tools your developers have written to solve problems. There are other benefits to doing this as well. Beyond the great "we have an open source arm" press you get a free army of bugfixers (mindshare may drag in new customers too). This gets your name out there, which attracts new top talent. This lets your developers run something, which keeps them motivated in other areas (for example, the work you pay them to do).

That's just one possible path to creating a self-retaining flywheel of talented, motivated developers, one of many paths, but the goal is the same. Internal motivation. You can't buy it, and you can't force it, but you certainly can build it. It will take some effort and pain on the front end, but by the time that wheel is moving, you'll wonder how you ever survived before.

Wednesday, June 11, 2008

The Worst of Crimes

There is no sin so great for a programmer as lack of curiosity. A sufficiently curious programmer can overcome a whole range of deficiencies, including laziness. Traits detrimental to normal jobs such as lack of: intelligence, social skills, or hygiene. Some of the greatest coders I know are lazy, smelly, and not necessarily "smart" in the common meaning. But they are curious about how things work - which is why they're great. These are the people that run into defects, and rather than sit on their hands, find ways to fix them. Curiosity drives research, and from self-driven research comes experience - which can even close the gap to their more intelligent peers.

Industrious coders will stay at work all night because it is their duty to do so. Curious coders will stay at work all night because they can't sleep without knowing why the database is running so damn slowly. Which would you rather hire?

I bring this up, because at my new job we're looking to hire, and I want to ensure that we aren't just going to hire someone with lots of education, or intelligence (both important attributes) but someone who has a general curiosity about how the world works.

Question: What activities do you do outside of work?
I know it's a cliche to say "I want a well-rounded person", but in this case, I do. Not because I particularly care if my co-workers like to play street-hockey, or are into beat poetry, but because curious people tend to have a lot of interests. It's all part and parcel to wanting to know about the world. Even the biggest dullard on Earth has at least one interest. If the best they can say is "I like to watch movie", move on...

Question: What non-school- or non-work- related programming project are you most proud of?
Of course, lots of people have varied interests. This doesn't mean they are necessarily relevant to the job. My buddy Jim mentioned this good question. It's the python paradox in another form: the best programmers do it outside of work. I think it's a misnomer to claim that good programmers program for fun, because I don't care if you have fun or not. I didn't learn Ruby for fun, I learned it because I needed a good scripting language to write code generators for crappy J2EE/EJB 1.0 projects. I wasn't having fun, but I was driven by a curiosity to know how Ruby worked. And that's enough. Again, no answer equals no employment...

Specify a real problem and try and gauge interest.
This isn't a question, but a strategy - and one that I prefer. Body language will tell you when someone is really interested in a problem - they'll sit up straighter, perk up a little, and give an obvious effort to solve the problem. Remember, in theory this person wants to work for you - so trial by fire. OK, at least mediation by lighter. The point is, try explain a specific technical problem you face, just minus the domain-specific jargon. For example: we run Hibernate on top of Oracle and some of the queries are going very slow. It's a good technical question, but don't - as an interviewer - get caught in the trap of just listening for technical correctness. Curious people will ask questions before working out a possible formulation - dullards will either have no answer (take a hike!), or ramble off a list of template answers (indexing, data model, bad query generation). I look for questions, even if it's as simple as "What version of Hibernate are running?". If someone can't get excited about solving a real problem (which involves attempting to understand it) during an interview, how excited will they be after 6 months of 8 [ahem, or 10... or 12] hour days?

Games aside, good technical interview questions and strategies can be useful - but never foolproof. I assume you are a curious person, and are merely seeking to find others of our kind. You only meet with a person for a short time, so like Bram Cohen (creator of BitTorrent) says: "In an interview you can tell if a person is a pleasant conversationalist, and you can give some technical questions to rule out the truly inept, but beyond that you might as well be rolling dice." That's true, but if you look for specific personality aspect - which I suggest curiosity is a must - you can at least get a good feel for this characteristic.

Good to be back!