The Flywheel and the Dead Sea

First thing's first. Some definitions:

The Flywheel Effect

Now picture a huge, heavy flywheel. It's a massive, metal disk mounted horizontally on an axle. It's about 100 feet in diameter, 10 feet thick, and it weighs about 25 tons. That flywheel is your company. Your job is to get that flywheel to move as fast as possible, because momentum -- mass times velocity -- is what will generate superior economic results over time.

Right now, the flywheel is at a standstill. To get it moving, you make a tremendous effort. You push with all of your might, and finally, you get the flywheel to inch forward. After two or three days of sustained effort, you get the flywheel to complete one entire turn. You keep pushing, and the flywheel begins to move a bit faster. It takes a lot of work, but at last the flywheel makes a second rotation. You keep pushing steadily. It makes three turns, four turns, five, six. With each turn, it moves faster, and then -- at some point, you can't say exactly when -- you break through. The momentum of the heavy wheel kicks in your favor. It spins faster and faster, with its own weight propelling it. You aren't pushing any harder, but the flywheel is accelerating, its momentum building, its speed increasing.

- Jim Collins in Good to Great

Now consider the above analogy concerning your software development team. Is it descriptive? Do they move via internal momentum, or need cattle-prodded to get work done? Moreover, consider this in terms of hiring software engineers. Good people want to work together - and they want to be valued. This is my positive analogy concerning the makeup of a development organization: if your group is filled with smart people it will attract other smart people, if they are empowered it will attract others looking to make a difference. Keep momentum high, and those incapable of keeping up will be flung off by the sheer force of the flywheel... what an attractive metaphor! Keep this in mind while reading the next one.

The Dead Sea Effect

The Dead Sea, of course, is a large body of water between Israel and Jordan, located well below sea level. The Jordan River empties into it; water leaves only by evaporation, which means that over the eons, the Dead Sea has become very salty (e.g., 8x saltier than the ocean). As such, it is generally unable to support life, except when spring floods temporarily lower the salinity.

Many large corporate/government IT shops — and not a few small ones — work like the Dead Sea...what happens is that the more talented and effective IT engineers are the ones most likely to leave — to evaporate, if you will. They are the ones least likely to put up with the frequent stupidities and workplace problems that plague large organizations; they are also the ones most likely to have other opportunities that they can readily move to.

What tends to remain behind is the ‘residue’ — the least talented and effective IT engineers.

- Bruce Webster

This is another common disease that plagues, not just IT shops, but companies that make their money on software. It tends to perpetuate when upper management has no real development experience, but I've seen it happen when those in charge are former developers themselves - the latter has always puzzled me, but considering middle-management's tendency to mask organizational problems that occur below them, I like to think that upper management is merely blissfully ignorant of the problem. but I digress.

Minding the Gap
What is at issue here is that, surprisingly, most development teams I encounter gravitate toward either end of the spectrum. The center of gravity is bimodal. Where one might conclude there is a bell curve associated with any particular company's talent pool, real productivity distribution is a little more lopsided. Certainly, overall talent may land somewhere around the middle, but a company entirely consisted of mid-range developers is still a dead sea. A pool of mediocre developers won't give you average results - you'll just get a decent maintenance staff with occasional updates - all the while lagging behind your more agile competitors. You must be a flywheel to move forward. It's like trying to break the Earth's gravitational field in a rocket... anything less than the threshold velocity and you'll tumble back to Earth. You must have a top developer to advance, and you must have a field of them to attract and retain others, and to keep each other motivated.

Attraction
Google started out with top notch talent. The two founders were computer science PHDs, and naturally that helped bootstrap their organization. They were able to hire the top talent, because they recognized them. Only talent can recognize talent (Corollary 2 of the Dunning-Kruger Effect).

But this is not all you require. You need to have a reason for talent to want to work with you. Remember: at this point in time, skilled developers are at a premium. They are not desperate for your job. Anyone who is, you don't want. This has a couple corollaries:

You must pay competitive wages for today. This should go unsaid - but it can't be. Don't be cheap. Equity sharing is also good. Give your people a reason to care. Don't fall into the trap of believing because you hired your first crop of developers three years ago at 60k, that a 5% raise is sufficient to match today. Don't wait for your top developers to put in their notice to start offering the raises. You must pay competitive wages to keep talent.

Money is ok, but the most attractive thing you can offer great developers is the opportunity to work with other smart, motivated people. Along with that, from personal experience, the ability to work with new technologies is also a plus (though not always necessary or possible - if you can offer it, make this known).

Attracting good developers is only part of the issue - then you must retain them. Attraction without retention of talent creates a dead sea.

Retention
ATG started out great. Also founded by two PHDs they invented JHTML, later licensed by Sun and converted to JSP. But anymore - well - how many great developers have heard of ATG? This may sound like an unfair comparison, but I don't think so. They both started out attracting great talent, but not anymore. Having worked rather closely with ATG, as well as others who have worked in and with the company, there is nothing attractive about working there. Innovation is stifled, and that knowledge floats around the aether.

Retaining developers is easy - it's all about empowerment. Beyond that, assuming you are not attracting new developers by lying (about their situation, benefits, etc.), simply allowing talented people to be valued is often enough.

Retention of talent without motivation stops a flywheel.

Motivation
Motivation is, of course, the force that causes the flywheel to turn. Keep the bar high, give talent ownership in the outcome. If you can attract top talent, and keep them motivated, retention handles itself. If you can keep talented people empowered, they will motivate themselves. And this is precisely the point. You don't need to do anything - hence the flywheel. Once it's up and running, it moves on it's own - the momentum makes it an unstoppable force.

Starting the Flywheel
Up to here, I've been pretty vague - purposefully brushing with large strokes. I want to drive home to idea that attraction, retention, and motivation are all required to create a self-sustaining flywheel of talent. So what if you're a company leaking talent, or are already a dead sea, what specifically can you do?

Step 1: Pinpoint internal talent. Ask around. Don't just ask the bosses, ask the developers' peers. All developers will know who the badasses of your organization are. Leveraging your developers allows them to have a voice in the business process - it's a great motivator, which is always great for retention. But what if you are new company, or don't have any top talent? This is more difficult - since as stated above, I believe only talent can spot talent. The next best thing to do is ask around, multiple sources, and rely on reputation - though this is the worst of the two options. It's like trying to find a good auto mechanic: you either be an expert yourself, or rely on reputation. Obviously the former is much better - and you'll probably go through several mechanics before you find a good one anyway. Reputations can be lies.

Step 2: Once you've found some top developers to meet with, leverage that talent to attract new people. Smart people tend to know other smart people. Despite the rumors, great developers are social - there are good ol' boys clubs everywhere. Chances are, your top developers are involved in other projects - either small side projects with others, or top open source projects. You never can tell. Find out. This is how you will attract the best.

Step 3: Generate mindshare. It's well known that Google Engineers are alloted 20% of their time to work on pet projects. That's where gmail, maps, movies, and other projects came from. Why let Google have all the fun? Sure, you may not be able to spare a whole day a week - but I don't care what you do, or how tight your deadlines are, every business can spare 4 hours a week for their developers to write open source tools, corporate blog, manage an external corporate bulletin board.

Make your place a fun and interesting place to work for developers - and great developers like to brag (who knows why... it must be written in our awesome DNA). Create an open source arm and release the code of internal tools your developers have written to solve problems. There are other benefits to doing this as well. Beyond the great "we have an open source arm" press you get a free army of bugfixers (mindshare may drag in new customers too). This gets your name out there, which attracts new top talent. This lets your developers run something, which keeps them motivated in other areas (for example, the work you pay them to do).


That's just one possible path to creating a self-retaining flywheel of talented, motivated developers, one of many paths, but the goal is the same. Internal motivation. You can't buy it, and you can't force it, but you certainly can build it. It will take some effort and pain on the front end, but by the time that wheel is moving, you'll wonder how you ever survived before.

The Power of Skunkworks

When you think of the long and gloomy history of man, you will find more hideous crimes have been committed in the name of obedience than have ever been committed in the name of rebellion. -- C. P. Snow

Or what about the definition of insanity? Doing the same thing over and over, but expecting a different result.

Sometimes underhanded measures are the best. I would never recommend making a habit of it, but there are times where obeying the letter of your job description (or contract) is the worst thing you can do for all involved - yourself and your client. As a programmer, architect, or manager of programmers, it is your job to create software that fulfills a business purpose. But sometimes the forest gets lost for the trees. Sometimes the people in charge have been given bad advice, sometimes they have pre-conceived notions about how things must be, or sometimes they are just plain incompetent. At times like these, desperate measures are called for. Fork an alternate reality where software makes sense. Skunkworks.

Some Valid Reasons for Skunkworks

The existing technology would be far more complex to change than to just rewrite. For example, Boss won't allow it, since he thinks because it took his 16 year old "genius" nephew 6 months to write that PHP script, it will take you just as long to rewrite it. Ignore him and crank your own code in a couple days.

Management wants to purchase an expensive system, where you know a perfect FOSS replacement that is easier to manage and free. You asked, but they claim the OSS system isn't "enterprisey" enough - you point out that the Brazilian Healthcare System runs millions of hits a day, and you're only looking at hundreds. Boss still refuses. Try it out, prove it works, and make the move. When you encounter resistance (and you will) keep escalating the issue. I've done this - moving up to the CIO before my change was accepted. Upper management is always happy to save money. I even let my boss claim it was his idea, just to save face.

An out-of-touch former engineer who designed the system (and happens to now be in charge) irrationally insists the system he wrote 10 years ago was perfect and nothing needs changed. You have inherited said system and all headaches contained therein. This is a tough sell, but again: rewrite, prove it works, escalate until you get the answer you want. The average sale requires seven contacts before something is actually sold. The average salesman stops at five. Persistence is shockingly effective.

Methodology; Least intrusive to most

Add new code
Inject alternate chunks of code, and some sort of switch to decide which will be executed. It's simple enough, allows a lot of reuse, but only really works if no-one is paying attention to the codebase.

Fork the code
Create an alternate trunk and make changes in that. If you use the same SCM it may make merging the code easier in the future.

Create a new project
Obviously the biggest endeavor, since this is effectively starting from scratch. But sometimes you can create a new project in a few days, where it could take weeks or months to hammer their square changes into the existing round system.

While at LPA me and a guy from Amentra were told to make the Blaze Rules Engine to work in ways it was not built to be used. After banging our heads on it for a month, countless conference calls with Fair Isaac, and general disgust with the complexity - I spent a weekend learning Drools and replaced it. It was perfect. Just what we needed. I switched their coffee with mine, and they never noticed. I told them a week later what happened. Told my boss. Then his boss. Several conference calls later my system was officially in - Blaze was out, saving our company $250k/year in license and support fees.

Fire people
This isn't really skunkworks, except for the fact that you may have to hide the real reason for firing someone. So in that case it isn't "official". This is damn intrusive. Also only works if you're a project manager. Sometimes the problem isn't the system - it's the people. Start over with "smarter" people. Probably won't work. Last resort.

Aftermath

Huzzah, hurray, congratulations! You've succeeded in keeping your project on the down-low, proved it worked, and moved up the chain until you got the greenlight to use your elegant brain-child. Now you have an easier life, and your clients get their features that much faster. Epic win. Everyone's happy, right? Fat chance. Someone's toes got stepped on - someone is is going to be pissed off.

The idea of skunkwork'd projects lies in the fact that it's easier to ask forgiveness than permission. If you ask beforehand, you are likely to be turned down. But if you do it and get caught, you can always beg forgiveness. Downplay your plan. Act meek. When your architect finds out what you did behind his back, tell him that his system was great, you're just more comfortable working with yours - you used it as a learning experience. When your project manager is angry at the hours you spent, say it took only a couple hours, and you worked on it over the weekend.

The truth is, unsolicited projects are dangerous. Only attempt them if you are confident in your 1337 h4Xing skillz. If you do it, you WILL be noticed. Eventually. This can be a good or bad thing. This may be a feather in your promotion-cap, or you may be signing your own pink-slip. But to quote Ferris Bueller: Only the meek get pinched. The bold survive.

To My Parents

As a member of the forgotten generation - that gap between the Gen-X slackers and the post-9/11 Millennials - those of us born between 1974 and 1989 - I ask you a favor: support Barack Obama.

This man is the John Kennedy of my generation - the only hope we've ever had. Do this for me - your child - let me stop living in your shadows.

You - booming babys - who stunted my growth with a steady diet of your youth. Down my throat went stories of how you were hippies and war heros and civil rights champions and creators of love and music.

You - lucky you - who held Dr. King, and JFK, and Lennon, and Carl Sagan, and Ginsberg as your own - leaving me with Jesse Jackson, and Clinton, and Madonna, and Bill Gates, and fuck if I know any poets.

You who witnessed, live, a man on the moon - while I witnessed, live, a teacher explode in the stratusphere.

You, who forced your children to grow in a world of television, demographics, and video games - yet dare blame me for my uninvolvement in the real world, when I've never confronted real, never seen or even smelled real. You grew up when history was static in books by experts, and left me to paddle ill-equipped in the historical fluid of wikipedia.

You had art - I have remixes, remakes and mash-ups.

You leave me insurmountable problems. You watched our debt grow from 0 to 9 trillion in your lifetime. You watched the skys blacken and the globe warm - and ask why I'm not doing more to fix it? And I will - but give me the political tools.

You gave me the burden of healthcare for you and my children, leaving me to pay a social security that I will never see - leaving the country bankrupt to pay you your pittance when you retire. Security indeed.

We have no hope. But don't blame yourselves, don't cry, mom. It's not gone - hope is a feeling I've never known.

I grew up knowing the problems you caused rested upon me to fix. I know I will never retire, and my children will be worse off than you, and that technology is our last great hope - and must be built by my hands. Yet you continue to push against my ideals like open-source voting or digital medical records or electric cars.

The truth is your generation drank up all the hope and wonder and left me thirsty for so long I feel born jaded. I have nothing left to believe in but iPhones and American Idol - which are more likely to stick around than the local grocery you complain I don't buy from.

Ramen is ramen, I say.

Dear, parents - you have fought some good battles in your lives. You can stop fighting now - you won. I grew up in a time where all people are equal, thanks. It never occurred to me to make my wife a homemaker, or disregard an employee because he's not caucasian. My battles are not black versus white or young versus old or rich versus poor. My battle is simple: the past versus the future. You must understand: things cannot stay the same.

The sheer size of your generation ensures that all votes bend to your desires. I don't know if I can change your minds in time, so I ask only for a small favor. Trust me - trust your children - if only this once. Vote for Barack Obama - my chance at hope.

Please. Get out of the spotlight. It's my turn to effect change - and I need Obama.

Stop User Group Assholery!

We've all seen it - and I'm not talking about being a flaming jerkwad. I can ignore that, and hence via my powers of displacement, forgive it. But this is an unforgivable sin in this modern age of search engines and public forums. Don't be this guy, though I know you have witnessed at one time or another, mining the very dark corners of the web for an esoteric answer, scrambling to find a single relevant hit on google.co.jp:

Hey guys! I'm having this problem and can't find an answer
please help!!! :(

[Some trace or dump]

Thanks;
Random Asshole

Only to be responded to hours later:

Random Asshole wrote:
> 
> Hey guys! I'm having this problem and can't find an answer
> please help!!! :(
> 
> [Some trace or dump]
>
> Thanks;
> Random Asshole

Ok, I figured it out and it works... ;)

Do you not see the problem here? Irony aside, you're just plain being a dick. I have a rule. Let's call it - oh, I don't know - the Golden Rule: Treat others how you want to be treated. Remember that one? If not, learn it. Its reach extends beyond the elementary school playground.

I propose a list of people who ask for help but do not reciprocate. Sort of like a "Do Not Call" list or "Sex Offender" registry. But instead of for telemarketers and frat guys, respectively, it's for assholes. Any names appearing on that list are barred from having their questions answered until such time as proving themselves nice little Netizens and sowing what they reap. Until then - need to find an answer to my problem: "Internal protocol error: No valid method call - missing method name", no thanks to this guy.

Sustainable, Sushmainable

Here's the thing: "A Standish Group research report shows a staggering 31.1% of projects will be canceled before they ever get completed. Further results indicate 52.7% of projects will cost 189% of their original estimates... On the success side, the average is only 16.2% for software projects that are completed on time and on budget. In the larger companies, the news is even worse: only 9% of their projects come in on time and on budget."

Well, don't freak out - or nod in knowing agreement for that matter. This report is over a decade old (The CHAOS Report). So, times have changed, right? A 2004 update shows that project cancellation has actually declined to 18%, and project success has risen to 28%, supposedly.

Big flippin' deal.

No matter what side of the fence you side on, can we at least agree that - oh, say - only 50% of software projects actually succeed? In other words, any given project you are on is really a coin-flip on whether it will ever be used.

This leads to my argument: I hate Java and .NET - bloated, confusing framework garbage. I've ranted about Java-esque static-typed OO languages before, and still point a cold boney finger at them as the culprit. Despite any philosophical musings, the above facts are the real reason I now stick mainly to agile languages. The numero uno reason I hear in favor of Java and .NET? "They are sustainable." Again:

Big flippin' deal.

I cannot stress enough how important it is to just get software out the door. Sustainability? Puhleeze - your project, if it does survive, won't last 10 years. Scalability? Don't kid yourself - your Killer App will likely never get that popular. I got SnipSnipe (check it out) done in Rails in a week. If it were Java, I'd still be working on it. Sure, I am still working on it, but that doesn't mean you can't use it.

$ rails snipsnipe; cd snipsnipe
$ mysqladmin create snipsnipe_development
$ script/plugin install http://svn.techno-weenie.net/projects/plugins/acts_as_authenticated
$ script/plugin install http://svn.viney.net.nz/things/rails/plugins/acts_as_taggable_on_steroids
$ script/plugin install http://juixe.com/svn/acts_as_commentable
$ script/plugin install http://juixe.com/svn/acts_as_voteable
$ script/generate model Snippet
$ script/generate authenticated user account

Make a few migrations and db:migrate, hang the acts_as_* gang onto the Snippet model.

class Snippet < ActiveRecord::Base
  acts_as_taggable
  acts_as_voteable
  acts_as_commentable

  belongs_to :user

  validates_presence_of :title, :body_raw
  validates_format_of :credit_url, 
      :with => /^http(s|):\/\/|^$/, 
      :message=> "must begin with http://"
end

There. That's the framework of SnipSnipe. The rest was just html/javascript and about 80 lines of controller code. Most of my time spent was fighting with Prototype and CSS. But, back to the task at hand.

Forget that fact that your project has only a 50% likelyhood of ever getting used at all, what are the odds of sustainability or scalability ever becoming a problem? If you ever write software that gets to these next levels - then, and only then, should you worry about such navel-gazing. In the interim, you can work on more new projects. Yay! Until then the best advice I can give is, just get the bitch out the door! (You can quote me on that)

Sorry for This

I don't like blogging about blogs or bloggers - the whole enterprise smacks of, oh I don't know, a chain letter. "I blog about you, you blog about me, pass it on to 10 people and you will get traffic! If you don't you'll be relegated to obscurity." You know - lots of text, no substance.

That said - I can't let gold like this pass. Bravo.

The Curve is a Circle

Jim and I have talked about this at length, and considering our obsession with meme-fabrication, this is one we both felt the need to discuss.

The phrase "the curve is a circle" is one we concocted to deal with the inadequacy of the illustrative ability of the term "learning curve" to describe the movement from beginner to expert. Such a viewpoint cannot appreciate the actions of one who has surpassed such sophomoric phases and ascended to the ranks of utter simplicity. Such states are almost Zen-like - moving along the circle only to discover there is no active difference - only an epistemic one. Sometimes the beginner and expert find themselves in agreement, where the intermediate believes himself superior to both. Consider the example JW gives: moving from plain text editor, to fancier IDEs, eventually venturing back to plain text editors. Corinthians 13:11 didn't quite have it right - sometimes being a man requires embracing childish things.



So how can one tell what side of the circle someone, or yourself, is on? That's the bad news. Unless you are on the other side it's damn near impossible to effectively argue a point to one on a different cycle. And believe me - it doesn't do well for an argument to declare, "you just don't understand the situation - I'm so far ahead I've lapped you." It's a good way to get punched in the face.

It's not new or a leap to say that the very language of a discussion has a different meaning depending on your point of view. This really makes communication difficult between an expert and sophomore - a little knowledge is a dangerous thing.

Here is an illustration of a conversation between me as a beginner and you as someone with intermediate knowledge on static versus dynamic typing.

You: Static typing is awesome!
Me: Dynamic typing is better.
You: How do you figure? Static typing gives compile-time checks.
Me: But, dynamic type is easier to code in.
You: Static typing is safer for large development teams.
Me: Yeah, but uh, dynamic is easier?
... ad infinitum ...

Now, let's say you make a claim as an expert - and I make a counter-claim as an intermediate fan of static typing.

You: Dynamic typing is awesome.
Me: No, static typing is the best. You are a n00b.
You: I like dynamic-typing because it makes it faster to develop and manage code. Once the use-cases are satisfied, you'll find a lot of unnecessary time that would have been spent on static-type frameworks has been saved.
Me: Static typing is safer for large development teams.
You: Yeah, I know - but without all of that overhead (and frameworks) to manage types, you don't need as many developers involved in a project - so you don't really need to worry about team size.
Me: Naw - they'll get bigger eventually - you've just worked on trivial projects.
You: No, Eric, you are an ass - I've worked teams of various sizes.
... and scene ...

In both cases, the fan of static typing is an intermediate level, while the dynamic typing fan is a newbie in the first (unable to articulate his position) and expert in the second (able to make an experienced, non-theoretical case). But look at it from the static-typing fan's perspective: they're both wrong, and both inexperienced. The curve is a circle.

Of course, maybe this is all just hypothetical.

Linus Agrees with Me

Well, not personally, and possibly not at all. But - as you may have read his rancorous rant against C++ - one line in particular caught my eye:

C++ leads to really really bad design choices. You invariably start using the "nice" library features of the language like STL and Boost and other total and utter crap, that may "help" you program, but causes ... inefficient abstracted programming models where two years down the road you notice that some abstraction wasn't very efficient, but now all your code depends on all the nice object models around it, and you cannot fix it without rewriting your app.

Wow - and to think I bother to pull punches occasionally.

The only difference is, the point he makes about C++ I extend to static typing in general. But he is correct - premeditated abstraction is my point #3... static typing requires you to know the future.

Linus agrees with me => QED

(note: put away your blow torches, flamers, it's called witticism)

5 Reasons Static Typing Sucks

Static typing is a pedantic construct, bred from experiments of the first Object Oriented languages, in the same way that the Vienna Circle was an experiment with the Tractus - a confused failure (for those robbed of a formal philosophical education I'm not just being an asshole here, the case is apt and expanded below). I am not against all forms of static typing, and for example, do appreciate how ECMAScript 2.0 uses a mix of static and dynamic types (adopted by ActionScript 2+).

It is not type-safety I am against, per se, it is the mindless parroting of virtues of "compile-time checking" while citing none of its drawbacks. I am against being forced to statically type everything in a language when it is not always necessary (consider in Java giving an Object a type, just to put it into a Map, where its type is lost, and convert it, at runtime, back into a type - though to be fair, generics go a long way to fixing that mess. But I maintain that a real type-safe system would have no need for ClassCastException). In my opinion type safety has one shining virtue: to solidify APIs. But I feel the trade-offs between the dangers of dynamically-typed languages versus the constraints and verbosity of statically-typed ones is an easy choice to make. I will create my own safety, thankyouverymuch. For those craving a visual experience, here is my belief codified:


Perhaps it would be hyperbolic to claim - being forced back into wrestling with a static type system - I'd prefer to wrestle with the business end of a shotgun. But I do have reasons - a mix of Ivory Tower and practical reasons which are sure to sway no die-hards on either side of the isle, but must be made mention for posterity - so at least our progeny can rest in the knowledge we weren't all mad.

1. Static typing requires extra typing

Keyboard typing, that is. As far as I am concerned - anything that adds more code is a Bad Thing^TM. It slows down development and increases the potential for defects. Working around the rules of a type system requires extra code that detracts from the problem at hand. All I ever hear in response to this is: yeah, but the compiler catches type errors. Have you ever run into a ClassCastException in Java code? How is that type-safety? Moreover, how common are type errors - I mean, really? I so rarely come across them at my job (where we have a whole staff of developers working on the same Ruby project) that it is hardly ever a consideration. All I can figure is that type-safety stops bad developers from doing dumb things, which my stance is quite simple on bad developers: fire them.

But the larger point is that programming will never get away from the need to give variables meaningful names and some sort of rudimentary documentation (even if that documentation is merely simple and readable code). If you use meaningful names, you can always figure out a type. In fairness, I have, in Ruby, been forced to resort to the following when running across unfamiliar code:

puts var.class

Before you start yelling "fowl!" I'll have you know I've typed my fair share of this kind of crap, too:

System.out.println(list.get(0).getClass().getName());

Have I ever written dynamically typed code that has run for two hours, only to return and find that it broke because of a typeo or a wrongly-used type? Yes. But in almost every case it could have been fixed by better unit tests. Besides, those few situations pale in comparison to the man-months saved in typing, digging through verbose code, and wrestling with the type system.

2. Static types are unreal

Real things are not statically typed. If you accept Phenomenology's basic views of reality (which - if you are an existentialist, Eastern philosopher, or relativist - you do by fiat, and you are an Objectivist - well, there is no hope for you), "objects" are only signaled (or named) by their utility. For example, we have a word "hammer" because in our system of perception we have a use for declaring it as a single class of objects with a common utility. What do you do if you need to bang on a nail and you don't have a "hammer"? You find a substitute - a rock for instance. You use the rock "as a hammer" in which case it too becomes a "hammer".

Even if you do not buy the above statement - in every day life, as long as we can successfully use something for which it is needed (as Heidegger would say Zuhandensein, it is "at hand"), that is good enough.

Consider that I am building a deck behind my house. My requirements for materials are: sturdy, cuttable, nailable, and looks_nice. Now I'm given objects of type "Wood" and some objects of type "Plastic". As long as my objects work for all criteria of which I require, it hardly matters what they are actually made of - even if I did not know in advance what I would be using those objects for.

Static typing not only requires you to know what you want to do with those objects in advance, it requires you to codify it (via either common inheritance or codifying for all known desired types - think if...instanceof in Java). This brings up the next problem:

3. Static typing requires you to know the future

Logical Positivism has failed. This is the philosophy that a comprehensive view of the world (via proven scientific statements - note the key word: proven) can be built up from simple to more complex statements like mathematical theorems. Of course, this was quickly shown unworkable via the criticisms of guys like Karl Popper and W. V. Quine - basically, science cannot "prove" anything (only disprove and give a best guess - or theory) and that since language can change with new discoveries, previous theories are all suspect (the term "gravity" has a different meaning after Einstein than it did after Newton, so building mathematical statements from the proofs of "gravity" would be to build on a house-of-cards).

Yet - despite the fundamental inability for a workable theory to be built off of a language of static terms - this is precisely what statically typed languages attempt. For example, consider that we create a software system that models a ball manufacturing company. Undoubtedly, we have an object named "Ball" that has a property "radius" and specialize (extend) that type to "SoccerBall" and "BasketBall" with their own properties.

abstract class Ball {
  Float radius;
}

class SoccerBall extends Ball {
  Boolean boringToPlay;
}

class BasketBall extends Ball {
  Boolean boringToWatch;
}

Everything looks fine, so we build our entire system based on the "Ball" type including manufacturing and billing. Everything looks fine - that is - until the company decides to start making rugby balls. So we extend the "Ball" model with "RugbyBall" - now what does the inherited property "radius" mean? At best we retrofit a meaning, at worst we ignore it. Whatever you do - your manager may still chastise you for your short-sightedness. Just let her know that you fell victim to a shortcoming of the logical positivism of which static typed programming is too similar - it is not your fault.

If you do attempt to re-define what "radius" means we need to retrofit the term (this is a common phenom in linguistics. For example, consider the word "oven" - when "microwave ovens" became popular a new term was required to differentiate the two: "conventional oven"). Static-termed languages are not equipped to deal with this change so we create substitute tricks like Aspects or Dependency Injection. But whatever we do - we tend to require a lot more typing to fix the fundamental flaw: such languages are ill-equipped at dealing with reality except in the narrowest sense, and can only grow via retrospect. So-called dynamic or "duck type" languages minimize such problems, since they make no assumptions on what terms (types) mean, only how they are used. You change where you need to, and elsewhere leave well enough alone. Our temporal frailties aside, the fact that the future can bring about unexpected changes have birthed one of the most frightening words in the developer's lexicon: "refactoring". So much for the idea that terms are static - yet delusional, we lumber on.

4. People want to work around it

I consider this the best evidence against (required) static typing. Theories are good, but like the old quote goes, "in theory, theory and practice are the same. In practice, they are not."

People want to work around static type systems. This is evinced by the rise of frameworks such as Spring in Java (configuration and run-time object override) or Reflection in C# (run-time view and replacement). As time goes on these languages keep adding more and more features to loosen the static typing (Java did not originally have Reflection - now they are considering adding named arguments for easier integration to dynamically typed scripting languages). I've read James Gosling claim that "safety is freedom" since you always know what type you are getting back - but is that really the case?

Orwellian-sounding phrases aside ("safety is freedom", "war is peace", "static is dynamic"), many powerful modern Java frameworks are built on some sort of Inversion of Control principle (like Eclipse or Maven) - which completely removes compile-time safety since types are not resolved until run-time - and work by voodoo such as runtime reflection, bytecode manipulation, and killing chickens. (Now, don't come to me and say: you're confusing the compiler and the type-checker. True, the JVM verifies type safety at runtime, but by then it's too late... you may as well have scripted your code.)

My single question to these frameworks is: why all the hassle? If static typing is so great, why do these frameworks work so hard trying to circumvent it? Perhaps it is a cultural artifact of the C++ and Self days, and a lingering confusion of class versus prototype-based programming. Is it an idea that John Q. Programmer holds, that class-equals-static and prototype-equals-dynamic? Do fans of class-based languages think they need to support static typing to support the concept of classes? I wish I knew - but the love-hate relationship between a programmer and their favorite language's type system astounds me.

5. Valid logic is rejectable

class C {
  Integer getDefault() {
    return new Integer(0);
  }

  int func(Float x) {
    if( x == null ) {
      x = getDefault();
    }
    return x.intValue();
  }
}

The above program will not compile in Java, despite the fact that the above program would not fail at runtime (ignoring JVM checks), since both Float and Integer inherit intValue from Number. But the type system requires you to rewrite the above perfectly logical code. This time - this one time - I want Float and Integer to be compatible. But no, the Mighty Type System has spoken! This is somewhat related to the first annoyance - but is not so much about the extra code required to rewrite the above into something acceptable by the type system, but the fact that it needs rewritten at all.

Here is a rewrite of the above function func to agree with the Java type system.

  int func(Float x) {
    if( x == null ) {
      x = new Float( getDefault().floatValue() );
    }
    return x.intValue();
  }

It adds no real descriptive value as to what the program does - its extra verbosity makes the type system happy, sure, but does it make the code any more functional, more readable, or less error prone? The answer to all three questions is "no". Are cases like the above worth throwing away a whole type system - just for a few coding shortcuts? I claim yes, it is. At the very least drop such a rigid static type system... dare I say a static static type system? No, I suppose not. Coming up with such bad puns is unbecoming.

So that's my list. Did it change your mind? Unlikely - indoctrination is hard to unravel, hence the difficulty with Zen. Don't take my word for it, try creating a large-scale project without a statically typed language. Am I assuming you haven't? Yes. So few people get the opportunity. Try it, get back to me. Please, no stories older than 5 years ago: "Eric, we tried using Perl 15 years ago and it was a disaster!" Try again with modern techniques. Perl has grown - shouldn't we all?

Housecleaning

First: If you have not noticed, I've started restructuring the blog to reduce its ugliness quotient. First and foremost was the addition of dp.SyntaxHighlighter - truly mana from heaven. However - it requires code to exist in textareas, not pre, so RSS feeds will look kind of ugly until I figure out a fix (blame blogger - they like to injects <br/> after lines). I also added some better RSS (Feedburner) and rocked out the linkwhoring at the bottom of each article.

Second: I finally got added to the Terracotta committers page! Now whose leg do I have to hump to get my name added onto Maven?

Third: Apparently my post about overriding nil in Ruby ruffled a few feathers amongst the Java crowed. Who'd have known null was a sacred cow? If that is the case, I'm curious as to the backlash concerning my next post "5 Reasons Static Typing Sucks".

Fourth: A belated congratulations to Jimbojw for his (fairly) recent promotion as a MediaWiki committer!

Convention over Configuration

Convention over configuration has been out in mind-space for a couple years now, yet unlike the Agile Manifesto, I have not seen any roadmap on how a project can achieve it. This is a gap in human knowledge. Since I love nothing more than to fill gaps on my free time, here are a few principles recurrent in my work with two of the top CoC-based projects: Ruby on Rails and Apache Maven.

I. Conventions are emergent.

Sitting down beforehand and designing a convention is only useful as a touchstone, a point of reference, and should only appear out of practical experience with many real-world examples. Any convention not born of real use-cases is doomed to failure. Don't be afraid to throw away "flexibility" in the beginning. Maven 2 had to fight through the pain of Maven 1 before it got it's modern architecture, and Rails stood on the backs of years and years of experience, the dynamics of the Ruby language, and still several versions.

II. Lack of a real example for a convention is the strongest argument against it.

The burden for denying change should not depend on counter-examples - the lack of an example is enough for denying a convention creation/change.

Here is an example: someone recently wanted to make a fundamental change to Maven but could not provide a valid use-case for it to be so. The arguer would not let the matter drop until someone came up with a counter-example on why the change would not be good. This should not be the case. The fact that the arguer could not come up with an example for the change should always be enough to discount the idea. Hypothetical "well - in the future we may want to..." answers are not examples - they are hypotheticals... refer to point 1.

III. A convention should provide as few ways to do a single task as possible.

This is the exact opposite of configuration - which states that the best configurations allow you the flexibility to do whatever you want. Convention should have an opinion - this is the correct way to do this action.

Fewer options are easier to communicate and learn. There are already a Vast scope of options - it is simpler to conceive and convey when there is only one or two ways to execute a specific task.

IV. Conventions should have a narrow scope.

Don't go too far. A large complex convention must be managed, and managing them requires more configuration, which negates the purposes of the conventions, to wit.

Ruby on Rails is a great example of this. They provided a few useful tools and a way to extend those tools via third party plugins. Rails itself is surprisingly simple - it is the large number of plugins that makes it powerful.

V. When in doubt, simplicity.

This should go without saying - but it doesn't. A simple convention should always win out over an equal but more complex one - despite the perceived add power. Assuming conventions are emergent and should have a narrow scope then conventions will, over time, become more focused on solving precise use-cases. Complex conventions which attempt to solve a wide-berth of problems are harder to manage, and thus, the convention is wiped-out due to necessary configuration complexity. Good conventions should work out of the box.

80 Columns, Unwillingly Continued...

Wow - who knew I'd have to write about this again so soon - if ever. Please understand, I'm not advocating lack of style. I'm not an anarchist, just a fan of practicality. And practically speaking, most people code on monitors made past 1988.

Time for an example.

I didn't write this code, but here is an example - I searched for all of 10 seconds to find it, I'm sure there are worse ones in our system:
<br /> # ... blah blah blah ...<br /> body = self.get_client_accounts_body(config, ppc_id)<br /> data = self.send_soap_message(config, ppc_id, body,<br /> config['soap']['urls']['account_service'], false)<br /> out = data.scan(/<ns\d+:getClientAccountsReturn xmlns:ns\d+="">(\S+)<\/ns\d+:getClientAccountsReturn>$/)<br /> # ... blah blah blah continued ...<br />
Do you see that? One < 80 column wrapped line - one unwrapped line.

The "data" line is 56 columns wide. Can anyone tell me what the point of wrapping this line was? I know! habit. Were the second line not wrapped, it would have been 104 columns wide - still shorter than the next line. The "out" line is 107 columns.

Let's first try and cram that into an 80-column-width cheerleader's wet dream (I'll even be kind and add spaces):
<br /> # ... blah blah blah ...<br /><br /> body = self.get_client_accounts_body(config, ppc_id)<br /><br /> data = self.send_soap_message(config, ppc_id, body, <br /> config['soap']['urls']['account_service'], false)<br /><br /> re_str = "<ns\d+:getClientAccountsReturn xmlns:ns\d+="""<br /> re_str += ">(\S+)<\/ns\d+:getClientAccountsReturn>$"<br /><br /> out = data.scan( Regexp.new( re_str ) )<br /><br /> # ... blah blah blah continued ...<br />
Now let's keep the above spacing. Some people seem to think I'm advocating no coding standards whatsoever - well, plain not true.

<br /> # ... blah blah blah ...<br /><br /> body = self.get_client_accounts_body(config, ppc_id)<br /><br /> data = self.send_soap_message(config, ppc_id, body, config['soap']['urls']['account_service'], false)<br /><br /> out = data.scan(/<ns\d+:getClientAccountsReturn xmlns:ns\d+="">(\S+)<\/ns\d+:getClientAccountsReturn>$/)<br /><br /> # ... blah blah blah continued ...<br />
One operation per line. Concise - easy to understand. Get client body, send that body as a soap message, parse the data. No added lines, no added breaks, no added complication just to keep everything crammed into a weird artificial and archaic constraint.

When looking through thousands of lines of code for the first time, I always prefer chunks of code that each execute a single step. I don't care that the regular expression is one long line - because in normal maintenance, I'll never need to know. In the off chance I do need to know what the expression does, well, I'm going to have to read it anyway - but I have to find it first.

Just don't miss the forest (overall project code) for the trees (individual lines).

Horse Asses are 80 Columns Wide

When I was younger my dad told me a story - which I deign to say sounds somewhere between half- and un-true (as most of his stories tended to be). It went something like this:

When designing the first rockets, early engineers had to consider transportation along the ground to their destinations. They chose to make the boosters half the width of the road so they could be loaded onto a truck and transported after manufacturing.

Trucks - as it were - are limited to the width of the road and were originally built to be of similar width to train tracks, which until the advent of automobiles were the only long-distance terrestrial transport available and a spacing-unit comfortable to those who manufactured the first cars and roads.

Trains and carriages were of a similar width - having descended from old European standards which were based on their roads which were patterned after the roads first carved out by the Roman Empire.

The Roman roads were little more than solidified paths which were packed by the grooves made by chariots - which of course were merely the width of two horses.

The punchline of the joke was something like "This goes to show that engineers just follow the horses asses that came before them."

Good story, eh? Its truthfulness notwithstanding - it makes a good point. I have a term for it - I call it "trimming the ham". Since I already have my slippers and pipe, and you, nestled by the fireplace in love-worn jammies, I'll spin another yarn.

There were once two college roommates, preparing their first big meal out in the world on a crisp fall day. Like any intelligent and innocent country girls, they decided that a ham would make a super-duper centerpiece to their array of delicious and healthful accouterments: baked beans, salad and cornbread. Before placing the ham in the pot, the sweet, amateur cook took a large knife and proceeded to cut the ends off - safely of course.

Her roommate - a curious girl with golden locks and a chocolate brown complexion inquired innocently: "What the fuck are you doing?"
"That is how you cook a ham. You trim the ends off before cooking."
"That's fucking crazy. I've never seen that shit," she replied coyly.
"Watch your fucking mouth," the cook suggested.
"Fuck you!" The golden-hair girl objected.
"Fine - I'll call my mom and we'll settle this. I bet 20 bucks I'm right."
"You're on, bitch!" Goldilocks agreed to the proposal.

So they skipped off to the public phone where the girl with golden braids and the cook called home.
"Hey mom, is that you?" She said into the phone. "Shut the hell up Sissy, and put mom on the phone you little brat!" she added, cheerily.
"Mom - you cut the ends off of your ham, right?" pause, "OK, so you do?" The black girl with the golden hair frowned. "I see - grandma does it that way too? Uh huh... she is a great cook... OK... yeah, thanks mom - bye... Please send money," she added with love.

The ham chef beamed with pride and informed her lovely roommate that, "You see? I told you, I fucking told you... now give me my twenty bucks."
"Listen," she nodded, "Your mom, you, even your granny are all nuts."
"Twenty... dollars," she repeated, ready for her congratulations.
"OK - enough of that shit," Goldie interjected. "Let's call your Granny - I gotta know what's wrong with your family."

So back to the phone the happy chef turned - this time to ring her loving grandmother. This time, she held the phone up so both could hear.

Ring ring... ring ring... ring ring

"Old people," she said with familial pride, "sleep all the goddamn time." Eventually the phone picked up.
"Hello"
"Hi Granny!" She shouted courteously
"Oh hello Deary! You know, I was just talking about you the other day to Mildred, you remember Mildred, right? Well - you know what her grandson is doing..."
"Sounds good Granny," she interjected, "Hey, I have a cooking emergency I need some help with."
Grandma cooed on the other end of the line.
"You cut the ends off of hams, right? Mom said you did."
Grandma laughed, "Oh dear - I haven't done that in years! I don't do that anymore."
The cooks grin and Goldie's frown switched places. "Why not?"
"I have a larger pan now. You see, Dear, I cut the ends off to fit in my old small pan."

Did you enjoy that? I hope so... a modern classic if I say so myself (which I don't). The moral is quite clear, though - "because it's always been that way" is no excuse to continue doing stupid things. Find out why, then adapt to new circumstances. Common wisdom - it seems to me - is rarely very wise.

Do I have a point? Sure. 80 column width programming. Stop. Please, oh please, for the love of Jesus Helguera (Google it) stop adhering to this outdated standard. I have worked with - oh, about 25 different programming teams over the years - and a disturbing majority are obsessed with that 80 column thing. It made sense back with old fixed-width displays, it even made sense a few years ago when people coded via telnet+vi. But not anymore. I hear the same damn arguments: but what if someone needs to connect and edit code in a terminal? You want to know my answer? Fuck 'em. It's 2007 - even terminals can stretch further than 80 column width anymore. Besides, why let those highly unlikely, exceedingly rare situations dictate standards? All editors these days can scroll right or autowrap - only retards and Satanists code remotely. Seriously... all non-trivial projects should have version control. Check out the damn code. If you find yourself modifying code via ssh+emacs (et al), you're probably modifying live code which is punishable by defenestration in some circles.

I'd rather a line be a single operation, even if it is 300 columns long (I have a 23 inch cinema display... are you a professional programmer? yeah, then you should too) even if I had to scroll. If you are a halfway decent coder those rare long lines should be far and few between - making the rest of your code more readable overall.

"But Eric, I travel a lot and have to code on my tiny laptop." Yeah, me too, but most people don't. Don't you care about them, you selfish punk? Please, think of the children.

I just started a new job last week, and I see a weird mix. Some lines are 150 columns wide, and some are broken into max 80 column chunks. Why?

Justkeeptypinguntiltheactionthatmakessenseasasingleoperationisdoneevenifitmightbealittlelongatleastthenitsasinglelineandifpeoplereallycaretolookatallofhtedynamicsoftheactionthentheycanscrolloffthesideofhtescreenjustthatonetime.

Of course, that whole thing fits on my screen - but even if it didn't at least I know it's one operation. Now if you'll excuse me, I need to find my medication.

Avoiding Meeting Creep

I wish I could take credit for the name - but I can proudly say I need not take credit for the phenomenon. I can hear the sounds of knowing nods from here by the mere mention of the term: "meeting creep". Anyone involved knows it with no further description. These are meetings akin to the more famous "scope creep" with many of the same downfalls: soul-sucking wheel-spinning wastes of time and resources.

Like scope-creep, these types of meetings arise from the same source: everyone -- and can be halted by the same source: everyone, but more particularly management. These are meetings with several flavors. Those that take an hour where 5 minutes would suffice. Those that containing far more people than necessary (the invitations creep out). Those that span more than a single meeting ("Let's schedule another meeting to discuss that point"). Although everyone is ultimately to blame for allowing meetings to creep - almost invariably they are propgated by two kinds of people: 1) those who just plain enjoy the sounds of their own voice and 2) those who have nothing substantial to contribute and so drag meetings to fill their days.

The first group is simple to spot via consistent catch-phrases, such as: "Just to reiterate... (followed by the entire meeting relayed back to the conversation)" or "I just want to clear up a point for the group... (followed by something that everyone already understood)". This type of person often talks constantly, incessantly, at any time - meetings are merely another stage to let everyone know how insightful they are. This person will often ask obvious questions that do not need asking, simply to talk. Or he or she will make random offtopic points that never needed made, and repeat himself or herself ad nauseum. Whatever the vehicle - this person will end up talking at great length. Often times these guys are management - having been promoted for being "great communicators" - as if not being able to shut up were a virtue.

So how can we avoid this? The first group are actually difficult to stop, due to their seeminlgy innocent reasons for talking (how can you blame someone for not understanding? I find the simplest method for dealing with the first type are to keep notes and sit by them. When they begin to talk - before they can start to "re-iterate" or "clear-up points" stop them and hand off notepad, or tell the group that you will send out the meetings notes, and let the leader know that it's ok to continue - if the leader is saavy enough he'll catch your cue and keep going.

The second group are slightly less annoying - but tend to be management so may be harder to deal with. These are the kind of creepers that like to invite waaaay too many people to a meeting, or to split off meetings ad infinitum like some kind of Mandelbrot branch. If you have the choice avoid these meetings altogether. If not, your best recourse is to be proactive enough to stop a single meeting from becomming five. Oftentimes a single meeting will split because this kind of manager likes to "keep everyone involved" - translated as "I manage people, so the more people involved the more work I must be doing". If the meeting starts to sway into a direction outside of your domain - do not allow anyone to reschedule with someone else involved, for example: "We really need to get Johnson in here to ask about that DB issue - reschedule this meeting for Tuesday". You should grab the phone and call Johnson now. He may be available for a 2 minute question. BAM. You just saved yourself and everyone else another meeting.

There are certainly other types of meeting-creepers, and other ways to handle them. Ideas?

Handling Slippery Managers

This is something I've wanted to write about for a while. I hear too many horror stories from great programmers around getting screwed by office politics: a project getting pulled; being strong-armed into bad technology decisions; forced to stay late-nights while the boss take 100% of the credit. So, this post is the first in a series about simple things programmers can do to deal with office politics.

Get it in writing. If you are promised by your manager (or anyone) something - anything - get it in writing before you commit. Especially important things: a better job title, the choice of the next project's technology, more money, or deciding on the new coffee