Friday, November 30, 2007

Mac OS X Curl +SSL Support via DarwinPorts

If you attempt to use the default curl install through port, you may be disappointed to find that https et al won't work. The default install does not include support for ssl, which is a variant.

However, if you already have curl installed you may have to uninstall it first.
sudo port uninstall curl
Moreover, if you have multiple versions installed, you may have to uninstall the default - in my case 7.17.0_0 (don't forget the @ symbol)
sudo port uninstall curl @7.17.0_0
Finally, reinstall with the ssl variant:
sudo port install curl +ssl

Thursday, November 29, 2007

Cheap Tricks XIII - Reading CSV Files using Header Keys

I read a lot of CSV file, most of which come with a one-line header describing the successive lines. For example:
ID,TEXT,COST
1,Gumball,0.05
2,Pack of Gum,0.50
3,Pepsi,1.00
In Ruby there are a few ways to go about parsing this file. You can install a gem like fastercsv, but I prefer just to parse it directly. You can also just skip the first line and assume a standard layout of data, but this is a little error prone if anyone decides to add or remove columns. It's easy to build a header lookup from that first line, like so:
h = line.split(',')
headermap = Hash[*h.zip((0...h.length).to_a).flatten]
This splits up the string by commas into an array, then uses the array's "zip" method to pair each element with a number - from the range 0 to length-1, then flattening, giving us an array that looks like this:
["ID",0,"TEXT",1,"COST",2]
Then we pass in the array as an unbounded param list and use the Hash[] function to create a map like this:
{"COST"=>2,"ID"=>0,"TEXT"=>1}
Pretty nifty for two line, eh? That way we can split up each row by comma, and access the array position by the header name, rather than a magic number:
SEPERATOR = ','
data_ary = []
hm = nil
IO.foreach(filename) do |line|
unless hm
h = line.split(SEPERATOR)
hm = Hash[*h.zip((0...h.length).to_a).flatten]
else
d = line.split(SEPERATOR)
data_ary << {
:id => d[hm["ID"]].to_i,
:text => d[hm["TEXT"]].to_s.downcase,
:cost => d[hm["COST"]].to_f,
}
end
end

# An array of hashes
p data_ary #=> [{:id=>1, :text=>"Gumball", :cost=>0.05}, ... ]

Tuesday, November 27, 2007

Cheap Tricks XII - Read SSL Email and Attachments

I need to connect to gmail to grab reports sent to me as attachments -- I'm also not using ActionMailer. What to do? Ruby/Gmail has two issues working against it: SSL required (POP3 SSL only available via trunk) and POP3 doesn't have built-in attachment-grabbing support. This is a quick-and-dirty hack to download the attachments as a file.
require 'net/pop'

BASE_DIR = "/tmp/attachements"

Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_NONE)
Net::POP3.start('pop.gmail.com', Net::POP3.default_pop3s_port, 'email@gmail.com', 'passwd') do |pop|
if pop.mails.empty?
puts 'No mail.'
else
pop.each_mail do |email|
msg = email.pop

msg.scan(/content-type: application\/octet-stream;\s*name=(.*?)\s*content-transfer-encoding: base64(.+?)(---)?/m) do |v|
filename = $1.to_s
data = $2.to_s.strip.unpack('m*')
File.open("#{BASE_DIR}/#{filename}", File::CREAT|File::TRUNC|File::WRONLY) do |f|
f.write(data)
end
end
end
end

Monday, November 26, 2007

Micromanaging - Tasks versus Goals

This is something I've wanted to write about for a while, but held-off for fear of another 10 page rant. But, taking a page from Seth Godin, I've decided to try writing less. Here goes.

I think it is well understood the weaknesses of task-based software development processes - hence the popularity of agile frameworks. Telling a programmer to code a small little piece of a large framework out of context is error-prone, slow, and most importantly a difficult way to keep talent. Give me a piece of a system and say "work with teams A, B and C to make D" is entirely different than saying "Do tasks X, Y and Z" without any basis for them. Good managers want the input of the expert paid to do the job - even if that expert is a 21 year old pimply-faced programmer punk. Sure, I might have more overall smarts and education than my mechanic - but when he's elbow deep in grease with my car taken apart I'm not going to give him tasks, just a goal: make it work. Anything more is micromanaging and gives substandard results (since I'm not an expert but he is) -- that doesn't mean I can't keep an eye out on him from dumping metal shavings into my oil-pan.

Micro-managers give "tasks"; Good managers give "goals".

It may sound like a mere semantic difference, but there is some real value in conceptualizing the distinction. Shortly, a task is a step toward reaching a goal. Sadly, there are people who cannot tell the difference. If you have one, or are one, my condolences. Some good things about goal-based over task-based management:

  • Easier on the manager (my favorite reason). If a manager has to take a collection of goals from higher-ups or customers (the ultimate higher-ups) and distill them into tasks, that puts a great strain on the manager. (S)he becomes a point of failure in the organization, since all new goals must travel through that point. However, when a manager spends time giving out goals rather than managing tasks, (s)he can spend the time ensuring that the goals are reached satisfactorily.

  • Gives a sense of ownership to employees. Great for morale, sure, but also keeps talent around. Smart people don't like feeling like cogs, and almost always prefer to have a stake in the outcome. When given a goal "get project A done in two weeks - by any means necessary" rather than a series of tasks "come in between 8am and 5pm; 3 days of design, 4 days of coding, 3 days of testing; utilize technology X" they are more likely to care about the outcome - it becomes their baby.


Are these benefits psychological? Sure. Companies are groups of people working toward a common set of goals - psychology is an important organizational component. But it has another bonus beyond the sanity of employees: goal-based directives are developed faster. My feelings on fast initial development are no secret, mainly because most projects have such short lives (if any at all) building "perfect" architectures are sinkholes of money and time. Complex top-down architecture (as opposed to emergent) take a high-minded view of an oligarchy which are suited for dividing labor into discreet tasks: create a caching mechanism, create a logging framework, create the model layer, create the DAO layer, etc. Oftentimes (I'll go as far as say usually, in my experience) these tasks are divided up to several people. Can you spot the vicious waste? If person A is working on component 1a, and person B is working on 1b you have now succeeded in failing in two ways:

  1. If person A quits, person B can hardly take on 1a without a learning curve. Perfect interchangeability of programmers is a pipedream - you have gained nothing by way of componentizing the org chart.

  2. Person A and B must cross artificial barriers to communicate (viz, between 1a and 1b). If person A and B were given a goal (work on component 1) they could leverage each others strengths, as well as cooperate in a manner comfortable to themselves to avoid redundancy. But as it stands, A will work on 1a, B will work on 1b, and any common effort not outlined by the oligarchy will be duplicated (1ax and 1bx - versus 1x).

Proponents of the top-down, task-based method will argue: "Iterations fix this problem!" True, after the effort has been expended once, retrospect will come into play and fix the original faulty analysis. But why go through the effort? Just let A and B loose to develop 1, fixing any architectural defects on later iterations. A simple shift in tact frees up a manager to work with the employees by removing obstacles rather than managing a check-list. Either that or going golfing.

Tuesday, November 20, 2007

Code Te Ching - Verse 40

Good developers are often defiant.
    But the best understand the true role of management.
Good managers are often decisive.
    But the best understand the true role of management.

What is the true role of management?
It cannot be defined, only described:
    To shield good developers from the chaotic world of business.
    To ensure good developers get what they need.
    To empower good developers making good decisions.
    To entitle good developers to the credit they deserve.
    To help good developers stay centered.

The wise manager is ambitionless.
He does his work and quietly steps back.

Friday, November 16, 2007

Stop User Group Assholery!

We've all seen it - and I'm not talking about being a flaming jerkwad. I can ignore that, and hence via my powers of displacement, forgive it. But this is an unforgivable sin in this modern age of search engines and public forums. Don't be this guy, though I know you have witnessed at one time or another, mining the very dark corners of the web for an esoteric answer, scrambling to find a single relevant hit on google.co.jp:
Hey guys! I'm having this problem and can't find an answer
please help!!! :(

[Some trace or dump]

Thanks;
Random Asshole
Only to be responded to hours later:
Random Asshole wrote:
>
> Hey guys! I'm having this problem and can't find an answer
> please help!!! :(
>
> [Some trace or dump]
>
> Thanks;
> Random Asshole

Ok, I figured it out and it works... ;)
Do you not see the problem here? Irony aside, you're just plain being a dick. I have a rule. Let's call it - oh, I don't know - the Golden Rule: Treat others how you want to be treated. Remember that one? If not, learn it. Its reach extends beyond the elementary school playground.

I propose a list of people who ask for help but do not reciprocate. Sort of like a "Do Not Call" list or "Sex Offender" registry. But instead of for telemarketers and frat guys, respectively, it's for assholes. Any names appearing on that list are barred from having their questions answered until such time as proving themselves nice little Netizens and sowing what they reap. Until then - need to find an answer to my problem: "Internal protocol error: No valid method call - missing method name", no thanks to this guy.

Thursday, November 15, 2007

Faster Page Loads With Image Aggregation

I'm so rarely impressed, that when the stars align with the correct ingredients - newness, cleverness, utility (and stardust) - I'll be recharged with enough school-girl giddiness for a good week. Today I realized the beauty of faster page loads with image aggregation. Naturally I had to run home and try this trick on SnipSnipe, it's now 2am and I'm still not sick of it! It works like this:

Take all of the standard images on your website and turn them into div/span elements of matching width/height. In SnipSnipe, I replaced the upper-right Firefox image:
<img src="/images/firefox.gif" />
With:
<div id="ffimg" />
Using CSS, we can create an object of similar height and width, setting the background image as the image we wish to see:
#ffimg {
width:211px;
height:92px;
background-image: url('/images/firefox.gif');
background-repeat: no-repeat;
background-position: 0px 0px;
}
This is such an old trick you can find it painted on cave walls. A user will see the image in their browser as though it were inline via the img tag. Sure, you gain flexibility, but the way this ball lays we don't exactly gain any strokes. If we do this with 15 images, the browser still must make 15 requests to the server - but we are one step closer.

Our next task is to create an image that contains all images we want loaded onto the page. The SnipSnipe version is right here.

Now for every image it is a simple matter of altering the css to point to the same big image, only reposition the image to the correct pixel location.
#ffimg {
width:211px;
height:92px;
background-image: url('/images/big.gif');
background-repeat: no-repeat;
background-position: -208px 0px;
}
This example shows a 211x92 pixel area of the big.gif image starting 208 pixels from the left side of the div (you must make them negative numbers since they shift left).

A quick tip: if you set the background of an 'a' tag to one image, it's a fast way to do a rollover - just shift the background-position on a:hover.

And that's it! The example I used contains 12 separate images aggregated into one. So rather than a dozen requests totaling 21k of data transfered, instead there is only one request of similar data size. As we like to say in the biz, it's chunky not chatty - reduces strain on your server and serves up images faster than piecemeal.

Thursday, November 8, 2007

Maven Information

I've decided to start dumping my massive Maven knowledge piecemeal into SnipSnipe, tagged under "maven" (http://snipsnipe.com/tag/maven) before I forget it all in favor of Ruby trivia. For starters I populated from the main site's FAQ.

New SnipSnipe Features

Oh Ferret, how awesome are you? For those who don't know, Ferret is a Lucene port in Ruby which, speed-wise, gives Lucene a run for it's money. More than that, it also has integration with memcached which, if it weren't running on a shared instance I would sooo be all over that. SnipSnipe isn't quite that popular yet, so I have a little time before worrying about dedicated hosting :)

Anyway, thanks to Ferret, SnipSnipe now has searchability. Actually, I like Ferret so much, I'm considering replacing acts_as_taggable entirely with acts_as_ferret. The benefit here is two fold: one, it reduces model complexity, which currently looks something like this (seriously):
class Code < ActiveRecord::Base
acts_as_taggable
acts_as_voteable
acts_as_commentable
acts_as_favorite
acts_as_ferret

end
More importantly, two, it increases the flexibility of searches. I like the idea of giving the model semantic metadata, which I already do somewhat (tags pull double-duty as a desciptor of the snippet as well as specifies how the code is to be highlighted - code tagged 'xml' will be highlighted differently than code tagged 'java' - it currently won't let you choose both). I'd like to be able to choose multiple tags, like "maven", "java", "plugin" and let the system drill down for me. Currently you can only sort by one snippet at a time.

Back to the new feature. First, notice the search bar in the upper right:

When you perform a search, it does what you'd expect. It searches matching terms in the snippet title, description, tags, or the code itself. Naturally this search is bookmarkable - as one would expect.

The second new feature is favorites. If you are logged in, you will notice a heart icon when hovering over a snippet. Clicking it will add that to your favorites list.

Later, if you click on the FAVORITES menu item you'll get a list of snippets you've marked. You can actually share your favorites list with other people. Clicking on "my"...

...will direct you to the page that others would see.

Favorites lists are normally searchable, so you have to specifically give the url to someone. This may change in the future.

Also, someone (thanks John) noticed that the old Firefox plugin no longer works, since I altered/simplified the URLs. Please update to version 0.6

Any other comments/suggestions are welcome.

I'll later post on my progress using Ferret as a replacement for taggable, as well as how I'm fixing pagination.

Monday, November 5, 2007

Cheap Tricks XI - OSX Image Compression?

All of the images I had used on SnipSnipe were huge. Not HUGE, just not as small as they could be. I do most of my image creation in Fireworks, where an average image will be around 16k, with the smallest around 4k. I opened each image and re-saved it using Mac Preview, and it compressed the images over 10x! Does anyone know - does Preview use awesome compression, or does Fireworks just suck at it?

Thursday, November 1, 2007

Code Te Ching - Verse 39

Computers are an extension of intention.
Therefore, the sage is One with the machine.

What do I mean when I say:
    "One with the machine"?
Do not think like the machine, only,
let it be an extension
        of your intention