Wednesday, May 6, 2009

Google App Engine Scales and Develops Fast

Really. It's amazing. After a quick chat about the concept with Jim, I bet my wife I could get a web app knocked out which accepted text messages, stored and printed them in an hour ( It took 35 minutes. In fact, I have a feeling this write-up will take longer. Now THAT's progress.

(FYI - sauce:

Getting started
First, create a Google App Engine account. If you already have gmail, just use that. You can have up to 10 apps.

If you download the SDK and install it for your OS, you'll get a nice little launcher with it. Right-click new, and you're on your way. Launching is just as simple... just click "Deploy", enter your account login, and your app is launched. In my case, (all GAE apps are subdomains of appspot).

The source code couldn't be simpler. First, appengine db models are mapped to Google's BigTable - a columnar (not relational) database. The nice thing about this kind of dataset is that OO models map fairly directly to the data on the backend (no ORM). The downside is that you'll have lots of duplicate data - since joins are really expensive. But I digress. All we need to do is store a text message, the phone number it came from, and the datetime the txt was made. Ready? Java people cover your eyes - you may cry from jealousy:
class Barf(db.Model):
text = db.StringProperty(required=True)
phone = db.PhoneNumberProperty()
date = db.DateTimeProperty(auto_now_add=True)
That's it. GAE creates a BigTable type which maps to this design (as well as automatically create any necessary indexes based on given queries, which then populates the index.yaml file. I could have put this in an external module (normally I'd put it in a file called and import it), but for the sake of speed and simplicity, I just put it in the file generated by the GAE SDK tools.

Next, we need to deal with request. GAE webapp maps URLs to files (in app.yaml - which already points root to by default), and uses 'WSGIApplication' to map internally URLs to RequestHandler classes. In this case, there are two urls: '/' and '/txtback' - one is the website URL, the other is so the SMS service can ping back data to the server. They are mapped in the main module
def main():
app = webapp.WSGIApplication([('/', MainHandler),
('/txtback', TxtBackHandler)])
Let's look at the root URL.
class MainHandler(webapp.RequestHandler):
def get(self):
barfs = Barf.all().order('-date').fetch(250)

path = os.path.join(os.path.dirname(__file__), 'templates/index.html')
self.response.out.write(template.render(path, {'barfs': barfs}))
Again - hopefully straightforward (isn't python so readable?). If a GET request happens, first fetch 250 Barf objects reverse ordered by date. Next, get the path to an external template file and write the rendered template (passing in the barfs objects) to the response output. Most of the magic is in the template.

The template is mostly a standard HTML file, with a little bit of server-side markup.
{% for barf in barfs %}
{% ifchanged %}
<li><div class="date">{{ }}</div></li>
{% endifchanged %}
<div class="time">{{|date:"P" }}</div>
<div class="quote">{{ barf.text|escape }}</div>
{% endfor %}
Again - it should be self explanatory. Iterate through each "barf" object. If the date has changed since the previous loop, output it. Then, output the formatted by the date formatter with type "P" (simple time) and the given text. That's all you need to output stored data. Google manages the rest for you - all with a scalable base.

Accepting Text Messages
Next, we want to accept text messages to populate the text Barf objects. There's no need to create a template here, a simple text response it sufficient. Our main goal is to store the input.
class TxtBackHandler(webapp.RequestHandler):
def get(self):
text = self.request.get('message')
if text: text = text.strip().lower()
phone = self.request.get('min')
if phone: phone = phone[-10:]

b = Barf(text=text, phone=phone)

We expect two request attributes, create a Barf object, save it and write back the URL. Simple as that.

But that was only the callback. To accept text messages, I signed up for a 41411 shortcode account (named "tbarf") through Once you sign up and verify the account, you need to provide the callback URL. Before you can do that, there must be a URL to hit. So, I click the deploy button. 5 seconds later, the app is up an running. All that's left is to give the URL to TextMarks - in my case:\p&message=\0 (where \p and \0 are interpolated with the calling phone and message data). 41411 is great, free, and they make money by ads.

Seriously - that's it. That took a grand total of 30 minutes. So, I registered a URL and pointed it at the app (, then just for good measure, slapped in Google Analytics. This whole app cost a total of $10, for the sweet, sweet domain.

Voilà! Eat it, EC2! Just for the record - this post took about an hour.