October 6, 2007 will

Being Contrary with Turbogears

I promised to write up a report on my experiences building becontrary.com with Python and the TurboGears framework. An earlier post explains the concept of the site, and what it is intended for.

I wanted to use Python to build a site, rather than something like php because Python has always made me significantly more productive as a programmer, and frankly because I am a huge Python fan-boy. Before I started on 'debate site', as I called it then, I dithered over which of the frameworks to use. The two major contenders at the time were Turbogears and Django, although I did consider other Python frameworks, and even Ruby on Rails. I was swayed towards Turbogears because it uses CherryPy, which I had had used previously and found ridiculously easy to use. To be honest though, my choice of framework was fairly arbitrary, but I do tend to suffer from analysis paralysis and I needed to make a decision. Ultimately I suspect that the choice of framework is less important than it may appear to be.

Turbogears was designed with the Model View Controller (MVC) paradigm in mind, which separates the web application in to three logically separate components. The Model is the database, the View is how the database information is presented and the Controller is the code that manulates the database. Its a well-established pattern that works well for web apps.

Getting Started

Getting started with Turbogears is a breeze. The command line interface takes a few parameters and generates a working application for you, which is very encouraging. Unfortunately I ran in to a few bugs very early on, and they weren't insignificant bugs either, they were kind of show-stoppers at the time. Fortunately help was to be found on the mailing list and I was able to work around them. I can't even remember what they were now, and I imagine they were fixed a long time ago.

The Model

One of the major attractions of a Python framework is that it can hide the details of working with a database behind an Object Relational Mapper which enables you to work with the database as if it were a collection of Python objects. Or at least it tries to. I found that SQLObject, the ORM used by Turbgears, didn't create a perfect abstraction and to get the best out of it I needed to understand what was going on behind the magic. For the majority of tasks though, SQLObject made it trivial to work with the database.

Incidently, I don't like the way that Python ORMs specify the database schema as Python code. Since the schema doesn't change when the application is running, I would prefer a simple xml definition that would be used by the ORM to dynamically build classes for each table that could be derived from in the model code. That way the schema definition could be ORM independant and also kept separate from the code that manipulates it. Perhaps I'm not seeing the big picture, having never tacked writing a database layer.

The View

Turbogears uses Kid templates, which are basicaly XHTML files with additional tags that are processed with Python to generate viewable output. I really didn't like the idea of hand editing XML to generate content, and I had intended to create the templates with Dreamweaver. However, I found that with a good XML editor it was not quite as error-prone as I had expeced. The Kid language is very expressive and I didn't find any limitations in the design, but I was frustrated by the fact that mistakes in the template didn't generate useful error messages. This one issue lead me to switch to Genshi, which is similar to Kid but gave far more specific error messages. The translation was painless and with experience I became very adept at writing the templates.

Although templates are considered the View component of the framework, I found that the CSS code deserved that title more than Genshi because I could generate very simple XHTML content with templates, and make it look nice with CSS. This made me feel a little better about occasionaly abusing the MVC purity by including code snippets in the templates. Practicality beats purity, right?

The Controller

CherryPy maps HTTP requests on to methods of a controller class, which makes handling the functionality for requests as simple as writing Python code. Most of the methods simply validated some parameters and pulled a few entries from the database. Even complex pages only required a few pages of code. This surprised me actualy, because the majority of the work was not Python, it was in the templates and CSS.

AJAX

I used AJAX in several places, to make the application seem more responsive. Creating an account for example. Such pages are usual tedious and error prone and, as a user, I often give up. I hate it when mistakes in the form throw you back to the same form, but the contents aren't as you left it (e.g passwords fields are blank). The account creation form in BeContrary.com is a single page and you get feedback from errors quickly.

The only place where AJAX was essential was in the voting. The arguments to each debate can be given a vote between -2 and +2. If the user had to wait for a page refresh when voting it would be a tedious process, and not to mention taxing on the server. As it is, a tiny AJAX request is sent for each vote that updates the server. The controller handles AJAX requests in the same way as other pages, which made the server part of the system nice and simple.

Technically though, I didn't use AJAX because I didn't use XML to return information. I didn't even use JSON, because for most things all I needed was an 'ok' or a 'fail' message.

Conclusion

Its was a great hobby project. Once I had gotten over the initial learning curve, not just of Turbgears but of CSS and AJAX, then I became remarkably productive. I found that I could sit down for an hour (sometimes less) and implement a new feature. The quick turnaround of new features gave me the impetuous to finish it. Something that is far from certain with my hobby projects!

Deployment

I considered the hosts shown on the Turbogears website, but settled on webfaction.com. They are very dynamic language oriented and offer a competitive deal. The site has been up for a few weeks without a hitch.

Promotion

Now I need to think about promoting the site. Something which I have never been particularily good at. I tried a bit of shameless self promotion, but I never get any love from Reddit with self-submitted links. Typically anything I submit gets voted down immediately without a single referer showing up in my logs. *sigh*

Use Markdown for formatting
*Italic* **Bold** `inline code` Links to [Google](http://www.google.com) > This is a quote > ```python import this ```
your comment will be previewed here
gravatar
Stan

Minor nit: I think you mean JSON (as in json.org) and not JASON.
(Feel free to delete this comment if you fix it.)

gravatar
Will

You are absolutely correct, of course!

gravatar
Jamie

Hi. As a heavy Plone user, I have been wanting to do some web app development with something easier (for me) to manipulate at the lowest possible level, and so I have started doing some tests projects with TG. I appreciate thsi blog entry, as it has helped me see that someone else can bootstrap themselves into TG. A couple questions: How much did you use the widgets, and if you did, are they for normal pages as well as forms? I am doing a bit of Ajax with Mochikit, but I am having problems learning how to use the documentation - it seems there are not many examples to learn from - how did you approach the Ajax part of learning?
I have a much more extensive site in mind that also has a ranking process involved in it, so I will try your site out to pick up the flavor of what is possible with TG. Great job!

gravatar
Tennessee Leeuwenburg

You might like to consider submitting an article to The Python Papers. I love the idea of becontrary.com and would be happy to carry the article.

Cheers,
-T

gravatar
Will

Jamie,

I didn't use widgets. I was so eager to start the site that I didn't learn about all TG had to offer, and by the time I read up on widgets I had done a lot of work. I would definitely use widgets if I started another project. There are a few places where template code is repeated that I would like to refactor out at some point.

I didn't use Mochikit either! But this time because I wanted to learn how to do Ajax manually before I used a library to help me. I learned Ajax techniques from an Apress book and from tutorials I googled for. I found Firefox + Firebug to be an excellent tool for working with Ajax, because you can see exactly what is being sent to the server and what was returned. I don't know how people managed without it!

Will

gravatar
Simon Willison

It sounds like you used Ajax rather than AJAX. Capitalised, "AJAX" means "Asynchronous JavaScript and XML", but actually most people avoid using XML these days so that acronym didn't make much sense. More recently people have started calling it "Ajax" instead, which doesn't stand for anything but means "any technique which lets the client talk to the server without refreshing the whole page".

gravatar
Matt Wilson

Instead of writing XML that gets turned into python, why not parse SQL directly and build up the data structures from that? Then we could use SQL to construct our tables.

gravatar
Will

With an XML based schema for the model, the ORM could still hide the differences between platforms. Besides, isn't one of the benefits of an ORM that you don't have to work with SQL?