It's All Geek to Me

What to do with Locidesktop?

So what to do with locidesktop.com? It's a desktop-like website bookmarking tool – if you haven't seen it, take a quick look at this example desktop.

I built Loci Desktop a few months ago and promoted it on a few geek sites. It's been running ever since, with no maintenance from myself, happily serving up start pages to a small number of regular users. There was a buzz when I promoted it, people were largely impressed, some were indifferent, but few ended up using it regularly. So now I'm left with a quandary.

I could try and promote it. But to what end? It's not like I need a certain number of visitors to cover the hosting. I'm using the same VPS as I am for my blog, and I designed Locidesktop to be ultra-low bandwidth anyway – so it effectively costs me nothing to run.

One option would be to sell the entire site outright, as the domain and technology rights. But there is currently no way of monetizing it and I doubt anyone would be interested as a commercial venture.

I could try and license it as a b2b service. A few people have commented that it would be a useful intranet service. I'm not sure about this, but it sounds plausible.

Alternatively, I could give back to the Django community and release it as open source, which I imagine would be the popular thing to do. Money isn't my primary motivator (a close second perhaps), so I wouldn't be averse to doing this. Thing is though, it would require work on my part to document it and maintain it, and I have other open source projects I would prefer to concentrate on. But I can't deny that it would be cool to see Locidesktop sites popping up over the interwebs.

Finally, I could just leave it as is. I'm pleased with how it turned out, and I have a few loyal users. Maybe I should just be satisfied.

Any options I haven't considered? Leave a comment…

PyFilesystem 0.3 released

I am pleased to announce a new version of PyFilesystem (0.3), which is a Python module that provides a common interface to many kinds of filesystem. Basically it provides a way of working with files and directories that is exactly the same, regardless of how and where the file information is stored. Even if you don't plan on working with anything other than the files and directories on your hard-drive, PyFilesystem can simplify your code and reduce the potential of error.

PyFilesystem is a joint effort by myself and Ryan Kelly, who has created a number of new FS implementations such as Amazon S3 support and Secure FTP, and some pretty cool features such as FUSE support and Django storage integration.

As an example of how awesome this package is, take a look at the following 6 lines of code, which creates a ramdrive:

from fs.osfs import OSFS
from fs.memoryfs import MemoryFS
from fs.expose import fuse

home_fs = OSFS('~/')
home_fs.makedir('ramdrive', allow_recreate=True)
fuse.mount(MemoryFS(), home_fs.getsyspath('ramdrive'))

If you run this, a directory called ramdrive will appear in your home folder, the contents of which are stored purely in memory.

I prepared a screencast that gives a quick demonstration of some features – because if a picture is worth a thousand words, this video must be worth fifteen thousand words a second:

PyFilesystem screencast from Will McGugan on Vimeo.

See the project page on google code for more information, including API docs. There are also a couple of blog posts that will give a some more context.

This release has reached a good level of stability and maturity. I'd like to invite as many Pythonistas as possible to check out this module and possibly contribute to the project.

Review of Django 1.2 E-commerce

I've worked with Django for more than two years now. The majority of the sites I have worked on have been social-networking or content based, but I have yet to do any serious work on a site where the main purpose is to advertise and sell products. So I when a copy of ‘Django 1.2 e-commerce’ landed on my desk I was intrigued by what it might cover that I hadn't been exposed to with other fields of Django development.

Django 1.2 E-commerce

The book starts out with a brief run-down of Django. The first chapter is more of a explanation of the philosophy behind Django, and definitely not a tutorial. Which I think is fair enough; if you are building an e-commerce site, you are probably a professional Python developer and there are plenty of books to get you up to speed with Django. In the second chapter, the author runs through a simple web-shop application with an inventory and a ‘buy now’ button – which seemed more like a confidence building exercise than anything else, but it does do a good job of demonstrating how simple it can be to build this kind of application with Django.

The subsequent chapters go in to detail regarding managing users, shopping-carts and taking payments with Google Checkout and Amazon services. I've never used these payment services, so I found the information particularly interesting. I like the author's approach of making generic views in order to share functionality across payment services, but I would have liked some more detail in to the APIs involved.

Chapter 6 was a surprise, it covers a variety of modules that you can use to add powerful search capabilities to your application. Django's database querying will only get you so far with searching, if you need more sophisticated searching of the kind you would expect from Google then you will need to integrate one of a number of external modules and services, which chapter 6 covers pretty well. The following chapter covers exposing data via several APIs, and creating PDF reports with ReportLab, but not in any great detail.

I was impressed with chapter 8, which covers writing JavaScript to create rich AJAX interfaces – something which is pretty much expected in a modern web-site. I would have preferred a more detail here, but only because I have a particular interest in front-end technologies. The next chapter explains how to integrate a Django application with Amazon payment services and S3 storage to sell digital goods, and goes it to more detail than other chapters.

The final chapter covers a number of options you will have for deploying your application. If your application is moderately sophisticated and has many components, deployment can be a tricky affair. This chapter explains how Python technologies such as Fabric, Buildout and Virtualenv can ease deployment headaches. It also covers serving the site with Apache and ‘mod_wsgi’.

The code snippets in this book are pretty good at demonstrating the subjects covered, but I did notice some quite glaring syntax errors in several of the code examples. The errors weren't subtle either; they would result in the code not even running. I suspect that many of them were likely to be the result of a non-technical editor re-formating the code and not a mistake on the part of the author, but there were also a number of programming errors and bad practices which were a little disapointing to see in a book aimed at professionals. For instance, the author consistently used the ‘is’ operator in place of the equality operator (they are not interchangable even though they may appear to be).

Overall, my impression of this book was favourable. It's definitely not a tutorial book in that its not going to teach you any new skills – since it covers so many technologies and doesn't go in to great detail about any of them. What it will do is give you a grounding of the components in an e-commerce system. If you are looking to build some kind of web-shop in Django then I would recommend this book. It's less of an essential purchase if you aren't working with e-commerce, but since many of the topics discussed in Django 1.2 e-commerce are applicable to other web-sites you may still want to check this book out.

Announcing Sore Thumb, a thumbnail and image processing module for Django

I recently worked on the re-design of 2 Degrees, which required a lot of image processing on thumbnails. The thumbnails where to be in a variety of different sizes, all with rounded corners and keylines on a selection of virtually identical off-white backgrounds and gradients. And they all had to work on IE6 *spit* without the transparency hack.

A variety of thumbnails generated by Sore Thumb

A lesser engineer may have told the front-end developer where to stick his rounded corners, but I didn't want see a grown man cry, so I built Sore Thumb, an on-the-fly thumbnail and image processing system for Django.

Sore Thumb uses a declarative method of defining thumbnails, similar to Django's model and form definitions. Here's an example of how to declare a thumbnail processor that produces a 120x100 pixel thumbnail with 10 pixel rounded corners and a dark grey keyline:

from sorethumb.djangothumbnail import DjangoThumbnail
from sorethumb.filters.defaultfilters import ThumbnailFilter
from sorethumb.filters.drawfilters import RoundedCornerFilter

class RoundedCornersEdged(DjangoThumbnail):
    format = 'png'
    filters = [ThumbnailFilter(120, 100),
               RoundedCornerFilter(10, border='#333')]

Once this class has been imported, the thumbnail processor will be available in templates via the sorethumb filter which takes a Django FileField and returns the url to the thumbnail.

For example:

{% load sorethumb %}

<img src="{{ profile.photo|sorethumb:"rounded_corners_edged" }} />

You will also need to add sorethumb to your INSTALLED_APPS for this to work.

That's pretty much all there is to working with Sore Thumb, see these examples for inspiration and the documentation for the details.

You can install sorethumb with easy_install, PIP, or directly from source:

easy_install -U sorethumb

Please let me know what you think of Sore Thumb. I'd rather not spend too much time maintaining it, but since most of the code has been in production for a number of months already, there shouldn't be many changes required. If you have any feature requests or bug-reports, now would be a good time to raise them while I have the time to do the work!

Django E-Commerce

Pakt publishing have released a new Django book called Django 1.2 E-Commerce, written by Jess Legg. I'll have a copy soon and post a review here.

In the meantime they are offering a free chapter of the book; Chapter 2 Setting up Shop in 30 minutes.

Exciting Python Developer Job

Meebo, the instant messenger in your browser company, are seriously looking for Python developers right now to work in Mountain View, California. From what I can gather they are expanding, and are building a number of sites in the Django framework.

Damn foreign keys, stealing our jobs and women

Django has support for Generic Foreign Keys, which let you reference one model instance from another, without knowing up-front what that model type is. The classic use for something like this is for a commenting system; you need generic foreign keys – or something like them – because you wouldn't want a commenting system that only worked with a single model.

If you have ever used generic foreign keys in Django, you will know that it is not quite transparent to the developer; a little effort is required to manage the various content types. I'll present here an alternative method to achieve this late binding of foreign keys that doesn't require storing the type of the object (as generic foreign keys do) and is completely transparent to the developer. I'm sure I'm not the first to think of this method, but I haven't yet seen it used in other Django projects.

Rather than store the type of object in a separate field, we can create a new model for each foreign key type we want to reference. For example; lets say we have a Rating model, and we want to rate Articles and Images – we could do this by generating a ArticlesRating model and a ImagesRating model with appropriate foreign keys. The easiest way to do this is with a function that returns a parameterized class definition.

Here's a snippet of code from a project I'm working on, that does just that:

rating.py

from django.db.models import Model, ForeignKey, IntegerField, Count, Avg
from django.db import IntegrityError
from django.contrib.auth.models import User

def make_rating_model(rated_model, namespace):

    class Rating(Model):

        user = ForeignKey(User)
        rated_object = ForeignKey(rated_model)
        vote = IntegerField(default=0, blank=True, null=False)

        class Meta:
            abstract=True
            db_table = u'rating_%s_%s' % (namespace, unicode(rated_model).lower())
            unique_together = ('user', 'rated_object')

        def __unicode__(self):
            return u"%s's rating of %s" % (self.user.username, unicode(self.rated_object))

        # Rest of the methods snipped for brevity
        # Contact me if you would like the whole class

    return Rating

This isn't a model definition, rather it is a function that create a model definition. You can call it multiple times to return a Rating model for each object you want a rating for. The function, make_rating_model takes two parameters; the name of the model you want to rate, and a string that is used to generate the table name, to avoid naming conflicts.

To create a rating object you would import ratings in your models.py file and add the following:

class ArticleRating(ratings.make_rating_model('Article', 'mysite')):
    pass

class ImageRating(ratings.make_rating_model('Image', 'mysite')):
    pass

Now if you syncdb you will get two completely independent models with essentially the same interface – which means you can write code that works equally well with model instances of either type.

This method doesn't quite replace generic foreign keys; if you don't know until runtime what model to reference, or if you require the objects to be in a single table, then you will still need generic foreign keys, but in my experience this is rarely the case.

Python developer programming tests

I recently administered a programming test to a number of applicants for a Python developer job at 2degreesnetwork.com. Since we now have a new developer (hi Gustavo!), I figured I would post the test and see what kind of response I get to them.

There are two parts to the test, neither are hugely difficult, but there is enough scope in the solutions to understand how the candidate approaches a problem.

In the first part of the test I asked the candidate to look at the following code and implement the thousands_with_commas function, which should take an integer and return a string representation of that integer with commas separating groups of three digits:

def thousands_with_commas(i):

    return str(i)


if __name__ == '__main__':

    assert thousands_with_commas(1234) == '1,234'
    assert thousands_with_commas(123456789) == '123,456,789'
    assert thousands_with_commas(12) == '12'

I think there is a way of doing this with the standard library, and there is also an implementation in Django, but I was looking for a solution from scratch.

It worked quite well as an interview problem, because there is no one obvious solution and there are a few potential gotchas to tackle.

In the second part of the test, I asked the candidate to implement a function that uses a word list to return the anagrams of a given word.

I started them off with the following code:

def annograms(word):

    words = [w.rstrip() for w in open('WORD.LST')]

    raise NotImplementedError


if __name__ == "__main__":

    print annograms("train")
    print '--'
    print annograms('drive')
    print '--'
    print annograms('python')

This part of the test gave a good indication of how much a candidate new about data structures and performance.

Feel free to post your solutions in the comments, although I suspect I've seen pretty much all variations on possible solutions!

When did I become the Javascript guy?

I have been busy working on locidesktop.com lately. One popular feature request from the beta testing was to use website favicons, so that links become more readily identifiable – which I have implemented after several late nights, and abusing my quad core.

A few of the 3D Favicons used in locidesktop.com

I built a pipeline that downloads favicons, extracts PNGs which are then processed in to a 3D scene and rendered for missing resolutions (.ICO files can contain multiple resolutions). I've tweaked the way that the 3D icons are produced, and they are significantly better than my first attempt – although some do come out better than others.

The 3D icons are integrated in to locidesktop so that they are pulled in whenever you add a site to your desktop. At the moment I have about 20,000 of them, and I'll be adding more in the future.

The desktop editing interface has been polished considerably, and I think that it is pretty slick now, although there are still plenty of improvements to be made, especially in the area of context sensitive help.

I've also created a test account so that I can demo the features of locidesktop.com without a login. The url is http://locidesktop.com/test/ to view the test desktop, and http://locidesktop.com/test/default/edit/ to play with desktop editing (but you won't be able to save).

As always, I welcome feedback…

Talented front-end developer job for Web 2.0 company in Oxford

The company I work for, 2Degrees, is looking for a front-end developer to join our team.

Are you a CSS monkey?

We need a CSS monkey with a good working knowledge of browser quirks and the ability to get even IE6 looking good (although you don't have to like it). It would help if you don't run away screaming from Javascript and can play well with the code monkeys.

More details are below. Email the address at the bottom of the job description, and mention this blog!